Infrared image enhancement method and device, equipment, storage medium and program product

Abstract

The invention relates to an infrared image enhancement method and device, equipment, a storage medium and a program product, and the method comprises the steps: decomposing and generating a basic layer image, a main detail layer image and a secondary detail layer image based on an input infrared image; respectively carrying out image enhancement processing on the basic layer image, the main detail layer image and the secondary detail layer image to obtain an enhanced basic layer image, an enhanced main detail layer image and an enhanced secondary detail layer image; and fusing the enhanced base layer image, the main detail layer image and the secondary detail layer image to obtain an enhanced infrared image. Therefore, the problems that most infrared image enhancement methods depend on global contrast enhancement or a simple image processing algorithm, image detail features and texture information cannot be fully kept, the target contrast is low, the detail performance is poor, the enhancement effect is poor, and subsequent image analysis and target detection performance are affected are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an infrared image enhancement method, device, equipment, storage medium and program product. Background Art

[0002] Among related technologies, infrared image enhancement technology can significantly improve the visual effects and subsequent analysis performance of original infrared images by enhancing image contrast, suppressing noise, and enhancing detail features, thereby more effectively assisting target detection and recognition. It has been widely used in military reconnaissance, security monitoring, industrial detection, medical diagnosis and other fields.

[0003] However, existing infrared image enhancement methods rely heavily on global contrast enhancement or simple image processing algorithms. While these methods can improve the overall visual quality of an image, they are limited in preserving local details and edge information. Especially for target areas in complex scenes, these methods often fail to fully preserve detailed features and texture information, resulting in low target contrast, poor detail representation, and poor enhancement effects, impacting subsequent image analysis and target detection performance. These issues urgently need to be addressed. Summary of the Invention

[0004] The present application provides an infrared image enhancement method, apparatus, device, storage medium, and program product to address the technical problem in related technologies that most infrared image enhancement methods rely on global contrast enhancement or simple image processing algorithms, which cannot fully preserve image detail features and texture information, resulting in low target contrast, poor detail representation, poor enhancement effect, and impact on subsequent image analysis and target detection performance.

[0005] The first aspect of the present application provides an infrared image enhancement method, comprising the following steps: generating a base layer image, a main detail layer image, and a secondary detail layer image based on decomposition of an input infrared image; performing image enhancement processing on the base layer image, the main detail layer image, and the secondary detail layer image, respectively, to obtain enhanced base layer image, main detail layer image, and secondary detail layer image; and fusing the enhanced base layer image, the main detail layer image, and the secondary detail layer image to obtain an enhanced infrared image.

[0006] Through the above technical means, the base layer image, main detail layer image and sub-detail layer image generated by decomposition are respectively subjected to targeted enhancement processing to achieve the enhancement of infrared images. Differential enhancement of features at different levels can be achieved, effectively improving the structural clarity and detail richness of the image, while suppressing background noise interference, bringing higher image contrast and more delicate texture expression, thereby significantly improving the application effect and reliability of infrared images in target detection, feature extraction and subsequent visual tasks.

[0007] Optionally, in one embodiment of the present application, the base layer image, the main detail layer image and the secondary detail layer image are respectively subjected to image enhancement processing, including: based on the base layer image, using a contrast-constrained adaptive histogram equalization strategy to enhance the overall contrast information; based on the main detail layer image, using a multi-scale detail-guided enhancement strategy to enhance important details of different scales; based on the secondary detail layer image, using a Gaussian-Laplacian operator to extract detail features.

[0008] Through the above technical means, different enhancement methods are used to process the base layer image, main detail layer image and secondary detail layer image respectively, which can specifically enhance the low-frequency structural information, main-scale detail features and micro-scale texture features. While fully preserving the overall structure of the image, the clarity and contrast of local details are significantly improved, and noise diffusion and artifact interference are effectively suppressed, ultimately achieving multi-level and precise enhancement of infrared images, and improving the reliability and accuracy of subsequent visual analysis tasks.

[0009] Optionally, in one embodiment of the present application, the decomposition based on the input infrared image generates a base layer image, a main detail layer image and a sub-detail layer image, including: calculating a first linear coefficient and a second linear coefficient based on a preset dynamically adjustable weighting parameter; and generating the main detail layer image and the sub-detail layer image based on the first linear coefficient and the second linear coefficient.

[0010] Through the above technical means, based on the preset dynamically adjustable weighting parameters, the first linear coefficient and the second linear coefficient are calculated, which can flexibly adjust the enhancement weights of features of different scales and effectively capture the local detail features in the image, including but not limited to high spatial frequency components and local contrast information. In this way, while suppressing noise interference, the detail structure and edge characteristics are highlighted, thereby improving the fineness of image enhancement and the overall visual quality.

[0011] The calculation formulas of the first linear coefficient and the second linear coefficient can be expressed as:

[0012]

[0013] b k =(1-a k )μ k ,

[0014] Among them, μ k and σ k Respectively represent the mean and standard deviation of the grayscale value of the kth pixel in the image within the specified window; w k represents an adjustable weighting coefficient; ε represents a regularization parameter; and k represents the position index of the k-th pixel.

[0015] By using the above technical means, combined with the mean and standard deviation of the image pixel grayscale values, the linear coefficient is calculated. The enhancement parameters can be adaptively adjusted according to the overall brightness distribution and local contrast changes of the image, thereby more accurately extracting the structural features and detail information of the image, and improving the local dynamic range and detail clarity of the image.

[0016] Optionally, in one embodiment of the present application, the calculation formula of the weighted parameter can be expressed as:

[0017]

[0018] Among them, λ represents the adjustment coefficient; f k represents the spatial frequency parameter; c tk represents the local contrast parameter; K represents the total number of pixels in the window.

[0019] By using the above technical means, combined with parameters such as spatial frequency and local contrast, the weighted parameters are calculated to measure the richness of image details and the degree of texture change. The weighted parameters of each area can be dynamically adjusted to achieve adaptive enhancement of features at different scales, thereby achieving a better balance between detail enhancement and noise suppression.

[0020] The second aspect of the present application provides an infrared image enhancement device, including: a decomposition module, which is used to decompose and generate a base layer image, a main detail layer image and a secondary detail layer image based on an input infrared image; a processing module, which is used to perform image enhancement processing on the base layer image, the main detail layer image and the secondary detail layer image respectively to obtain enhanced base layer image, main detail layer image and secondary detail layer image; and an enhancement module, which is used to fuse the enhanced base layer image, the main detail layer image and the secondary detail layer image to obtain an enhanced infrared image.

[0021] Through the above technical means, the base layer image, main detail layer image and sub-detail layer image generated by decomposition are respectively subjected to targeted enhancement processing to achieve the enhancement of infrared images. Differential enhancement of features at different levels can be achieved, effectively improving the structural clarity and detail richness of the image, while suppressing background noise interference, bringing higher image contrast and more delicate texture expression, thereby significantly improving the application effect and reliability of infrared images in target detection, feature extraction and subsequent visual tasks.

[0022] Optionally, in one embodiment of the present application, the processing module includes: a first enhancement unit, used to enhance the overall contrast information based on the base layer image using a contrast constrained adaptive histogram equalization strategy; a second enhancement unit, used to enhance important details of different scales based on the main detail layer image using a multi-scale detail guided enhancement strategy; and an extraction unit, used to extract detail features based on the secondary detail layer image using a Gaussian-Laplacian operator.

[0023] Through the above technical means, different enhancement methods are used to process the base layer image, main detail layer image and secondary detail layer image respectively, which can specifically enhance the low-frequency structural information, main-scale detail features and micro-scale texture features. While fully preserving the overall structure of the image, the clarity and contrast of local details are significantly improved, and noise diffusion and artifact interference are effectively suppressed, ultimately achieving multi-level and precise enhancement of infrared images, and improving the reliability and accuracy of subsequent visual analysis tasks.

[0024] Optionally, in one embodiment of the present application, the decomposition module includes: a calculation unit for calculating a first linear coefficient and a second linear coefficient based on a preset dynamically adjustable weighting parameter; and a generation unit for generating the main detail layer image and the secondary detail layer image based on the first linear coefficient and the second linear coefficient.

[0025] Through the above technical means, based on the preset dynamically adjustable weighting parameters, the first linear coefficient and the second linear coefficient are calculated, which can flexibly adjust the enhancement weights of features of different scales and effectively capture the local detail features in the image, including but not limited to high spatial frequency components and local contrast information. In this way, while suppressing noise interference, the detail structure and edge characteristics are highlighted, thereby improving the fineness of image enhancement and the overall visual quality.

[0026] Optionally, in one embodiment of the present application, the calculation formulas of the first linear coefficient and the second linear coefficient can be respectively expressed as:

[0027]

[0028] b k =(1-a k )μ k ,

[0029] Among them, μ k and σ k Respectively represent the mean and standard deviation of the grayscale value of the kth pixel in the image within the specified window; w k represents an adjustable weighting coefficient; ε represents a regularization parameter; and k represents the position index of the k-th pixel.

[0030] By using the above technical means, combined with the mean and standard deviation of the image pixel grayscale values, the linear coefficient is calculated. The enhancement parameters can be adaptively adjusted according to the overall brightness distribution and local contrast changes of the image, thereby more accurately extracting the structural features and detail information of the image, and improving the local dynamic range and detail clarity of the image.

[0031] Optionally, in one embodiment of the present application, the calculation formula of the weighted parameter can be expressed as:

[0032]

[0033] Among them, λ represents the adjustment coefficient; f k represents the spatial frequency parameter; c tk represents the local contrast parameter; K represents the total number of pixels in the window.

[0034] By using the above technical means, combined with parameters such as spatial frequency and local contrast, the weighted parameters are calculated to measure the richness of image details and the degree of texture change. The weighted parameters of each area can be dynamically adjusted to achieve adaptive enhancement of features at different scales, thereby achieving a better balance between detail enhancement and noise suppression.

[0035] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the infrared image enhancement method as described in the above embodiment.

[0036] A fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program that implements the above infrared image enhancement method when executed by a processor.

[0037] A fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above infrared image enhancement method.

[0038] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0040] Figure 1 This is a flow chart of an infrared image enhancement method provided according to an embodiment of the present application;

[0041] Figure 2A schematic diagram of a two-step image decomposition process according to an embodiment of the present application;

[0042] Figure 3 This is a schematic diagram of the process of enhancing the master-detail layer according to one embodiment of the present application;

[0043] Figure 4 This is a schematic diagram of the infrared image enhancement process according to one embodiment of the present application;

[0044] Figure 5 A schematic diagram of ablation experiment results of a decomposition method according to an embodiment of the present application;

[0045] Figure 6 A schematic diagram of detail enhancement ablation experiment results according to an embodiment of the present application;

[0046] Figure 7 This is a schematic diagram of enhanced comparative experimental results of scenario 1 according to an embodiment of the present application;

[0047] Figure 8 This is a schematic diagram of enhanced comparative experimental results for scenario 2 according to an embodiment of the present application;

[0048] Figure 9 This is a schematic diagram of enhanced comparative experimental results of scenario 3 according to an embodiment of the present application;

[0049] Figure 10 Schematic diagram of a block diagram of an infrared image enhancement device provided according to an embodiment of the present application;

[0050] Figure 11 The figure is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application.

[0051] Reference numerals:

[0052] 10-infrared image enhancement device; 100-decomposition module, 200-processing module and 300-enhancement module; 1101-memory, 1102-processor and 1103-communication interface. DETAILED DESCRIPTION

[0053] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0054] The following describes an infrared image enhancement method, device, electronic device, and storage medium according to an embodiment of the present application with reference to the accompanying drawings. In response to the technical problem that most infrared image enhancement methods mentioned in the above background art rely on global contrast enhancement or simple image processing algorithms, which cannot fully preserve image detail features and texture information, resulting in low target contrast, poor detail representation, poor enhancement effect, and affecting subsequent image analysis and target detection performance, the present application provides an infrared image enhancement method, in which an infrared image is decomposed into multiple scales to extract detail information at each scale, and local detail features are used to perform weighted optimization on guided filtering, thereby performing targeted enhancement processing on the layered images separately, thereby finely strengthening structural features and texture details at different scales, while improving image contrast and detail clarity, effectively suppressing noise diffusion and artifact enhancement, maintaining the overall structural integrity and detail authenticity of the image, thereby obtaining a high-quality infrared image with rich information, distinct layers, clear visual effects, and significantly improving subsequent target detection and feature extraction performance, and significantly improving the robustness and practicality of the infrared image enhancement method under complex environmental conditions. This solves the problem that most infrared image enhancement methods in related technologies rely on global contrast enhancement or simple image processing algorithms, which cannot fully maintain image detail features and texture information, resulting in low target contrast, poor detail performance, poor enhancement effect, and affecting subsequent image analysis and target detection performance.

[0055] Specifically, Figure 1 A flowchart of an infrared image enhancement method provided in an embodiment of the present application.

[0056] like Figure 1 As shown, the infrared image enhancement method includes the following steps:

[0057] In step S101 , a base layer image, a primary detail layer image, and a secondary detail layer image are generated based on decomposition of an input infrared image.

[0058] Among them, infrared image decomposition refers to the process of separating different information components in infrared images. Specifically, through decomposition, useful information in the image (such as heat sources, target objects) can be distinguished from unnecessary parts (such as background noise), which facilitates subsequent image enhancement, target detection or analysis. Image decomposition can be based on a variety of technologies, including but not limited to scale, frequency or physical component decomposition, so as to effectively extract key features and remove background noise or irrelevant information. There are many decomposition methods for infrared images, including but not limited to guided filtering decomposition, wavelet transform decomposition or sparse representation decomposition. These methods each have their own advantages, and the appropriate decomposition method can be selected according to different application scenarios.

[0059] It should be noted that the base layer image is mainly the overall structural information extracted from the original image through low-pass filtering or smoothing, and mainly retains large-scale, low-frequency image content, such as the overall brightness distribution of the region, the main contour shape, etc. The main detail layer image is mainly the medium and high-frequency components extracted after removing the base layer image information. It contains the main edge information, texture features, and important detail changes between the cultivated land area and the background in the image, which can highlight the contour characteristics of key objects. The secondary detail layer image is mainly the subtle change information obtained after further separation of the main detail layer image, usually including local small-scale textures, weak boundaries or noise features, which helps to improve the segmentation and classification accuracy at a fine-grained level.

[0060] The embodiment of the present application can extract different components of the image (such as background, details, heat sources, etc.) by finely decomposing the image, so that each component can be processed independently as needed, providing important preparation for subsequent image enhancement.

[0061] Optionally, in one embodiment of the present application, a base layer image, a main detail layer image and a sub-detail layer image are generated based on the decomposition of an input infrared image, including: calculating a first linear coefficient and a second linear coefficient based on a preset dynamically adjustable weighting parameter; and generating a main detail layer image and a sub-detail layer image based on the first linear coefficient and the second linear coefficient.

[0062] It should be noted that the embodiment of the present application mainly adopts the guided filtering decomposition method, which has excellent edge preservation ability, can accurately identify the edge information in the image, and retain these edge features to the greatest extent during the filtering process, and can automatically adjust the filtering effect according to the characteristics of the input image.

[0063] Specifically, the guided filtering method calculates the first linear coefficient a based on the mean and variance within the local window. k and the second linear coefficient b k , thereby generating a detail layer. However, for areas with complex textures and drastic lighting changes in the image, the guided filtering method will over-smoothe, resulting in the loss of detailed information. As a result, the calculated detail layer cannot accurately reflect the true detail characteristics of the image, thereby reducing the credibility of the detail layer information.

[0064] Furthermore, the embodiment of the present application proposes a guided filter for decomposing the weighted local features of the input infrared image, and calculating the linear coefficient a k and b k A dynamically adjustable weighting parameter w is introduced k .

[0065] Optionally, in one embodiment of the present application, the calculation formulas for the first linear coefficient and the second linear coefficient can be respectively expressed as:

[0066]

[0067] b k =(1-a k )μ k ,

[0068] Among them, μ k and σ k Respectively represent the mean and standard deviation of the grayscale value of the kth pixel in the specified window; w k represents the adjustable weighting coefficient; ε represents the regularization parameter.

[0069] Optionally, in one embodiment of the present application, the calculation formula of the weighted parameter can be expressed as:

[0070]

[0071] Among them, λ represents the adjustment coefficient; f k represents the spatial frequency parameter; c tk represents the local contrast parameter; K represents the total number of pixels in the window.

[0072] The embodiment of this application introduces the weighted parameter w k It can effectively capture the local detail features of the image, including high spatial frequency components and contrast information. These information can jointly reflect the texture, edge and brightness changes in the local area, and can effectively avoid the detail loss problem caused by excessive smoothing of the filtering method, thereby improving the integrity of the detail layer information and the credibility of the expression.

[0073] Furthermore, the parameter f k The spatial frequency domain feature weight is obtained by performing FFT (Fast Fourier Transform) on the image in the local window. The calculation expression can be expressed as:

[0074]

[0075] in, is a local image window centered at the kth pixel and of size M×M; It is a local window FFT of the internal image for frequency domain analysis.

[0076] Furthermore, the window The local contrast parameter c within tk The calculation formula can be expressed as:

[0077]

[0078] Where I(j) represents the grayscale value of the j-th pixel in image I.

[0079] As a possible implementation method, the basic algorithm flow of the guided filtering with local feature weighting can be shown in Algorithm 1.

[0080]

[0081] In the embodiments of the present application, the guided filtering method based on local feature weighting can fully consider the texture intensity, contrast change and spatial frequency characteristics of the local area during the filtering process, realize differentiated processing of edge and detail areas, more effectively retain the detail information and structural features in the image, suppress invalid interference in the background area, avoid detail loss caused by excessive smoothing, and ultimately achieve a balance between image structure preservation and detail enhancement, thereby improving the visual quality of the enhanced image and the accuracy of subsequent processing.

[0082] The following is a detailed explanation of the infrared image decomposition of this application using a specific example. Figure 2 As shown, the embodiment of the present application includes the following steps:

[0083] Step S201: input an infrared image.

[0084] Step S202: Apply LDF-GF (Local Detail Feature-Guided Filter) to the input original infrared image I to obtain the base layer I. b , the formula can be shown as follows:

[0085]

[0086] Among them, the function Indicates applying LDF-GF to the input image.

[0087] Specifically, the obtained base layer mainly contains low-frequency information, which helps to maintain the overall structure and contrast of the image.

[0088] Step S203: By subtracting the base layer from the original image, the main detail layer I can be obtained. pd :

[0089] I pd =II b ,

[0090] This step separates high-frequency details from the input image, ensuring that important fine details are preserved for further processing.

[0091] Step S204: To further extract details, LDF-GF is applied to the main detail layer for filtering. Through this process, important fine details are retained and can be used for subsequent processing. This operation can obtain the secondary detail layer I ad , as shown below:

[0092]

[0093] Specifically, these two consecutive filtering operations capture and separate multi-scale details, producing a base layer and two detail layers. This decomposition enables subsequent image enhancement to apply layer-specific algorithms that preserve the overall image structure while enhancing fine details, thereby improving visual quality.

[0094] In step S102, image enhancement processing is performed on the base layer image, the main detail layer image and the secondary detail layer image respectively to obtain enhanced base layer image, main detail layer image and secondary detail layer image.

[0095] Among them, there are many methods for image enhancement, and different enhancement strategies can be adopted in different application scenarios. For example, the overall contrast of the image can be improved through histogram equalization, local details and brightness consistency can be enhanced through the multi-scale Retinex method, image edge features can be highlighted through nonlinear filtering, or image quality can be adaptively improved through an enhancement network based on deep learning, thereby effectively improving the usability and accuracy of infrared images in tasks such as detection, recognition or analysis.

[0096] Specifically, the embodiments of the present application respectively adopt adaptive enhancement methods for different levels of images obtained by decomposition to give full play to the characteristics of each layer of images.

[0097] Optionally, in one embodiment of the present application, image enhancement processing is performed on the base layer image, the main detail layer image and the secondary detail layer image respectively, including: based on the base layer image, using a contrast constrained adaptive histogram equalization strategy to enhance the overall contrast information; based on the main detail layer image, using a multi-scale detail guided enhancement strategy to enhance important details of different scales; based on the secondary detail layer image, using a Gaussian-Laplacian operator to extract detail features.

[0098] Specifically, the enhancement of images at different levels in the embodiment of the present application mainly includes the following:

[0099] (1) Base layer enhancement

[0100] In the embodiment of the present application, the base layer contains low spatial frequency information that is crucial for maintaining image structure and contrast. Therefore, CLAHE (Contrast Limited Adaptive Histogram Equalization) is used to enhance the base layer, aiming to improve the overall contrast of the image while preserving key structural information. This process can be expressed as follows:

[0101]

[0102] in, represents the enhanced base layer image; Indicates the base layer I b Applied CLAHE operation.

[0103] (2) Master-detail layer enhancement

[0104] In an embodiment of the present application, the primary detail layer of an infrared image contains a wide range of multi-scale features, including broad textures and structural outlines at different scales. Conventional enhancement methods fail to fully consider the scale differences in structural information. When applied to the primary detail layer, this can easily lead to over-enhancement of details, uneven enhancement effects, the introduction of artifacts, and even the loss of valuable detail information. To address these issues, an embodiment of the present application provides an MSDGE (Multi-scale Detail Guided Enhancement) strategy to preserve and enhance details of different scales in the primary detail layer.

[0105] The following is a detailed explanation of the main detail enhancement of this application using a specific example. Figure 3 As shown, the embodiment of the present application includes the following steps:

[0106] Step S301: input the main detail layer image.

[0107] Step S302 : applying a multi-scale scaling operation to the infrared image master-detail layer.

[0108] Specifically, the embodiment of the present application can be based on various scaling factors s q The input image is scaled to obtain image representations at multiple different scaling factors. The scaled main layer can be shown as follows:

[0109]

[0110] in, Represents the image The scaling operation applied; q Represents the scaling factor; q is the index of each scale. Scaling operation uses bilinear interpolation.

[0111] Specifically, the embodiment of the present application adopts five different scales, and common scaling factors mainly include s1 = 0.50, s2 = 0.75, s3 = 1.00, s4 = 1.25, and s5 = 1.50. Furthermore, multi-scale detail feature extraction is performed on the image at each scale.

[0112] Step S303 : Apply the LDF-GF method to each scaled image to extract internal detail information.

[0113] Specifically, the operations at each scale can be as follows:

[0114]

[0115] in, Represents the detail layer extracted at a given scale.

[0116] Step S304: for each detail image obtained at each scale, Apply a contrast stretch enhancement to increase its contrast and make details more visible.

[0117] Specifically, the contrast enhancement process can be as follows:

[0118]

[0119] in, Represents the enhanced scaled image at the qth scale The j-th pixel value of L max and L min They are the maximum and minimum values ​​of the grayscale range of the target image, respectively. For 8-bit images, they are usually 255 and 0, respectively. max and I min are the maximum and minimum grayscale values ​​of the input image, respectively.

[0120] Step S305: The enhanced detail layer at each scale is converted to Rescaling to the original size can be done as follows:

[0121]

[0122] Furthermore, the final reconstruction formula of the master-detail layer can be expressed as:

[0123]

[0124] Step S306: outputting the enhanced main-detail layer image.

[0125] As a possible implementation method, the multi-scale guided detail enhancement algorithm of the main detail layer can be shown as Algorithm 2.

[0126]

[0127] In an embodiment of the present application, the multi-scale guided detail enhancement of the main detail layer can adaptively fuse image detail information at different scales, thereby effectively enhancing the expression of local features such as texture and edges while retaining the overall structure of the image. It can suppress the problem of detail loss or uneven enhancement caused by single-scale processing, improve the visual clarity and local contrast of the image, and thus provide higher quality image input for subsequent image recognition, target detection and other tasks.

[0128] (3) Sub-detail layer enhancement

[0129] In the embodiment of the present application, the sub-detail layer of the infrared image contains local small details of the original image. In order to fully highlight these small details, the LoG (Laplacian of Gaussian) edge extraction operator is used to enhance its edge and texture information.

[0130] First, for the sub-detail layer I ad Apply a Gaussian filter:

[0131]

[0132] in, represents the result of applying a Gaussian filter; Represents the image I ad Perform Gaussian filtering. This process can smooth the image, reduce noise, and prepare for edge enhancement in the next step.

[0133] Furthermore, the Gaussian filtered image Apply a Laplacian operation to enhance edges:

[0134]

[0135] Furthermore, the enhancement result of the sub-detail layer can be obtained by adding the original image to the Gaussian-Laplacian result:

[0136]

[0137] The embodiment of the present application can fully utilize the characteristics of each layer of the image by performing different enhancements on different layers, effectively improving the overall contrast and brightness performance of the base layer, highlighting the target contours and key features of the main detail layer, while suppressing the noise that may exist in the secondary detail layer and retaining the necessary tiny details.

[0138] In step S103, the enhanced base layer image, the primary detail layer image and the secondary detail layer image are fused to obtain an enhanced infrared image.

[0139] In the embodiment of the present application, the base layer and the two enhanced detail layers have significant differences in low-frequency overall information and high-frequency detail information. In order to make the merged enhanced image detail information more comprehensive and fully utilize the unique information advantages of each layer, the merged enhanced image I e It can be expressed as:

[0140]

[0141] Among them, w b 、w pd and w ad Represent the weighting coefficients of the enhanced base layer, main detail layer, and sub-detail layer respectively. This weighting coefficient can balance the proportion of information in different image layers and can be calculated based on local gradient and contrast.

[0142] Specifically, the calculation formulas for the three weighting coefficients can be shown as follows:

[0143]

[0144] w b =1-w pd -w ad ,

[0145] Among them, G pd and G ad Represent the enhanced main detail layer and enhanced sub-detail layer image local gradients respectively; C pd and C ad They represent the enhanced local contrast of the main detail layer and the enhanced sub-detail layer images respectively; α is an adjustment parameter.

[0146] The following is a detailed explanation of the infrared image enhancement process of this application using a specific example. Figure 4 As shown, the embodiment of the present application includes the following steps:

[0147] Step S401: input an original infrared image.

[0148] Step S402: Two-step image decomposition. A guided filtering decomposition method based on local features is used on the input multispectral infrared image. Through two-step decomposition, a single infrared image is decomposed into a base layer, a main detail layer, and a secondary detail layer, preparing for the subsequent layered enhancement step.

[0149] Step S403: Image enhancement. The obtained base layer and detail layer are enhanced separately. CLAHE is used on the base layer to enhance overall contrast information. A multi-scale detail-guided enhancement method is used on the primary detail layer to enhance important details at different scales. The LoG operator is applied to the secondary detail layer to extract detail features. This can highlight detail information such as edges and textures in the detail layer, thereby improving the detail richness of the final enhanced image.

[0150] Step S404: Multi-layer enhancement and reconstruction: A weighted fusion scheme based on local gradient and contrast is used to merge the enhanced base layer, primary detail layer, and secondary detail layer into an enhanced image.

[0151] Step S405: output the enhanced image.

[0152] Combine Figure 5 、 Figure 6 、 Figure 7 、 Figure 8 、 Figure 9 , using a specific example to verify and evaluate the effectiveness of the framework and algorithm proposed in this application in infrared image enhancement.

[0153] The embodiment of the present application divides the result evaluation into two parts: enhanced ablation experiment and enhanced contrast experiment.

[0154] Specifically, in order to demonstrate the universality of the infrared image enhancement method provided in the present application for the infrared image enhancement effect of different spectral bands and scenes, the embodiments of the present application respectively take infrared images of a scene from different data sets for enhancement experiments.

[0155] Specifically, scenes 1 to 3 use infrared images selected from Multispectral Imaging Datasets 1, Multispectral Imaging Datasets 2, and the airborne hyperspectral dataset of the Washington, D.C. Mall as source images. The corresponding infrared image wavelengths of the above three scenes are 800nm, 1050nm, and 1700nm, respectively.

[0156] (1) Enhanced ablation experiment

[0157] To verify the effectiveness of the improvements to the guided filter and the master-detail layer enhancement scheme, the embodiment of the present application selected a 1700nm single-spectral band image from the airborne data of a shopping mall in Washington, DC for enhancement experiments, namely, scene 3.

[0158] It should be noted that, in order to effectively compare the results, the embodiment of the present application uses the evaluation indicators EN (entropy) and AG (average gradient) of the enhanced image for comparative display.

[0159] EN reflects the statistical uncertainty of pixel values ​​in an image. A higher entropy value indicates a richer image. AG measures the severity of local changes in an image (i.e., clarity or edge strength). A higher gradient indicates a clearer image.

[0160] (A) Impact of filtering algorithms

[0161] In order to verify the effectiveness of the LDF-GF proposed in this application, the embodiment of this application replaces the filter with BF (Bilateral Filter) and GF (Guided Filter) respectively, while keeping other parts unchanged.

[0162] The enhancement effects and evaluation indicators of these three methods can be as follows Figure 5 (a) and summarized in the second row of Table 1, which is a table of evaluation indicators for decomposition and ablation experiment enhancement results. Figure 5 The results in (a) and Table 1 can demonstrate the superiority of the LDF-GF method proposed in this application.

[0163] Table 1

[0164]

[0165] (B) Impact of Decomposition Strategy

[0166] In order to verify the effectiveness of the two-step decomposition scheme based on LDF-GF, the embodiment of the present application keeps the three filtering methods in the previous section unchanged, changes the decomposition from two steps to one step (generating only one detail layer), and keeps other parts unchanged.

[0167] The enhancement effects and evaluation metrics for all these cases are as follows: Figure 5 (b) and are summarized in the first row of Table 1. Figure 5 The results in Table 1 can prove the effectiveness of the two-step decomposition strategy proposed in the embodiment of the present application.

[0168] (C) Impact of detail layer enhancement methods

[0169] In the enhancement experiment, first, the embodiment of the present application fixed the proposed MSGDE method as the enhancement method for the main detail layer, and combined the sub-detail layer with the other four schemes. Secondly, the embodiment of the present application fixed the LoG method as the enhancement method for the sub-detail layer, and obtained a total of 7 cases. Among them, other enhancement methods include: DDE (Dynamic Detail Enhancement), DRP (Detail Reconstruction Processing) and MSDF (Multi-Scale Detail Fusion).

[0170] The enhancement effects and evaluation indicators of all these cases can be respectively as follows Figure 6 As shown in the table below, Table 2 shows the evaluation index table of detail enhancement ablation experiment. Figure 6 From Table 2, we can see that the results obtained by the framework and method proposed in this application are optimal in terms of detail preservation and enhancement effects.

[0171] Table 2

[0172]

[0173]

[0174] (2) Enhanced contrast experiment

[0175] The embodiments of the present application are comprehensively compared to verify the reliability and effectiveness of the proposed enhancement method.

[0176] Specifically, the embodiment of the present application uses four enhancement methods for comparison, namely BF-DDE, GF-DDE, BF-DRP and GF-MSDF, and conducts enhancement comparison experiments on images of all three scenes. All enhancement indicators can be shown in Table 3 below, which is an evaluation index table of image enhancement comparison experiment results.

[0177] Table 3

[0178]

[0179] At the same time, the enhanced effect diagram of the infrared band image of each scene can be shown as follows: Figure 7 、 Figure 8 and Figure 9 Taking into account the evaluation indicators and visual effects, it can be proved that the enhancement method proposed in the embodiment of the present application can achieve optimal enhancement of the detail information and visual effects of the infrared image.

[0180] According to the infrared image enhancement method proposed in the embodiment of the present application, the base layer image, main detail layer image and secondary detail layer image generated by decomposing the infrared image are enhanced respectively to obtain an enhanced infrared image. The structural features and texture details at different scales can be fine-grainedly enhanced. While improving the image contrast and detail clarity, the noise diffusion and artifact enhancement can be effectively suppressed, and the overall structural integrity and detail authenticity of the image can be maintained, thereby obtaining a high-quality infrared image with rich information, distinct layers, clear visual effects and significantly improved subsequent target detection and feature extraction performance.

[0181] Next, the infrared image enhancement device proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0182] Figure 10 Schematic diagram of an infrared image enhancement device according to an embodiment of the present application.

[0183] like Figure 10 As shown, the infrared image enhancement device 10 includes: a decomposition module 100 , a processing module 200 and an enhancement module 300 .

[0184] The decomposition module 100 is used to generate a base layer image, a main detail layer image and a secondary detail layer image based on the decomposition of the input infrared image.

[0185] The processing module 200 is used to perform image enhancement processing on the base layer image, the main detail layer image and the secondary detail layer image respectively to obtain enhanced base layer image, main detail layer image and secondary detail layer image.

[0186] The enhancement block 300 is used to fuse the enhanced base layer image, the primary detail layer image and the secondary detail layer image to obtain an enhanced infrared image.

[0187] Optionally, in one embodiment of the present application, the processing module 200 includes: a first enhancement unit, a second enhancement unit and a decomposition unit

[0188] Among them, the first enhancement unit is used to enhance the overall contrast information based on the base layer image using a contrast constrained adaptive histogram equalization strategy; the second enhancement unit is used to enhance important details of different scales based on the main detail layer image using a multi-scale detail guided enhancement strategy.

[0189] The extraction unit is used to extract detail features based on the sub-detail layer image using a Gaussian-Laplacian operator.

[0190] Optionally, in one embodiment of the present application, the decomposition module 100 includes: a calculation unit and a generation unit.

[0191] The calculation unit is used to calculate the first linear coefficient and the second linear coefficient based on a preset dynamically adjustable weighting parameter.

[0192] A generating unit is configured to generate a primary detail layer image and a secondary detail layer image based on the first linear coefficient and the second linear coefficient.

[0193] Optionally, in one embodiment of the present application, the calculation formulas for the first linear coefficient and the second linear coefficient can be respectively expressed as:

[0194]

[0195] b k =(1-a k )μ k ,

[0196] Among them, μ k and σ k Respectively represent the mean and standard deviation of the grayscale value of the kth pixel in the image within the specified window; w k represents an adjustable weighting coefficient; ε represents a regularization parameter; and k represents the position index of the k-th pixel.

[0197] Optionally, in one embodiment of the present application, the calculation formula of the weighted parameter can be expressed as:

[0198]

[0199] Among them, λ represents the adjustment coefficient; f k represents the spatial frequency parameter; c tk represents the local contrast parameter; K represents the total number of pixels in the window.

[0200] It should be noted that the aforementioned explanation of the infrared image enhancement method embodiment is also applicable to the infrared image enhancement device of this embodiment, and will not be repeated here.

[0201] According to the infrared image enhancement device proposed in the embodiment of the present application, the base layer image, main detail layer image and secondary detail layer image generated by decomposing the infrared image are enhanced respectively to obtain an enhanced infrared image. The structural features and texture details at different scales can be fine-grainedly enhanced. While improving the image contrast and detail clarity, the noise diffusion and artifact enhancement can be effectively suppressed, and the overall structural integrity and detail authenticity of the image can be maintained, thereby obtaining a high-quality infrared image with rich information, distinct layers, clear visual effects and significantly improved subsequent target detection and feature extraction performance.

[0202] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0203] A memory 1101 , a processor 1102 , and a computer program stored in the memory 1101 and executable on the processor 1102 .

[0204] When the processor 1102 executes the program, the infrared image enhancement method provided in the above embodiment is implemented.

[0205] Furthermore, the electronic device further includes:

[0206] The communication interface 1103 is used for communication between the memory 1101 and the processor 1102 .

[0207] The memory 1101 is used to store computer programs that can be run on the processor 1102 .

[0208] The memory 1101 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0209] If the memory 1101, processor 1102, and communication interface 1103 are implemented independently, the communication interface 1103, memory 1101, and processor 1102 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0210] Optionally, in a specific implementation, if the memory 1101, the processor 1102 and the communication interface 1103 are integrated on a chip, the memory 1101, the processor 1102 and the communication interface 1103 can communicate with each other through an internal interface.

[0211] The processor 1102 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0212] This embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the above infrared image enhancement method is implemented.

[0213] An embodiment of the present application further provides a computer program product, including a computer program, which can run computer instructions. When the computer instructions are executed by a processor, the infrared image enhancement method provided in the embodiment of the present application is implemented.

[0214] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0215] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0216] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0217] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0218] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, it can be implemented using any one or a combination of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0219] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0220] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0221] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for enhancing infrared images, characterized in that: The following steps are involved: Decomposing the input infrared image to generate a base layer image, a main detail layer image and a sub-detail layer image; performing image enhancement processing on the base layer image, the main detail layer image and the secondary detail layer image respectively to obtain enhanced base layer image, main detail layer image and secondary detail layer image; The enhanced base layer image, the primary detail layer image and the secondary detail layer image are fused to obtain an enhanced infrared image.

2. The method according to claim 1, characterized in that The performing image enhancement processing on the base layer image, the main detail layer image and the secondary detail layer image respectively includes: Based on the base layer image, enhancing the overall contrast information by using a contrast constrained adaptive histogram equalization strategy; Based on the main detail layer image, a multi-scale detail guided enhancement strategy is used to enhance important details of different scales; Based on the sub-detail layer image, detail features are extracted using a Gaussian-Laplacian operator.

3. The method according to claim 1, characterized in that The step of generating a base layer image, a main detail layer image, and a sub-detail layer image based on the decomposition of the input infrared image includes: Calculating a first linear coefficient and a second linear coefficient based on a preset dynamically adjustable weighting parameter; The primary detail layer image and the secondary detail layer image are generated based on the first linear coefficient and the second linear coefficient.

4. The method according to claim 3, characterized in that The calculation formulas of the first linear coefficient and the second linear coefficient are respectively: b k =(1-a k )m k , Among them, μ k and σ k Respectively represent the mean and standard deviation of the grayscale value of the kth pixel in the image within the specified window; w k represents an adjustable weighting coefficient; ε represents a regularization parameter; and k represents the position index of the k-th pixel.

5. The method according to claim 3 or 4, characterized in that The calculation formula of the weighted parameter is: Among them, λ represents the adjustment coefficient; f k represents the spatial frequency parameter; c tk represents the local contrast parameter; K represents the total number of pixels in the window.

6. An infrared image enhancement device, characterized in that: include: A decomposition module, configured to generate a base layer image, a main detail layer image and a sub-detail layer image based on the decomposition of the input infrared image; a processing module, configured to perform image enhancement processing on the base layer image, the main detail layer image, and the secondary detail layer image, respectively, to obtain enhanced base layer image, main detail layer image, and secondary detail layer image; The enhancement module is used to fuse the enhanced base layer image, the primary detail layer image and the secondary detail layer image to obtain an enhanced infrared image.

7. The device according to claim 6, characterized in that The processing module includes: A first enhancement unit, configured to enhance overall contrast information based on the base layer image by using a contrast constrained adaptive histogram equalization strategy; A second enhancement unit is configured to enhance important details of different scales based on the main detail layer image using a multi-scale detail guided enhancement strategy; The extraction unit is configured to extract detail features based on the sub-detail layer image using a Gaussian-Laplacian operator.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the infrared image enhancement method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement an infrared image enhancement method according to any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that The computer program is executed to implement an infrared image enhancement method according to any one of claims 1 to 5.

Images