An infrared image enhancement method and device based on histogram matching and curvelet transform

Abstract

The application provides an infrared image enhancement method and device based on histogram matching and curvelet transform. The method comprises: acquiring an infrared image to be enhanced and a reference image; calculating the gray mean value and the gray standard deviation of the infrared image and the reference image respectively; calculating a correction coefficient and an average correction amount based on the gray mean value and the gray standard deviation; generating a histogram matching enhanced image of the infrared image based on the correction coefficient and the average correction amount; respectively performing curvelet transform decomposition on the histogram matching enhanced image and the infrared image to obtain corresponding multi-scale and multi-direction sub-band coefficients, and fusing corresponding sub-band coefficients at the same scale and the same direction to obtain fused sub-band coefficients; and performing curvelet inverse transform on the fused sub-band coefficients to obtain a fused enhanced image. Through the cooperation of the global gray correction of histogram matching and the local detail fusion of curvelet transform, the application realizes the synchronous improvement of the contrast, the definition and the detail information of the infrared image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to an infrared image enhancement method and apparatus based on histogram matching and curvelet transform. Background Technology

[0002] Image enhancement technology is one of the fundamental technologies in computer vision and image processing. Its goal is to improve the visual quality and usable information content of images, such as improving contrast, highlighting details and textures, enhancing edge structures, and suppressing noise interference, thereby providing more reliable input for subsequent tasks such as target detection, segmentation, recognition, and tracking. With the widespread application of infrared imaging technology in scenarios such as night vision surveillance, security patrols, autonomous driving, medical diagnosis, industrial temperature measurement, and disaster relief, infrared images possess irreplaceable advantages under low-light or complex lighting conditions. However, limited by the infrared imaging mechanism and sensor characteristics, infrared images often suffer from narrow dynamic range, insufficient contrast, limited grayscale levels, blurred details, and significant noise, making it difficult to distinguish key targets from the background and affecting the effectiveness of manual observation and algorithmic analysis.

[0003] Unlike visible light images, infrared images primarily reflect the thermal radiation information of a target. Their imaging results are influenced by factors such as target temperature distribution, material emissivity, ambient temperature variations, and atmospheric transmission. Because infrared images often exhibit non-uniform grayscale distribution and localized weak texture features, directly applying traditional enhancement methods designed for visible light images, such as global histogram equalization, simple linear stretching, or local contrast enhancement, can easily lead to localized over-enhancement, loss of detail in bright and dark areas, amplified background noise, and increased artifacts. It can even damage the target's edge structure and distort textures, thereby reducing the interpretability of infrared images and the performance of subsequent tasks.

[0004] To improve the visual quality and structural fidelity of infrared images, related methods often employ histogram-based enhancement or multi-scale transform domain enhancement strategies. Histogram-based methods can improve overall contrast and grayscale levels to some extent, but they lack the ability to characterize image structural information, making it difficult to simultaneously enhance contrast and preserve details and edges. On the other hand, transform domain methods such as wavelets, contourlets, and curvelets can characterize image details and edge structures at multiple scales and in multiple directions, exhibiting good preservation of texture and contour information. However, without a reasonable grayscale distribution correction or fusion strategy, they may still result in insufficient enhancement, lack of detail prominence, or residual noise. Summary of the Invention

[0005] In view of this, in order to address the common problems in related infrared image enhancement methods, such as the difficulty in balancing global enhancement and local details, the ease with which contrast enhancement introduces noise amplification and artifacts, the easy damage to edge and texture information, and the lack of visual naturalness of the enhancement results, this application proposes an infrared image enhancement method and apparatus based on histogram matching and curve transform.

[0006] Specifically, this application is implemented through the following technical solution:

[0007] According to a first aspect of the embodiments of this specification, an infrared image enhancement method based on histogram matching and curvelet transform is provided, comprising the following steps:

[0008] Step S1: Obtain the infrared image to be enhanced and the reference image of the infrared image;

[0009] Step S2: Calculate the mean gray level and standard deviation gray level of the infrared image and the reference image respectively;

[0010] Step S3: Based on the mean and standard deviation of the grayscale of the infrared image and the mean and standard deviation of the grayscale of the reference image, calculate the correction coefficient and the average correction amount; the correction coefficient is used to correct the grayscale range of the infrared image; the average correction amount is used to compensate for the brightness shift of the corrected infrared image.

[0011] Step S4: Based on the correction coefficient and the average correction amount, perform histogram matching enhancement on the infrared image to obtain a histogram matching enhanced image;

[0012] Step S5: Perform curvelet transform decomposition on the histogram matching enhanced image and the infrared image respectively to obtain the corresponding multi-scale and multi-directional sub-band coefficients, and fuse the corresponding sub-band coefficients of the two images at the same scale and in the same direction to obtain the fused sub-band coefficients.

[0013] Step S6: Perform inverse curvelet transform reconstruction on the fused subband coefficients to obtain the fused enhanced image of the infrared image.

[0014] According to a second aspect of the embodiments of this specification, an infrared image enhancement device based on histogram matching and curvelet transform is provided, the device comprising:

[0015] An image acquisition unit is used to acquire the infrared image to be enhanced and a reference image of the infrared image;

[0016] A statistical calculation unit is used to calculate the mean gray level and the standard deviation gray level of the infrared image and the reference image, respectively.

[0017] An intermediate calculation unit is used to calculate a correction coefficient and an average correction amount based on the grayscale mean and standard deviation of the infrared image and the grayscale mean and standard deviation of the reference image; the correction coefficient is used to correct the grayscale range of the infrared image; the average correction amount is used to compensate for the brightness shift of the corrected infrared image.

[0018] The histogram matching unit is used to perform histogram matching enhancement on the infrared image based on the correction coefficient and the average correction amount to obtain a histogram matching enhanced image.

[0019] The transformation and fusion unit is used to perform curvelet transform decomposition on the histogram matching enhanced image and the infrared image respectively to obtain the corresponding multi-scale and multi-directional sub-band coefficients, and to fuse the corresponding sub-band coefficients of the two images at the same scale and in the same direction to obtain the fused sub-band coefficients.

[0020] The inverse curvelet transform unit is used to perform inverse curvelet transform reconstruction on the fused subband coefficients to obtain the fused enhanced image of the infrared image.

[0021] According to a third aspect of the embodiments of this specification, an electronic device is provided, including a processor; and a computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method described in the first aspect.

[0022] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executed by a processor of the method described in the first aspect.

[0023] In this embodiment, the synergistic effect of global grayscale correction by histogram matching and local detail fusion by curvelet transform achieves simultaneous improvement in contrast, sharpness, and detail information of infrared images, while effectively suppressing noise amplification and artifact generation, resulting in visually natural and structurally faithful enhanced results. This embodiment first performs grayscale mapping on the infrared image using a linear transformation based on the statistics of the reference image and the original infrared image, achieving a more natural and visually desirable global contrast enhancement with lower computational complexity. Second, utilizing the multi-scale and multi-directional characteristics of curvelet transform, detail information is extracted from both the infrared image and the histogram-matched enhanced image. Selective enhancement is then performed through a coefficient fusion strategy to significantly sharpen the edges of targets and highlight weak textures, resulting in a final fusion result that possesses both excellent overall visual effects and rich local details. Attached Figure Description

[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0025] Figure 1 This is a schematic flowchart illustrating an infrared image enhancement method based on histogram matching and curvelet transform, as shown in an exemplary embodiment of this application.

[0026] Figure 2 This is a schematic diagram illustrating the principle of infrared image enhancement according to an exemplary embodiment of this application;

[0027] Figure 3 This is a schematic diagram illustrating the grayscale distribution change during a histogram matching enhancement process, as shown in an exemplary embodiment of this application.

[0028] Figure 4 This is a schematic diagram illustrating an infrared image fusion enhancement process based on curve transform, as shown in an exemplary embodiment of this application.

[0029] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment of this application;

[0030] Figure 6 This is a block diagram illustrating an infrared image enhancement device based on histogram matching and curvelet transform, as shown in an exemplary embodiment of this application. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0032] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0033] The embodiments described in this specification will now be described in detail.

[0034] This application provides an infrared image enhancement method based on histogram matching and curvelet transform. Figure 1 This is a schematic flowchart illustrating an infrared image enhancement method based on histogram matching and curvelet transform, as shown in an exemplary embodiment of this application. Figure 1 As shown, the infrared image enhancement method includes at least the following steps:

[0035] Step S1: Obtain the infrared image to be enhanced and the reference image of the infrared image.

[0036] The reference image is a high-quality reference image used for histogram matching to guide the grayscale distribution of the infrared image to be enhanced, thereby improving overall contrast. The reference image and the infrared image are similar in imaging band, sensor type, and ambient temperature range to avoid matching failure due to differences in physical mechanisms.

[0037] In some embodiments, a reference image is selected from a historical image sequence acquired by the same imaging device (e.g., an infrared camera) as the infrared image to be enhanced, characterized by high contrast, uniform grayscale distribution, and absence of significant noise or artifacts. Alternatively, a high-quality image similar to the current scene type is selected from a preset standard infrared image database as the reference image.

[0038] Step S2: Calculate the mean gray level and standard deviation gray level of the infrared image and the reference image respectively.

[0039] Step S3: Based on the grayscale mean and standard deviation of the infrared image and the grayscale mean and standard deviation of the reference image, calculate the correction coefficient and the average correction amount; the correction coefficient is used to correct the grayscale range of the infrared image so that the contrast of the infrared image is close to that of the reference image; the average correction amount is used to compensate for the brightness shift of the corrected infrared image so that the brightness of the compensated infrared image is consistent with the brightness of the infrared image.

[0040] Step S4: Based on the correction coefficient and the average correction amount, perform histogram matching enhancement on the infrared image to obtain a histogram matching enhanced image.

[0041] Step S5: Perform curvelet transform decomposition on the histogram matching enhanced image and the infrared image respectively to obtain the corresponding multi-scale and multi-directional sub-band coefficients, and fuse the corresponding sub-band coefficients of the two images at the same scale and in the same direction to obtain the fused sub-band coefficients.

[0042] Step S6: Perform inverse curvelet transform reconstruction on the fused subband coefficients to obtain the fused enhanced image of the infrared image.

[0043] It can be seen that, in Figure 1 The infrared image enhancement method shown achieves simultaneous improvement in contrast, sharpness, and detail information of the infrared image through the synergy of global grayscale correction by histogram matching and local detail fusion by curvelet transform. Simultaneously, it effectively suppresses noise amplification and artifact generation, resulting in a visually natural and structurally faithful enhancement. This embodiment first performs grayscale mapping on the infrared image using a linear transformation based on the statistics of the reference image and the original infrared image. This allows for a rapid and more natural global contrast enhancement that better meets visual expectations with lower computational complexity. Secondly, utilizing the multi-scale and multi-directional characteristics of curvelet transform, detailed information is extracted from both the infrared image and the histogram-matched enhanced image. A coefficient fusion strategy is then used for selective enhancement to significantly sharpen the edges of the target and highlight weak textures, resulting in a final fusion result that possesses both excellent overall visual effects and rich local details.

[0044] In some embodiments, step S3 includes:

[0045] Based on the grayscale standard deviation of the infrared image grayscale standard deviation of the reference image The correction coefficient is calculated. , ;

[0046] Gray-scale mean of infrared images The grayscale mean of the reference image and the correction coefficient The average correction amount is calculated. , .

[0047] In some embodiments, step S2 includes:

[0048] Determine whether the standard deviation of the infrared image is less than a preset lower threshold.

[0049] If the value is less than the preset lower threshold, the preset lower threshold is updated to the standard deviation of the infrared image;

[0050] Accordingly, step S3 calculates the correction coefficient based on the standard deviation of the updated infrared image.

[0051] In some embodiments, step S4 includes:

[0052] The pixel values ​​of each pixel in the infrared image are corrected based on the correction coefficient to obtain the corrected infrared image;

[0053] The pixel values ​​of each pixel in the corrected infrared image are compensated based on the average correction amount to obtain the histogram matching enhanced image.

[0054] In some embodiments, step S4 further includes:

[0055] The histogram matching enhanced image is subjected to a grayscale range constraint, which includes mapping pixel values ​​to [0, 255] and / or normalizing them to [0, 1]. This grayscale range constraint is used to avoid overflow.

[0056] In some embodiments, the method further includes the following steps prior to step S5:

[0057] The infrared image and the histogram matching enhanced image are registered based on a phase-correlation registration strategy so that the infrared image and the histogram matching enhanced image are aligned in spatial coordinates.

[0058] Accordingly, step S5 involves performing curvelet transform decomposition on the histogram-matched enhanced image after image registration and the infrared image.

[0059] Since there may be slight geometric shifts or scale differences between the infrared image and the histogram-matched enhanced image, this embodiment performs image registration processing on the two images to align them in spatial coordinates. The registration method is phase-correlation-based registration, and the registration result is based on satisfying pixel-level correspondence to ensure the accuracy of subsequent transform domain coefficient fusion.

[0060] In some embodiments, step S5 includes:

[0061] Based on the Curvelet function, a curvilinear transform decomposition is performed to obtain multi-scale, multi-directional subband coefficients, which can be expressed as follows:

[0062]

[0063] in, For scale indexing, For direction index, For spatial location index, Curvelet basis functions are used to enhance images or infrared images by histogram matching at pixel points. The pixel value at that location, where X and Y are the length and width of the image. This represents the sub-band coefficient.

[0064] Curves transform has the ability to represent sparse data at multiple scales and in multiple directions. It is particularly good at expressing structural information such as edges and curve contours, so it is suitable for enhancing details and preserving structure in infrared images.

[0065] In some embodiments, step S5 includes:

[0066] The coefficients of corresponding sub-bands with the same scale and direction are fused based on the maximum frequency fusion rule;

[0067] The maximum frequency fusion rule includes selecting the sub-band coefficient with the larger absolute value from the corresponding sub-band coefficients of two images at the same scale and in the same direction as the fusion sub-band coefficient.

[0068] Next, the infrared image enhancement method of the embodiments of this application will be described in detail.

[0069] In this embodiment, an infrared camera is used to acquire images of the target scene, including roads, buildings, pedestrians, and the surrounding background. The acquired infrared image has a resolution of 640×480, is a single-channel grayscale image, with each pixel being 8-bit grayscale and a grayscale value range of 0–255. The acquired raw infrared image is used as the input image to be enhanced, denoted as [image description missing]. .

[0070] To ensure the stability of subsequent statistical calculations and transformation operations, the input image is preprocessed. This preprocessing includes image format standardization and pixel value mapping. It can be used for range optimization, noise suppression, and bad pixel repair of images with poor quality.

[0071] like Figure 2 As shown, the infrared image enhancement method of this embodiment includes image steps:

[0072] Step S1: Select the reference image and the infrared image to be enhanced.

[0073] In this embodiment, a high-quality image with good histogram features is selected as the reference image, denoted as . The reference image is a frame with high contrast and more balanced grayscale distribution selected from the environmental image sequence captured by the same infrared camera, or an image similar to the current scene type selected from a preset standard infrared image database, in order to reduce unnatural brightness or over-enhancement after matching.

[0074] Step S2: Calculate the statistical values.

[0075] The grayscale mean of the reference image is calculated using the following formula (1). :

[0076] (1)

[0077] in, These are the length and width of the reference image, respectively. For pixels The pixel value, the grayscale mean Characterizes the overall brightness level of the reference image.

[0078] The grayscale standard deviation of the reference image is calculated using the following formula (2). :

[0079] (2)

[0080] The gray standard deviation Characterizes the contrast and grayscale dispersion of the reference image.

[0081] Similarly, the infrared image to be enhanced is calculated using the following formula (3). grayscale mean :

[0082] (3)

[0083] in, These represent the length and width of the input infrared image, respectively.

[0084] The infrared image to be enhanced is calculated using the following formula (4)). gray standard deviation :

[0085] (4)

[0086] The gray standard deviation This reflects the grayscale contrast level of the infrared image to be enhanced. For infrared images of a target scene, when the ambient temperature is relatively uniform or the temperature difference between the target and the ambient temperature is small, the grayscale standard deviation is [value missing]. A value that is too small may result in insufficient overall image contrast.

[0087] Step S3: Calculate the correction factor and the average correction amount.

[0088] After calculating the statistics of the infrared image to be enhanced and the reference image, the correction coefficient is calculated based on the following formula (5). :

[0089] (5)

[0090] in, Used to scale the dynamic range of the infrared image to be enhanced, bringing its overall contrast closer to that of the reference image.

[0091] Optionally, to avoid Too small a threshold value leads to unstable values; in some implementations, a lower threshold is set. ,make To ensure To ensure computational stability and avoid abnormal amplification.

[0092] And, the average correction amount is calculated based on the following formula (6). :

[0093] (6)

[0094] The average correction amount It is used to compensate for the brightness shift after scaling, so that the infrared image maintains a reasonable consistency in the overall brightness level and avoids problems such as being too bright, too dark or having significant grayscale shift after scaling.

[0095] Step S4: Generate histogram matching enhanced image.

[0096] After calculating the correction coefficients and the average correction amount, a histogram matching enhanced image is generated based on the following formula (7). :

[0097] (7)

[0098] refer to Figure 3 As shown, after histogram matching enhancement, the overall contrast of the infrared image to be enhanced tends to be similar to that of the reference image.

[0099] Optionally, histogram matching is used to enhance the image. Apply grayscale range constraints to avoid overflow and improve display quality.

[0100] Thus, through the above steps S2 to S4, the global grayscale distribution of the infrared image to be enhanced can be quickly corrected without relying on complex iterative optimization algorithms, significantly improving the overall contrast and grayscale levels.

[0101] Step S5, image registration.

[0102] After generating the histogram matching enhancement map, the infrared image to be enhanced is... Histogram matching to enhance images These are two images to be merged.

[0103] Since scaling, cropping, or interpolation operations may be introduced during the preprocessing stage, resulting in slight geometric deviations between the two images, this embodiment uses a phase-correlation-based registration method to align the two images at the pixel level, so that they satisfy a one-to-one correspondence in spatial coordinates, thereby ensuring the accuracy of subsequent transform domain coefficient fusion.

[0104] It is worth noting that, for cases where no geometric changes are introduced, the two images have the same spatial coordinate mapping, and the registration step can be simplified or omitted.

[0105] Step S6, curve transform.

[0106] like Figure 4 As shown, firstly, a curvelet transform is performed on the two registered images to decompose the images into different scales. Different directions Subband coefficient:

[0107] (8)

[0108] in, Infrared image or histogram matching to enhance images , For scale indexing, For direction index, For spatial location index, These are Curvelet base functions.

[0109] Curves transform has the ability to represent sparse data in multiple scales and directions, which can effectively express the edges, curve contours and texture details in infrared images. This allows for more targeted highlighting of the target structure and preservation of detailed information when selecting and fusing coefficients in the transform domain.

[0110] Next, this embodiment performs fusion on the corresponding sub-band coefficients of the two images at the same scale and in the same direction. To highlight edge and detail information, this embodiment adopts the maximum frequency fusion rule:

[0111] (9)

[0112] in, For the fusion subband coefficient, The subband coefficients corresponding to the infrared image are: The subband coefficients corresponding to the histogram-matched enhanced image are used.

[0113] Maximum frequency fusion rules can prioritize the retention of information containing stronger edge response, contour information or texture details, thereby improving the clarity of target boundaries and the discernibility of local structures.

[0114] Finally, an inverse curvelet transform is performed on the fused curvelet coefficients to reconstruct the final fused and enhanced image. .

[0115] Optionally, the fused and enhanced image can be further cropped in grayscale, normalized, or have its contrast fine-tuned to meet the format requirements of the display terminal or subsequent algorithm input.

[0116] Based on the above embodiments of this application, global grayscale distribution correction is first achieved through histogram matching, thereby improving the overall brightness levels and contrast. Then, edge and detail information is enhanced through curvelet transform decomposition and sub-band coefficient fusion, making the target outline clearer and the detailed texture easier to identify. Compared with the method of using only global histogram enhancement, the embodiments of this application can better preserve the structure and edge continuity; compared with the method of using only transform domain enhancement, the embodiments of this application can improve the overall contrast and visual naturalness while maintaining details, thereby improving the robustness and applicability of infrared images in target detection, recognition, and surveillance applications.

[0117] Figure 5 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 5 At the hardware level, the device includes a processor 502, an internal bus 504, a network interface 506, memory 508, a hardware acceleration device 510, and non-volatile memory 512, and may also include other hardware required for its functions. One or more embodiments of this application can be implemented in software, for example, the processor 502 reads the corresponding computer program from the non-volatile memory 512 into memory 508 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0118] Figure 6 This is a block diagram illustrating an infrared image enhancement device based on histogram matching and curvelet transform, as shown in an exemplary embodiment of this application. The infrared image enhancement device can be applied to, for example... Figure 5 The electronic device shown implements the technical solution of this application. The infrared image enhancement device includes: an image acquisition unit 610, a statistical calculation unit 620, an intermediate calculation unit 630, a histogram matching unit 640, a transformation and fusion unit 650, and a curvelet inverse transform unit 660, wherein:

[0119] Image acquisition unit 610 is used to acquire the infrared image to be enhanced and a reference image of the infrared image;

[0120] The statistical calculation unit 620 is used to calculate the gray mean and gray standard deviation of the infrared image and the reference image, respectively.

[0121] The intermediate calculation unit 630 is used to calculate a correction coefficient and an average correction amount based on the gray-scale mean and standard deviation of the infrared image and the gray-scale mean and standard deviation of the reference image; the correction coefficient is used to correct the gray-scale range of the infrared image; the average correction amount is used to compensate for the brightness shift of the corrected infrared image.

[0122] Histogram matching unit 640 is used to perform histogram matching enhancement on the infrared image based on the correction coefficient and the average correction amount to obtain a histogram matching enhanced image;

[0123] The transformation and fusion unit 650 is used to perform curvelet transform decomposition on the histogram matching enhanced image and the infrared image respectively to obtain the corresponding multi-scale and multi-directional sub-band coefficients, and to fuse the corresponding sub-band coefficients of the two images at the same scale and in the same direction to obtain the fused sub-band coefficients.

[0124] The inverse curve transform unit 660 is used to perform inverse curve transform reconstruction on the fused subband coefficients to obtain the fused enhanced image of the infrared image.

[0125] In some embodiments, the intermediate calculation unit 630 is used to perform calculations based on the grayscale standard deviation of the infrared image. grayscale standard deviation of the reference image The correction coefficient is calculated. , Gray-scale mean based on infrared images The grayscale mean of the reference image and the correction coefficient The average correction amount is calculated. , .

[0126] In some embodiments, the statistical calculation unit 620 is further configured to determine whether the standard deviation of the infrared image is less than a preset lower threshold; if it is less than the preset lower threshold, the preset lower threshold is updated to the standard deviation of the infrared image.

[0127] Accordingly, the intermediate calculation unit 630 is used to calculate the correction coefficient based on the standard deviation of the updated infrared image.

[0128] In some embodiments, the histogram matching unit 640 is configured to correct the pixel values ​​of each pixel in the infrared image based on the correction coefficient to obtain a corrected infrared image; and to compensate the pixel values ​​of each pixel in the corrected infrared image based on the average correction amount to obtain the histogram matching enhanced image.

[0129] In some embodiments, the histogram matching unit 640 is further configured to impose a grayscale range constraint on the histogram matching enhanced image, the grayscale range constraint including mapping pixel values ​​to [0, 255] and / or normalizing them to [0, 1].

[0130] In some embodiments, the apparatus further includes an image registration unit;

[0131] The image registration unit is used to perform image registration of the infrared image and the histogram matching enhanced image based on a phase-related registration strategy, so that the infrared image and the histogram matching enhanced image are aligned in spatial coordinates;

[0132] Correspondingly, the transformation and fusion unit 650 is used to perform curvelet transform decomposition on the histogram matching enhanced image after image registration and the infrared image.

[0133] In some embodiments, the transformation and fusion unit 650 is used to perform curvelet transform decomposition based on the Curvelet function to obtain multi-scale, multi-directional subband coefficients, wherein the subband coefficients can be expressed as:

[0134]

[0135] in, For scale indexing, For direction index, For spatial location index, For Curvelet base functions, Histogram matching enhances images or infrared images at pixel points. The pixel value at that location, where X and Y are the length and width of the image. This represents the sub-band coefficient.

[0136] In some embodiments, the transformation and fusion unit 650 is used to fuse corresponding sub-band coefficients of the same scale and the same direction based on the maximum frequency fusion rule; wherein, the maximum frequency fusion rule includes selecting the sub-band coefficient with the larger absolute value of the corresponding sub-band coefficients of two images at the same scale and the same direction as the fused sub-band coefficient.

[0137] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0138] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0139] Accordingly, embodiments of this application also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0140] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0141] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0142] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0143] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0144] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0145] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0146] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0147] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0148] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An infrared image enhancement method based on histogram matching and curvelet transform, characterized in that, Includes the following steps: Step S1: Obtain the infrared image to be enhanced and the reference image of the infrared image; Step S2: Calculate the mean gray level and standard deviation gray level of the infrared image and the reference image respectively; Step S3: Based on the mean and standard deviation of the grayscale of the infrared image and the mean and standard deviation of the grayscale of the reference image, calculate the correction coefficient and the average correction amount; the correction coefficient is used to correct the grayscale range of the infrared image. The average correction amount is used to compensate for the brightness shift of the corrected infrared image; Step S4: Based on the correction coefficient and the average correction amount, perform histogram matching enhancement on the infrared image to obtain a histogram matching enhanced image; Step S5: Perform curvelet transform decomposition on the histogram matching enhanced image and the infrared image respectively to obtain the corresponding multi-scale and multi-directional sub-band coefficients, and fuse the corresponding sub-band coefficients of the two images at the same scale and in the same direction to obtain the fused sub-band coefficients. Step S6: Perform inverse curvelet transform reconstruction on the fused subband coefficients to obtain the fused enhanced image of the infrared image; Step S3 includes: Based on the grayscale standard deviation of the infrared image grayscale standard deviation of the reference image The correction coefficient is calculated. , Gray-scale mean based on infrared images The grayscale mean of the reference image and the correction coefficient The average correction amount is calculated. , ; Step S4 includes: The pixel values ​​of each pixel in the infrared image are corrected based on the correction coefficient to obtain the corrected infrared image; the pixel values ​​of each pixel in the corrected infrared image are compensated based on the average correction amount to obtain the histogram matching enhanced image.

2. The method according to claim 1, characterized in that, Step S2 includes: Determine whether the standard deviation of the infrared image is less than a preset lower threshold. If the value is less than the preset lower threshold, the preset lower threshold is updated to the standard deviation of the infrared image; Step S3 calculates the correction coefficient based on the standard deviation of the updated infrared image.

3. The method according to claim 1, characterized in that, Step S4 further includes: The histogram matching enhanced image is subjected to grayscale range constraints, which include mapping pixel values ​​to [0, 255] and / or normalizing them to [0, 1].

4. The method according to claim 1, characterized in that, The steps preceding step S5 also include: The infrared image and the histogram matching enhanced image are registered based on a phase-correlation registration strategy so that the infrared image and the histogram matching enhanced image are aligned in spatial coordinates. Step S5 involves performing curvelet transform decomposition on the histogram-matched enhanced image after image registration and the infrared image.

5. The method according to claim 1, characterized in that, Step S5 includes: Based on the Curvelet function, a curvilinear transform decomposition is performed to obtain multi-scale, multi-directional subband coefficients, which can be expressed as follows: in, For scale indexing, For direction index, For spatial location index, For Curvelet base functions, Histogram matching enhances images or infrared images at pixel points. The pixel value at that location, where X and Y are the length and width of the image. This represents the sub-band coefficient.

6. The method according to claim 1, characterized in that, Step S5 includes: The coefficients of corresponding sub-bands with the same scale and direction are fused based on the maximum frequency fusion rule; The maximum frequency fusion rule includes selecting the sub-band coefficient with the larger absolute value from the corresponding sub-band coefficients of two images at the same scale and in the same direction as the fusion sub-band coefficient.

7. An infrared image enhancement device based on histogram matching and improved curvelet transform, characterized in that, The device includes: An image acquisition unit is used to acquire the infrared image to be enhanced and a reference image of the infrared image; A statistical calculation unit is used to calculate the mean gray level and the standard deviation gray level of the infrared image and the reference image, respectively. An intermediate calculation unit is used to calculate a correction coefficient and an average correction amount based on the grayscale mean and standard deviation of the infrared image and the grayscale mean and standard deviation of the reference image; the correction coefficient is used to correct the grayscale range of the infrared image; the average correction amount is used to compensate for the brightness shift of the corrected infrared image. The histogram matching unit is used to perform histogram matching enhancement on the infrared image based on the correction coefficient and the average correction amount to obtain a histogram matching enhanced image. The transformation and fusion unit is used to perform curvelet transform decomposition on the histogram matching enhanced image and the infrared image respectively to obtain the corresponding multi-scale and multi-directional sub-band coefficients, and to fuse the corresponding sub-band coefficients of the two images at the same scale and in the same direction to obtain the fused sub-band coefficients. The inverse curvelet transform unit is used to perform inverse curvelet transform reconstruction on the fused subband coefficients to obtain the fused enhanced image of the infrared image; The intermediate calculation unit is used to perform calculations based on the grayscale standard deviation of the infrared image. grayscale standard deviation of the reference image The correction coefficient is calculated. , Gray-scale mean based on infrared images The grayscale mean of the reference image and the correction coefficient The average correction amount is calculated. , ; The histogram matching unit is used to correct the pixel values ​​of each pixel in the infrared image based on the correction coefficient to obtain a corrected infrared image; and to compensate the pixel values ​​of each pixel in the corrected infrared image based on the average correction amount to obtain the histogram matching enhanced image.

8. An electronic device, characterized in that, include: A processor and a computer-readable storage medium, wherein computer program instructions are stored in the computer-readable storage medium, the computer program instructions causing the processor to perform the method as described in any one of claims 1 to 6 when executed by the processor.

Images