Method for processing an infrared image by merging several colorimetric treatments of the infrared image
The method addresses the challenge of balancing area detection and detail identification in infrared images by duplicating and processing the image for local and global enhancements, resulting in a fused output that effectively highlights both features.
Patent Information
- Application Number
- FR2023001664
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-02-23
- Publication Date
- 2025-01-10
- Estimated Expiration
- 2043-02-23
AI Technical Summary
Existing infrared image processing methods struggle to simultaneously achieve optimal detection of areas of interest and identification of details, as current histogram stretching techniques either enhance contrast at the expense of detail or preserve detail at the cost of detectability.
A method involving duplication of an infrared image into two grayscale images, applying local processing for detail enhancement and global processing for area detection, followed by colorization and weighted fusion to create a final image that combines these enhancements.
The method enables effective detection of areas of interest and identification of details in infrared images, enhancing both capabilities simultaneously through a weighted fusion of local and global processing results.
Smart Images
Figure 00000011_0000 
Figure 00000011_0001
Abstract
Description
Title of the invention: Method for processing an infrared image by merging several colorimetric treatments of the infrared image Technical field
[0001] The present invention relates to the processing of infrared images captured by a thermal camera.
[0002] In particular, the present invention relates to a processing method comprising a fusion of at least two processing operations of the same infrared image so as to bring out several types of information from the scene captured in the form of an infrared image.
[0003] Generally speaking, the invention applies to any type of image capable of providing different types of information depending on the image processing applied. Prior techniques
[0004] Existing imaging devices comprise, for example, a camera comprising a sensor and an optical system, as well as an image processing module enabling respectively capturing an image of a scene and processing said image by applying a processing function.
[0005] In particular, the image processing module comprises, for example, a module for optimizing the histogram of the captured images. The histogram of an image is a representation of the intensity distribution of the pixels of said image. The histogram optimization module is also called AHC for Automatic Histogram Control.
[0006] The histogram optimization module is intended to be able to analyze and optimize the processing of the image captured by the camera, for example by stretching the histogram globally and / or locally. Generally speaking, stretching a histogram consists, for example, in better distributing the intensity distribution of the pixels of the image, and more generally in giving another form of distribution of the intensities of the pixels. In particular, stretching the histogram globally is carried out by taking into account the value of the intensity of all the pixels of the image while stretching the histogram locally is carried out by taking into account the value of the intensity of pixels on one or more specific zones of the image, for example on zones having a particular spatial frequency.A local stretching of the image histogram is for example carried out by stretching the histogram for each spatial frequency of the image independently of the others, the superposition of these stretched histograms making it possible to obtain the image. treated locally.
[0007] An advantage of the global stretching of a histogram of an infrared image obtained with an infrared sensor, also called a thermal sensor, is to facilitate the detection by a computer or by an operator of areas of interest in the infrared image, for example areas emitting heat. Indeed, global stretching improves the contrast of the image and allows better detection of areas of interest at the expense of a loss of details. This advantage is useful for use day and night for surveillance, defense and machine vision applications.
[0008] One advantage of locally stretching a histogram of an infrared image is that it helps identify details in the scene observed by the infrared camera. However, the amount of detail causes the human eye to lose its ability to recognize and detect areas of interest, particularly hot spots.
[0009] The histogram optimization module also allows the histogram of an image to be stretched in a mixed manner, namely by applying a weighting between a global stretching and a local stretching.
[0010] However, the choice of a mixed compromise between local stretching and global stretching does not make it possible to obtain the best performances in each of the two cases, to both detect areas of interest (for example hot spots within the image) and to identify in a fine manner details of the scene represented in the image.
[0011] Imaging devices according to the state of the art comprising cameras Thermal imaging also includes a lightweight processing path in addition to the histogram optimization module in the image processing module. This processing path allows the image histogram to be stretched globally, or to not apply any histogram stretching processing. This processing path is typically used in particular to transmit tracking information or to recover a stream of unprocessed raw images.
[0012] In addition, imaging devices according to the state of the art may comprise an image fusion module, allowing imaging devices comprising an infrared sensor and a sensor in the visible wavelengths to combine thermal and visible information on the same image output from the imaging device. Statement of the invention
[0013] The present invention therefore aims to overcome the aforementioned drawbacks and to provide a method for processing an infrared image forming an image offering the best possible performance in both detection and identification.
[0014] The present invention relates to a method for processing an infrared image captured by an infrared camera, the method comprising the following steps:
[0015] - a step of duplicating the infrared image into a first level image grayscale and a second grayscale image;
[0016] - a step of applying a local treatment bringing out the details on the first image for each spatial frequency by an image processing module;
[0017] - a step of applying a global processing on the second image by the module image processing highlighting hot spots of interest in the second image;
[0018] - a step of coloring at least one area of the second image by means of of a correspondence table between gray levels and a computer color coding format;
[0019] - a step of converting the first image from a grayscale format to a computer color coding format; and
[0020] - a step of merging the first image with the second image by a module of merging images into a final image comprising for each pixel of the final image a weighted merging of the equivalent pixels respectively of the first image and of the area of the second image.
[0021] Thus, using pre-existing modules, in particular a pre-existing image processing module, the present invention makes it possible to form an infrared image grouping the information from two different processing operations to the maximum of their capacity and thus to allow an operator or a computer to easily detect an area of interest, for example via colorimetric information contained in the final image, while being able to identify the details of the scene observed, for example via information from the local processing applied.
[0022] In one embodiment, the method further comprises a step of displaying the final image in real time by a display screen.
[0023] Advantageously, the step of applying local processing comprises stretching the histogram for each spatial frequency of the first image independently of the other spatial frequencies, and comprises superimposing these stretched histograms so as to obtain the first locally processed image.
[0024] Advantageously, the step of applying a global processing to the second image comprises stretching the histogram of the second image by taking into account the value of the intensity of all the pixels of the second image.
[0025] In a particular embodiment, the method comprises a step of automatic or manual selection of the area of the second image.
[0026] Advantageously, the step of selecting the area of the second image comprises a step of automatic detection of the area as a function of the temperature and / or the movement and / or shape of said area.
[0027] In particular embodiments, the computer color coding format is an RGB or YCbCr format.
[0028] Advantageously, the coloring and / or merging steps are applied to the entire second image or only to one or more areas of the second image.
[0029] In one embodiment, the steps of applying local processing to the first image and applying global processing to the second image are performed simultaneously.
[0030] The present invention also relates to a computer program comprising program code instructions for executing the steps of the method as defined above, when said program is executed on a computer.
[0031] The present invention also relates to a computer-readable recording medium comprising instructions which, when executed by a computer, cause the latter to implement the steps of the method as defined above.
[0032] The present invention also relates to an imaging device comprising an image processing module configured to implement a method for processing an infrared image captured by an infrared camera, as defined above. Brief description of the drawings
[0033] Other aims, characteristics and advantages of the invention will appear on reading the following description, given solely by way of non-limiting example, and made with reference to the appended drawings in which:
[0034] [Fig.l] is a schematic view of an imaging device intended to implement the method according to the invention; and
[0035] [Fig.2] is a schematic illustration of the different steps of the infrared image processing method according to the invention. Detailed description of at least one embodiment
[0036] [Fig.l] schematically shows an imaging device 2 intended to implement the method described below.
[0037] The imaging device 2 comprises an infrared camera comprising an optical system and an infrared sensor 4 as well as an image processing module 6 capable of be similar to a computer program embedded in the camera. The image processing module 6 comprises on the one hand a local optimization module 8 for the histogram of images captured by the infrared sensor 4, and on the other hand a lightweight global stretching module 10 for images captured by the infrared sensor 4.
[0038] The image processing module 6 also comprises a fusion module 12 of an image output from the local histogram optimization module 8 with an image output from the light global stretching module 10.
[0039] In particular, the local histogram optimization module 8 allows stretching of the histogram of an image locally. The lightweight global stretching module 10 allows stretching of the histogram of an image globally.
[0040] The imaging device 2 optionally comprises a display screen 14 for displaying an image. For example, the displayed image is the image obtained at the output of the fusion module 12.
[0041] [Fig. 2] schematically shows the different steps of a method for processing an infrared image 16 captured by an infrared sensor 4 in one embodiment. The infrared sensor is, for example, the infrared sensor 4 of the imaging device 2 illustrated in [Fig. 1] and the method is, for example, implemented by the imaging device 2 illustrated in [Fig. 1].
[0042] To implement the processing method, a step 18 of duplicating the infrared image 16 captured by the infrared sensor 4 is first carried out. The infrared image 16 is, for example, in grayscale.
[0043] The duplication thus generates a first image 20 and a second image 22, identical to the captured infrared image 16, and in grayscale. In particular, the infrared image 16 represents an indoor or outdoor scene. The duplication step 18 can be carried out more particularly with a duplication module (not shown) included in the imaging device.
[0044] The duplication step 18 can also be carried out by duplicating the infrared image 16 into a number N of images. This makes it possible to subsequently apply a number N of different image processing operations respectively to said N images. Preferably, N is between 2 and 4 duplicated images. The present invention is therefore not limited to the duplication, processing and merging of two processing operations of an image.
[0045] A step 24 is then carried out for applying local processing to highlight the details in the first image 20. The details correspond to the high spatial frequencies of an image. This step 24 is for example implemented with the image processing module 6 by the local histogram optimization module 8.
[0046] In particular, step 24 of applying local processing comprises, for example, stretching the histogram for each spatial frequency of the first independent image. depending on the others, and includes the superposition of these stretched histograms in order to obtain the first locally processed image. In other words, the first image is processed spatial frequency by spatial frequency, the local processing enhances the high spatial frequencies and lowers the low spatial frequencies for different resolutions of the image. This then makes it possible to study the different spatial components of the image. Thus, the intensity value of the pixels for a given spatial frequency is modified and stretched over the entire spectrum of possible intensities. The histogram stretching is carried out for each spatial frequency of the first image, and all the virtual sub-images are combined into a first processed image, we obtain a first image with many intensity variations, and therefore a greater number of details.
[0047] A step 26 of applying a global processing to the second image 22 is also carried out. This step 26 is for example implemented with the image processing module 6 and more particularly with the lightweight global stretching module 10. In one embodiment, this step 26 comprises the stretching of the histogram of the second image. In other words, the intensity value of all the pixels of the second image is modified so that the histogram of these values is stretched over all the possible intensity values of the pixels. More precisely, the stretching of the histogram is carried out by applying an intensity gain to each of the pixels so as to completely fill the intensity dynamics of the image, then by carrying out a histogram recalibration around the average value of said intensity dynamics of the image.
[0048] Advantageously, the steps 24 and 26 of applying a local processing and applying a global processing are carried out simultaneously. This is made possible by the fact that the first image 20 and the second image 22 are processed independently, for example respectively by the local histogram optimization module 8 on the one hand and by the light global stretching module 10 on the other hand.
[0049] Alternatively, N images are duplicated, for example four images, steps 24 and 26 being applied to two images and different processing steps being applied to the two remaining images to modify them, for example by additional processing bricks.
[0050] A step 28 of colorizing at least one area of the second processed image 22 is then carried out. This step 28 is carried out by means of a correspondence table between gray levels and a computer color coding format.
[0051] The computer color coding format is for example an RGB or YCbCr format.
[0052] Thus, each gray level corresponds to a precise color combination defined in the correspondence table. This correspondence makes it possible to better highlighting areas of interest, including hotspots in the infrared image. An advantage is that the colorimetric information obtained highlights for the observer the elements of interest in the scene, for example hotspots, while being superimposable with another type of information, such as intensity variations illustrating the details of the image.
[0053] The coloring step 28 is applied for example to the entirety of the second processed image 22.
[0054] As a variant, before the colorization step 28, a step 30 of automatic or manual selection of one or more zones of said second image 22, processed or unprocessed, is carried out. The colorization step can then be applied only to one or more selected zones, the unselected zones still being converted into a computer color coding format, without however changing their visual rendering. Thus, in a computer color coding format of the RGB type, namely with three red, green and blue components, the unselected zones retain a grayscale type appearance while having their RGB components such that the intensity of the red component is equal, for each pixel of an unselected zone, to the intensity of the green component, as well as to the intensity of the blue component, noted R=G=B.
[0055] The selection is for example manual and is carried out by an operator who selects, for example on a display screen, the areas of interest of the second image 20.
[0056] Optionally, the step of selecting an area of the second image 22 comprises a step 32 of automatic detection of the area as a function of the temperature and / or the movement and / or the shape of said area. The detection is carried out automatically by a computer and / or an on-board electronic card as a function of temperature criteria, and / or movement and / or shape of the area to be selected.
[0057] Thus, the detection of a hot spot of interest then makes it possible to colorize it in order to make it more visible.
[0058] A step 34 of converting the first image 20 from its grayscale format to a computer color coding format is carried out in parallel. This step 34 is similar to the conversion of the unselected areas mentioned above. In preparation for merging the first and second images 20 and 22, the first image 20 is converted into a computer color coding format while retaining the same visual appearance of grayscale. Thus, in an RGB type computer color coding format, the intensity of the red component is equal, for each pixel of the first image, to the intensity of the green component, as well as to the intensity of the blue component, denoted R=G=B.
[0059] A step 36 of merging the first modified image 20 is then carried out after step 34 with the second image 22 modified after step 28 into a final image 38, the first and second images being in the same color computer coding format. If necessary, the merging step 36 makes it possible to merge the N modified images. The merging step 36 is for example carried out by the image merging module 12 of the imaging device 2.
[0060] The final image 38 more particularly comprises for each pixel of said final image 38 a weighted fusion of the equivalent pixels respectively of the first image 20 and of the second image 22. Equivalent pixels are understood to mean the pixels of the first and second images which correspond to the same pixel in the captured infrared image.
[0061] According to an example of weighted fusion, by noting x and y the coordinates of each pixel of the final image 38, a the weighting factor between 0 and 1, R, G, B the intensities of the colors for an RGB color computer coding format, we obtain the RGB intensities for each pixel of the final image 38, such that:
[0062] R(x, y) - a *Rimi(x, y) + (1 - a) *Rim2(x, y) ■ G(x, y) = a *Gim {x, y) + (1 - a) *Gim2(x, y). B(x, y) = a*Bjml(x, y) + ( 1- a) *Bim£x, y)
[0063] The weighting factor can be modified by an operator. When it is equal to 0, only the second modified image 22 is visible on the final image 38. When it is equal to 1, only the first modified image 20 is visible on the final image 38.
[0064] Optionally, the weighting factor is different depending on the x and y coordinates, for example for coordinates included in selected areas.
[0065] Step 36 of merging the first modified image 20 with the second modified image 22 into a final image 38 makes it possible to combine the respective interests of local processing and global processing, and therefore to facilitate identification and detection respectively on the same final image 38.
[0066] Optionally, a step 40 of displaying the final image 38 is carried out by the display screen 14 of the imaging device 2, the display step 40 preferably being carried out in real time so that the display step 40 is carried out at most one second after the infrared image 16 has been duplicated, for example in a time corresponding to three processing frames, i.e. 60 milliseconds for an acquisition frequency of 50 Hz.
Claims
Claims
1. Method for processing an infrared image (16) captured by an infrared camera, characterized in that it comprises the following steps: - a step of duplicating (18) the infrared image (16) into a first image (20) in gray levels and a second image (22) in gray levels; - a step of applying a local processing (24) highlighting the details on the first image (20) for each spatial frequency by an image processing module (6); - a step of applying a global processing (26) on the second image (22) by the image processing module (6) highlighting hot spots of interest in the second image (22); - a step of colorizing (28) at least one area of the second image (22) by means of a correspondence table between gray levels and a computer color coding format;- a step of converting (34) the first image (20) from a grayscale format to a color computer coding format; and - a step of merging (36) the first image (20) with the second image (22) by an image merging module (12), the merging resulting in a final image (38) comprising for each pixel of the final image (38) a weighted merging of the equivalent pixels respectively of the first image (20) and of the area of the second image (22).;
2. Method according to claim 1, further comprising a step of displaying (40) in real time the final image (38) by a display screen (14).
3. A method according to any one of claims 1 and 2, wherein the step of applying local processing (24) comprises stretching the histogram for each spatial frequency of the first image (20) independently of the other spatial frequencies, and comprises superimposing these stretched histograms so as to obtain the first image (20) processed locally.
4. A method according to any one of claims 1 to 3, wherein the step of applying a global processing (26) to the second image (22) comprises stretching the histogram of the second image (22) taking into account the intensity value of all the pixels of the second image (22).
5. Method according to any one of claims 1 to 4, comprising a step of automatic or manual selection (30) of the area of the second image (22).
6. Method according to claim 5, in which the step of selecting (30) the area of the second image (22) comprises a step of automatically detecting (32) the area as a function of the temperature and / or the movement and / or the shape of said area.
7. A method according to any one of claims 1 to 6, wherein the computer color coding format is an RGB or YCbCr format.
8. A method according to any one of claims 1 to 7, wherein the colorizing (28) and / or merging (36) steps are applied to the entire second image (22) or only to one or more areas of the second image (22).
9. A method according to any one of claims 1 to 8, wherein the steps of applying a local processing (24) to the first image (20) and applying a global processing (26) to the second image (22) are performed simultaneously.
10. A computer program comprising program code instructions for carrying out the steps of the method according to any one of claims 1 to 9, when said program is executed on a computer.