Image optimization method and apparatus for thermal imaging

Abstract

This provides an image optimization method for thermal images. [Solution] The thermal image optimization method according to the present invention includes the steps of: a data capture circuit capturing a raw image of a detection area using a thermal imaging device; a processor updating a neural network model using a plurality of sample images stored in storage and a plurality of sample correspondence tables corresponding to each of the plurality of sample images; a processor inputting the raw image into the neural network model to generate an optimized correspondence table; and a processor converting the raw image into an optimized image using the optimized correspondence table. The plurality of sample correspondence tables show the correspondence between the pixel values ​​of a plurality of sample images generated by the thermal imaging device in a plurality of training areas and the pixel values ​​of the optimized plurality of sample images.

Description

Technical Field

[0001] The present invention relates to the field of thermal image processing technology, and more particularly to an image optimization method for thermal imaging and a device for thermal imaging.

Background Art

[0002] The target objects detected by thermography are diverse, and the detection purposes and personal subjective cognitions are also different. Therefore, in the conventional thermal image technology, it is difficult to construct a standardized adjustment flowchart for adjusting parameters such as the contrast and brightness of the images generated by thermography.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, in the conventional thermal image technology, often, the user has to manually adjust complex parameters for the images generated by thermography to obtain an optimized thermal image (that is, it is necessary to reduce the influence of the environmental area or surrounding articles when detecting the temperature of each target object). Also, for various different regions, in the conventional thermal image technology, the user has to readjust the parameters to obtain the parameters applicable to the target region. Therefore, eliminating the need for the complex flowchart of manually adjusting parameters and eliminating the need to readjust the parameters over time for various different regions is a problem that engineers in this field eagerly hope to solve.

[0004] The present invention has been made by the intensive research of the inventor in view of the above problems, and its object is to provide an image optimization method for thermal imaging and a device for thermal imaging that solve the problems that in the prior art, a complex flowchart for manually adjusting parameters is required and it is necessary to readjust the parameters over time for various different regions. [Means for solving the problem]

[0005] To achieve the above objective, an image optimization method for thermal images, which is one aspect of the present invention, The data capture circuit performs step a, in which it captures the original image of the detection area using a thermal imaging device. Step b, the processor updates a neural network model using a plurality of sample images stored in storage and a plurality of sample correspondence tables corresponding to each of the plurality of sample images, wherein the plurality of sample correspondence tables show the correspondence between the pixel values ​​of the plurality of sample images generated by the thermal imaging device in a plurality of training regions and the optimized pixel values ​​of the plurality of sample images. The processor inputs the original image into the neural network model to generate an optimized correspondence table (step c), The processor includes step d, which converts the original image into an optimized image using the optimized correspondence table.

[0006] To achieve the above objective, another aspect of the present invention, an image optimization apparatus for thermal images, is provided. A data capture circuit is positioned to capture the original image of the detection area using a thermal imaging device, A storage device arranged to store multiple sample images, multiple sample correspondence tables corresponding to each of the multiple sample images, and multiple instructions, wherein the multiple sample correspondence tables show the correspondence between the pixel values ​​of the multiple sample images generated by the thermal imaging device in multiple training areas and the optimized pixel values ​​of the multiple sample images. The system comprises a data capture circuit and a processor connected to the storage and configured to operate a neural network model, The processor accesses a plurality of the instructions, Step a, updating the neural network model using multiple sample images and multiple sample correspondence tables; Step b involves inputting the original image into the neural network model to generate an optimized correspondence table. Step c is performed, which converts the original image into an optimized image using the optimized correspondence table. [Effects of the Invention]

[0007] Thus, the present invention has the following effects. This invention utilizes a large number of pre-stored sample images and a sample correspondence table corresponding to specific parameters to train a neural network model. The trained neural network model then converts a new image into a new correspondence table and optimizes all pixels of the new image using a lookup table method. This eliminates the need for complex flowcharts that require manual parameter adjustment and the time-consuming process of readjusting parameters for various different domains.

[0008] Other features of the present invention will be made clearer by description in this specification and the accompanying drawings. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing an image optimization apparatus 100 for thermal images in some embodiments of the present invention. [Figure 2] This is a schematic diagram illustrating a sample correspondence table and sample images in several embodiments of the present invention. [Figure 3] This flowchart shows a thermal image optimization method in some embodiments of the present invention. [Figure 4] This is a schematic diagram illustrating optimized images in several embodiments of the present invention. [Modes for carrying out the invention]

[0010] The present invention will be described below through embodiments of the invention, but these embodiments are not intended to limit the invention as defined in the claims. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0011] Figure 1 is a block diagram showing an image optimization apparatus 100 for thermal images in several embodiments of the present invention. In this embodiment, the image optimization apparatus 100 for thermal images comprises a data capture circuit 110, a storage 120, and a processor 130. The processor 130 is connected to the data capture circuit 110 and the storage 120 (see Figure 1).

[0012] In some embodiments, the thermal image optimization device 100 is composed of any data processing device (e.g., desktop PC, laptop PC, tablet terminal, etc.) or server (e.g., cloud server, virtual server, or rack server, etc.). In this embodiment, the data capture circuit 110 captures the original image img1 of the detection area using the thermal imaging device 200. In other words, the user generates the original image img1 of the detection area by photographing the detection area with the thermal imaging device 200. Next, the thermal image optimization device 100 is connected to the thermal imaging device 200 via the data capture circuit 110, and the original image img1 of the detection area is captured by the thermal imaging device 200.

[0013] In some embodiments, the detection area is an area captured by the thermal imaging device 200 to generate the source image img1 (e.g., a street, factory, garage, etc.). In some embodiments, the thermal imaging device 200 is implemented by any thermal imaging device (e.g., a general infrared thermograph, quantum imaging device, optical and infrared combined imaging device, etc.). In some embodiments, the source image img1 may be any unoptimized type of image generated by the thermal imaging device 200 (e.g., a grayscale image or an RGB image, etc.). In some embodiments, the data capture circuit 110 may be any wireless communication circuit for communication (e.g., a Wi-Fi communication circuit or a Bluetooth® communication circuit) or a wired communication circuit (e.g., an Ethernet communication circuit).

[0014] In some embodiments, storage 120 is used to store multiple sample images si1 to siN, multiple sample mapping tables st1 to stN corresponding to each of the sample images si1 to siN, and multiple instructions. N may be any positive integer, and the present invention is not particularly limited. In some embodiments, sample images si1 to siN are multiple unoptimized images previously captured by the thermal imaging device 200 in multiple training areas (i.e., the pixel values ​​of all pixels in the images are unprocessed thermal sensing data), and sample images si1 to siN and the original image img1 are all images of the same type (e.g., all grayscale images). The training area may be the same as or different from the sensing area. In some embodiments, the multiple instructions may be implemented in arbitrary firmware or software, and the processor 130 accesses these instructions to execute the thermal image optimization method described later. In some embodiments, storage 120 may be implemented in flash memory, ROM, hard disk, or any equivalent storage element.

[0015] In some embodiments, sample correspondence tables st1 to stN show the correspondence between the pixel values ​​of sample images si1 to siN generated by the thermal imaging device 200 in multiple training regions and the pixel values ​​of the optimized sample images si1 to siN. In some embodiments, sample correspondence tables st1 to stN may be multiple correspondence tables generated by arbitrary image optimization algorithms (e.g., histogram equalization algorithms) that optimize contrast, brightness, etc., based on each of the sample images si1 to siN. Specifically, the user sets the optimal contrast, brightness, and other parameters of a particular image optimization algorithm for sample images si1 to siN that have been previously captured in multiple training regions, and converts each of the sample images si1 to siN into sample correspondence tables st1 to stN using the particular image optimization algorithm (i.e., they have a one-to-one conversion relationship).

[0016] The following explanation uses sample correspondence table st1 and sample image si1 as examples. Refer also to Figure 2, which is a schematic diagram illustrating sample correspondence table st1 and sample image si1 in several embodiments of the present invention. Sample image si1 is first converted into sample correspondence table st1 using a histogram equalization algorithm (for example, statistics are taken for the pixel values ​​of all pixels in sample image si1 to generate statistical histograms of multiple pixel value intervals, and sample correspondence table st1 is generated from the equalized statistical histograms). Sample correspondence table st1 includes multiple pixel value intervals for all pixels of sample image si1 and the optimized pixel values ​​corresponding to each pixel value interval. The degree of histogram equalization described above may be adjusted by a suitable alpha parameter pre-set by the user for multiple training regions. In this way, the pixel values ​​of all pixels in sample image si1 are each converted to their corresponding optimized pixel values ​​by sample correspondence table st1.

[0017] For details, as can be seen from Sample Correspondence Table st1, if the pixel value of one pixel coordinate in sample image si1 (for example, (10,20)) falls within the pixel value range of 0 to 5, the optimized pixel value corresponding to this pixel value is 6, and the pixel value of the same pixel coordinate (for example, (10,20)) in the optimized sample image osi1 is set to 6. If the pixel value of another pixel coordinate in sample image si1 (for example, (50,10)) falls within the pixel value range of 6 to 10, the optimized pixel value corresponding to this pixel value is 8, and the pixel value of the same pixel coordinate (for example, (50,10)) in the optimized sample image osi1 is set to 8. By analogy, the pixel values ​​of all pixels in sample image si1 are converted to the pixel values ​​of all pixels in sample image osi1 optimized by Sample Correspondence Table st1 using a look-up table method. Other sample correspondence tables st2 to stN also contain data similar to sample correspondence table st1, and convert the pixel values ​​of all pixels in each sample image si2 to siN to the pixel values ​​of all pixels in the corresponding optimized sample image. It should be noted that the maximum pixel value range int of sample correspondence table st1 is determined by the data size of each sample image si1 to siN stored (for example, if the data size is 256 bits, the maximum pixel value range int may be in the pixel value range of 250 to 255. If the data size is 16384 bits, the maximum pixel value range int may be in the pixel value range of 16380 to 16383).

[0018] Returning to FIG. 1, in this embodiment, the processor 130 further executes one neural network model 131. In some embodiments, the neural network model 131 may be implemented by any neural network model that has been image processed (e.g., a convolutional neural network model, a deep neural network model, a YOLO model, or a transformer model, etc.). In some embodiments, the processor 130 may be implemented by a central processing unit (CPU), a micro control unit (MCU), a programmable logic controller (PLC), a system on chip (SoC), or a field programmable gate array (FPGA), etc.

[0019] Referring also to FIG. 3, FIG. 3 is a flowchart showing an image optimization method for thermal images in some embodiments of the present invention. This image optimization method for thermal images is applied to the image optimization device 100 for thermal images shown in FIG. 1. The image optimization method for thermal images includes steps S310 to S340 (see FIG. 3).

[0020] First, in step S310, the data capture circuit 110 captures the original image img1 of the detection area by the thermal image device 200 of the detection area. In step S320, the processor 130 updates the neural network model 131 by using the sample images si1 to siN and the sample correspondence tables st1 to stN stored in the storage 120. [[ID=]10]

[0021] In some embodiments, the processor 130 uses each sample image as a training sample and a sample correspondence table corresponding to each sample image as training labels. Next, the processor 130 inputs each training sample into the neural network model 131, generates a corresponding result correspondence table as result labels, and calculates the loss value between the corresponding result label and the corresponding training label. Next, the processor 130 uses the calculated loss value to execute the backpropagation algorithm of the neural network model 131 and updates the parameters of the neural network model 131 (i.e., the weight values ​​of each of the multiple neural network layers of the neural network model 131). In this way, the processor 130 completes the update to the neural network model 131 (i.e., the training phase is completed).

[0022] In some embodiments, the processor 130 first performs any type of normalization (e.g., Min-Max normalization or Z-score normalization) on the sample images si1 to siN to generate normalized sample images si1 to siN, and uses each normalized sample image as a training sample.

[0023] In step S330, the processor 130 inputs the original image img1 into the neural network model 131 to generate an optimized correspondence table. In other words, each time the thermal imaging device 200 captures a detection area and generates an original image img1, the processor 130 inputs this original image img1 into the neural network model 131 and converts the original image img1 into an optimized correspondence table (i.e., the usage stage). In some embodiments, the optimized correspondence table shows the correspondence between the pixel values ​​of the original image img1 and the optimized pixel values. Specifically, if the pixel value of one pixel in the original image img1 belongs to one pixel value interval, the optimized correspondence table shows the optimized pixel value corresponding to this pixel value interval. In some embodiments, the processor 130 first performs any type of normalization (e.g., Min-Max normalization or Z-score normalization, etc.) on the original image img1 to generate a normalized original image img1, and then inputs the normalized original image img1 into the neural network model 131.

[0024] It is important to note that the trained neural network model 131 automatically adjusts various parameters (i.e., brightness and contrast, etc.) corresponding to the original image img1 for any detection region, and generates an optimized correspondence table. Therefore, with this method, there is no need to manually adjust various parameters to generate an optimized correspondence table corresponding to the original image img1.

[0025] In step S340, the processor 130 converts the original image img1 into an optimized image using an optimized lookup table. In some embodiments, the processor 130 converts the original image img1 into an optimized image by converting the pixel values ​​of multiple pixel coordinates of the original image img1 into optimized pixel values ​​using an optimized lookup table. In some embodiments, the processor 130 identifies which pixel value interval each pixel value of the pixel coordinate belongs to from the optimized lookup table and obtains the optimized pixel value corresponding to each pixel value interval from the optimized lookup table. Next, the processor 130 converts the original image img1 into an optimized image by setting each pixel coordinate of the original image as the corresponding optimized pixel value. In some embodiments, the optimized lookup table shows the correspondence between multiple pixel value intervals of the original image and the optimized pixel values.

[0026] Specifically, if the pixel value of one pixel in the original image img1 belongs to a single pixel value interval, the processor 130 retrieves the optimized pixel value corresponding to this interval using a lookup table against an optimized correspondence table, and converts the pixel value of this pixel in the original image img1 to the corresponding optimized pixel value. By analogy, the processor 130 converts the pixel values ​​of other pixels in the original image img1 in the same manner. In this way, the processor 130 can convert the original image img1 into an optimized image. In some embodiments, the optimized image belongs to the same type of image as the original image img1 (for example, both the optimized image and the original image img1 are grayscale images).

[0027] The generation of optimized images will be explained below with actual examples. Please also refer to Figure 4, which is a schematic diagram showing optimized image img2 in some embodiments of the present invention. The processor 130 inputs the original image img1 into the neural network model 131 to generate an optimized correspondence table ot1, which includes multiple pixel value intervals for all pixels of the original image img1 and optimized pixel values ​​corresponding to each pixel value interval (see Figure 4).

[0028] Next, by searching the optimized lookup table ot1, the processor 130 identifies which pixel value interval in the optimized lookup table ot1 each pixel of the original image img1 belongs to. Then, the processor 130 converts the pixel values ​​of pixels in the original image img1 that belong to the pixel value interval of 0 to 5 to 3, and converts the pixel values ​​of pixels in the original image img1 that belong to the pixel value interval of 6 to 10 to 9. By analogy, the processor 130 converts the pixel values ​​of all pixels in the original image img1 to their respective optimized pixel values ​​using the lookup table optimized lookup table ot1.

[0029] In other words, if the pixel value of one pixel coordinate (e.g., (11,21)) in the original image img1 falls within the pixel value range of 0 to 5, the processor 130 identifies the optimized pixel value corresponding to this pixel value as 3 using the lookup table optimized correspondence table ot1, and sets the pixel value of the same pixel coordinate (e.g., (11,21)) in the optimized image img2 to 3. If the pixel value of another pixel coordinate (e.g., (55,15)) in the original image img1 falls within the pixel value range of 6 to 10, the processor 130 identifies the optimized pixel value corresponding to this pixel value as 9 using the lookup table optimized correspondence table ot1, and sets the pixel value of the same pixel coordinate (e.g., (55,15)) in the optimized image img2 to 9.

[0030] By analogy, the processor 130 converts the pixel values ​​of all pixels in the original image img1 to the pixel values ​​of all pixels in the optimized image img2 using the lookup table optimized correspondence table ot1. In this way, each time the thermal imaging device 200 captures a new detection area and generates a new original image, the processor 130 converts the new original image into a new optimized correspondence table, and then rapidly converts the new original image into a new optimized image using the lookup table optimized correspondence table. Unlike conventional techniques, there is no need for flowcharts to manually adjust parameters for the area to be captured in accordance with the diversity of target objects, differences in detection objectives, and differences in individual subjective perception. Furthermore, the method of optimizing pixels using the generated optimized correspondence table can achieve optimization effects similar to those of sample correspondence tables st1 to stN (i.e., similar contrast and brightness optimization). It should be noted that the maximum pixel value range int in the optimized correspondence table ot1 is determined by the data size of the original image img1 (for example, if the data size is 256 bits, the maximum pixel value range int may be in the pixel value range of 250 to 255, and if the data size is 16384 bits, the maximum pixel value range int may be in the pixel value range of 16380 to 16383).

[0031] In some embodiments, the thermal image optimization apparatus 100 may further include a display device (not shown). The processor 130 controls the display device to display the optimized image img2 described above, allowing the user to view the optimized image img2 optimized by the neural network model 131. The displayed optimized image img2 has optimal contrast and brightness, etc. It should also be noted that the optimization characteristics of the optimized correspondence table ot1 generated by the neural network model 131 are similar to the optimization characteristics of the sample correspondence tables st1 to stN (i.e., similar adjustments are made to parameters such as contrast and brightness).

[0032] In summary, the thermal image optimization method and apparatus described herein trains a neural network model using a large number of pre-stored sample images and a sample correspondence table corresponding to specific parameters. In this way, each time the thermal imaging device captures a new image, the thermal image optimization method and apparatus described herein converts the new image into a new correspondence table using the trained neural network model and optimizes all pixels of the new image using a lookup table method (i.e., obtaining optimal contrast and brightness). This eliminates the need for complex flowcharts for manually adjusting parameters and the need to readjust parameters over time in various different environments. Furthermore, it achieves image optimization and maintains optimal performance without using complex image optimization algorithms.

[0033] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention. [Explanation of symbols]

[0034] 100 Image optimization device for thermal imaging 110 Data Capture Circuit 120 storage 130 Processors 131 Neural Network Models si1~siN Sample Images Sample correspondence table for st1~stN img1 Original image int maximum pixel value range osi1 optimized sample image S310~S340 Step ot1 Optimized correspondence table img2 Optimized image

Claims

1. The data capture circuit performs step a, in which it captures the original image of the detection area using a thermal imaging device. Step b, the processor updates a neural network model using a plurality of sample images stored in storage and a plurality of sample correspondence tables corresponding to each of the plurality of sample images, wherein the plurality of sample correspondence tables show the correspondence between the pixel values ​​of the plurality of sample images generated by the thermal imaging device in a plurality of training regions and the optimized pixel values ​​of the plurality of sample images. The processor inputs the original image into the neural network model to generate an optimized correspondence table (step c), A thermal image optimization method characterized by comprising the step d, in which the processor converts the original image into an optimized image using the optimized correspondence table.

2. The optimized correspondence table shows the correspondence between multiple pixel value intervals of the original image and multiple optimized pixel values. The thermal image optimization method according to claim 1, characterized in that each of the sample correspondence tables includes a plurality of pixel value intervals of the sample image corresponding to each of the sample correspondence tables and a plurality of optimized pixel values ​​corresponding to each of the plurality of pixel value intervals.

3. Step b above is, The processor performs the steps of using multiple sample images as multiple training samples, The processor takes the step of setting a plurality of sample correspondence tables, each corresponding to a plurality of sample images, as a plurality of training labels, each corresponding to a plurality of training samples. The thermal image optimization method according to claim 1, characterized in that the processor updates the neural network model using a plurality of training samples and a plurality of training labels corresponding to each of the plurality of training samples.

4. Step d is, The thermal image optimization method according to claim 1, characterized in that the processor includes the step of converting the pixel values ​​of multiple pixel coordinates in the original image to the pixel values ​​of multiple pixel coordinates in the optimized image, respectively, using the optimized correspondence table in a lookup table manner.

5. The optimized correspondence table includes a plurality of pixel value intervals of the original image and a plurality of optimized pixel values ​​corresponding to each of the plurality of pixel value intervals, Step d is, The processor identifies from the optimized correspondence table which pixel value interval each pixel coordinate belongs to, and obtains the optimized pixel value corresponding to each pixel value interval from the optimized correspondence table. The thermal image optimization method according to claim 4, further comprising the step of the processor setting the pixel value of each of the pixel coordinates of the optimized image as the corresponding optimized pixel value.

6. A data capture circuit is positioned to capture the original image of the detection area using a thermal imaging device, A storage device arranged to store multiple sample images, multiple sample correspondence tables corresponding to each of the multiple sample images, and multiple instructions, wherein the multiple sample correspondence tables show the correspondence between the pixel values ​​of the multiple sample images generated by the thermal imaging device in multiple training areas and the optimized pixel values ​​of the multiple sample images. The system comprises a data capture circuit and a processor connected to the storage and configured to operate a neural network model, The processor accesses a plurality of the instructions, Operation a, which updates the neural network model using multiple sample images and multiple sample correspondence tables, Operation b involves inputting the original image into the neural network model to generate an optimized correspondence table. A thermal image image optimization apparatus characterized by performing operation c, which converts the original image into an optimized image using the optimized correspondence table.

7. The optimized correspondence table shows the correspondence between multiple pixel value intervals of the original image and multiple optimized pixel values. The thermal image optimization apparatus according to claim 6, characterized in that each of the sample correspondence tables includes a plurality of pixel value intervals of the sample image corresponding to each of the sample correspondence tables and a plurality of optimized pixel values ​​corresponding to each of the plurality of pixel value intervals.

8. In operation a, the processor The operation of using multiple sample images as multiple training samples, The operation of setting multiple sample correspondence tables corresponding to multiple sample images as multiple training labels corresponding to multiple training samples, The thermal image optimization apparatus according to claim 6, characterized in that it is configured to perform an operation to update the neural network model using a plurality of training samples and a plurality of training labels corresponding to each of the plurality of training samples.

9. In operation c, the processor The thermal image optimization apparatus according to claim 6, characterized in that it is configured to perform an operation in which the pixel values ​​of multiple pixel coordinates of the original image are converted to the pixel values ​​of multiple pixel coordinates of the optimized image, respectively, using the optimized correspondence table in a lookup table manner.

10. The optimized correspondence table includes a plurality of pixel value intervals of the original image and a plurality of optimized pixel values ​​corresponding to each of the plurality of pixel value intervals, In operation c, the processor The operation involves identifying which pixel value interval each pixel coordinate belongs to from the optimized correspondence table, and obtaining the optimized pixel value corresponding to each pixel value interval from the optimized correspondence table. The thermal image optimization apparatus according to claim 9, characterized in that it is configured to perform an operation of setting the pixel value of each of the pixel coordinates of the optimized image as the corresponding optimized pixel value.

Images