Method and apparatus for processing thermal image by using determination of occurrence of non-uniformity

Abstract

A method for processing a thermal image, according to the present invention, comprises the steps of: determining whether an execution condition for non-uniformity correction is satisfied; if it is determined that the execution condition is satisfied, determining whether non-uniformity occurs in a captured image received from a thermal imaging camera; if it is determined that the execution condition is satisfied, dividing each frame of the captured image into a target block and an exclusion block, and deriving a non-uniformity correction value for each block on the basis of pixel values of the target block accumulated in a plurality of frames; determining whether to update the non-uniformity correction value, according to a result of the determination of whether the non-uniformity occurs; and outputting a corrected image by correcting the captured image by using the updated non-uniformity correction value.

Description

Thermal image processing method and apparatus using determination of non-uniformity occurrence

[0001] The present invention relates to a thermal image processing method and apparatus using a determination of non-uniformity occurrence. Specifically, the present invention relates to a thermal image processing method and apparatus using a determination of non-uniformity occurrence that determines whether non-uniformity occurs in a thermal image camera based on an image captured by a thermal image camera, and generates a corrected image by correcting the non-uniformity.

[0002]

[0003] The content described in this section merely provides background information regarding the present embodiment and does not constitute prior art.

[0004] Non-Uniformity Correction (NUC) is a technology primarily used in infrared (IR) and thermal imaging cameras. Each pixel in a sensor can react differently depending on temperature or other environmental factors, and these differences can cause distortion or noise in the image. NUC is the process of correcting such non-uniform responses to make the output image uniform.

[0005] NUC is a critical technology that corrects the non-uniform pixel response of infrared and thermal imaging cameras in various ways. Methods include pre-calibration through offline calibration or automatic real-time calibration based on environmental changes. Each calibration method is selected based on performance, cost, and the application environment, and plays a vital role in maintaining high-quality images.

[0006] This specification is a deliverable derived from the "Automotive Industry Technology Development (R&D)" project (Project Title: Development of ISP (Image Signal Processor) Semiconductors for XGA-Class Thermal Imaging Night Vision Cameras, Project No.: RS-2024-00405892) organized by the Korea Institute of Industrial Technology Planning and Evaluation under the Ministry of Trade, Industry and Energy.

[0007]

[0008] The objective of the present invention is to provide a thermal image processing method and apparatus using a non-uniformity occurrence determination that corrects the non-uniformity of a captured image taken through a thermal imaging camera and generates a corrected image.

[0009] The objects of the present invention are not limited to those mentioned above, and other unmentioned objects and advantages of the present invention may be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

[0010]

[0011] A thermal image processing method according to an embodiment of the present invention may include: a step of determining whether a condition for performing non-uniformity correction is satisfied; a step of determining whether non-uniformity occurs in a captured image received from a thermal image camera when the condition for performing is determined to be satisfied; a step of dividing each frame of the captured image into a target block and an exclusion block when the condition for performing is determined to be satisfied, and deriving a non-uniformity correction value for each block based on the pixel value of the target block accumulated from a plurality of frames; a step of determining whether to update the non-uniformity correction value according to the result of determining whether non-uniformity occurs; and a step of correcting the captured image using the updated non-uniformity correction value and outputting a corrected image.

[0012] Additionally, the step of determining whether non-uniformity occurs may include: a step of calculating the pixel standard deviation of the frame; a step of determining whether the pixel standard deviation of the frame is below a first threshold; a step of determining whether the difference between the maximum pixel value and the minimum pixel value of the frame is below a second threshold; a step of determining whether at least one of the cases where the pixel standard deviation is below the first threshold or where the difference between the maximum pixel value and the minimum pixel value is below the second threshold persists for n consecutive frames (n is a natural number); and a step of determining whether non-uniformity occurs based on the result of determining persistence for the n frames.

[0013] Additionally, the above execution conditions may include at least one of the following: when the operation of the thermal imaging camera is started; when the operation of the thermal imaging camera is terminated; when a first time has elapsed after the operation of the thermal imaging camera; when non-uniformity correction of the thermal imaging camera is not performed for a second time; and when the pixel standard deviation of the captured image is less than or equal to a predetermined value.

[0014] Additionally, the step of deriving the non-uniformity correction value may include: dividing each frame included in the captured image into a plurality of blocks; distinguishing each block into a target block and an exclusion block based on at least one of the difference in pixel values ​​for corresponding blocks of two adjacent frames or the average pixel value of each block of the frame; determining whether to mark the block location based on the cumulative quantity of the target block in the plurality of frames; and calculating the non-uniformity correction value for the block location that is not marked.

[0015] Additionally, the step of dividing each of the above blocks into target blocks and exclusion blocks may include: a step of calculating the difference in pixel values ​​between corresponding pixels for a predetermined block of an adjacent first frame and a second frame; a step of calculating a block difference value by accumulating the difference in pixel values ​​for each pixel; and a step of dividing each block of the second frame into the target block or the exclusion block based on the block difference value.

[0016] In addition, the step of distinguishing between the target block or the exclusion block based on the block difference value may set the corresponding block of the second frame as the exclusion block if the block difference value exceeds a third threshold value.

[0017] Additionally, the step of dividing each of the above blocks into target blocks and exclusion blocks may include: a step of calculating the average pixel value of pixels included in a specific block of a specific frame; and a step of dividing the specific block into the target block or the exclusion block based on the difference between the pixel value of each pixel included in the specific block and the average pixel value of the specific block.

[0018] In addition, the step of distinguishing the specific block as the target block or the exclusion block may set the block as the exclusion block when the difference between the pixel value of a predetermined pixel and the average pixel value exceeds a fourth threshold.

[0019] Additionally, the step of determining whether to mark the target block may include, for each block location, a step of counting the cumulative quantity of the target block over a plurality of frames; and a step of adding a marking to the block location when the cumulative quantity of the block location exceeds a fifth threshold.

[0020] Additionally, the step of determining whether the target block is marked may further include, for each block location, a step of calculating the average pixel value of each pixel included in the block location using the cumulative quantity of the target block during the plurality of frames; and a step of outputting the average pixel value of each pixel for the block location and whether the block location is marked.

[0021] Additionally, the step of calculating the non-uniformity correction value may include: a step of determining whether each of the block positions is marked; and, for the unmarked block positions, a step of calculating a pixel-by-pixel non-uniformity correction value of the block position based on the average pixel value of each pixel included in the block position.

[0022] Additionally, the step of calculating the non-uniformity correction value for each pixel may include: a step of calculating the block average pixel value of the block location based on the average pixel value of each pixel; and a step of calculating the non-uniformity correction value for each pixel of the block location based on the difference between the average pixel value of each pixel included in the block location and the block average pixel value of the block location.

[0023] A thermal image processing method according to an embodiment of the present invention may include: a step of determining whether a condition for performing non-uniformity correction is satisfied; a step of determining whether non-uniformity occurs in a captured image received from a thermal imaging camera when the condition for performing non-uniformity is determined to be satisfied; a step of dividing each frame of the captured image into a target block and an exclusion block when it is determined that non-uniformity occurs in the captured image, and deriving a non-uniformity correction value for each block based on the pixel value of the target block accumulated from a plurality of frames; a step of determining whether to update the non-uniformity correction value according to the result of determining whether non-uniformity occurs; and a step of correcting the captured image using the updated non-uniformity correction value and outputting a corrected image.

[0024] A thermal image processing device according to an embodiment of the present invention comprises: a memory storing a thermal image processing program; and a processor for executing the thermal image processing program, wherein the thermal image processing program may include: a step of determining whether a condition for performing non-uniformity correction is satisfied; a step of determining whether a condition for performing non-uniformity correction is satisfied; a step of determining whether non-uniformity occurs in a captured image received from a thermal image camera when the condition for performing non-uniformity is determined to be satisfied; a step of dividing each frame of the captured image into a target block and an exclusion block when the condition for performing non-uniformity is determined to be satisfied, and deriving a non-uniformity correction value for each block based on the pixel value of the target block accumulated from a plurality of frames; a step of determining whether to update the non-uniformity correction value according to the result of determining whether non-uniformity occurs; and a step of correcting the captured image using the updated non-uniformity correction value and outputting a corrected image.

[0025] Additionally, in the thermal image processing program, the step of determining whether non-uniformity occurs may include: a step of calculating the pixel standard deviation of the frame; a step of determining whether the pixel standard deviation of the frame is below a first threshold; a step of determining whether the difference between the maximum pixel value and the minimum pixel value of the frame is below a second threshold; a step of determining whether at least one of the cases where the pixel standard deviation is below the first threshold or where the difference between the maximum pixel value and the minimum pixel value is below the second threshold persists for n consecutive frames (n is a natural number); and a step of determining whether non-uniformity occurs based on the result of determining persistence for the n frames.

[0026]

[0027] The thermal image processing method and apparatus using the determination of non-uniformity occurrence of the present invention determines the non-uniformity of the thermal image camera based on an actual captured image, so that non-uniformity can be reflected in real time.

[0028] In addition, the thermal image processing method and apparatus using the determination of non-uniformity occurrence according to the present invention can increase the accuracy of non-uniformity correction by calculating non-uniformity correction values ​​for each block.

[0029] In addition to the above, the specific effects of the present invention are described together with the following explanation of specific details for implementing the invention.

[0030]

[0031] FIGS. 1 to 4 are conceptual diagrams schematically illustrating a thermal image processing system including a thermal image processing device according to an embodiment of the present invention.

[0032] FIGS. 5 to 7 are exemplary diagrams for explaining the operation of a thermal image processing device.

[0033] FIG. 8 is a flowchart illustrating a thermal image processing method according to an embodiment of the present invention.

[0034] FIG. 9 is a flowchart for explaining the process of determining whether non-uniformity occurs in the captured image shown in FIG. 8.

[0035] FIG. 10 is a flowchart for explaining the process of deriving non-uniformity correction values ​​of the captured image illustrated in FIG. 8.

[0036] FIG. 11 is a flowchart for explaining the process of classifying each block into a target block or an exclusion block based on the first condition illustrated in FIG. 10.

[0037] FIG. 12 is a flowchart for explaining the process of classifying each block into a target block or an exclusion block based on the second condition illustrated in FIG. 10.

[0038] FIG. 13 is a flowchart for explaining the process of determining whether to mark the block location shown in FIG. 10.

[0039] FIG. 14 is a flowchart illustrating the process of calculating non-uniformity correction values ​​for unmarked block locations shown in FIG. 10.

[0040] FIG. 15 is a flowchart illustrating a thermal image processing method of the present invention according to another embodiment.

[0041]

[0042] Terms and words used in this specification and claims shall not be interpreted as being limited to their general or dictionary meanings. In accordance with the principle that an inventor may define the concept of a term or word to best describe their invention, they shall be interpreted in a meaning and concept consistent with the technical spirit of the invention. Furthermore, since the embodiments described in this specification and the configurations illustrated in the drawings are merely one embodiment of the invention and do not represent the entire technical spirit of the invention, it should be understood that various equivalents, modifications, and applicable examples capable of replacing them may exist at the time of filing this application.

[0043] The terms first, second, A, B, etc., as used in this specification and claims may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0044] The terms used in this specification and claims are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" should be understood as not precluding the existence or addition of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification.

[0045] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this invention pertains.

[0046] Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0047] In addition, each component, process, procedure, or method included in each embodiment of the present invention may be shared within a scope that is not technically contradictory to one another.

[0048] Hereinafter, with reference to FIGS. 1 to 15, a thermal imaging processing apparatus and method according to embodiments of the present invention will be described in detail.

[0049]

[0050] FIGS. 1 to 4 are conceptual diagrams schematically illustrating a thermal imaging processing system to which the thermal imaging processing device of the present invention is applied. FIGS. 5 to 7 are exemplary diagrams for explaining the operation of the thermal imaging processing device.

[0051] First, a thermal image processing device is described with reference to FIGS. 1 to 4. A thermal image processing system (1001, 1002, 1003, 1004) includes a thermal image camera (100) and a thermal image processing device (200). The thermal image processing device (200) receives a captured image from the thermal image camera (100), determines whether non-uniformity occurs based on the captured image, and corrects the non-uniformity to output a corrected image.

[0052] The first thermal image processing system (1001) according to the first embodiment shown in FIG. 1 and the second thermal image processing system (1002) according to the second embodiment shown in FIG. 2 represent a structure in which a thermal image camera (100) and a thermal image processing device (200) are provided as separate devices.

[0053] The first thermal image processing system (1001) and the second thermal image processing system (1002) receive a captured image taken by a thermal image processing device (200) through a thermal image camera (100), determine whether non-uniformity occurs in the thermal image camera (100) based on the captured image, generate a non-uniformity correction value according to the determination result, and output a corrected image in which the captured image is corrected using the non-uniformity correction value.

[0054] At this time, the thermal image processing device (200) can generate a correction image for both the image captured by the thermal image camera (100) including a shutter as in FIG. 1 and the image captured by the thermal image camera (100) not including a shutter as in FIG. 2.

[0055] In addition, the third thermal image processing system (1003) according to the third embodiment shown in FIG. 3 and the fourth thermal image processing system (1004) according to the fourth embodiment shown in FIG. 4 represent a structure in which a thermal image camera (100) and a thermal image processing device (200) are configured as a single device.

[0056] The third thermal image processing system (1003) and the fourth thermal image processing system (1004) can omit the process of converting the measured values ​​of the thermal image sensor into images and directly correct the non-uniformity of each pixel of the thermal image sensor to output a corrected image. At this time, the third thermal image processing system (1003) may be provided with a structure including a shutter, and the fourth thermal image processing system (1004) may be provided with a structure not including a shutter.

[0057] Next, a thermal image processing device (200) will be described with reference to FIGS. 1 and FIGS. 5 to 7.

[0058] The thermal image processing device (200) may include an input interface, memory, a processor, and an output interface. The input interface receives a captured image from a thermal image camera (100). The memory stores a thermal image processing program, and the processor can generate a corrected image by executing the thermal image processing program stored in the memory. The output interface can output the corrected image generated by the thermal image processing program.

[0059] To specifically describe the operation of the thermal image processing program, the thermal image processing program determines whether the conditions for performing non-uniformity correction are satisfied. Here, the conditions for performing non-uniformity correction may include at least one of the following: when the operation of the thermal image camera (100) is started; when the operation of the thermal image camera (100) is terminated; when a first time has elapsed after the operation of the thermal image camera (100); when non-uniformity correction of the thermal image camera (100) has not been performed for a second time; when the pixel standard deviation of the captured image is less than or equal to a predetermined value; and when a signal for performing non-uniformity correction is received from an external source. During the operation of the thermal image camera (100), the thermal image processing program may continuously perform the operation of calculating the pixel standard deviation of the captured image and determining whether the pixel standard deviation of the captured image is less than or equal to a predetermined value.

[0060] And, when it is determined that the conditions for performing non-uniformity correction are satisfied, it is determined whether non-uniformity occurs in the captured image received from the thermal imaging camera (100). In addition, when it is determined that the conditions for performing non-uniformity correction are satisfied, each frame of the captured image is divided into target blocks and exclusion blocks, and a non-uniformity correction value for each block location is calculated based on the pixel values ​​of the target blocks accumulated from multiple frames.

[0061] Subsequently, the thermal image processing program determines whether to update the non-uniformity correction value based on the result of judging whether non-uniformity has occurred, corrects the captured image based on the updated non-uniformity correction value, and outputs a corrected image.

[0062] Here, when it is determined that the execution conditions are satisfied, the thermal image processing program may perform an operation to determine whether non-uniformity occurs and an operation to calculate a non-uniformity correction value, respectively. At this time, the thermal image processing program may perform the operation to calculate a non-uniformity correction value even if no non-uniformity occurs in the captured image.

[0063] Additionally, the thermal imaging processing program may perform operations to calculate a non-uniformity correction value, update the non-uniformity correction value, and generate a corrected image based on the updated non-uniformity correction value if it is determined that non-uniformity has occurred during the operation to determine whether non-uniformity has occurred in the captured image. In this case, if it is determined that no non-uniformity has occurred, the subsequent operations are omitted.

[0064]

[0065] Next, each operation of the thermal image processing program will be explained.

[0066] First, the operation of a thermal imaging processing program determining whether non-uniformity occurs in a captured image is explained.

[0067] The thermal image processing program calculates the pixel standard deviation for a frame (F) of the captured image and determines whether the pixel standard deviation of the frame is less than or equal to a first threshold. If the pixel standard deviation of the frame is less than or equal to the first threshold, it determines whether the difference between the maximum pixel value and the minimum pixel value of the frame is less than or equal to a second threshold.

[0068] If the difference between the maximum pixel value and the minimum pixel value is less than or equal to the second threshold, it is determined whether the difference between the maximum pixel value and the minimum pixel value being less than or equal to the second threshold persists for n consecutive frames (n is a natural number), and if it persists for n frames, it is determined that non-uniformity has occurred.

[0069] On the other hand, if the pixel standard deviation exceeds the first threshold or the difference between the maximum and minimum pixel values ​​exceeds the second threshold, it is determined whether this state persists for m consecutive frames (m is a natural number); if it persists for m consecutive frames, it is determined that no non-uniformity has occurred; and if it does not persist for m consecutive frames, the above operation is repeated for continuously received frames.

[0070] Here, the operation of determining whether the pixel standard deviation of the frame is below a first threshold and the operation of determining whether the difference between the maximum pixel value and the minimum pixel value is below a second threshold may be performed either individually or both.

[0071] Next, with reference to FIGS. 5 to 7, the operation of a thermal image processing program to calculate a non-uniformity correction value for a captured image is described. The thermal image processing program divides each frame (F) of a captured image received from a thermal image camera (100) into a plurality of blocks (B).

[0072] Then, based on the distinction condition, each block (B) is divided into target blocks and exclusion blocks, and a decision is made on whether to mark each block location based on the cumulative number of target blocks for each block location within a plurality of frames included in the captured video.

[0073] Next, each operation for calculating non-uniformity correction values ​​will be explained in detail.

[0074] First, the operation of a thermal image processing program classifying each block into target blocks and exclusion blocks based on a classification condition is described. At this time, the classification condition may include a first condition based on the difference in pixel values ​​for corresponding blocks of two adjacent frames and a second condition based on the average pixel value of each block of a frame.

[0075] To explain the operation of distinguishing each block into a target block and an exclusion block according to a first condition based on the difference in pixel values ​​for corresponding blocks of two adjacent frames, the thermal image processing program calculates the difference in pixel values ​​between corresponding pixels (P) for a predetermined block (B) of two adjacent frames in a plurality of frames (F) of a captured image, and calculates a block difference value by accumulating the difference in pixel values ​​for each pixel (P).

[0076] And, if the block difference value exceeds the third threshold, the corresponding block (B) of the second frame (F2) is set as an exclusion block, and otherwise, the corresponding block (B) is set as a target block.

[0077] The operation of setting the first block (B11) of the first frame (F1) and the second frame (F2) as a target block or an exclusion block is explained as an example. The first pixel (P) of the first block (B11) of the first frame (F1) 1_1 ) and the first pixel (P) of the first block (B11) of the second frame (F2). 2_1 Calculate the difference in pixel values ​​of ), and the second pixel (P) of the first block (B11) of the first frame (F1). 1_2 ) and the second pixel (P) of the first block (B11) of the second frame (F2). 2_2 Calculate the difference in pixel values ​​of ), and perform this process for all pixels (P).

[0078] And, through mathematical formula 1, the block difference value (F t Calculate ) and block difference value (F t If ) exceeds the third threshold, the first block (B11) of the second frame (F2) is set as an exclusion block, and otherwise, the first block (B11) of the second frame (F2) is set as a target block.

[0079]

[0080] [Mathematical Formula 1]

[0081]

[0082]

[0083] Next, an operation to distinguish each block into target blocks and exclusion blocks based on a second condition based on the average pixel value of each block of the frame is described.

[0084] The thermal image processing program calculates the average pixel value of the pixels (P) included in each block of the frame (F), and if the difference between the pixel value of each pixel (P) included in the block and the average pixel value of the block exceeds a fourth threshold, the block is set as an exclusion block, otherwise, the block is set as a target block.

[0085] The process of setting the first block (B11) of the first frame (F1) as a target block or an exclusion block is explained as an example. The first pixel (P) of the first block (B11) of the first frame (F1). 1-1 From ) the nth pixel (P 1-n Calculate the average pixel value of the first block (B11) using the pixel value of each pixel (P) up to ).

[0086] And, through mathematical formula 2, the first pixel (P 1_1 From ) the nth pixel (P 1_n Calculate the pixel value difference (Diff_pixel(x)) between each pixel's pixel value (x) and the average pixel value (Average(Block)) of the first block (B11), and if there is at least one pixel (P) where the pixel value difference (Diff_pixel(x)) exceeds the fourth threshold, set the first block (B11) as an exclusion block.

[0087]

[0088] [Mathematical Formula 2]

[0089]

[0090]

[0091] A thermal image processing program can classify each block of a frame into a target block or an exclusion block based on at least one of the first and second conditions described above.

[0092] Next, the operation of marking each block based on the cumulative quantity of target blocks within multiple frames is described.

[0093] The thermal imaging processing program measures the cumulative quantity of target blocks for each block location across multiple frames. If the cumulative quantity of target blocks exceeds a fifth threshold, the program calculates the pixel-by-pixel average value for that block location using the accumulated target blocks. Conversely, if the cumulative quantity of target blocks is less than or equal to the fifth threshold, the program adds a marking to that block location. Subsequently, the thermal imaging processing program outputs the pixel-by-pixel average value and whether a marking is present for each block location. At this time, the fifth threshold for the cumulative quantity for each block location may be set identically for all block locations or may be set differently for each block location.

[0094] Next, the operation of a thermal imaging processing program to calculate non-uniformity correction values ​​for unmarked block locations is described.

[0095] The thermal imaging processing program determines whether each block location is marked and can calculate a pixel-by-pixel non-uniformity correction value for the corresponding block location based on the pixel-by-pixel average value for unmarked blocks. More specifically, the thermal imaging processing program calculates the block average pixel value for the corresponding block location using the pixel-by-pixel average value of the block location, calculates a non-uniformity correction value for each pixel based on the difference between the pixel average value and the block average pixel value, and thereby calculates the pixel-by-pixel non-uniformity correction value for the corresponding block location.

[0096] Next, the operation of updating non-uniformity correction values ​​and the operation of outputting a corrected image are explained. If the thermal image processing program determines that non-uniformity has occurred in the captured image, it can update the non-uniformity correction value currently set at the corresponding block location using the calculated non-uniformity correction value. On the other hand, if it determines that no non-uniformity has occurred in the captured image, it can maintain the non-uniformity correction value currently set at each block location without using the calculated non-uniformity correction value. Then, it can correct the captured image based on the non-uniformity correction value set at each block location and output a corrected image.

[0097]

[0098] FIG. 8 is a flowchart illustrating a thermal image processing method according to an embodiment of the present invention. FIG. 9 is a flowchart illustrating a process for determining whether non-uniformity occurs in a captured image illustrated in FIG. 8. FIG. 10 is a flowchart illustrating a process for deriving a non-uniformity correction value for a captured image illustrated in FIG. 8. FIG. 11 is a flowchart illustrating a process for classifying each block into a target block or an exclusion block based on a first condition illustrated in FIG. 10. FIG. 12 is a flowchart illustrating a process for classifying each block into a target block or an exclusion block based on a second condition illustrated in FIG. 10. FIG. 13 is a flowchart illustrating a process for determining whether to mark a block location illustrated in FIG. 10. FIG. 14 is a flowchart illustrating a process for calculating a non-uniformity correction value for an unmarked block location illustrated in FIG. 10. FIG. 15 is a flowchart illustrating a thermal image processing method according to another embodiment of the present invention.

[0099] Referring to FIGS. 1 and FIG. 8, a thermal image processing method using a thermal image processing device (200) is described. The thermal image processing method determines whether the conditions for performing non-uniformity correction are satisfied (step S100). Here, the conditions for performing non-uniformity correction may include at least one of the following: when the operation of the thermal image camera (100) is started; when the operation of the thermal image camera (100) is terminated; when the pixel standard deviation of the captured image is less than or equal to a predetermined value; and when a signal for performing non-uniformity correction is received from the outside. The thermal image processing device (200) may periodically or in real time perform the operation of calculating the pixel standard deviation of the captured image during the operation of the thermal image camera (100) and determining whether the pixel standard deviation of the captured image is less than or equal to a predetermined value.

[0100] Then, if it is determined that the conditions for performing non-uniformity correction are satisfied, it is determined whether non-uniformity occurs in the captured image received from the thermal imaging camera (100) (step S200). Additionally, if it is determined that the conditions for performing non-uniformity correction are satisfied, each frame of the captured image is divided into target blocks and exclusion blocks, and a non-uniformity correction value for each block location is calculated based on the pixel values ​​of the target blocks accumulated from multiple frames (step S300).

[0101] Subsequently, based on the result of determining whether non-uniformity occurs, it is determined whether to update the non-uniformity correction value (step S400), and the captured image is corrected based on the updated non-uniformity correction value to output the corrected image (step S500).

[0102]

[0103] Next, each step of the thermal imaging processing method is explained.

[0104] First, with reference to FIG. 9, a process (step S200) in which a thermal image processing device (200) determines whether non-uniformity occurs in a captured image is described.

[0105] The thermal image processing device (200) calculates the pixel standard deviation for a frame (F) of a captured image (step S210) and determines whether the pixel standard deviation of the frame is less than or equal to a first threshold value (step S220). If the pixel standard deviation of the frame is less than or equal to the first threshold value, it determines whether the difference between the maximum pixel value and the minimum pixel value of the frame is less than or equal to a second threshold value (step S230).

[0106] If the difference between the maximum pixel value and the minimum pixel value is less than or equal to the second threshold, it is determined whether the case where the difference between the maximum pixel value and the minimum pixel value is less than or equal to the second threshold persists for n consecutive frames (n is a natural number) (step S240), and if it persists for n frames, it is determined that non-uniformity has occurred (step S250).

[0107] On the other hand, if the pixel standard deviation exceeds the first threshold or the difference between the maximum pixel value and the minimum pixel value exceeds the second threshold, it is determined whether this state persists for m consecutive frames (m is a natural number) (step S260), and if it persists for m consecutive frames, it is determined that no non-uniformity has occurred (step S270), and if it does not persist for m consecutive frames, the above operation is continuously repeated for the received frames.

[0108]

[0109] Next, with reference to FIGS. 5 to 7 and FIG. 10, the process (step S300) of the thermal image processing device (200) calculating a non-uniformity correction value for a captured image is described.

[0110] The thermal image processing device (200) divides each frame (F) of the captured image received from the thermal image camera (100) into a plurality of blocks (B) (step S310), and separates each block (B) into target blocks and exclusion blocks based on a separation condition (steps S320, S330). Then, it determines whether to mark each block location based on the cumulative number of target blocks for each block location within the plurality of frames included in the captured image (step S340), and calculates and stores a non-uniformity correction value for the unmarked block locations (step S350).

[0111] Next, each step of calculating the non-uniformity correction value is explained in detail.

[0112]

[0113] Referring to FIGS. 11 and 12, the operation of a thermal image processing device (200) distinguishing each block into a target block and an exclusion block based on a distinction condition is described. At this time, the distinction condition may include a first condition based on the difference in pixel values ​​for corresponding blocks of two adjacent frames and a second condition based on the average pixel value of each block of the frame.

[0114] Referring to FIG. 11, a process (step S320) of dividing each block into a target block and an exclusion block according to a first condition based on the difference in pixel values ​​for corresponding blocks of two adjacent frames is described.

[0115] The thermal image processing device (200) calculates the difference in pixel values ​​between corresponding pixels (P) for a predetermined block (B) of two adjacent frames in a plurality of frames (F) of a captured image (step S321), and calculates a block difference value by accumulating the difference in pixel values ​​for each pixel (P) (step S322).

[0116] Then, it is determined whether the block difference value exceeds the third threshold (step S323), and if the block difference value exceeds the third threshold, the corresponding block (B) of the second frame (F2) is set as an exclusion block (step S324), and if not, the corresponding block (B) is set as a target block (step S325).

[0117] The process of setting the first block (B11) of the first frame (F1) and the second frame (F2) as a target block or an exclusion block is explained as an example. The first pixel (P) of the first block (B11) of the first frame (F1) 1_1 ) and the first pixel (P) of the first block (B11) of the second frame (F2). 2_1 Calculate the difference in pixel values ​​of ), and the second pixel (P) of the first block (B11) of the first frame (F1). 1_2 ) and the second pixel (P) of the first block (B11) of the second frame (F2). 2_2 Calculate the difference in pixel values ​​of ), and perform this process for all pixels (P).

[0118] And, through mathematical formula 1, the block difference value (F t Calculate ) and block difference value (F t If ) exceeds the third threshold, the first block (B11) of the second frame (F2) is set as an exclusion block, and otherwise, the first block (B11) of the second frame (F2) is set as a target block.

[0119]

[0120] [Mathematical Formula 1]

[0121]

[0122]

[0123] Next, referring to FIG. 12, a process (step S330) of dividing each block into a target block and an exclusion block according to a second condition based on the average pixel value of each block of the frame is described.

[0124] The thermal image processing device (200) calculates the average pixel value of the pixels (P) included in each block of the frame (F) (step S331), and calculates the difference between the pixel value of each pixel (P) included in the block and the average pixel value of the block (step S332). Then, it determines whether the difference value exceeds a fourth threshold value (step S333), and if the difference value exceeds the fourth threshold value, sets the block as an exclusion block (step S334).

[0125] On the other hand, if the difference value in step S333 does not exceed the second threshold, it is determined whether step S333 has been performed for all pixels (step S335), and if step S333 has been performed for all pixels, the corresponding block is set as the target block (step S336), and if not, step S332 can be performed for other pixels.

[0126] The process of setting the first block (B11) of the first frame (F1) as a target block or an exclusion block is explained as an example. The first pixel (P) of the first block (B11) of the first frame (F1). 1-1 From ) the nth pixel (P 1-n Calculate the average pixel value of the first block (B11) using the pixel value of each pixel (P) up to ).

[0127] And, through mathematical formula 2, the first pixel (P 1_1 From ) the nth pixel (P 1_n Calculate the pixel value difference (Diff_pixel(x)) between each pixel's pixel value (x) and the average pixel value (Average(Block)) of the first block (B11), and if there is at least one pixel (P) where the pixel value difference (Diff_pixel(x)) exceeds the fourth threshold, set the first block (B11) as an exclusion block.

[0128]

[0129] [Mathematical Formula 2]

[0130]

[0131]

[0132] A thermal image processing program can classify each block of a frame into a target block or an exclusion block based on at least one of the first and second conditions described above.

[0133] The thermal image processing device (200) can classify each block of the frame into a target block or an exclusion block based on at least one of step S320 and step S330.

[0134]

[0135] Next, referring to FIG. 13, a process (step S340) of marking each block based on the cumulative quantity of target blocks within a plurality of frames is described.

[0136] The thermal image processing device (200) counts the accumulated quantity of target blocks for each block location in a plurality of frames (step S341). Then, it determines whether the accumulated quantity of target blocks exceeds a fifth threshold (step S342), and if the accumulated quantity exceeds the fifth threshold, it calculates the pixel-by-pixel average value for the corresponding block location using the accumulated target blocks (step S343). On the other hand, if the accumulated quantity of target blocks is less than or equal to the fifth threshold, it adds a marking to the corresponding block location (step S344). Subsequently, the thermal image processing program outputs the pixel-by-pixel average value based on the accumulated target blocks and whether a marking is applied for each block location (step S345).

[0137]

[0138] Next, with reference to FIG. 14, the process (step S350) of the thermal image processing device (200) calculating a non-uniformity correction value for an unmarked block location is described.

[0139] The thermal imaging processing device (200) determines whether each block location is marked (step S351), and if the block location is not marked, calculates the block average pixel value for the corresponding block location based on the pixel-by-pixel average pixel value for the corresponding block location calculated in step S343 (step S352). Subsequently, the pixel-by-pixel non-uniformity correction value for the corresponding block location can be calculated (step S353) based on the difference between the pixel-by-pixel average pixel value and the block average pixel value. On the other hand, if the block location is marked in step S351, the process of calculating the non-uniformity correction value for the corresponding block location is omitted.

[0140] Next, to explain the process of updating the non-uniformity correction value (step S400), if the thermal image processing device (200) determines in step S200 that non-uniformity has occurred in the captured image, it can update the currently set non-uniformity correction value with the non-uniformity correction value calculated in the process of calculating the non-uniformity correction value (step S300).

[0141] That is, if step S250 is performed and it is determined that non-uniformity has occurred in the captured image, the non-uniformity correction value currently set at the corresponding block location can be updated with the non-uniformity correction value calculated in S353. On the other hand, if step S270 is performed and it is determined that no non-uniformity has occurred in the captured image, the non-uniformity correction value currently set at each block location can be maintained without updating with the non-uniformity correction value calculated in S353.

[0142] And, in the process of correcting the captured image based on the non-uniformity correction value and outputting the corrected image (step S500), the thermal image processing device (200) can correct the captured image based on the non-uniformity correction value set for each block position and generate the corrected image.

[0143] Additionally, if the thermal image processing method determines that the execution conditions are satisfied in step S100, it may perform a process of determining whether non-uniformity occurs (step S200) and a process of calculating a non-uniformity correction value (step S300) as shown in FIG. 8, respectively. If it is determined that non-uniformity has occurred in the process of determining whether non-uniformity occurs (step S200) as shown in FIG. 15, it may perform a process of calculating a non-uniformity correction value (step S300), a process of updating the calculated non-uniformity correction value (step S400), and a process of correcting the captured image based on the updated non-uniformity correction value to output a corrected image (step S500).

[0144]

[0145] The above description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment.

[0146]

[0147] [Explanation of the symbol]

[0148] 1001, 1002, 1003, 1004: Thermal imaging processing system

[0149] 100: Thermal imaging camera

[0150] 200: Thermal imaging processing device

Claims

1. A thermal image processing method for correcting images captured by a thermal imaging camera A step of determining whether the execution conditions for non-uniformity correction are satisfied; When it is determined that the above execution conditions are satisfied, a step of determining whether non-uniformity occurs in the captured image received from the thermal imaging camera; When it is determined that the above execution conditions are satisfied, a step of dividing each frame of the captured image into a target block and an exclusion block, and deriving a non-uniformity correction value for each block based on the pixel values ​​of the target block accumulated from a plurality of frames; A step of determining whether to update the non-uniformity correction value based on the result of determining whether the above non-uniformity occurs; and A thermal image processing method comprising the step of correcting the captured image using the updated non-uniformity correction value and outputting a corrected image.

2. In Paragraph 1, The step of determining whether the above-mentioned non-uniformity occurs is, A step of calculating the pixel standard deviation of the above frame; A step of determining whether the pixel standard deviation of the above frame is less than or equal to a first threshold; A step of determining whether the difference between the maximum pixel value and the minimum pixel value of the above frame is less than or equal to a second threshold; A step of determining whether at least one of the cases where the pixel standard deviation is below a first threshold or where the difference between the maximum pixel value and the minimum pixel value is below a second threshold persists for n consecutive frames (n is a natural number); and A thermal image processing method comprising a step of determining whether non-uniformity occurs based on the result of determining whether the n frames persist.

3. In Paragraph 1, The above execution conditions are, A thermal image processing method comprising at least one of the following: when the operation of the thermal imaging camera is started; when the operation of the thermal imaging camera is terminated; when a first time has elapsed after the operation of the thermal imaging camera; when non-uniformity correction of the thermal imaging camera is not performed for a second time; and when the pixel standard deviation of the captured image is less than or equal to a predetermined value.

4. In Paragraph 1, The step of deriving the above non-uniformity correction value is, A step of dividing each frame included in the above-mentioned captured image into a plurality of blocks; A step of dividing each block into a target block and an exclusion block based on at least one of the difference in pixel values ​​for corresponding blocks of two adjacent frames or the average pixel value of each block of said frame; A step of determining whether to mark the corresponding block location based on the cumulative quantity of the target block in a plurality of frames; and A thermal image processing method comprising the step of calculating a non-uniformity correction value for the above-mentioned block location that is not marked.

5. In Paragraph 4, The step of dividing each of the above blocks into target blocks and exclusion blocks is: For a predetermined block of adjacent first and second frames, a step of calculating the difference in pixel values ​​between corresponding pixels; A step of calculating a block difference value by accumulating the pixel value difference for each pixel; and A thermal image processing method comprising the step of classifying each block of the second frame into the target block or the exclusion block based on the block difference value.

6. In Paragraph 5, The step of classifying into the target block or the exclusion block based on the block difference value is: A thermal image processing method that sets the corresponding block of the second frame as the exclusion block when the block difference value exceeds a third threshold.

7. In Paragraph 4, The step of dividing each of the above blocks into target blocks and exclusion blocks is: A step of calculating the average pixel value of pixels included in a specific block of a specific frame; and A thermal image processing method comprising the step of classifying a specific block into a target block or an exclusion block based on the difference between the pixel value of each pixel included in the specific block and the average pixel value of the specific block.

8. In Paragraph 7, The step of classifying the above specific block into the above target block or the above exclusion block is, A thermal image processing method that sets the block as the exclusion block when the difference between the pixel value of a predetermined pixel and the average pixel value exceeds a fourth threshold.

9. In Paragraph 8, The step of determining whether to mark the above target block is, For each block location, a step of counting the cumulative quantity of the target block over a plurality of frames; and A thermal image processing method comprising the step of adding a marking to a block location when the accumulated quantity of the above block location exceeds a fifth threshold.

10. In Paragraph 9, The step of determining whether to mark the above target block is, For each block location, a step of calculating the average pixel value of each pixel included in the corresponding block location using the cumulative quantity of the target block during the plurality of frames; and A thermal image processing method further comprising the step of outputting the average pixel value of each pixel for the above block location and whether the above block location is marked.

11. In Paragraph 4, The step of calculating the above non-uniformity correction value is, A step of determining whether to mark the position of each block; and A thermal image processing method comprising the step of calculating a pixel-by-pixel non-uniformity correction value for the block location based on the average pixel value of each pixel included in the block location, for the unmarked block location.

12. In Paragraph 11, The step of calculating the non-uniformity correction value for each pixel above is, A step of calculating the block average pixel value of the target block based on the average pixel value of each of the above pixels; and A thermal image processing method comprising the step of calculating a non-uniformity correction value for each pixel of the target block based on the difference between the average pixel value of each pixel included in the target block and the block average pixel value of the target block.

13. A thermal image processing method for correcting images captured by a thermal imaging camera A step of determining whether the execution conditions for non-uniformity correction are satisfied; When it is determined that the above execution conditions are satisfied, a step of determining whether non-uniformity occurs in the captured image received from the thermal imaging camera; If it is determined that non-uniformity has occurred in the above-mentioned captured image, the step of dividing each frame of the above-mentioned captured image into a target block and an exclusion block, and deriving a non-uniformity correction value for each block based on the pixel values ​​of the target block accumulated from a plurality of frames; A step of determining whether to update the non-uniformity correction value based on the result of determining whether the above non-uniformity occurs; and A thermal image processing method comprising the step of correcting the captured image using the updated non-uniformity correction value and outputting a corrected image.

14. A thermal image processing device for correcting images captured by a thermal imaging camera, Memory where a thermal imaging processing program is stored; and It includes a processor that executes the above thermal image processing program, The above thermal imaging processing program is, A step of determining whether the conditions for performing non-uniformity correction are satisfied; A step of determining whether the execution conditions for non-uniformity correction are satisfied; When it is determined that the above execution conditions are satisfied, a step of determining whether non-uniformity occurs in the captured image received from the thermal imaging camera; When it is determined that the above execution conditions are satisfied, a step of dividing each frame of the captured image into a target block and an exclusion block, and deriving a non-uniformity correction value for each block based on the pixel values ​​of the target block accumulated from a plurality of frames; A step of determining whether to update the non-uniformity correction value based on the result of determining whether the above non-uniformity occurs; and A thermal image processing device comprising the step of correcting the captured image using the updated non-uniformity correction value and outputting a corrected image.

15. In Paragraph 14, In the above thermal image processing program, The step of determining whether the above-mentioned non-uniformity occurs is, A step of calculating the pixel standard deviation of the above frame; A step of determining whether the pixel standard deviation of the above frame is less than or equal to a first threshold; A step of determining whether the difference between the maximum pixel value and the minimum pixel value of the above frame is less than or equal to a second threshold; A step of determining whether at least one of the cases where the pixel standard deviation is below a first threshold or where the difference between the maximum pixel value and the minimum pixel value is below a second threshold persists for n consecutive frames (n is a natural number); and A thermal image processing device comprising a step of determining whether non-uniformity occurs based on the result of determining whether the n frames persist.

Images