Body heat tester and method of operating the same

Abstract

The present invention relates to an automatic optimal mapping method between a real image and a thermal imaging image of a fever screening device, and a fever screening device applying the same, in which the automatic optimal mapping method applies a frame matching method so that a real image obtained from a real image camera has a wider angle than a thermal imaging image obtained from a thermal imaging camera, in order to maximize the use of thermal imaging pixels without omission, thereby reconstructing the real image by automatically stretching or shortening the top, bottom, left, and right sides of the real image so that the thermal imaging image can be maximally used without omission of the thermal imaging pixels of the fever screening device using an infrared imaging device. The fever screening device of the present invention includes a thermal imaging camera that captures a thermal imaging image of an area including a plurality of objects, a real image camera that simultaneously captures a real image of the area including the plurality of objects with the thermal imaging camera, and an operation processing unit that matches the thermal imaging image received from the thermal imaging camera with the real image received from the real image camera, and stretches or shortens the top, bottom, left, and right sides of the real image based on the thermal imaging image to obtain a reconstructed real image matched with the thermal imaging image, and detects a body heat (temperature) of the object by using the thermal imaging image and the reconstructed real image. The fever screening device of the present invention can maximize the use of thermal imaging pixels without omission by obtaining an image of a real image camera captured at a wider angle than a thermal imaging camera, thereby reducing errors and improving accuracy.

Description

TECHNICAL FIELD

[0001] The present invention relates to a method for optimal automatic mapping between a real image and a thermal image in a body heat tester (a fever screening device) and a body heat tester using the same. The present invention maximizes the use of a thermal image without missing thermal image pixels in a thermal inspection device using an infrared imaging device. In order to obtain most of the thermal image without missing pixels, a real image obtained from a real imaging camera (a visual camera) should have a wider angle of view than a thermal image obtained from a thermal imaging camera. In order to maximize the use of thermal image pixels without missing thermal image pixels, the present invention reconstructs a real image by automatically stretching (expanding) or shortening the top, bottom, left, and right sides of the real image via application of a frame matching method that makes the real image obtained from a real image camera have a wider angle than the thermal image obtained from a thermal image camera. BACKGROUND

[0002] A thermal imaging camera is a camera that can measure the surface temperature of an object in a non-contact manner. The standard for determining fever in humans through body temperature screening for a specific infectious disease is 37.3°C. In order to compare with a person with a normal body temperature of 36.5°C, the temperature should be expressed in 0.1°C units, and a temperature accuracy of ±0.4°C should be maintained at room temperature. These are requirements for operating a thermal imaging camera for medical purposes.

[0003] If these requirements are not met (i.e., if the temperature accuracy is low), a feverish person can be judged to be a normal person, or the opposite result can occur. Therefore, a conventional thermal imaging camera is operated as an accurate temperature reference point by additionally installing a black body source as an additional device for improving temperature accuracy.

[0004] In order to accurately identify and track a feverish person, a real imaging camera must be additionally used, and both a real image and a thermal image of a person should be simultaneously photographed. In this case, when a real image output screen and a thermal image output screen are configured, it is necessary to provide the same location coordinate information.

[0005] In general, in temperature inspection as a public screening tool, two or more heat sources should be simultaneously detected, and moving objects should be analyzed and displayed in real time on a screen.

[0006] In addition, for an infrared image, data based on temperature information is received, and in order to visualize the data, it is necessary to determine what color is expressed through a color mapping technique for a corresponding temperature. This is necessary for diagnostic efficiency. For color mapping of a temperature range, the mapping of results should be completed by analyzing a histogram.

[0007] If a temperature screening product for the public is composed of only a thermal imaging camera, there is a disadvantage that it cannot easily distinguish a feverish person from a normal person or track the feverish person when there are many feverish people.

[0008] This is because a black and white image or a color mapping image only emphasizes temperature differences, and thus it is not easy to distinguish facial features and appearances. To solve this problem, the International Organization for Standardization has developed a recommendation for fever screening that should simultaneously photograph both a real image and a thermal image of a person. In this case, when the resolutions of the two images (real image and thermal image) are different, or when the viewing angles are different even if the resolutions are the same, this can cause user confusion. In the case of displaying a measured temperature on a screen, the temperature value of the correct coordinates is not confused only when the positions of the objects are the same. Therefore, a temperature screening tool requires a 1:1 mapping of a real image and a thermal image.

[0009] In general, there are three different types of temperature measurement instruments. They include a method of measuring temperature values of all temperatures entering a screen, face recognition at a plurality of regions, and a method of reducing errors (malfunctions) by measuring different temperature values that distinguish a background, an object, and a person entering or leaving. In the case of using face recognition, there is a problem that the number of persons entering and leaving cannot be accurately determined according to image detection accuracy. Specifically, the object must look straight at the camera, and the influence of wearing a mask or an accessory such as a hat adversely affects the detection ability. In addition, ideal positions for accurate temperature measurement are points at which an artery passes (such as the corner of the eye and the temple region of the face) and points at which a carotid artery passes through the neck and the skin. These regions are regions in which accurate body temperature can be measured. Since face recognition excludes these positions (including side views or rear views of the object), there is a limitation on measuring body temperature. Therefore, multi-region body temperature detection is a method of compensating for the disadvantages of face recognition.

[0010] In the present invention, thermal image pixels are maximally used without missing pixel values. To do so, real imaging of a wider angle than thermal imaging is automatically obtained by applying a frame matching method (i.e., the top, bottom, left, and right of the real image can be automatically stretched or shortened). The present invention proposes an optimal automatic mapping method of a real image and a thermal image of a body heat tester using an infrared imaging device, and the body heat tester applies the method to maximally use thermal image pixels without missing pixel values.

[0011] In the present invention, the body heat tester stores six real image frames each at a time before and after the thermal image frame in a memory unit based on the thermal image frame; selects a real image frame having the largest matching rate among the saved real image frames; detects the boundary of the object in the thermal image frame and the selected real image frame; and matches the thermal image frame and the result of expanding (stretching) or reducing (shortening) the selected real image frame by 1 pixel in each direction of the top, bottom, left, and right to select a point having the largest matching rate. By doing so, it is possible to obtain the result of optimized mapping in all directions.

[0012] Korean Patent Publication No. 10-2016-0056023 "Apparatus for providing thermal image using thermopile array sensor and real imaging camera and method of using the same" as the prior art describes creating a high-resolution thermal image of an object by matching a thermal image with a high-resolution real image through the characteristics of the analyzed thermal distribution and the high-resolution real image.

[0013] However, Korean Patent Publication No. 10-2016-0056023 does not have the technology of detecting the boundary of the object (real image and thermal image) like the present invention; matching the stretched and shortened boundary result by 1 pixel in each direction; comparing the matching rate; re-executing the same process to find a point having the largest matching rate; and thus fully utilizing the thermal image without missing the thermal image pixel. SUMMARY

[0014]

Technical Problem

[0015] The issue to be addressed by the present invention is to provide an optimal automatic mapping method between a real image and a thermal image in a body heat tester, in which the real image from a real imaging camera (vision camera) has a wider angle of view than the thermal image from a thermal imaging camera, to maximize the use of thermal imaging without missing the thermal imaging pixel in a thermal inspection device using an infrared imaging device, and a body heat tester using the same.

[0016] Another issue to be addressed by the present invention is to provide an optimal automatic mapping method between a real image and a thermal image in a body heat tester, in which six real image frames each at a time before and after the thermal image frame are stored in a memory unit based on the thermal image frame storage; a real image frame having the largest matching rate is selected from among the saved real image frames; the boundary of the object in the thermal image frame and the selected real image frame is detected; and the thermal image frame and the result of expanding (stretching) or reducing (shortening) the selected real image frame by 1 pixel in each direction of the top, bottom, left, and right is matched to select a point having the largest matching rate.

[0017]

Technical Solution to the Problem

[0018] To solve the above problems and achieve the above objects, a body heat tester according to the present application is characterized by having a real thermal image mapping unit. The body heat tester includes a thermal imaging camera for photographing a thermal image of a region having a plurality of subjects, a real imaging camera for photographing a real image of the region having the plurality of subjects simultaneously with the thermal camera, and a data processing unit.

[0019] The data processing unit performs the following operations: matching a thermal image received from the thermal imaging camera and a real image received from the real imaging camera; obtaining a reconstructed real image matched with the thermal image by stretching or shortening a top, a bottom, a left side, and a right side of the real image based on the thermal image; and detecting a body heat (temperature) of a subject using the thermal image and the reconstructed real image.

[0020] The data processing unit stores six real image frames each at a time before and after a thermal image frame in a memory unit based on the thermal image frame; selects a real image frame having a maximum matching rate with the thermal image among the saved real image frames; and detects a boundary of a subject in the thermal image frame and the selected real image frame. Then, the thermal image frame and a result of expanding (stretching) or reducing (shortening) the selected real image frame by 1 pixel in each direction of the top, the bottom, the left, and the right are matched to select a point having a maximum matching rate.

[0021] The real image and thermal image matching unit inverts a real image screen and a thermal image screen for edge detection; compares a thermal image profile on a horizontal axis in the inverted predetermined thermal image with a real image profile on the horizontal axis in the inverted real image; obtains a real image profile on the horizontal axis closest to the thermal image profile on the horizontal axis as a reconstructed real image profile on the horizontal axis; compares a thermal image profile on a vertical axis in the inverted predetermined thermal image with a real image profile on the vertical axis in the inverted real image; obtains a real image profile on the vertical axis closest to the thermal image profile on the vertical axis as a reconstructed real image profile on the vertical axis; calculates a ratio and an increment (position) of the obtained real image profiles (horizontal and vertical); adjusts a ratio and a position of the real image using the ratio and the increment values of the real image profiles (horizontal and vertical); stores the adjusted real image as a reconstructed real image in a memory unit; and outputs the same to a display unit.

[0022] Further, the data processing unit further includes a multi-region body temperature detection unit. The multi-region body temperature detection unit divides the thermal image and the reconstructed real image into a preset number of horizontal x vertical; obtains a total average from a region of interest (ROI) that is a divided region of the thermal image; excludes a divided region in which a maximum value of each divided region of the thermal image is less than the total average from the ROI; excludes a region in which a maximum value of each divided region of the thermal image exceeds a preset threshold temperature from the ROI; compares maximum values from adjacent regions of the ROI; excludes remaining adjacent regions from the ROI, leaving only a divided region having a maximum maximum value among the adjacent regions; calculates coordinates of the remaining ROI (i.e., XY coordinates) and a highest temperature of the corresponding ROI; and displays the calculated highest temperature on the thermal image and the real image at the coordinates of the corresponding ROI.

[0023] The data processing unit further includes a level automatic adjustment unit. The level automatic adjustment unit calculates a histogram from the thermal image; detects a background peak (a peak value higher than a reference value of a preset background peak) in the histogram; detects a body surface area (BSA) peak (a peak value lower than a reference value of a preset BSA peak) in the histogram; sets a point in which a highest temperature region becomes 3% of a total thermal image area as an upper limit value that is a maximum expression temperature; sets a divided point in which the background and the BSA intersect as a minimum point (a lower limit value); performs pseudo-colorization on the BSA in the histogram by creating color mapping data using the lower limit and the upper limit; and outputs the obtained pseudo-colorized image to the display unit.

[0024] Another feature of the present application is a method of operating a body heat tester including a thermal image camera and a real image camera that simultaneously detect body heat of a plurality of subjects. The method of operating the body heat tester includes: a data processing unit matching a thermal image received from the thermal imaging camera and a real image received from the real imaging camera; based on the thermal image, obtaining a reconstructed real image matched with the thermal image by stretching or shortening a top, a bottom, a left, and a right of the real image; and detecting body heat (body temperature) of the subjects using the thermal image and the reconstructed real image.

[0025] In order to obtain the reconstructed real image, the data processing unit imports six real image frames acquired at a time before / after a thermal image frame from a memory unit based on the thermal image frame; selects a real image frame having a maximum matching rate with the thermal image frame among the real image frames; detects a boundary of the subject in each of the thermal image frame and the selected real image frame; matches the thermal image frame with a result of expanding (stretching) and reducing (shortening) the selected real image frame by 1 pixel in each direction of the top, the bottom, the left, and the right; and selects a point having a maximum matching rate.

[0026] To match the thermal image and the real image, the method of operating the body heat tester includes a screen inversion step, a horizontal axis matching rate detecting step, a step of comparing with a maximum matching value of the horizontal axis, a step of comparing with a maximum matching value of the vertical axis, a step of storing matching data of the horizontal axis and the vertical axis, and a real image adjusting step. In the screen inversion step, the data processing unit inverts the real image screen and the thermal image screen for edge detection of the data processing unit. In the horizontal axis matching rate detecting step, the data processing unit obtains a horizontal axis matching rate by comparing a horizontal axis thermal image profile in a predetermined thermal image inverted in the screen inversion step with a real image profile along the horizontal axis in the inverted real image in the screen inversion step. In the step of comparing with the maximum matching value of the horizontal axis, the data processing unit compares the horizontal axis matching rate with a pre-stored maximum matching rate value of the horizontal axis; and if the horizontal axis matching rate is not equal to the maximum matching rate value of the horizontal axis, imports a next horizontal axis real image profile of the inverted real image from the screen inversion step from the memory unit, and returns to the horizontal axis matching rate detecting step. In the vertical axis matching rate detecting step, the data processing unit obtains a vertical axis matching rate by comparing a vertical axis thermal image profile in a predetermined thermal image inverted in the screen inversion step with a real image profile along the vertical axis in the inverted real image in the screen inversion step. In the step of comparing with the maximum matching value of the vertical axis, the data processing unit performs the following steps: comparing the vertical axis matching rate with a pre-stored maximum matching rate value of the vertical axis; and if the vertical axis matching rate is not equal to the maximum matching rate value of the vertical axis, importing a next vertical axis real image profile of the inverted real image from the screen inversion step from the memory unit, and returning to the vertical axis matching rate detecting step. In the step of storing matching data of the horizontal axis and the vertical axis, if the horizontal axis matching rate is equal to the maximum matching value of the horizontal axis in the step of comparing with the maximum matching value of the horizontal axis, the data processing unit uses the horizontal axis real image profile as a horizontal axis real image profile for reconstruction; if the vertical axis matching rate is equal to the maximum matching rate value of the vertical axis in the step of comparing with the maximum matching value of the vertical axis, uses the vertical axis real image profile as a vertical axis real image profile for reconstruction; and obtains a rate and a position (increment) of the horizontal axis real image profile for reconstruction and the vertical axis real image profile for reconstruction, and stores the rate and the position (increment) in the memory unit. In the real image adjusting step, the data processing unit adjusts the position and the rate of the vertical real image profile and the horizontal real image profile with the rate and the increment value of the vertical real image profile and the horizontal real image profile for reconstruction; saves the adjusted real image as a reconstructed real image in the memory unit; and outputs it to the display unit.

[0027] To detect the body heat of a subject, a method of operating a body heat tester includes a region division step, an outermost region exclusion step, an average value detection step, a maximum value comparison step, a neighboring region comparison step, a highest temperature detection step, and a step of displaying a corresponding value on an image. In the region division step, a data processing unit divides a region of a thermal image on a screen into a preset number of horizontal divisions (number of horizontal division lines) x number of vertical divisions (number of vertical division lines), and sets the divided region as an ROI. In the outermost region exclusion step, the data processing unit excludes the outermost region in each of the divided regions in the region division step. In the average value detection step, the data processing unit calculates an average value of the thermal image in all of the regions of the thermal image divided in the region division step. In the maximum value comparison step, the data processing unit compares a maximum value of the regions of the thermal image divided in the region division step with the average value detected in the average value detection step, and excludes a region in which the maximum value is smaller than the average value from the ROI. In the neighboring region comparison step, the data processing unit compares the maximum values between neighboring regions connected to each other in the ROI, and leaves only a region having the largest maximum value and excludes the remaining neighboring regions from the ROI. In the highest temperature detection step, the data processing unit takes the remaining ROI after the neighboring region comparison step as a final ROI, and calculates a coordinate (i.e., XY coordinate) and a highest temperature of the final ROI. In the step of displaying a corresponding value on an image, the data processing unit displays the highest temperature of the final ROI detected in the highest temperature detection step on a thermal image and a real image of the coordinate of the final ROI.

[0028] Further, a method of operating a body heat tester includes a histogram calculation step, a background peak detection step, a BSA peak detection step, a lower limit value and upper limit value setting step, and a pseudo-coloring step. In the histogram calculation step, a data processing unit calculates a histogram from a thermal image. In the background peak detection step, the data processing unit detects a background peak in the histogram, in which a background peak value higher than a preset background peak reference value is detected. In the BSA peak detection step, the data processing unit detects a BSA peak in the histogram, in which a peak value lower than a preset BSA peak reference value is detected. In the lower limit value and upper limit value setting step, the data processing unit calculates a point at which a highest temperature region becomes 3% of a total area of the thermal image as a maximum expression temperature (upper limit value), and calculates a point at which a background and a BSA intersect as a minimum point (lower limit value). In the pseudo-coloring step, the data processing unit creates color mapping data using the lower limit value and the upper limit value obtained in the lower limit value and upper limit value setting step, and performs pseudo-coloring on the BSA in the histogram, and outputs the obtained pseudo-colored image to a display unit.

[0029] The horizontal axis matching rate in the horizontal axis matching rate detecting step is an absolute value of a value obtained by subtracting the length of the thermal image profile and the length of the real image profile on the horizontal axis. The vertical axis matching rate in the vertical axis matching rate detecting step is an absolute value of a value obtained by subtracting the length of the thermal image profile and the length of the real image profile on the vertical axis. The maximum matching value of the horizontal axis and the vertical axis can be 0.

[0030]

[0031] In the present application, the best automatic mapping method between a real image and a thermal image in a body heat tester and the body heat tester using the method maximizes the use of thermal imaging without missing thermal imaging pixels in a thermal inspection device using an infrared imaging device, since a real image from a real imaging camera (a vision camera) has a wider angle of view than a thermal image from a thermal imaging camera. By doing so, errors can be reduced, and precision can be improved.

[0032] The present application stores six real image frames each at a time before and after a thermal image frame in a memory unit based on the thermal image frame; selects a real image frame having a maximum matching rate from among the saved real image frames; detects the boundary of an object in the thermal image frame and the selected real image frame; and matches the result of expanding (stretching) or reducing (shortening) the selected real image frame by 1 pixel in each direction of the top, bottom, left, and right of the thermal image frame to select a point having a maximum matching rate. The present application repeats the process of comparing the matching rate several times to set a point having a maximum matching rate, wherein repeating 4 times in the up, down, left, and right directions allows it to be set as a result of optimized mapping in all directions. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a block diagram showing a schematic configuration of a body heat tester of the present application.

[0034] Figure 2 shows example images before and after applying the frame matching method of the present application.

[0035] Figure 3 is a flowchart schematically showing a best automatic mapping method of a real image and a thermal image of a body heat tester of the present application.

[0036] Figure 4 is an explanatory diagram showing the matching rate detecting step and the step of comparing with the maximum matching value in the flowchart of Figure 3

[0037] Figure 5 is an explanatory diagram for showing the real image adjusting step in the flowchart of Figure 3

[0038] ​​​Figure 6 An example of detecting and displaying a body temperature by the multi-region body temperature detection method of the present application is shown.

[0039] Figure 7 is a flowchart showing an example of a method of detecting a multi-region body temperature according to the present application.

[0040] Figure 8 An example image obtained after the region division step of Figure 7 is shown.

[0041] Figure 9 An image as a result of applying the image of Figure 8 to the outermost region exclusion step is shown.

[0042] Figure 10 An image as a result of applying the image of Figure 9 to the maximum value comparison step is shown.

[0043] Figure 11 An image as a result of applying the image of Figure 10 to the adjacent region comparison step is shown.

[0044] Figure 12 Example images before and after the level adjustment are shown.

[0045] Figure 13 is a flowchart showing histogram equalization (level automatic adjustment) for diagnostic purposes in the present application.

[0046] Figure 14 An example of a background portion and a BSA portion is shown.

[0047] Figure 15 An ROI selected from the BSA of Figure 14 is shown. DETAILED DESCRIPTION

[0048] The optimal automatic mapping method of the body heat tester of the present application of real images and thermal images in a body heat tester, and a body heat tester applying the same will be described in detail below with reference to the accompanying drawings.

[0049] Figure 1 is a block diagram showing the schematic configuration of the body heat tester of the present application.

[0050] The thermal image detected by the (infrared) thermal imaging camera (120) and the real image detected by the (general-purpose) real imaging camera (160) are transmitted to the data processing unit (200). In other words, the simultaneously taken thermal image and real image are transmitted to the data processing unit (200).

[0051] The data processing unit (200) includes a real thermal image mapping unit (201), a multi-zone body temperature detection unit (205), and a level automatic adjustment unit (207).

[0052] The real thermal image mapping unit (201) of the data processing unit (200) stores six front / back real image frames in the memory unit (220) based on the thermal image frames, and then selects a real image frame having the largest matching rate with the thermal image frame. The boundaries of the objects in the real image frame and the thermal image frame selected in this way are detected. The thermal image frame and the selected real image frame are matched by expanding or reducing the thermal image frame and the selected real image frame by 1 pixel in both the horizontal (left-right) direction and the vertical (up-down) direction to select a point having the largest matching rate. This process is repeated several times to set a point having the highest selected matching rate. This process is repeated four times in the up, down, left, and right directions. Then, as a result of the mapping optimized in all of the up, down, left, and right directions, the process is fixed.

[0053] Accordingly, the data processing unit (200) can obtain a real image having a wider angle than the thermal image. As a result, the multi-zone body temperature detection unit (205) detects the body heat (body temperature) of a plurality of objects using the thermal image without missing the pixel values of the thermal image. The level automatic adjustment unit (207) performs pseudo-colorization on the BSA, and outputs the body heat image and the temperature value to the display unit (210).

[0054] The optimal automatic mapping method of the body heat tester of the present application will be described in detail below.

[0055] <1:1 mapping method of real image and thermal image>

[0056] The 1:1 mapping method of real image and thermal image is operated by the real thermal image mapping unit (201) of the data processing unit (200).

[0057] In order to maximize the use of the thermal image without missing the pixel values, real imaging from the real imaging camera should be obtained to have a wider angle than thermal imaging from the thermal imaging camera. To this end, in the present application, the data processing unit (200) has a function of stretching and shortening the real image in both the horizontal (left-right) direction and the vertical (up-down) direction.

[0058] This is done by automation of the task, and a frame matching method is applied.

[0059] The data processing unit (200) stores six real image frames each at a time before / after the thermal image frame in the memory unit (220) based on the thermal image frame, selects a real image frame having a maximum matching rate with the thermal image frame among the real image frames, and detects a boundary of an object in the selected real image frame and the thermal image frame.

[0060] The thermal image frame is matched with a result of expanding or reducing the selected real image frame by 1 pixel in each direction of the top, the bottom, the left, and the right, and the matching rates are compared. The same process is repeatedly performed to find and fix a real image frame and a thermal image frame of a point having a maximum matching rate. The same process is repeated four times in each direction of the top, the bottom, the left, and the right to find and set an optimized mapping structure in all directions.

[0061] Figure 2 Before and after applying the frame matching method of the present application are shown. Figure 2 (a) and (b) show real images and thermal images before applying the frame matching method of the present application, and Figure 2 (c) and (d) show real images and thermal images after applying the frame matching method of the present application.

[0062] It can be seen that, compared to images before applying the present application, the matching rate of real images and thermal images after applying the present application can be significantly improved, and the real images have been stretched or shortened.

[0063] Figure 3 is a flowchart schematically showing a best automatic mapping method of real images and thermal images in a body heat tester of the present application.

[0064] In the screen inversion step, the data processing unit (200) performs inversion of the real image screen for edge detection (S110) and inversion of the thermal image screen for edge detection (S115).

[0065] In the horizontal axis matching rate detection step, the data processing unit (200) compares a preset thermal image profile (on the horizontal axis) in the inverted thermal image in the screen inversion step with a real image profile on the horizontal axis (i.e., it can be called an Nth real image profile on the horizontal axis because the real image profiles on the horizontal axis are compared in turn) in the inverted real image in the screen inversion step to obtain a horizontal axis matching rate (S120). Here, an absolute value of a value obtained by subtracting a length of the preset thermal image profile and a length of the real image profile on the horizontal axis can be obtained as the horizontal axis matching rate (a value indicating a matching degree of the horizontal axis).

[0066] In the step of comparing with the maximum matching value of the horizontal axis, the data processing unit 200 compares the horizontal axis matching rate detected in the horizontal axis matching rate detecting step with the maximum matching value (i.e., the maximum matching rate value) of the horizontal axis previously stored (S125). If the horizontal axis matching rate is less than or equal to the maximum matching value of the horizontal axis, the maximum matching value of the horizontal axis has not been detected. Then, before entering the horizontal axis matching rate detecting step, the next (i.e., N+l) real image profile on the horizontal axis of the inverted real image from the screen inversion step is imported from the memory unit (S127). Here, the maximum matching value of the horizontal axis can be 0.

[0067] That is, in the horizontal axis matching rate detecting step and the step of comparing with the maximum matching value of the horizontal axis, the result of the profile analysis for the edge of the real image on the horizontal axis is compared with the value of the profile analysis for the edge of the thermal image. This process is repeated to find the maximum matching value of the horizontal axis.

[0068] In the vertical axis matching rate detecting step, the data processing unit 200 compares the preset thermal image profile (on the vertical axis) in the inverted thermal image in the screen inversion step with the real image profile on the vertical axis (i.e., can be referred to as the Nth real image profile on the vertical axis because the real image profile on the vertical axis is compared sequentially) in the inverted real image in the screen inversion step to obtain the vertical axis matching rate (S130). Here, the absolute value of the value obtained by subtracting the length of the preset thermal image profile and the length of the real image profile on the vertical axis can be obtained as the vertical axis matching rate (a value indicating the matching degree of the vertical axis).

[0069] In the step of comparing with the maximum matching value of the vertical axis, the data processing unit 200 compares the vertical axis matching rate detected in the vertical axis matching rate detecting step with the maximum matching value (i.e., the maximum matching rate value) of the vertical axis previously stored (S135). If the vertical axis matching rate is less than or equal to the maximum matching value of the vertical axis, the maximum matching value of the vertical axis has not been detected. Then, before entering the vertical axis matching rate detecting step, the next (i.e., N+l) real image profile on the vertical axis of the inverted real image from the screen inversion step is imported from the memory unit (S137). Here, the maximum matching value of the vertical axis can be 0.

[0070] In the step of storing the matching data on the horizontal and vertical sides, if the horizontal axis matching rate is equal to or less than the maximum matching value of the horizontal axis in the step of comparing with the maximum matching value of the horizontal axis, the maximum matching value on the horizontal axis is detected. The real image profile on the horizontal axis at this time is stored in the memory unit (220) as the horizontal axis real image profile at the time of the maximum matching with the preset thermal image profile (i.e., the horizontal axis real image profile for reconstruction). If the vertical axis matching rate is equal to or less than the maximum matching value of the vertical axis in the step of comparing with the maximum matching value of the vertical axis, the maximum matching value on the vertical axis is detected. The real image profile on the vertical axis at this time is stored in the memory unit (220) as the vertical axis real image profile at the time of the maximum matching with the preset thermal image profile (i.e., the vertical axis real image profile for reconstruction). The ratio and the increment value of the horizontal axis real image profile at the time of the maximum matching and the vertical axis real image profile at the time of the maximum matching are stored in the memory unit (220) (S150).

[0071] For example, when the matching rate is the same as the maximum matching value and thus the preset thermal image profile matches the length of the Nth real image profile, the Nth profile is stored in the memory unit (220). The ratio and the increment value of the horizontal axis real image profile and the vertical axis real image profile at this time are stored in the memory unit (220).

[0072] In the real image adjustment step, the ratio and the position of the real image are adjusted using the ratio and the increment value of the real image profiles of the horizontal axis and the vertical axis obtained in the step of storing the matching data on the horizontal and vertical sides. The adjusted real image is stored in the memory unit (220) and is output to the display unit (210). Here, the position (increment) can mean the distance between the preset thermal image profile and the real image profile on the horizontal axis.

[0073] After the real image adjustment step, a step of saving the set value can be further included.

[0074] In the step of saving the set value, the ratio and the increment value of the real image profiles (for reconstruction) on the horizontal axis and the vertical axis are stored as the set value. The set value can be used later for adjusting the real image.

[0075] Figure 4 is an explanatory diagram illustrating the matching rate detection step and the step of comparing with the maximum matching value in the flowchart of Figure 3 Figure 5 is an explanatory diagram for illustrating the real image adjustment step in the flowchart of Figure 3

[0076] Figure 4 (a) illustrates an example of a real image and a thermal image before the screen inversion step.​​

[0077] Figure 4 (b) shows the real image and the thermal image after the screen inversion step. Figure 4 (a) and the result of the thermal image.

[0078] Figure 4 (c) shows the comparison of the real image profile and the thermal image profile Figure 4 (c) and the selection of the real image profile Figure 4 (c) that is identical to the thermal image profile

[0079] Figure 5 (a) is a diagram for showing the real image adjustment step. The real image is adjusted by using the ratio and the increment value of the real image profile of the vertical axis (real vertical image profile) saved in the step of storing the matching data on the horizontal side and the vertical side of the flowchart of Figure 3

[0080] Figure 5 (b) shows the real image and the thermal image after the real image adjustment step. Figure 4 (b) and the thermal image.

[0081] <Multi-zone body temperature detection>

[0082] The multi-zone body temperature detection is detected by the multi-zone body temperature detection unit (205) of the data processing unit (200).

[0083] The thermal image screen is divided into a preset number of horizontal x vertical (for example, 4 x 3 or 8 x 6). The divided region in which the pixel value is greater than a preset value (for example, the overall average value) in each divided region is detected as a maximum point (peak, i.e., the maximum point of the divided region). When the peak exceeds a preset threshold temperature (for example, 40℃), the divided region having a peak exceeding the threshold temperature is excluded from the ROI. Then, at the divided region which is not excluded, the peaks of adjacent divided regions (i.e., the divided regions connected to each other) are compared with each other, and the divided region having a small peak (i.e., a local peak) among them is excluded. The temperature value of the maximum point of each divided region obtained in this way is displayed in the corresponding part of the real image and the thermal image.

[0084] Figure 6 is an example of detecting and displaying the body temperature by the multi-zone body temperature detection method of the present application.

[0085] The body temperature value detected for each person is displayed in the real image and the thermal image, respectively.

[0086] Figure 7 ​This is a flowchart illustrating an example of the multi-region body temperature detection method of the present invention. Figure 8 This is shown after the region segmentation step. Figure 7 Example image. Figure 9 This shows the steps after the outermost region exclusion step. Figure 8 The image. Figure 10 This is shown after the maximum value comparison step. Figure 9 The image. Figure 11 This shows the steps following the adjacent region comparison. Figure 10 The image.

[0087] In the region segmentation step, the data processing unit (200) segments the thermal image screen into a preset number of horizontal × vertical divisions (e.g., 4 × 3 or 8 × 6) (S210). An example image following the region segmentation step is shown in... Figure 8 middle.

[0088] In the outermost region exclusion step (S215), the outermost region is excluded from each segmented region divided in the region segmentation step. That is, the value of the outermost region is set to "0", and the outermost region is excluded from the ROI. Figure 9 Show as will Figure 8 The image is applied to the result of the outermost region exclusion step.

[0089] In the average value detection step, the data processing unit (200) obtains the average value of the thermal image in all regions (i.e., all segmented regions) of the segmented thermal image in the region segmentation step (S220) (the average temperature of the segmented region).

[0090] In the maximum value comparison step, the data processing unit (200) compares the maximum value of each segmented region with the average value detected in the average value detection step (S230). If the maximum value is less than the average value, the segmented region is excluded from the ROI (S235). In this way, regions with an average value less than the maximum value among all segmented regions are excluded from the ROI. Figure 9 The image shows a case where a region is excluded from the ROI during the maximum value comparison step because the maximum value of the region is less than the average value. Figure 10 For ease of explanation, the term "divided region" will be referred to as "region" in the following text.

[0091] In some cases, a threshold temperature comparison step (not shown) may also be included between the maximum value comparison step and the adjacent region comparison step.

[0092] In the threshold temperature comparison step (not shown), regions exceeding a preset threshold temperature (e.g., 40°C) in the regions (segmented regions) that were not excluded in the maximum value comparison step are excluded from the ROI.

[0093] In the adjacent region comparison step, the maximum values between adjacent regions in the regions remaining after the maximum value comparison step or the threshold temperature comparison step are compared (S250). Among the adjacent regions, only the region having the largest maximum value is left, and the adjacent regions other than the region having the largest maximum value are excluded from the ROI (S255). In this way, the maximum values of the remaining regions in the regions that have passed through the maximum value comparison step or the threshold temperature comparison step are compared, and only the region having the largest maximum value is left, and the remaining adjacent regions are excluded from the ROI (S260). In other words, the process of comparing each other between the adjacent regions of all the adjacent regions of the ROI is repeated, the region having the largest maximum value is selected, and the remaining adjacent regions other than the region having the largest maximum value are excluded.

[0094] In Figure 10 In the adjacent region comparison step, the maximum values between adjacent regions in the regions remaining after the maximum value comparison step or the threshold temperature comparison step are compared (S250). Among the adjacent regions, only the region having the largest maximum value is left, and the adjacent regions other than the region having the largest maximum value are excluded from the ROI (S255). In this way, the maximum values of the remaining regions in the regions that have passed through the maximum value comparison step or the threshold temperature comparison step are compared, and only the region having the largest maximum value is left, and the remaining adjacent regions are excluded from the ROI (S260). In other words, the process of comparing each other between the adjacent regions of all the adjacent regions of the ROI is repeated, the region having the largest maximum value is selected, and the remaining adjacent regions other than the region having the largest maximum value are excluded. Figure 11

[0095] In the highest temperature detection step, the highest temperature of the corresponding region of the coordinates (i.e., XY coordinates) of the final ROI detected in the adjacent region comparison step is detected (S265).

[0096] In the real image and the thermal image, the highest temperature of the corresponding region is displayed at the position of the coordinates of the final ROI detected in the highest temperature detection step (S270). That is, the temperature value is displayed on the image.

[0097] <Histogram equalization for diagnostic purposes (automatic adjustment of the winning level)>

[0098] The histogram equalization for diagnostic purposes (automatic adjustment of the winning level) is performed by the automatic adjustment of the winning level unit (207) of the data processing unit (200).

[0099] When colors are mapped to the infrared image, it is necessary to appropriately distribute the color expression over the entire histogram of the temperature region of interest in order to maximize the diagnostic efficiency.

[0100] Since the overall histogram of the image has a characteristic that can divide the background region and the BSA, the region to be observed can be selected by adjusting the winning level. In the histogram, the horizontal axis is the temperature range, and the vertical axis is the accumulated amount of the number of pixels.

[0101] ​The temperature distribution is determined in the interval between the minimum value and the maximum value at the adjusted level. Pseudo-coloring is performed by defining the point at which the highest temperature region becomes 3% of the total area as the maximum expression temperature and by defining the intersection point between the background and the BSA as the minimum point. In other words, pseudo-coloring is performed according to the window level by defining the point at which 3% of the total area has the highest temperature region as the maximum expression temperature and by defining the intersection point between the background and the BSA as the minimum expression temperature.

[0102] Figure 12 The before and after level adjustment is shown.

[0103] Figure 12 (a) shows an example image before level adjustment. Figure 12 (b) shows an image obtained by dividing Figure 12 the background region and the BSA in the histogram of (a). Figure 12 (c) shows a pseudo-colored image on the image of (b). Figure 12

[0104] Figure 13 is a flowchart showing diagnostic histogram equalization (level automatic adjustment) in the present application. Figure 14 Examples of the background region and the BSA are shown. Figure 15 The ROI of the BSA selected from the BSA of Figure 14 is shown.

[0105] In the histogram calculation step, a histogram is calculated from the thermal image (S510).

[0106] In the background peak detection step, a background peak is detected from the histogram calculated in the histogram calculation step (S520).

[0107] In the condition detection execution step of the background peak, it is checked whether the background peak detected in the background peak detection step is a local peak lower than a preset background peak reference value (S525). If it is a local peak lower than the background peak reference value, the background peak is removed (S530), and the process returns to the background peak detection step.

[0108] In the BSA peak detection step, a BSA peak is detected from the histogram calculated in the histogram calculation step (S520). Figure 14 (b) shows the BSA peak detected in the BSA peak detection step of Figure 13 .

[0109] ​In the BSA peak condition detection execution step, it is checked whether the BSA peak detected in the BSA peak detection step is a local peak higher than a preset BSA peak reference value (S555). If it is a local peak higher than the BSA peak reference value, the BSA peak is removed (S560), and the process returns to the BSA peak detection step. Figure 15 A BSA in which a detected body surface area of interest is shown after the BSA peak condition detection execution step is performed. Figure 14 A BSA in which a detected body surface area of interest is shown after the BSA peak condition detection execution step is performed.

[0110] In the lower limit value and upper limit value setting step, a point having a lower limit (S570) and an upper limit (S575) and values of the lower limit and the upper limit are set. That is, a highest temperature region that will become 3% of the total area is set as the maximum (upper limit value) expression temperature, and an intersection point between the background and the BSA is set as the minimum point (lower limit value).

[0111] In the pseudo-coloring step, color mapping data is generated using the BSA peak, the background peak, and the lower limit value and the upper limit value. Pseudo-coloring of the BSA is performed using a histogram (S585). The pseudo-coloring image performed in this way is output to the display unit (210).

[0112] In the present specification, details that those skilled in the art in the technical field of the present application can fully recognize and infer are omitted. In addition to the specific examples described in the present specification, various modifications can be made within the scope in which the technical idea or the basic configuration of the present application is not changed. Therefore, the present application can be implemented in a manner different from that specifically described and shown in the present specification, which can be understood by those skilled in the art in the technical field of the present application having ordinary knowledge and skills.

[0113]

INDUSTRIAL APPLICABILITY

[0114] The optimal automatic mapping method of real images and thermal images in a body heat tester and a body heat tester applying the method of the present application reduce errors and improve accuracy. The body heat tester of the present application can accurately measure the body temperature of many people at the same time in airports, schools, restaurants, government offices, and companies.

Claims

1. A body temperature testing device, comprising: A thermal imaging camera, used to capture thermal images of an area containing multiple objects; A real imaging camera, used to simultaneously capture real images of an area with multiple objects in conjunction with the thermal imaging camera; as well as The data processing unit performs the following operations: The thermal images received from the thermal imaging camera are matched with the real images received from the real imaging camera. A reconstructed real image matching the thermal image is obtained by stretching or shortening the top, bottom, left, and right sides of the real image based on the thermal image. The thermal image and the reconstructed real image are used to detect the body heat (temperature) of the object. The data processing unit further includes a real thermal image mapping unit, which is configured as follows: Based on the thermal image frame, six real image frames, each at a time before and after the thermal image frame, are stored in the memory unit. Select the real image frame with the highest matching rate to the thermal image from the saved real image frames; Detect the boundary of the object in the thermal image frame and the selected real image frame; and The thermal image frame is matched with the result of expanding (stretching) or shrinking (shortening) the selected real image frame by 1 pixel in each of the top, bottom, left, and right directions to select the point with the maximum matching rate.

2. The body temperature testing instrument according to claim 1, wherein, The real thermal image mapping unit is configured as follows: Invert the real image screen and the thermal image screen for edge detection; The thermal image contour on the horizontal axis in the inverted thermal image is compared with the real image contour on the horizontal axis in the inverted real image; The real image contour on the horizontal axis that is closest to the thermal image contour on the horizontal axis is obtained as the reconstructed real image contour on the horizontal axis. The thermal image contour on the vertical axis in the inverted thermal image is compared with the real image contour on the vertical axis in the inverted real image; The real image contour on the vertical axis that is closest to the thermal image contour on the vertical axis is obtained as the reconstructed real image contour on the vertical axis; Calculate the ratio and increment (position) of the reconstructed horizontal ground truth contour and the reconstructed vertical ground truth contour; The ratio and increment (position) of the horizontal and vertical true image are adjusted using the ratio and increment values ​​of the reconstructed horizontal true image contour and the reconstructed vertical true image contour. The adjusted real image is stored as the reconstructed real image in the memory unit; as well as The adjusted real image is output to the display unit.

3. The body temperature testing instrument according to claim 1, wherein, The data processing unit further includes a multi-region body temperature detection unit, which is configured as follows: The thermal image is divided into a preset number of horizontal lines × a preset number of vertical lines to obtain segmented regions of the thermal image, and the segmented regions are set as regions of interest (ROIs). Calculate the overall average of the thermal images across all segmented regions; Exclude segmentation regions in the thermal image whose maximum value is less than the overall average value from the region of interest; Exclude regions from the region of interest from which the maximum value of each segmented region of the thermal image exceeds a preset threshold temperature; The maximum values ​​between adjacent regions (adjacent segmented regions) from the region of interest are compared. Exclude the remaining adjacent regions from the region of interest, leaving only the segmented region with the largest maximum value among the adjacent regions; Calculate the coordinates (i.e., XY coordinates) of the remaining segmented regions in the region of interest and the highest temperature of the remaining segmented regions; as well as The calculated highest temperature is displayed on the thermal and ground truth images at the coordinates of the remaining segmented regions.

4. The body temperature testing instrument according to claim 3, wherein, The data processing unit further includes a win-level automatic adjustment unit, which is configured to: A histogram is calculated from the thermal image, where the horizontal axis of the histogram represents the temperature range and the vertical axis represents the cumulative number of pixels in the thermal image. Detect background peaks in the histogram whose peak values ​​are higher than a reference value for background peaks; The surface area peak in the histogram is detected if its peak value is lower than a preset reference value. The highest temperature region on the horizontal axis is set as the upper limit value of the point representing the maximum temperature, which is 3% of the total thermal image area. The point where the background and the body surface area intersect on the horizontal axis is set as the lower limit value for the minimum point representing temperature. The body surface area in the thermal image is pseudo-colorized by creating color mapping data using the lower and upper limits; as well as The obtained pseudo-colorized image is output to the display unit.

5. A method for operating a body heat testing instrument, the body heat testing instrument comprising a thermal imaging camera for simultaneously detecting the body heat of multiple objects and a real imaging camera, the method comprising: The reconstructed real image detection step includes a data processing unit receiving a thermal image from the thermal imaging camera and a real image from the real imaging camera, and performing matching by stretching or shortening the top, bottom, left, and right sides of the real image based on the thermal image to obtain a reconstructed real image that matches the thermal image. The body heat detection step involves the data processing unit using thermal images and reconstructed real images to detect the object's body heat (body temperature). In the reconstructed real image detection step, the data processing unit includes: Import six real image frames from the memory cell, based on the thermal image frame and the time before / after the thermal image frame; Select the real image frame that has the highest matching rate with the thermal image frame from the real image frames; Detecting the boundaries of objects in each of the thermal image frames and the selected real image frames; and The thermal image frame is matched with the result of expanding (stretching) or shrinking (shrunk) the selected real image frame by 1 pixel in each of the top, bottom, left, and right directions to select the point with the maximum matching rate.

6. The method of the thermal testing instrument for an operating body according to claim 5, wherein the step of detecting the reconstructed real image includes: The screen inversion step, wherein the data processing unit inverts the real image screen and the thermal image screen for edge detection by the data processing unit; The horizontal axis matching rate detection step, wherein the data processing unit obtains the horizontal axis matching rate by comparing the horizontal axis thermal image contour in the reversed thermal image and the real image in the screen reversal step with the horizontal axis real image contour; The step of comparing with the maximum matching value of the horizontal axis, wherein the data processing unit compares the horizontal axis matching rate with the pre-stored maximum matching rate value of the horizontal axis, and if the horizontal axis matching rate is not equal to the maximum matching rate value of the horizontal axis, then imports the next horizontal axis real image contour from the screen inversion step of the inverted real image from the memory unit and returns to the horizontal axis matching rate detection step; The vertical axis matching rate detection step, wherein the data processing unit obtains the vertical axis matching rate by comparing the vertical axis thermal image contour in the thermal image and the real image in the screen reversal step with the vertical axis real image contour; The step of comparing with the maximum matching value of the vertical axis, wherein the data processing unit compares the vertical axis matching rate with the pre-stored maximum matching rate value of the vertical axis; if the vertical axis matching rate is not equal to the maximum matching rate value of the vertical axis, then imports the next vertical axis real image contour from the screen inversion step of the inverted real image from the memory unit and returns to the vertical axis matching rate detection step; The step of storing the matching data of the horizontal axis and the vertical axis, wherein the data processing unit performs the following steps: If the horizontal axis matching rate is equal to the maximum matching value of the horizontal axis in the step of comparing with the maximum matching value of the horizontal axis, then the horizontal axis true image contour is used as the horizontal axis true image contour for reconstruction. If the vertical axis matching rate is equal to the maximum matching rate value of the vertical axis in the step of comparing with the maximum matching value of the vertical axis, then the true vertical axis image contour is used as the true vertical axis image contour for reconstruction; and The ratio and increment (position) of the horizontal axis true image contour used for reconstruction to the vertical axis true image contour used for reconstruction are obtained, and the ratio and the increment (position) are stored in the memory unit. The real image adjustment step involves the data processing unit adjusting the position and ratio of the vertical and horizontal real image contours using the ratio and increment values ​​of the reconstructed vertical real image contours, storing the adjusted real image as a reconstructed real image in the memory unit, and outputting it to the display unit.

7. The method of the operating body thermal testing instrument according to claim 5, wherein, The body heat detection step includes: In the region segmentation step, the data processing unit divides the region of the thermal image on the screen into a preset number of horizontal lines × a preset number of vertical lines to obtain the segmented region of the thermal image, and sets the segmented region as the region of interest. The outermost region exclusion step, wherein the data processing unit excludes the outermost region among the segmented regions divided in the region segmentation step; The average value detection step, wherein the data processing unit calculates the average value of the thermal images in all segmented regions of the thermal image divided in the region segmentation step; The maximum value comparison step involves the data processing unit comparing the maximum value of each segmented region divided in the region segmentation step with the average value detected in the average value detection step, and excluding segmented regions from the region of interest whose maximum value is less than the average value. The adjacent region comparison step involves the data processing unit comparing the maximum values ​​between interconnected adjacent regions (adjacent segmented regions) in the region of interest, retaining only the segmented region with the largest maximum value, and excluding the remaining adjacent regions from the region of interest. The highest temperature detection step involves the data processing unit taking the remaining segmented region after the adjacent region comparison step as the final region of interest, and calculating the coordinates (i.e., XY coordinates) and the highest temperature of the segmented region of the final region of interest. The step of displaying the corresponding value on the image includes, wherein the data processing unit displays the highest temperature of the segmented region of the final region of interest on the thermal image and the ground image of the coordinates of the segmented region of the final region of interest.

8. The method for operating a body heat testing instrument according to claim 5, further comprising a pseudo-colorized image output step after the body heat detection step, wherein, The pseudo-colorization image output step includes: The histogram calculation step involves the data processing unit calculating a histogram from the thermal image. The background peak detection step involves detecting background peaks in the histogram, wherein the data processing unit detects background peak values ​​that are higher than a preset background peak reference value. The body surface area peak detection step detects the body surface area peak in the histogram, wherein the data processing unit detects the peak value that is lower than the preset body surface area peak reference value. The lower limit and upper limit setting steps include: the data processing unit calculates the point where the highest temperature region becomes 3% of the total area of ​​the thermal image as the maximum expressed temperature (upper limit value), and calculates the point where the background and the body surface area intersect as the minimum point (lower limit value). In the pseudo-colorization step, the data processing unit uses the lower limit and upper limit values ​​obtained in the lower limit and upper limit setting step to create color mapping data, performs pseudo-colorization on the body surface area in the thermal image, and outputs the pseudo-colorized image to the display unit.

9. The method of the operating body thermal testing instrument according to claim 6, in, The horizontal axis matching rate in the horizontal axis matching rate detection step is the absolute value obtained by subtracting the length of the thermal image contour and the length of the real image contour on the horizontal axis. In the vertical axis matching rate detection step, the vertical axis matching rate is the absolute value obtained by subtracting the length of the thermal image contour and the length of the real image contour on the vertical axis.

10. The method of the operating body thermal testing instrument according to claim 9, wherein, The maximum matching value between the horizontal axis and the vertical axis is 0.

Images