Image processor, method for processing image, and program

Abstract

To provide an image processor, a method for processing an image, and a program which can properly process thermal image data.SOLUTION: An image processor 10 includes: a thermal image data acquisition unit 14 for acquiring thermal image data taken by a sensor; a weather information acquisition unit 11 for acquiring weather information on the weather; a situation change detection unit 12 for detecting change of the situation of the weather; and a control unit 16 for adjusting the sensor output value of the thermal image data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus, an image processing method, and a program.

Background Art

[0002] Patent Document 1 discloses an object detection device that detects an object based on sensing results of a camera and a radar mounted on a vehicle. In this object detection device, the rainfall situation is determined based on the movable state of either the wiper or the washer of the vehicle. The object detection device adjusts the reliability of the camera and the radar based on the rainfall situation. The object detection device derives the rainfall amount based on the operation frequency of the wiper. Then, the reliability is adjusted so that the higher the rainfall amount, the lower the reliability of the camera and the higher the reliability of the radar.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, an infrared camera may be used as a surveillance camera or an in-vehicle camera. An infrared camera has a bolometer or the like and can capture a thermal image corresponding to a temperature distribution. For example, in a far-infrared camera, the entire screen may become low illuminance due to the influence of rain. Also, in rainy weather, the temperatures of the ground, the road surface, people, vehicles, etc. become uniform. Therefore, in rainy weather, the contrast of the thermal image data becomes low.

[0005] In view of the above problems, an object of the present disclosure is to provide an image processing apparatus, an image processing method, and a program capable of appropriately processing thermal image data.

Means for Solving the Problems

[0006] An image processing apparatus according to an aspect of the present embodiment includes a thermal image data acquisition unit that acquires thermal image data captured by a sensor, a weather information acquisition unit that acquires weather information regarding the weather, a situation change detection unit that detects a change in the situation of the weather, and a control unit that adjusts the sensor output value of the thermal image data when a change in the situation of the weather is detected.

[0007] An image processing method according to an aspect of the present embodiment includes a step of acquiring thermal image data captured by a sensor, a step of acquiring weather information regarding the weather, a step of detecting a change in the situation of the weather, and a step of adjusting the sensor output value of the thermal image data when a change in the situation of the weather is detected.

[0008] A program according to an aspect of the present embodiment is a program that causes a computer to execute an image processing method, and the image processing method includes a step of acquiring thermal image data captured by a sensor, a step of acquiring weather information regarding the weather, a step of detecting a change in the situation of the weather, and a step of adjusting the sensor output value of the thermal image data when a change in the situation of the weather is detected.

Advantages of the Invention

[0009] According to the present embodiment, it is possible to provide an image processing apparatus, an image processing method, and a program that can appropriately process thermal image data.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying out the Invention

[0011] Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiments. Also, for clarity of explanation, the following description and drawings are simplified as appropriate.

[0012] Embodiment 1 The imaging system according to the present embodiment can be mounted on a moving body such as a vehicle. For example, the imaging system is used as an in-vehicle device such as a drive recorder or a driving assistance sensor. In this case, the imaging system 1 is mounted on the vehicle. Then, the thermal image data captured by the imaging system is used for pedestrian detection and the like. Also, the imaging system may be used as a surveillance camera or a security camera.

[0013] FIG. 1 is a block diagram showing a schematic system configuration of the imaging system 1. The imaging system 1 includes an image processing device 10, an infrared sensor 20, a raindrop sensor 30, a rainfall sensor 40, and a display 50. The image processing device 10 includes a weather information acquisition unit 11, a situation change detection unit 12, a thermal image data acquisition unit 14, a control unit 16, and a display image data output unit 18.

[0014] The far-infrared sensor 20 is a camera that detects far-infrared light (also referred to as far-infrared rays) and captures a thermal image of a subject. The far-infrared sensor 20 captures a moving image. Alternatively, the far-infrared sensor 20 captures a series of still images. The data of the image captured by the far-infrared sensor 20 is used as thermal image data. The far-infrared sensor 20 is communicatively connected to the image processing device 10 wirelessly or by wire. The far-infrared sensor 20 captures a subject at a predetermined angle of view. Note that the sensor that captures thermal image data is not limited to the far-infrared sensor 20 that detects far-infrared rays. For example, an infrared camera that detects infrared rays other than far-infrared rays may capture thermal image data.

[0015] The far-infrared sensor 20 includes a lens unit 21 and an image sensor 22. The image sensor 22 is a light detector such as a microbolometer. The image sensor 22 may be a thermal (uncooled) element or a quantum (cooled) element. The image sensor 22 includes a plurality of pixels arranged in the horizontal and vertical directions. For example, the image sensor 22 has 640×480 pixels.

[0016] The lens unit 21 is disposed on the incident side of the image sensor 22. The lens unit 21 forms an image of the subject on the image sensor 22. The image sensor 22 detects the far-infrared light refracted by the lens unit 21. The lens unit 21 may include a plurality of lenses such as a zoom lens and a focus lens. The output bit width of the far-infrared sensor 20 is, for example, 10 bits to 14 bits. The value of each pixel of the thermal image data captured by the far-infrared sensor 20 is used as the sensor output value.

[0017] The rain drop sensor 30 detects rain drops adhering to a window or the like. The rain drop sensor 30 is, for example, an optical sensor and includes a light emitting element, a light receiving element, and the like. Then, the light reception amount of the light receiving element of the rain drop sensor 30 becomes a measured value. For example, when the imaging system 1 is an in-vehicle system, the rain drop sensor 30 is attached to the front window. The rain drop sensor 30 detects the presence or absence of rain drops based on the change in the light reception amount that occurs when the rain drops adhering to the front window pass through. The measured value of the rain drop sensor 30 changes according to the rain drops adhering to the window. Then, the rain drop sensor 30 determines whether it is raining by comparing the measured value with a threshold value.

[0018] When the measured value of the rain drop sensor 30 is equal to or greater than the threshold value, it is determined that it is raining. When it is determined that it is raining, the rain drop sensor 30 may output a signal for operating the wiper of the vehicle. When the measured value of the rain drop sensor 30 is less than the threshold value, it is determined that it is not raining. Therefore, when it is not raining, the rain drop sensor 30 may output a signal for stopping the wiper of the vehicle. The rain drop sensor 30 outputs the detection result of the rain drops to the image processing device 10 as weather information. The rain drop sensor 30 outputs weather information indicating whether it is raining to the image processing device 10.

[0019] The rain gauge sensor 40 detects the amount of rainfall. For example, the rain gauge sensor 40 is a rain gauge equipped with a container for storing water and detects the precipitation amount. The rain gauge sensor 40 detects the precipitation amount by measuring the amount of rain water stored in the container. The rain gauge sensor 40 detects the precipitation amount per unit time. Alternatively, the rain gauge sensor 40 may be a radar type rain gauge. The rain gauge sensor 40 outputs the detected precipitation amount to the image processing device 10 as weather information.

[0020] The image processing device 10 is, for example, an information processing device having a memory, a processor, and the like. The memory stores a processing program, various parameters, measurement data, and the like. The processor executes the processing program stored in the memory. By the processor executing the processing program, the processing described later is executed.

[0021] The weather information acquisition unit 11 acquires weather information from the raindrop sensor 30 and the rainfall sensor 40. Further, the weather information acquisition unit 11 may acquire weather information only from the rainfall sensor 40, or may acquire weather information only from the raindrop sensor 30. Note that the weather information acquired by the weather information acquisition unit 11 is not limited to precipitation, rainfall, etc. The weather information acquisition unit 11 may acquire, for example, information indicating the weather such as sunny, rainy, or cloudy as weather information.

[0022] Furthermore, the weather information acquisition unit 11 may acquire information regarding the sunlight condition as weather information. For example, the sunlight condition changes according to time such as daytime, nighttime, or evening and the current position. The weather information acquisition unit 11 acquires the information regarding this sunlight condition as the weather condition. Also, the weather information may be acquired from sources other than the raindrop sensor 30 and the rainfall sensor 40. For example, the weather information acquisition unit 11 may acquire information regarding the weather shown on the Internet as weather information. Specifically, the weather information acquisition unit 11 acquires information indicating rainy, cloudy, sunny, etc. as weather information. When the imaging system 1 is an in-vehicle device, the weather information acquisition unit 11 can acquire weather information based on the current position of the vehicle obtained from a satellite positioning system such as GPS.

[0023] The situation change detection unit 12 detects a change in the weather situation based on the weather information. For example, the situation change detection unit 12 detects the start of rain, the end of rain, and a change in precipitation as changes in the weather situation. For example, when the detection result of the raindrop sensor 30 changes, the situation change detection unit 12 detects the start or stop of precipitation as a situation change. Alternatively, when the measured value at the rainfall sensor 40 changes significantly, the situation change detection unit 12 detects a change in the weather situation.

[0024] The thermal image data acquisition unit 14 acquires thermal image data captured by the far-infrared sensor 20. In the thermal image data, the sensor output value of each pixel is represented by 10 bits to 14 bits.

[0025] The control unit 16 adjusts the gain for the thermal image data. Specifically, when the weather conditions change, the control unit 16 adjusts the gain based on the average value of the thermal image data. Specifically, the control unit 16 adjusts the gain so that the average value of the thermal image data becomes the reference value. The reference value may be a fixed value or a variable value that changes according to the environment and situation. Note that an initial setting value of the gain is set in the control unit 16. The initial setting value of the gain is set based on the thermal image data such as in sunny weather.

[0026] The control unit 16 performs gain processing on the thermal image data using the gain. Specifically, the control unit 16 multiplies the sensor output value by the gain. The thermal image data obtained by the gain processing is used as the display image data. The control unit 16 can generate the display image data by digital processing. Note that the gain processing may be executed in the far-infrared sensor 20. In this case, the value on which the gain processing has been performed becomes the sensor output value.

[0027] The display image data output unit 18 outputs the display image data to the display 50. The image processing apparatus 10 is communicably connected to the display 50. The image processing apparatus 10 may be connected to the display 50 by wire or wirelessly. As described above, the image processing apparatus 10 executes image processing on the thermal image data. Then, the image processing apparatus 10 may display the thermal image data on the display 50.

[0028] The display 50 is a display device including, for example, a liquid crystal panel or an organic electroluminescence panel. The display 50 is provided at a position visible to a user who is, for example, a driver of a vehicle. The display 50 displays the thermal image data image-processed by the image processing apparatus 10.

[0029] The image processing method of the present embodiment will be described with reference to FIG. 2. FIG. 2 is a flowchart showing the image processing method.

[0030] First, the raindrop sensor 30 measures the presence or absence of raindrops (S101). As described above, the amount of light received by the light-receiving element of the raindrop sensor 30 becomes the measured value. The weather information acquisition unit 11 acquires the measurement result of the raindrop sensor 30 as weather information. The situation change detection unit 12 determines whether the measured value is equal to or greater than the threshold value (S102). The situation change detection unit 12 detects a change in the precipitation situation based on the measurement result. The situation change detection unit 12 can detect the start and end of rain based on the comparison result between the measured value and the threshold value.

[0031] When the measured value of the raindrop sensor 30 is less than the threshold value (NO in S102), the control unit 16 sets the gain to the initial set value (S103). A gain value based on sunny weather is preset in the control unit 16 as the initial set value. When the measured value is less than the threshold value, since it is not raining, the control unit 16 sets the gain to the initial set value. Then, the control unit 16 multiplies the gain of the initial set value by the sensor output value to generate display image data.

[0032] When the output value of the raindrop sensor 30 is equal to or greater than the threshold value (YES in S102), the rainfall sensor 40 measures the precipitation amount (S104). Here, the rainfall sensor 40 measures, for example, the precipitation amount in 6 minutes. The weather information acquisition unit 11 acquires the measurement result of the rainfall sensor 40 as weather information.

[0033] Next, the situation change detection unit 12 determines whether the weather situation has changed (S105). Here, the situation change detection unit 12 detects a change in the precipitation situation based on the precipitation amount. For example, the precipitation amount is classified into a plurality of ranges. For example, the precipitation amount in 6 minutes less than 1 mm is the first range, 1 mm or more and less than 2 mm is the second range, 2 mm or more and less than 3 mm is the third range. 3 mm or more and less than 5 mm is the fourth range, 5 mm or more and less than 8 mm is the fifth range, and 8 mm or more is the sixth range. The situation change detection unit 12 obtains the range to which the measured precipitation amount belongs. The situation change detection unit 12 stores which of the first range to the sixth range the precipitation amount per 6 minutes belongs to.

[0034] The situation change detection unit 12 compares the current range of the precipitation amount with the previous range to detect a change in the weather situation. When the current range and the previous range are the same, the situation change detection unit 12 determines that the weather situation has not changed (NO in S105), and returns to step S101. As described above, the situation change detection unit 12 detects a change in the precipitation situation regarding the precipitation amount.

[0035] When the current range and the previous range are different, the situation change detection unit 12 determines whether there has been a change in the weather situation (YES in S105). For example, if the previous range was the first range and the current range is the second range, since the precipitation amount has increased, the situation change detection unit 12 determines that there has been a change in the weather situation. Or, if the previous range was the third range and the current range is the second range, since the precipitation amount has decreased, the situation change detection unit 12 determines that there has been a change in the weather situation. Thus, when the range to which the measured precipitation amount belongs changes, the situation change detection unit 12 determines that there has been a change in the weather situation.

[0036] When a change in the weather situation is detected (YES in S105), the thermal image data acquisition unit 14 acquires thermal image data (S106). Then, the control unit 16 acquires the average value of the thermal image data (S107). Here, the control unit 16 calculates, for example, the average value of the sensor output values of each pixel in one frame of thermal image data. That is, the control unit 16 calculates the average value of the sensor output values by dividing the sum of all pixel values of the acquired one-frame thermal image data by the number of pixels in one frame. Also, the thermal image data here may be gain-processed using the previous gain.

[0037] Next, the control unit 16 determines whether the calculated average value is less than or equal to the reference value (S108). When the average value is less than or equal to the reference value (YES in S108), the control unit 16 increases the gain (S109). That is, since the illuminance is low when the average value is less than or equal to the reference value, the control unit 16 sets the gain high. Then, the process returns to step S105.

[0038] When the average value is greater than the reference value (NO in S108), the control unit 16 decreases the gain (S110). That is, when the average value is greater than the reference value, the illuminance is high, so the control unit 16 sets the gain low. Thereby, the control unit 16 can bring the average value of the thermal image data closer to the reference value. Then, the process returns to step S105.

[0039] In this way, the control unit 16 adjusts the gain so that the average value approaches the reference value. The display image data output unit 18 outputs the display image data obtained by the gain process to the display 50. Thereby, the display 50 can display the display image data on which the gain process has been performed with an appropriate gain.

[0040] For example, when the average value of the thermal image data is low, the entire screen becomes low illuminance. Due to the temperature becoming uniform, such as the temperature of the ground, road surface, people, cars, etc., the display image with low contrast is obtained. Therefore, when the average value of the thermal image data is low, the control unit 16 can obtain high-contrast thermal image data by setting the gain high. Thus, the image processing apparatus 10 can accurately detect targets such as pedestrians and vehicles based on the output image data.

[0041] FIG. 3 is a diagram showing the display image data obtained with the gain before adjustment, and FIG. 4 is a diagram showing the display image data obtained with the gain after adjustment. By adjusting the gain, the contrast of the thermal image data can be improved. Therefore, the user can clearly visually recognize targets such as pedestrians. The recognition performance of the target by image processing can be improved.

[0042] Also, when the situation change detection unit 12 detects a change in the weather situation, the control unit 16 adjusts the gain. Therefore, when the weather situation does not change, the control unit 16 maintains the previous gain. By doing so, the control unit 16 can stably use an appropriate gain. That is, even when the imaging direction or the subject changes, the gain does not change if there is no change in the weather situation.

[0043] Furthermore, when the average value of the thermal image data is higher than the reference value, the control unit 16 decreases the gain. Thereby, noise can be suppressed. After the control unit 16 adjusts the gain, the process returns to step S105. In step S105, the situation change detection unit 12 detects a change in the weather situation based on the precipitation amount. If no situation change is detected (NO in S105), the process returns to step S101. Therefore, when the situation change detection unit 12 detects that the rain has stopped, the weather information acquisition unit 11 sets the gain to the initial setting value. Thereby, an appropriate gain can be used even in a state where it is not raining.

[0044] When acquiring the average value of the thermal image data, the control unit 16 may calculate the average value of the sensor output values for several frames. In this case, for example, the control unit 16 calculates the average value of the sensor output values for each frame. Then, the control unit 16 calculates the average value of the sensor output values by dividing the sum of the average values for a plurality of frames by the number of frames. Thereby, even when the imaging direction or the subject changes over time, the control unit 16 can obtain an appropriate average value. Therefore, it becomes possible to set an appropriate gain.

[0045] Also, the situation change detection unit 12 detects a change in the precipitation situation as a change in the weather situation. For example, in the situation change detection unit 12, the precipitation amount levels are set in several ranges. When the current precipitation amount range deviates from the previous range, the situation change detection unit 12 determines that the precipitation situation has changed. By doing so, when the rainfall situation changes, the control unit 16 performs gain adjustment. Therefore, the control unit 16 can perform gain processing using an appropriate gain. The display 50 can display display image data with high contrast, improving visibility for the user. Furthermore, image recognition processing such as pedestrian detection can be performed with high accuracy.

[0046] In addition, the weather information includes precipitation information regarding the precipitation situation. When the precipitation situation changes, the control unit 16 adjusts the gain. Specifically, in FIG. 2, as the weather information, the measured value of the precipitation amount by the rain gauge sensor 40 is used. Of course, the weather information is not limited to the measured value of the precipitation amount. The measurement result of the raindrop sensor 30 may be used as the weather information. For example, the situation change detection unit 12 may detect a situation change based on the measurement result of the raindrop sensor 30. When the presence or absence of rainfall changes, the situation change detection unit 12 detects the change in the rainfall situation. Of course, the situation change detection unit 12 may use the information indicating the precipitation amount from the rain gauge sensor 40 or the information indicating the presence or absence of precipitation of the raindrop sensor 30 as the weather information, or may use the information from two or more sensors, etc. as the weather information.

[0047] Also, when there is no rainfall, the control unit 16 performs gain processing using the initial setting value of the gain. When the precipitation amount changes, the control unit 16 adjusts the gain so that the average value approaches the reference value. The reference value may be set according to the average value of the thermal image data when there is no rain. By doing so, the gain can be adjusted so as to approach the average value of the display image data when there is no rainfall. The control unit 16 compares the latest average value with the previous average value and adjusts the gain up or down according to the comparison result. The adjustment amount of the gain here may be set according to the difference value between the current average value and the reference value.

[0048] For example, the initial setting value and the reference value are set based on the average value on a sunny day. When the illuminance decreases due to rainfall, the gain increases. Therefore, the control unit 16 can adjust the gain so that the average value of the thermal image data approaches the average value on a sunny day. Thereby, since display image data with high contrast can be generated, the visibility of the user can be improved.

[0049] The situation change detection unit 12 may detect a situation change using information indicating the weather such as sunny, rainy, or cloudy days as weather information. The weather information acquisition unit 11 may acquire the weather information from the Internet or the like. That is, the weather information is not limited to that obtained by the measurement of sensors, and may be obtained from information acquired from the Internet or the like. Further, when the imaging system 1 is mounted on a vehicle, the weather information acquisition unit 11 may acquire the weather information based on the operation status of the wiper or the lighting status of the vehicle lights. That is, the situation change detection unit 12 may detect a change in the operation status of the wiper or a change in the lighting status of the vehicle lights as a situation change.

[0050] Furthermore, in the thermal image data, when the average value of the sensor output values in some regions is high, the control unit 16 may perform gain adjustment. For example, as shown in FIG. 3, assume that the average value of the region R including a person is higher than the average value of the surrounding regions. In this case, since it is highly likely that the region R includes a target such as a pedestrian, by increasing the gain by the control unit 16, the detection accuracy and recognition accuracy of the target can be increased.

[0051] Embodiment 2 In the present embodiment, the control unit 16 adjusts the dynamic range of the far-infrared sensor 20 instead of the gain, which is different from Embodiment 1. The image processing apparatus and the image processing method according to Embodiment 2 will be described with reference to FIG. 5. FIG. 5 is a flowchart showing the image processing method by the image processing apparatus.

[0052] Note that the configurations of the image processing apparatus 10 and the imaging system 1 are the same as those in Embodiment 1, and thus the description thereof will be omitted. Also, points other than the adjustment of the dynamic range are the same as those in Embodiment 1, and thus the description will be omitted as appropriate. Further, the processes of steps S201, S202, S204, S205, and S206 are the same as the processes of steps S101, S102, S104, S105, and S206 in FIG. 2, and thus the detailed description thereof will be omitted.

[0053] First, the raindrop sensor 30 measures the presence or absence of raindrops (S201). The situation change detection unit 12 determines whether the measured value is equal to or greater than the threshold value (S202). When the measured value of the raindrop sensor 30 is less than the threshold value (NO in S202), the control unit 16 sets it to the initial setting value (S203). That is, since the weather situation has changed, the image processing apparatus 10 adjusts the display image data. In the present embodiment, since the control unit 16 adjusts the dynamic range, the initial setting value is the setting value of the dynamic range of the far-infrared sensor 20. The control unit 16 stores the initial setting value of the dynamic range based on sunny weather. The initial setting value includes, for example, the values of the offset voltage and gain voltage of the imaging element 22.

[0054] When the output value of the raindrop sensor 30 is equal to or greater than the threshold value (YES in S202), the rain gauge sensor 40 measures the precipitation amount (S204). Next, the situation change detection unit 12 determines whether the weather situation has changed (S205). When the current range and the previous range are the same, the situation change detection unit 12 determines that the weather situation has not changed (NO in S205), and returns to step S201. When the current range and the previous range are different, the situation change detection unit 12 determines whether there has been a change in the weather situation (YES in S205). When a change in the weather situation is detected (YES in S205), the thermal image data acquisition unit 14 acquires thermal image data (S206).

[0055] In the thermal image data acquired here, the dynamic range is the initial setting value. Also, the thermal image data here is the data before the gain process, but it may be the data after the gain process as in Embodiment 1. The control unit 16 creates a histogram from the thermal image data (S207). The control unit 16 adjusts the dynamic range based on the histogram (S208).

[0056] For example, in a histogram, the horizontal axis represents the sensor output value of the thermal image data, and the vertical axis represents the number of pixels having that sensor output value. Specifically, the horizontal axis is divided into classes by a predetermined width of the sensor output value, and the vertical axis is the number of pixels included in that class. The control unit 16 can determine whether the thermal image data has low contrast by using the histogram. The control unit 16 widens the dynamic range so that the contrast of the thermal image data increases.

[0057] For example, in rainy weather or drizzly weather, the image captured by the far-infrared sensor 20 has low illuminance. Since the temperature of the ground and the road surface and the temperature of targets such as people and vehicles become uniform, the contrast decreases. There is a possibility that the sharpness of the target decreases and the recognition performance deteriorates. Therefore, when the weather condition changes, the dynamic range is adjusted to an appropriate range. For example, the control unit 16 changes the set value from the initial set value so as to widen the dynamic range. By doing so, since the sharpness of the target can be increased, the recognition performance can be improved. For example, the control unit 16 may compare the sensor output values of the thermal image data before and after the change in the weather condition. When the sensor output value before the change is smaller than the sensor output value after the change, the control unit 16 adjusts the dynamic range with respect to the sensor output value. For example, the control unit 16 adjusts the offset voltage and the gain voltage.

[0058] By doing so, when the weather condition changes, the control unit 16 can adjust the dynamic range. For example, when the sensor output value after the change in the weather condition is small, the control unit 16 performs adjustment so as to widen the dynamic range. When the weather condition changes, appropriate thermal image data can be obtained. As a result, since a decrease in the sharpness of the target can be prevented, the detection accuracy and recognition accuracy of the target can be increased.

[0059] In addition, the change in weather conditions may be other than during rainy days. For example, the contrast may decrease even during light rain or when fog occurs. Alternatively, the contrast may decrease even during high temperatures. Specifically, during high temperatures, since the temperature around is higher than that of a person, the person may become blacked out. In such a case, by adjusting the dynamic range, thermal image data suitable for target recognition can be obtained. Then, the display image data output unit 18 outputs, as display image data, data obtained by multiplying the thermal image data acquired with the adjusted dynamic range by a gain to the display 50. The display 50 can display appropriate thermal image data.

[0060] Embodiment 3 In the present embodiment, the control unit 16 performs both the gain adjustment of Embodiment 1 and the dynamic range adjustment of Embodiment 2. The image processing apparatus and the image processing method according to the present embodiment will be described with reference to FIG. 6. FIG. 6 is a flowchart showing an image processing method by the image processing apparatus.

[0061] Note that the configurations of the image processing apparatus 10 and the imaging system 1 are the same as those in Embodiment 1, and thus the description thereof will be omitted. Note that the basic processes of the gain adjustment and the dynamic range adjustment are the same as those in Embodiments 1 and 2, and thus the overlapping content will be described as appropriate.

[0062] First, the rain drop sensor 30 measures the presence or absence of rain drops (S301). The situation change detection unit 12 determines whether the measured value is equal to or greater than the threshold value (S302). When the measured value of the rain drop sensor 30 is less than the threshold value (NO in S302), the control unit 16 sets it to the initial set value (S303). In the present embodiment, the initial set value includes the set value of the dynamic range and the set value of the gain of the far-infrared sensor 20. The control unit 16 stores the initial set values of the dynamic range and the gain based on sunny days.

[0063] When the output value of the raindrop sensor 30 is equal to or greater than the threshold value (YES in S302), the rainfall sensor 40 measures the precipitation amount (S304). Next, the situation change detection unit 12 determines whether the weather situation has changed (S305). If the current range and the previous range are the same, the situation change detection unit 12 determines that the weather situation has not changed (NO in S305), and returns to step S301. If the current range and the previous range are different, the situation change detection unit 12 determines that the weather situation has changed (YES in S305). When the weather situation has changed (YES in S305), the thermal image data acquisition unit 14 acquires thermal image data (S306).

[0064] In the thermal image data acquired here, the dynamic range is the initial set value. Also, the thermal image data is the data before the gain process is performed, but it may be the data after the gain process as in the first embodiment. The control unit 16 creates a histogram from the thermal image data (S307). The control unit 16 adjusts the dynamic range based on the histogram (S308).

[0065] The thermal image data acquisition unit 14 acquires the thermal image data with the dynamic range adjusted (hereinafter also referred to as DR-adjusted) (S309). That is, the thermal image data acquisition unit 14 acquires the thermal image data with the dynamic range adjusted in step S308. The control unit 16 calculates the average value of the DR-adjusted thermal image data (S310).

[0066] The control unit 16 determines whether the average value is less than the reference value (S311). When the average value is not less than the reference value (NO in S311), the digital gain is decreased (S312). When the average value is less than the reference value (YES in S311), the digital gain is increased (S313). In this way, the control unit 16 adjusts the gain based on the average value. The control unit 16 performs gain adjustment so that the average value approaches the reference value. The reference value can be the average value of the thermal image data acquired on a sunny day.

[0067] The thermal imaging data acquiring unit 14 acquires gain-adjusted (also referred to as G-adjusted) thermal imaging data (S314). That is, the thermal imaging data acquiring unit 14 acquires thermal imaging data that has been subjected to gain processing using the gain adjusted in step S312 or step S313.

[0068] The control unit 16 creates a histogram of the G-adjusted thermal imaging data (S315). Then, the control unit 16 adjusts the dynamic range based on the histogram (S316). For example, if the contrast of the G-adjusted thermal imaging data is low, the control unit 16 widens the dynamic range.

[0069] In this way, the control unit 16 compares the sensor output values of the thermal imaging data before and after the change in the situation. If the sensor output value before the change is small, the control unit 16 may adjust the dynamic range for the sensor output value, and adjust the gain for the sensor output value until the sensor output value becomes equal to the average value of the sensor output values acquired when the weather information is fine.

[0070] By doing this, the imaging system 1 can acquire appropriate thermal imaging data. That is, the image processing device 10 can acquire thermal imaging data with an appropriate dynamic range and digital gain. This allows thermal imaging data suitable for target recognition to be obtained. The display image data output unit 18 outputs the thermal imaging data acquired with the adjusted dynamic range and digital gain to the display 50 as display image data. The display 50 can display the appropriate thermal imaging data.

[0071] When the programs of the above-described image processing apparatus 10 and imaging system 1 are read into a computer, they include a set of instructions (or software code) for causing the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD), or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disc, or other optical disc storage, magnetic storage devices. The program may be transmitted on a transient computer-readable medium or communication medium. By way of example and not limitation, the transient computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0072] Note that the present invention is not limited to the above-described embodiments, and can be appropriately modified without departing from the spirit thereof.

Explanation of Reference Numerals

[0073] 1 Imaging system 10 Image processing apparatus 11 Weather information acquisition unit 12 Situation change detection unit 14 Thermal image data acquisition unit 16 Control unit 18 Display image data output unit 20 Far-infrared sensor 30 Raindrop sensor 40 Rainfall sensor 50 Display

Claims

1. A thermal image data acquisition unit that acquires thermal image data captured by a sensor; A weather information acquisition unit that acquires weather information regarding the weather; A situation change detection unit that detects a change in the situation of the weather; A control unit that adjusts the sensor output value of the thermal image data when a change in the situation of the weather is detected, and An image processing apparatus.

2. The weather information includes precipitation information indicating a precipitation situation, and The control unit adjusts the sensor output value when the situation change detection unit detects a change in the precipitation situation. The image processing apparatus according to Claim 1.

3. The control unit compares the sensor output value of the thermal image data before and after the change in the situation, and adjusts the gain with respect to the sensor output value when the sensor output value before the change is small. The image processing apparatus according to Claim 1 or 2.

4. The control unit compares the sensor output value of the thermal image data before and after the change in the situation, and adjusts the dynamic range with respect to the sensor output value when the sensor output value before the change is small. The image processing apparatus according to Claim 1 or 2.

5. The control unit compares the sensor output value of the thermal image data before and after the change in the situation, adjusts the dynamic range with respect to the sensor output value when the sensor output value before the change is small, and adjusts the gain with respect to the sensor output value until it becomes the average value of the sensor output values obtained during clear weather. The image processing apparatus according to Claim 1 or 2.

6. A step of acquiring thermal image data captured by a sensor; A step of acquiring weather information regarding the weather; A step of detecting a change in the situation of the weather; A step of adjusting the sensor output value of the thermal image data when a change in the situation of the weather is detected, and An image processing method.

7. A program for causing a computer to execute an image processing method, the image processing method including: A step of acquiring thermal image data captured by a sensor; A step of acquiring weather information regarding the weather; A step of detecting a change in the situation of the weather; A step of adjusting the sensor output value of the thermal image data when a change in the situation of the weather is detected, and A program. ​

Images