Control Systems and Programs

Abstract

To provide a control system and the like which reduces image quality deterioration of a thermal image and reduces interruption of a sensor function.SOLUTION: In a control system 11, an image data acquisition unit 160 acquires thermal image data from an infrared camera which captures a thermal image of the periphery of a vehicle. A saturation region detection unit 173 detects the presence of a saturation region in the thermal image. A low-temperature region detection unit 174 detects the presence of a low-temperature region equivalent to a clearance in the thermal image. A determination unit 151 determines whether a low-temperature region exists in the periphery of the saturation region. A designation unit 152 outputs a signal for protecting a sensor part of the infrared camera when the determination unit 151 determines that the low-temperature region exists in a predetermined determination region in the periphery of the saturation region.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a control system and a program. [Background technology]

[0002] Many object detection systems using infrared cameras that detect the heat of objects have been developed. It is expected that such object detection systems will be used to improve the safety of automobiles, for example. However, because infrared cameras have the characteristic of detecting the heat of objects, there is a possibility that the thermal sensor used as the infrared imaging element will malfunction when capturing sunlight. Therefore, technology to suppress the effects of sunlight is required.

[0003] For example, a method is disclosed for detecting burn-in events that cause defects in thermal images captured by a focal plane array using flat field correction (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-027380 A Summary of the Invention [Problem to be solved by the invention]

[0005] An object detection system that uses an infrared camera is expected to quickly detect when direct sunlight hits the infrared camera to protect the thermal sensor, while also continuing to function properly when the camera is not exposed to direct sunlight.

[0006] The present invention has been made to solve these problems, and provides a control system and the like that suppresses deterioration of the image quality of a thermal image and suppresses interruption of the sensor function. [Means for solving the problem]

[0007] The control system of the present invention has an image data acquisition unit, a saturated area detection unit, a low temperature area detection unit, a judgment unit, and an instruction unit. The image data acquisition unit acquires thermal image data from an infrared camera that captures thermal images of the area around the vehicle. The saturated area detection unit detects the presence of a saturated area in the thermal image. The low temperature area detection unit detects the presence of a low temperature area corresponding to the sky in the thermal image. The judgment unit judges whether or not a low temperature area exists around the saturated area. If the judgment unit judges that a low temperature area exists within a predetermined judgment area around the saturated area, the instruction unit outputs a signal instructing protection of the sensor unit of the infrared camera.

[0008] The program according to the present invention causes a computer to execute the following control method. The control method has an image data acquisition step, a saturated area detection step, a low temperature area detection step, a judgment step, and an instruction step. The image data acquisition step acquires thermal image data from an infrared camera that captures thermal images of the surroundings of the vehicle. The saturated area detection step detects the presence of a saturated area in the thermal image. The low temperature area detection step detects the presence of a low temperature area corresponding to the sky in the thermal image. The judgment step judges whether or not a low temperature area exists around the saturated area. The instruction step outputs a signal instructing protection of the infrared sensor of the infrared camera if it is judged in the judgment step that a low temperature area exists around the saturated area. Effect of the Invention

[0009] According to the present invention, it is possible to provide a control system, a control method, and a program that suppress deterioration of the image quality of a thermal image and suppress interruption of the sensor function. [Brief description of the drawings]

[0010] [Figure 1] 1 is a configuration diagram of a vehicle equipped with a control system according to a first embodiment. [Diagram 2] FIG. 1 is a diagram illustrating the configuration of an infrared camera. [Diagram 3]FIG. 1 is a block diagram of a control system according to a first embodiment. [Figure 4] FIG. 2 is a block diagram of an image processing unit according to the first embodiment. [Diagram 5] 4 is a flowchart of a control method according to the first embodiment. [Figure 6] FIG. 2 is a diagram for explaining functions of the control system according to the first embodiment. [Figure 7] 11 is a flowchart of a control method according to a second embodiment. [Figure 8] FIG. 11 is a diagram for explaining functions of a control system according to a second embodiment. [Figure 9] 13 is a flowchart of a control method according to a third embodiment. [Figure 10] FIG. 11 is a diagram for explaining the function of the control system according to the third embodiment. [Figure 11] 13 is a flowchart of a control method according to a fourth embodiment. [Figure 12] FIG. 13 is a diagram for explaining the functions of a control system according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present invention will be described below through the embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. In addition, not all of the configurations described in the embodiments are necessarily essential as means for solving the problems. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. In each drawing, the same elements are given the same reference numerals, and duplicate explanations are omitted as necessary.

[0012] <Embodiment 1> Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a configuration diagram of a vehicle equipped with a control system according to the first embodiment. The control system according to the present embodiment controls an infrared camera provided in the vehicle. A vehicle 90 shown in Fig. 1 includes an infrared camera 10, a control system 11, and a display 12.

[0013] The infrared camera 10 is provided on the vehicle 90 so as to capture an image of the periphery of the vehicle 90. The infrared camera 10 is fixed, for example, to the front of the vehicle 90 and captures an image of the area in front of the vehicle 90. The infrared camera 10 is communicably connected to a control system 11, receives a predetermined instruction signal from the control system 11, and operates according to the received instruction signal. The infrared camera 10 also supplies image data (also called thermal image data) related to an image (also called a thermal image) captured by the infrared camera 10 to the control system 11. The infrared camera 10 is a so-called far-infrared camera.

[0014] The control system 11 is fixed at a desired location on the vehicle 90, and is communicatively connected to each of the infrared camera 10 and the display 12. The control system 11 controls the infrared camera 10 to acquire image data generated by the infrared camera 10, and causes the display 12 to display the acquired image data.

[0015] The display 12 is a display device including, for example, a liquid crystal panel or an organic electroluminescence panel, and is provided at a position visible to the driver of the vehicle 90. The display 12 displays an image captured by the infrared camera 10 via the control system 11.

[0016] With the above-mentioned configuration, the control system 11 displays the image captured by the infrared camera 10 on the display 12 in a manner that is visible to the driver. In this way, the control system 11 can allow the driver to recognize objects around the vehicle 90.

[0017] Next, the configuration of the infrared camera 10 will be described with reference to Fig. 2. Fig. 2 is a configuration diagram of the infrared camera. The infrared camera 10 shown in Fig. 2 has, as its main components, a housing 101, an objective lens 102, a shutter 103, an infrared sensor 104, a camera control circuit 105, and a temperature sensor 106.

[0018] The housing 101 houses each component of the infrared camera 10 and is fixed to the vehicle 90. The objective lens 102 receives infrared light incident from the range captured by the infrared camera 10 and projects the infrared light onto the infrared sensor 104. The shutter 103 includes a light-shielding plate material and is interposed between the objective lens 102 and the infrared sensor 104 so as to be able to open and close.

[0019] When the shutter 103 is closed, it blocks light entering the infrared sensor 104 from the objective lens 102. On the other hand, when the shutter 103 is open, it does not block light entering the infrared sensor 104 from the objective lens 102. In addition, the shutter 103 is a black body as viewed from the infrared sensor 104, and the infrared sensor 104 is calibrated with the shutter 103 closed.

[0020] The infrared sensor 104 is composed of thermally sensitive elements arranged in an array, receives infrared light incident through the objective lens 102, and generates image data based on changes in the resistance value of each of the thermally sensitive elements. The infrared sensor 104 is communicably connected to a camera control circuit 105, and operates upon receiving a predetermined control signal from the camera control circuit 105. When the infrared sensor 104 generates image data, it supplies the generated image data to the camera control circuit 105.

[0021] When the infrared sensor 104 receives sunlight with a strength greater than a predetermined value for a predetermined period of time or longer, the output of the elements in the area receiving the sunlight becomes saturated. Also, the temperature of the elements in the area receiving the sunlight rises above a specified value, causing abnormal conditions such as irreversible deformation of the elements or changes in characteristics. For this reason, the infrared camera 10 has a shutter 103 to protect the infrared sensor 104 from direct sunlight.

[0022] The dynamic range of the infrared sensor 104 is set so that the resolution for detecting objects such as pedestrians is relatively high. Therefore, even in a temperature range where no abnormality occurs, the output of the elements of the infrared sensor 104 becomes saturated in the range where an object with a high temperature is photographed.

[0023] The camera control circuit 105 is a control circuit including an MCU (Micro Controller Unit) and controls the shutter 103 and the infrared sensor 104. The camera control circuit 105 is also communicably connected to the control system 11, receives control signals from the control system 11, and controls each component of the infrared camera 10 in response to the received control signals. The camera control circuit 105 controls the shutter 103 to be kept closed when the infrared camera 10 is not capturing an image. The camera control circuit 105 controls the shutter 103 to be open when the infrared camera 10 is capturing an image. At this time, the camera control circuit 105 also supplies image data generated by the infrared sensor 104 to the control system 11.

[0024] Furthermore, the camera control circuit 105 temporarily closes the shutter 103 under a predetermined condition. The predetermined condition is, for example, when the shutter 103 is temporarily closed to protect the infrared sensor 104 or when calibration is performed. In this case, for example, the camera control circuit 105 receives an instruction from the control system 11 to temporarily close the shutter 103.

[0025] The camera control circuit 105 is communicatively connected to the temperature sensor 106, and receives measurement data relating to the temperature inside the infrared camera 10 from the temperature sensor 106. The camera control circuit 105 also supplies the measurement data received from the temperature sensor 106 to the control system 11. The camera control circuit 105 may also receive the measurement data from the temperature sensor 106, and perform calibration in accordance with the received measurement data.

[0026] The temperature sensor 106 is installed inside the infrared camera 10 to measure the temperature of the infrared camera 10 and supply the measurement data to the camera control circuit 105. The temperature sensor 106 is preferably installed in the vicinity of the infrared sensor 104.

[0027] Next, the control system 11 will be described with reference to Fig. 3. Fig. 3 is a block diagram of the control system according to the first embodiment. The control system 11 has, as its main components, a control IF 120, a ROM 130, a RAM 140, a system control circuit 150, an image data acquisition unit 160, an image processing unit 170, an image recognition unit 180, and an image data output unit 190. These components are connected via a bus 110 so as to be able to communicate with each other as appropriate.

[0028] The control IF 120 is a communication line interface for controlling the infrared camera 10. The control IF 120 supplies the infrared camera 10 with a control signal for the control system 11 to control the infrared camera 10.

[0029] The ROM 130 (Read Only Memory) is a non-volatile memory that stores preset information or data. The ROM 130 stores in advance, for example, a program for the control system 11 to realize the functions according to the present embodiment.

[0030] The RAM 140 (RAM (Random Access Memory)) is a volatile memory having a storage area in which the control system 11 can temporarily expand data. The RAM 140 may be, for example, a DRAM (Dynamic Random Access Memory), and may include a register associated with the system control circuit 150 or the like. The RAM 140 includes an area in which the programs stored in the ROM 130 are expanded and executed. The RAM 140 may also be used, for example, when processing image data supplied from the infrared camera 10.

[0031] The system control circuit 150 includes an arithmetic unit such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The system control circuit 150 includes a determination unit 151 and an instruction unit 152 as functional blocks.

[0032] The determination unit 151 performs a predetermined determination on the positional relationship between a saturated region and a low temperature region when both are included in the thermal image based on the image data obtained from the control system 11. More specifically, the determination unit 151 determines whether or not a low temperature region exists within a predetermined range around the saturated region in the thermal image.

[0033] That is, the thermal image captured by the infrared camera 10 may include an image of the sun as a saturated region. The thermal image may also include an image of a high-temperature part such as a muffler of another vehicle as a saturated region. Furthermore, there is a high possibility that the sky is captured around the image of the sun. The temperature of the sky area included in the thermal image is very low. On the other hand, there is a low possibility that a low-temperature area showing the temperature of the sky exists around the image of a high-temperature part such as a muffler of another vehicle. Therefore, when a low-temperature area corresponding to the sky exists around the saturated region, that is, a low-temperature area showing the temperature of the sky, it can be estimated that the saturated region is an image of the sun. Therefore, the control system 11 judges whether or not a low-temperature area corresponding to the sky exists around the saturated region. The judgment unit 151 supplies the result of this judgment to the instruction unit 152.

[0034] The instruction unit 152 outputs a predetermined instruction to the infrared camera 10. More specifically, the instruction unit 152 receives, for example, the determination result of the determination unit 151. If the determination result is that a low temperature area indicating the temperature of the sky is present around the saturated area, the instruction unit 152 outputs a signal to the infrared camera 10 to instruct it to protect the infrared sensor 104. More specifically, the instruction unit 152 outputs an instruction to close the shutter 103 of the infrared camera 10. The control IF 120 supplies the instruction output by the instruction unit 152 to the infrared camera 10.

[0035] The image data acquisition unit 160 is an interface that acquires thermal image data (input thermal image data), which is data related to a thermal image, from the infrared camera 10. The image data acquisition unit 160 acquires image data periodically, for example, from the infrared camera 10. For example, the image data acquisition unit 160 receives one frame of an image every 1 / 15th of a second. When the image data acquisition unit 160 receives the image data, it supplies the received image data to the image processing unit 170.

[0036] The image processing unit 170 is realized by an image processing circuit including, for example, a GPU (Graphics Processing Unit). The image processing unit 170 performs predetermined processing on the thermal imaging data in cooperation with the RAM 140. The image processing unit 170 will be described in detail later.

[0037] The image recognition unit 180 receives thermal imaging data from the image data acquisition unit 160, and detects vehicles, people, etc. from the thermal image based on the received thermal imaging data. More specifically, the image recognition unit 180 has a recognition dictionary for detecting detection target objects such as vehicles and people, and detects vehicles, people, etc. by referring to the data in the recognition dictionary. When the image recognition unit 180 detects a vehicle or person, it generates information on the position and size of the detected vehicle or person, and supplies the generated information to the system control circuit 150.

[0038] The image data output unit 190 is an interface for outputting the image data (output thermal image data) processed by the image processing unit 170 to the display 12. The image data output unit 190 outputs the image data in a data format corresponding to the specifications of the display 12. This data format is, for example, HDMI (High-Definition Multimedia Interface) (registered trademark) or DVI (Digital Visual Interface).

[0039] Next, the image processing unit 170 will be described with reference to Fig. 4. Fig. 4 is a block diagram of the image processing unit 170 according to the first embodiment. The image processing unit 170 has a defective pixel correction unit 171, a NUC unit 172, a saturated region detection unit 173, and a low temperature region detection unit 174 as main components.

[0040] The defective pixel correction unit 171 stores defective pixels of the infrared sensor 104 in advance, and performs a process (interpolation process) of interpolating the pixel value of the stored defective pixel from the pixel values ​​of surrounding pixels. The defective pixel correction unit 171 receives input thermal image data from the infrared camera 10 via the image data acquisition unit 160, and performs the above-mentioned interpolation process on the received image data. The defective pixel correction unit 171 supplies the image data after the interpolation process to the NUC unit 172 and the saturated area detection unit 173.

[0041] The NUC unit 172 performs NUC (Non-Uniformity Correction), which is a calibration process for suppressing variations in pixel values ​​output in response to input light. The NUC unit 172 has in advance settings of gain and offset values ​​corresponding to the characteristics of each pixel of the infrared sensor 104. The NUC unit 172 calibrates each pixel value of the image data received from the defective pixel correction unit 171 according to the in advance settings.

[0042] The saturated region detection unit 173 receives image data in which defective pixels have been corrected from the defective pixel correction unit 171, and detects saturated regions in the received image data. More specifically, the saturated region detection unit 173 searches for saturated regions in the image data in order to detect the saturated regions. A saturated region is a region in which saturated pixels or pixels that are substantially saturated exist. A saturated pixel is a pixel in which the pixel data is at the upper limit value. For example, when the luminance level of each pixel in the imaging data is expressed in 14 bits from zero to 16383, a pixel at a coordinate having a luminance level of 16383 is called a saturated pixel.

[0043] The saturated region detection unit 173 determines that a saturated region exists when, for example, four adjacent pixels are detected as saturated pixels. Alternatively, the saturated region detection unit 173 may determine that a saturated region exists when, for example, nine or more adjacent pixels have a luminance value of 98 percent or more of the upper limit. When the saturated region detection unit 173 detects that a saturated region exists in the received imaging data, it outputs a signal indicating the detection result of the saturated region to the system control circuit 150. The signal indicating the detection result of the saturated region may include data regarding the position of the pixel determined to be a saturated region.

[0044] The low temperature area detection unit 174 detects a low temperature area in a predetermined area of ​​the image data. More specifically, the low temperature area detection unit 174 searches for a low temperature area in a predetermined area of ​​the image data in order to detect the low temperature area in the predetermined area. The predetermined area in which the low temperature area detection unit 174 searches for the low temperature area may be the entire image data, or may be an area set according to the saturation area.

[0045] The low temperature region is a region that includes pixel values ​​that indicate a relatively low temperature in the thermal image. More specifically, the low temperature region is a region in which pixels with relatively small brightness values ​​exist, and the difference in brightness value between the pixels and saturated pixels is greater than a predetermined threshold value. The low temperature region according to the present embodiment is set for the purpose of detecting the temperature of the sky. Generally, in a thermal image, the sky has a pixel value that corresponds to, for example, about minus 20 degrees Celsius or less. Therefore, the threshold value, which is the difference in brightness value between the saturated pixels, is set so that the above-mentioned temperature can be detected. The brightness value of the thermal image measured and output by the infrared sensor 104 changes depending on the environmental temperature. Therefore, the above-mentioned threshold value can be set according to the temperature measured by the temperature sensor 106.

[0046] Instead of determining that the difference in brightness value from the saturated pixel is greater than the above-mentioned threshold, the low temperature area detection unit 174 may determine that the brightness value of each pixel is below a predetermined lower limit. In this case, the lower limit may be set according to the temperature measured by the temperature sensor 106. When the low temperature area detection unit 174 detects that a low temperature area exists in the received imaging data, it outputs a signal indicating the detection result of the low temperature area to the system control circuit 150. The signal indicating the detection result of the low temperature area may include data on the position of the pixel determined to be in the low temperature area.

[0047] Next, the process executed by the control system 11 will be described with reference to Fig. 5. Fig. 5 is a flowchart of the control method according to the first embodiment. The flowchart shown in Fig. 5 is started, for example, when the control system 11 starts imaging using the infrared camera 10, and is repeatedly executed during the period when imaging using the infrared camera 10 is being performed.

[0048] First, the image data acquisition unit 160 of the control system 11 acquires thermal image data from the infrared camera 10 (step S10) and supplies the data to the image processing unit 170.

[0049] Next, the saturated area detection section 173 and the low temperature area detection section 174 of the image processing section 170 search for saturated areas and low temperature areas indicating the sky temperature from the defect-corrected image data (step S11). When the image processing section 170 detects saturated areas and low temperature areas as a result of the search, it supplies data on the positions of the detected saturated areas and low temperature areas to the system control circuit 150.

[0050] Next, the system control circuit 150 judges whether the image processing unit 170 has detected the presence of a saturation region and a low temperature region in the thermal imaging data (step S12). More specifically, the judgment unit 151 of the system control circuit 150 makes this judgment. If it is judged that the presence of a saturation region and a low temperature region in the thermal imaging data has been detected (step S12: YES), the system control circuit 150 proceeds to step S13. On the other hand, if it is not judged that the presence of a saturation region and a low temperature region in the image data has been detected (step S12: NO), the control system 11 ends the series of processes.

[0051] In step S13, the determination unit 151 of the system control circuit 150 determines whether or not a low temperature area is included around the saturated area in the thermal image (step S13). If it is determined that a low temperature area is included around the saturated area (step S13: YES), the control system 11 proceeds to step S14. On the other hand, if it is not determined that a low temperature area is included around the saturated area (step S13: NO), the control system 11 ends the series of processes.

[0052] In step S14, the instruction unit 152 of the system control circuit 150 outputs a signal instructing to protect the infrared sensor 104 of the infrared camera 10 (step S14). More specifically, for example, the instruction unit 152 outputs an instruction to close the shutter 103 of the infrared camera 10. The control system 11 supplies a signal related to this instruction to the infrared camera 10 via the control IF 120. When the control system 11 outputs the signal instructing to close the shutter 103, it ends the series of processes.

[0053] In step S14, when the shutter 103 is closed, the instruction unit 152 outputs a signal instructing to open the shutter 103 after a predetermined period of time has elapsed. The predetermined period in this case is, for example, a period for calibrating the infrared sensor 104. Alternatively, it may be a period set in advance, such as one second.

[0054] The above describes the control method, which is a process executed by the control system 11. In the above control method, the control system 11 outputs a signal instructing protection of the infrared sensor 104 according to the positional relationship between the saturated area and the low temperature area. That is, when a low temperature area corresponding to the sky exists around the saturated area, the control system 11 outputs a signal instructing protection of the infrared sensor 104. On the other hand, when a low temperature area corresponding to the sky does not exist around the saturated area, the control system 11 does not output a signal instructing protection of the infrared sensor 104. In this way, the above control method can suppress deterioration of the image quality of the thermal image and suppress interruption of the sensor function.

[0055] Next, functions of the control system 11 according to the present embodiment will be described with reference to Fig. 6. Fig. 6 is a diagram for explaining functions of the control system according to the first embodiment. Fig. 6 shows an example of an image captured by the infrared camera 10. An image D11 shown in Fig. 6 is an image of an area ahead of a traveling vehicle 90 captured by the infrared camera 10. The image D11 includes an image of another vehicle 990 traveling ahead of the vehicle 90. The image of the other vehicle 990 includes an image of a muffler 991. The image D11 also includes an image of the sun 800 and an image of the sky 801.

[0056] In the image D11, a saturated region may occur in pixels corresponding to the image of the muffler 991. That is, the saturated region detection unit 173 may detect saturated pixels in the region of the muffler 991 (the region of an ellipse hatched with a dashed line) as a saturated region. Similarly, in the image D11, a saturated region may occur in pixels corresponding to the image of the sun 800. That is, the saturated region detection unit 173 may detect saturated pixels in the region of the sun 800 (the region of a circle hatched with a dashed line) as a saturated region. Also, in the image D11, a low temperature region may occur in pixels corresponding to the image of the sky 801. That is, the low temperature region detection unit 174 may detect the region of the sky 801 (the region hatched with a solid line) as a low temperature region.

[0057] At this time, the judgment unit 151 judges whether or not a low temperature region exists around the saturated region. Specifically, the judgment unit 151 sets a predetermined judgment region around the saturated region. The judgment region is a predetermined region defined with the saturated region as its center. In FIG. 6, a rectangle indicated by a thick two-dot chain line around the sun 800 indicates the boundary of a judgment region A11 set around the saturated region corresponding to the sun 800. Similarly, a rectangle indicated by a thick two-dot chain line around the muffler 991 indicates the boundary of a judgment region A11 set around the saturated region corresponding to the muffler 991.

[0058] In the above situation, the judgment unit 151 judges whether or not a low temperature area exists within the judgment area set in the saturated area corresponding to each of the sun 800 and the muffler 991. As shown in FIG. 6, the judgment area A12 does not include a low temperature area. Therefore, the judgment unit 151 does not judge that a low temperature area corresponding to the sky exists around the saturated area corresponding to the image of the muffler 991. Therefore, the instruction unit 152 does not output a signal instructing protection of the infrared sensor 104 from the saturated area corresponding to the image of the muffler 991.

[0059] 6, the judgment area A11 includes a low temperature area corresponding to the image of the sky 801. Therefore, the judgment unit 151 judges that a low temperature area exists around the saturated area corresponding to the image of the sun 800. Therefore, the instruction unit 152 outputs a signal to instruct protection of the infrared sensor 104 from the saturated area corresponding to the image of the sun 800.

[0060] The above-mentioned infrared sensor 104 may be referred to as a thermal sensor or a bolometer. The above-mentioned control system 11 may be configured to include at least one of the infrared camera 10 and the display 12.

[0061] The above describes the embodiment 1. As described above, according to the embodiment 1, it is possible to provide a control system, a control method, and a program that suppress deterioration of the image quality of a thermal image and suppress interruption of the sensor function.

[0062] <Embodiment 2> Next, a description will be given of embodiment 2. In embodiment 2, the process executed by control system 11 is different from embodiment 1.

[0063] In the control system 11 according to the second embodiment, when the saturated region detection unit 173 of the image processing unit 170 detects a saturated region, the image processing unit 170 sets a judgment region for the detected saturated region. Furthermore, the image processing unit 170 searches for a low temperature region in the judgment region to detect a low temperature region in the set judgment region. Then, the low temperature region detection unit 174 supplies the search result to the system control circuit 150.

[0064] Fig. 7 is a flowchart of the control method according to the embodiment 2. The flowchart shown in Fig. 7 differs from the flowchart shown in Fig. 5 in that steps S21 to S24 are executed instead of steps S11 and S12.

[0065] First, the image data acquisition unit 160 of the control system 11 according to the second embodiment acquires thermal image data from the infrared camera 10 (step S10), and supplies the thermal image data to the image processing unit 170.

[0066] Next, the saturated region detection unit 173 of the image processing unit 170 searches for saturated regions from the image data that has been subjected to the defect correction (step S21).

[0067] Next, the image processing unit 170 judges whether or not the saturated region detection unit 173 has detected a saturated region (step S22). If the saturated region detection unit 173 has not detected a saturated region, the image processing unit 170 does not judge that the saturated region detection unit 173 has detected a saturated region. In this case (step S22: NO), the control system 11 ends the series of processes. If it is judged that the saturated region detection unit 173 has detected a saturated region (step S22: YES), the saturated region detection unit 173 supplies data regarding the position of the detected saturated region, etc. to the low temperature region detection unit 174, and the image processing unit 170 proceeds to step S23.

[0068] In step S23, the low temperature region detection unit 174 sets a judgment region for the saturated region of the image data, and searches for a low temperature region within the judgment region (step S23).

[0069] Next, the judgment unit 151 of the system control circuit 150 judges whether the low temperature area detection unit 174 has detected a low temperature area corresponding to the sky in the judgment area (step S24). If the low temperature area detection unit 174 has not detected a low temperature area, the judgment unit 151 does not judge that the low temperature area detection unit 174 has detected a low temperature area. That is, in this case, it means that although a saturated area exists in the image data, a low temperature area corresponding to the sky does not exist around the saturated area. In this case (step S24: NO), the control system 11 ends the series of processes. On the other hand, if it is judged that the low temperature area detection unit 174 has detected a low temperature area (step S24: YES), the image processing unit 170 supplies data on the positions of the detected saturated area and low temperature area to the system control circuit 150, and the system control circuit 150 proceeds to step S13.

[0070] In step S13, the judgment unit 151 of the system control circuit 150 judges whether or not a low temperature area is included around the saturated area in the thermal image (step S13). In this embodiment, the "surrounding of the saturated area" refers to the judgment area set by the low temperature area detection unit 174. If it is judged that a low temperature area is included around the saturated area, i.e., in the judgment area (step S13: YES), the control system 11 proceeds to step S14. On the other hand, if it is not judged that a low temperature area is included in this judgment area (step S13: NO), the control system 11 ends the series of processes.

[0071] Next, functions of the control system 11 according to the present embodiment will be described with reference to Fig. 8. Fig. 8 is a diagram for explaining functions of the control system according to the embodiment 2. An image D21 shown in Fig. 8 includes images of the sun 800, the sky 801, and another vehicle 990, similar to Fig. 6.

[0072] 8, the control system 11 according to the second embodiment searches for a low temperature region in the judgment region A11 and the judgment region A12 corresponding to the saturation region. Therefore, the low temperature region detection unit 174 according to the present embodiment detects a low temperature region in the region inside the judgment region A11, but does not detect a low temperature region in other regions.

[0073] With this configuration, the control system 11 according to the present embodiment only needs to search for a low temperature region in the judgment region corresponding to the saturation region, and therefore can speed up the process of outputting a signal instructing protection of the infrared sensor 104.

[0074] The above describes the second embodiment. According to the second embodiment, it is possible to provide a control system, a control method, and a program that efficiently suppress deterioration in image quality of a thermal image and suppress interruption of the sensor function.

[0075] <Embodiment 3> Next, a description will be given of embodiment 3. A control system 11 according to embodiment 3 differs from the above-described embodiments in the method of searching for a low temperature region and the method of detecting a low temperature region.

[0076] The determination unit 151 according to this embodiment measures the distance between the saturation region and the low temperature region in the determination region. In this case, for example, the determination unit 151 measures the distance from the center of the saturation region to the center of the low temperature region. Then, when the distance between the saturation region and the low temperature region is less than a predetermined threshold distance, the determination unit 151 determines that a low temperature region exists around the saturation region. Note that the determination unit 151 may measure the shortest distance between the end of the saturation region and the end of the low temperature region.

[0077] The image processing unit 170 according to the present embodiment may measure the size of the saturated region. In this case, the judgment unit 151 may set at least one of the size of the judgment region and the threshold distance according to the measured size of the saturated region. Furthermore, in this case, the judgment unit 151 may provide information on the set judgment region or the threshold distance to the low temperature region detection unit 174.

[0078] Next, the process of the control system 11 according to the third embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart of a control method according to the third embodiment. The control method according to the third embodiment differs from the flowchart shown in Fig. 7 in that it includes step S31 instead of step S13.

[0079] In step S24, the determination unit 151 of the system control circuit 150 determines whether or not the low temperature region detection unit 174 has detected a low temperature region (step S24). If it is determined that the low temperature region detection unit 174 has detected a low temperature region (step S24: YES), the system control circuit 150 proceeds to step S31.

[0080] In step S31, the judgment unit 151 judges whether the distance between the saturation region and the low temperature region is less than the threshold distance (step S31). If it is not judged that the distance between the saturation region and the low temperature region is less than the threshold distance (step S31: NO), the control system 11 ends the series of processes. On the other hand, if it is judged that the distance between the saturation region and the low temperature region is less than the threshold distance (step S31: YES), the judgment unit 151 proceeds to step S14, and the instruction unit 152 outputs a signal instructing protection of the infrared sensor 104 (step S14).

[0081] Next, the function of the control system 11 according to the third embodiment will be further described with reference to Fig. 10. Fig. 10 is a diagram for explaining the function of the control system according to the third embodiment. An image D31 shown in Fig. 10 includes images of the sun 800, the sky 801, and another vehicle 990, similar to Fig. 8.

[0082] The control system 11 according to the third embodiment searches for a low temperature region in the judgment region A31 and judgment region A32 corresponding to the saturated region. At this time, the low temperature region detection unit 174 divides the judgment region A31 and judgment region A32 into predetermined unit regions and judges whether the unit regions can be low temperature regions. In this case, for example, the low temperature region detection unit 174 counts up a histogram of pixels included in the unit regions, and judges the unit regions to be low temperature regions when the ratio of pixels corresponding to low temperature regions is, for example, 90 percent or more. Note that the above-mentioned 90 percent is an example. The percentage by which the low temperature region detection unit 174 judges a region to be a low temperature region is not limited to 90 percent.

[0083] 10, the low temperature region detection unit 174 sets a plurality of unit regions U31 and determines whether each unit region U311 is a low temperature region. The low temperature region detection unit 174 supplies data indicating whether each unit region U31 is a low temperature region to the system control circuit 150.

[0084] When the determination unit 151 of the system control circuit 150 receives data related to the saturation region and the low temperature region from the image processing unit 170, the determination unit 151 calculates the distance between the saturation region and the low temperature region using the received data. The determination unit 151 then determines whether the distance between the saturation region and the low temperature region is less than a threshold distance. In this case, the determination unit 151 calculates the distance between the saturation region and the low temperature region using, for example, the vector value of a vector connecting the center of the saturation region and the center of the unit region determined to be a low temperature region.

[0085] 10, the low temperature area detection unit 174 searches for a low temperature area in the judgment area A31. As a result, the low temperature area detection unit 174 judges a plurality of unit areas in the judgment area A31 to be low temperature areas. On the other hand, the low temperature area detection unit 174 searches for a low temperature area in the judgment area A32, but does not detect a low temperature area.

[0086] The dotted circle around the image of the sun 800 indicates a threshold distance of radius D800 from the center of the saturated region corresponding to the sun 800. The center of the unit region U311 indicated by the thick solid line is located at a position less than the threshold distance. In this case, the determination unit 151 determines that the saturated region corresponding to the sun 800 and the unit region U311 are less than the threshold distance.

[0087] The judgment unit 151 may perform judgments other than those described above. For example, the judgment unit 151 may calculate a predetermined threshold distance from the number of pixels existing between the saturation region and the low temperature region. More specifically, for example, when the thermal image data has 320 pixels in the horizontal direction and 240 pixels in the vertical direction, the judgment unit 151 may set a value of 10 pixels from the saturation region as the threshold and determine that the saturation region and the low temperature region are less than the predetermined threshold distance.

[0088] In the above example, the judgment region A31 is a square, and one side has a length D31. This length D31 can be set based on the size of the saturated region corresponding to the sun 800. Furthermore, one side of the judgment region A32 has a length D32. This length D32 can be set based on the size of the saturated region corresponding to the muffler 991. That is, the length of one side of the judgment region when the size of the saturated region is relatively small can be set to be shorter than the length of one side of the judgment region when the size of the saturated region is relatively large.

[0089] The third embodiment has been described above. The example shown in Fig. 10 shows an example in which a judgment region is set and a low temperature region is searched for within the set judgment region, similar to the example in Fig. 8 shown in the second embodiment. However, the control system 11 according to the third embodiment may search for a low temperature region in the entire image data, as shown in the first embodiment, and then determine the distance between the saturation region and the low temperature region.

[0090] The above describes the third embodiment. According to the third embodiment, it is possible to provide a control system, a control method, and a program that suitably suppress deterioration of the image quality of a thermal image and that suitably suppress interruption of the sensor function.

[0091] <Fourth embodiment> Next, a description will be given of embodiment 4. In the control system 11 according to embodiment 4, the process of the determination unit 151 is different from that of the above-mentioned embodiments.

[0092] The determination unit 151 according to this embodiment differs from the above-mentioned embodiment in that it further determines whether or not a low-temperature region exists at a position higher than the position where the saturated region exists in the thermal image. That is, when the determination unit 151 according to this embodiment determines that a low-temperature region exists around a saturated region, it determines whether or not this low-temperature region exists at a position higher than the position of the saturated region in the thermal image. Here, whether or not the low-temperature region exists at a position higher than the position of the saturated region is determined, for example, by whether or not a low-temperature region is included at a position higher in the height direction than the central pixel of the saturated region.

[0093] Fig. 11 is a flowchart of a control method according to the fourth embodiment. The control method according to this embodiment differs from the flowchart shown in Fig. 7 in that it has step S41 instead of step S13. The flowchart according to this embodiment also differs from the flowchart shown in Fig. 9 in that it has step S41 instead of step S31.

[0094] 11, the determination unit 151 of the system control circuit 150 determines whether or not the low temperature region detection unit 174 has detected a low temperature region (step S24). If it is determined that the low temperature region detection unit 174 has detected a low temperature region (step S24: YES), the system control circuit 150 proceeds to step S41.

[0095] In step S41, the judgment unit 151 judges whether or not a low temperature region exists above the saturation region (step S41). If it is not judged that a low temperature region exists above the saturation region (step S41: NO), the control system 11 ends the series of processes. On the other hand, if it is judged that a low temperature region exists above the saturation region (step S41: YES), the judgment unit 151 proceeds to step S14, and the instruction unit 152 outputs a signal instructing protection of the infrared sensor 104 (step S14).

[0096] Next, the function of the control system 11 according to the fourth embodiment will be further described with reference to Fig. 12. Fig. 12 is a diagram for explaining the function of the control system according to the fourth embodiment. An image D41 shown in Fig. 12 includes images of the sun 800, the sky 801, and another vehicle 990.

[0097] After detecting the saturated region, the low temperature region detection unit 174 according to the present embodiment sets a judgment region corresponding to the upper part of the saturated region, and searches for a low temperature region in this judgment region. In FIG. 12, the judgment region A41 is a judgment region corresponding to the upper part of the sun 800. Therefore, the upper part of the saturated region can be said to be the upper part continuing to the saturated region. Moreover, the judgment region A42 is a judgment region corresponding to the upper part of the muffler 991. The low temperature region detection unit 174 sets the judgment region in this way, and judges whether or not a low temperature region exists in the set judgment region. In the example shown in FIG. 12, the judgment unit 151 judges that a low temperature region exists in the judgment region A41. As a result, the control system 11 can efficiently detect a low temperature region when there is no sky below the sun, as shown in FIG. 12.

[0098] The fourth embodiment has been described above. The example shown in Fig. 12 shows an example in which a judgment region is set and a low temperature region is searched for within the set judgment region, similar to the example in Fig. 8 shown in the second embodiment. However, the control system 11 according to the fourth embodiment may search for a low temperature region in the entire image data, as shown in the first embodiment, and then determine that a low temperature region exists above a saturation region.

[0099] The functions described in the fourth embodiment may be executed according to the position of the saturated region. The control system 11 according to the fourth embodiment may perform the above-mentioned functions, for example, when the center of the saturated region is located within a range of two-thirds from the bottom of the thermal image. In this case, when the center of the saturated region is located above two-thirds from the bottom of the thermal image, the control system 11 may perform functions corresponding to the first or second embodiment.

[0100] The above describes the embodiment 4. According to the embodiment 4, it is possible to provide a control system, a control method, and a program that suppress deterioration of the quality of a thermal image and suppress interruption of the sensor function.

[0101] The above-mentioned program includes a set of instructions (or software code) for making the computer perform one or more functions described in the embodiment when the program is loaded into the computer. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or the tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, the transitory computer-readable medium or the communication medium includes electrical, optical, acoustic, or other forms of propagating signals.

[0102] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0103] 10. Infrared Camera 11 Control System 12 Display 90 vehicles 101 Case 102 Objective Lens 103 Shutter 104 Infrared Sensor 105 Camera control circuit 106 Temperature Sensor 110 Bus 120 Control IF 130 ROM 140 RAM 150 System control circuit 151 Judgment Department 152 Instruction section 160 Image data acquisition unit 170 Image Processing Unit 171 Defective pixel correction section 172 NUC Department 173 Saturation Area Detection Unit 174 Low temperature range detector 180 Image Recognition Unit 190 Image data output section 270 Image Processing Unit 800 sun 801 sky 990 Other vehicles 991 Muffler

Claims

1. an image data acquisition unit that acquires thermal image data from an infrared camera that captures a thermal image of the periphery of the vehicle; a saturated region detection unit for detecting the presence of a saturated region in the thermal image; a low temperature area detection unit that detects the presence of a low temperature area corresponding to the sky in the thermal image; a determination unit that determines whether the low temperature region exists around the saturation region; an instruction unit that outputs a signal instructing protection of a sensor unit of the infrared camera when the determination unit determines that the low temperature area exists within a predetermined determination area around the saturation area; A control system comprising:

2. The determination unit determines that the low temperature region is present around the saturated region when the saturated region and the low temperature region in the determination region are less than a predetermined threshold distance. The control system of claim 1 .

3. the determination unit sets at least one of a size of the determination region and the threshold distance in accordance with a size of the saturated region. The control system of claim 2.

4. The determination unit further determines whether the low temperature region is present at a position higher than a position where the saturated region is present in the thermal image. A control system according to any one of claims 1 to 3.

5. An image data acquisition step of acquiring thermal image data from an infrared camera that captures a thermal image of the periphery of the vehicle; a saturated region detection step of detecting a saturated region in the thermal image; a low temperature area detection step of detecting a low temperature area corresponding to the sky in the thermal image; a determination step of determining whether the low temperature region exists around the saturation region; an instruction step of outputting a signal for instructing protection of an infrared sensor of the infrared camera when it is determined in the determination step that the low temperature area exists around the saturation area; A control method comprising: program.

Images