Image processing device, image processing method, and program

Abstract

To provide an image processing apparatus configured to properly set gray levels of temperature zones of a desired object, an image processing method, and a program.SOLUTION: In an image processing apparatus 11, an image data acquisition unit 160 acquires thermal image data from an infrared camera which captures a thermal image of surroundings of a vehicle. A determination unit 151 determines whether to execute enhancement processing to enhance contrast, on the basis of information associated with the contrast of the thermal image. An image processing unit 170 generates, based on the determination, first and second local tone curves corresponding to first and second local regions, synthesizes the local tone curves using a predetermined weighting factor to generate blend tone curve, and generates contrast-enhanced image data from the thermal image data by applying the blend tone curve. An image data output unit 190 outputs the contrast-enhanced image data.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an image processing apparatus, an image processing method, and a program. [Background technology]

[0002] Many object detection systems have been developed that use infrared cameras to detect the heat of objects. It is desirable for such object detection systems to be able to effectively detect the heat of objects such as people and animals in a variety of environments.

[0003] By the way, the infrared light used by infrared cameras has a characteristic that it is easily absorbed by water. Therefore, the infrared light input to the camera is attenuated under environmental conditions such as rain or fog. Therefore, it is difficult for an infrared camera to generate a clear image under such conditions. Therefore, a commonly known method for processing such images is to create a blurred image from the direct current and low frequency components of the image and subtract it from the original image.

[0004] Furthermore, for example, a technique has been disclosed for converting the histogram of an image in accordance with environmental data acquired from a predetermined sensor (Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] US Patent No. 08077995 Summary of the Invention [Problem to be solved by the invention]

[0006] However, there is a demand for the development of a method for reducing the load of image processing rather than the above-mentioned image processing technology.

[0007] The present invention has been made to solve the above problems, and provides an image processing device and the like that improves the contrast of an image by simple processing. [Means for solving the problem]

[0008] The image processing device according to the present invention includes an image data acquisition unit, a judgment unit, an image processing unit, and an image data output unit. The image data acquisition unit acquires thermal image data from an infrared camera that captures a thermal image of the periphery of a vehicle. The judgment unit judges whether or not to perform an enhancement process for enhancing the contrast based on information associated with the contrast of the thermal image. The image processing unit generates contrast-enhanced image data from the thermal image data based on the judgment. The output unit outputs the contrast-enhanced image data. The image processing unit sets a local region including a plurality of first local regions and a plurality of second local regions that include the first local regions and are larger than the first local regions, for the thermal image. The image processing unit calculates a frequency distribution regarding pixel values ​​of pixels that constitute the first local region and the second local region. The image processing unit generates redistribution data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution to the frequency distribution, and accumulates the redistribution data from low pixel values ​​toward high pixel values ​​to generate a local tone curve. The image processing unit generates a blended tone curve by combining a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a weighting coefficient set in advance for each of them. The image processing unit generates contrast-enhanced image data by applying the blended tone curve to pixels corresponding to the local regions.

[0009] In the image processing method according to the present invention, a computer executes the following steps. The computer acquires thermal image data from an infrared camera that captures a thermal image of the periphery of a vehicle. The computer determines whether or not to perform an enhancement process for enhancing the contrast based on information associated with the contrast of the thermal image. Based on the determination, the computer sets a local area including a plurality of first local areas and a plurality of second local areas that include the first local areas and are larger than the first local areas for the thermal image. The computer calculates a frequency distribution regarding pixel values ​​of pixels that constitute the first local area and the second local area. The computer generates redistribution data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution to the frequency distribution. The computer accumulates the redistribution data from low pixel values ​​toward high pixel values ​​to generate a local tone curve. The computer generates a blended tone curve by combining the first local tone curve corresponding to the first local area and the second local tone curve corresponding to the second local area using a weighting coefficient that is preset for each of them. The computer generates contrast-enhanced image data by applying the blended tone curve to pixels corresponding to the local areas. The computer outputs the contrast-enhanced image data.

[0010] The program according to the present invention causes a computer to execute the following image processing method. In the image processing method, the computer executes the following steps. The computer determines whether or not to perform an enhancement process for enhancing contrast based on information associated with the contrast of the thermal image. Based on the determination, the computer sets local regions including a plurality of first local regions and a plurality of second local regions that include the first local regions and are larger than the first local regions, for the thermal image. The computer calculates a frequency distribution regarding pixel values ​​of pixels constituting the first local region and the second local region. The computer generates redistributed data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution to the frequency distribution. The computer accumulates the redistributed data from low pixel values ​​toward high pixel values ​​to generate a local tone curve. The computer generates a blended tone curve by combining the first local tone curve corresponding to the first local region and the second local tone curve corresponding to the second local region using a weighting coefficient that is preset for each of them. The computer generates contrast-enhanced image data by applying the blended tone curve to pixels corresponding to the local regions. The computer outputs the contrast-enhanced image data. Effect of the Invention

[0011] According to the present invention, it is possible to provide an image processing device, an image processing method, and a program for improving the contrast of an image by simple processing. [Brief description of the drawings]

[0012] [Figure 1] 1 is a configuration diagram of a vehicle equipped with an image processing device according to a first embodiment. [Diagram 2] FIG. 1 is a diagram illustrating the configuration of an infrared camera. [Diagram 3] 1 is a block diagram of an image processing device according to a first embodiment. [Figure 4] FIG. 2 is a block diagram of an image processing unit according to the first embodiment. [Diagram 5] 4 is a first flowchart of the image processing method according to the first embodiment. [Figure 6] 5 is a second flowchart of the image processing method according to the first embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of a local region according to the first embodiment. [Figure 8] 11A to 11C are diagrams illustrating an example of a process for generating a local tone curve. [Figure 9] FIG. 13 is a diagram showing an example of a blend tone curve. [Figure 10] 11 is a flowchart of an image processing method according to a second embodiment. [Figure 11] FIG. 11 is a frequency distribution diagram of thermal imaging data. [Figure 12] FIG. 13 is a diagram illustrating an example of a unit region. [Figure 13] FIG. 11 is a diagram illustrating an example of a local region according to the second embodiment. [Figure 14] FIG. 11 is a diagram showing the relationship between a local tone curve and a blend tone curve according to the second embodiment. [Figure 15] 13A to 13C are diagrams illustrating examples of a local tone curve and a blend tone curve according to the second embodiment. [Figure 16] 13 is a flowchart of an image processing method according to a modified example of the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present invention will be described below through the embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. In addition, not all of the configurations described in the embodiments are necessarily essential as means for solving the problems. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. In each drawing, the same elements are given the same reference numerals, and duplicate explanations are omitted as necessary.

[0014] <Embodiment 1> Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram of a vehicle equipped with an image processing device according to the first embodiment. The image processing device according to the present embodiment receives a thermal image from an infrared camera provided in the vehicle, performs a predetermined process, and displays the received thermal image on a display. The image processing device according to the present embodiment also has a function of detecting a predetermined object from the thermal image received from the infrared camera. For example, when the image processing device detects an animal or a person in front of the vehicle, it issues a warning to the driver. Therefore, the image processing device processes the thermal image in order to detect the predetermined object. The vehicle 90 shown in FIG. 1 includes an infrared camera 10, an image processing device 11, a display 12, and an ECU 13.

[0015] The infrared camera 10 is provided on the vehicle 90 so as to capture an image of the periphery of the vehicle 90. The infrared camera 10 is fixed, for example, to the front of the vehicle 90 and captures an image of the area in front of the vehicle 90. The infrared camera 10 is communicably connected to an image processing device 11, receives a predetermined instruction signal from the image processing device 11, and operates according to the received instruction signal. The infrared camera 10 also supplies the image processing device 11 with image data (also called thermal image data) related to the image captured by the infrared camera 10 (also called thermal image).

[0016] The image processing device 11 is fixed at an arbitrary location of the vehicle 90, and is communicatively connected to each of the infrared camera 10 and the display 12. The image processing device 11 acquires image data generated by the infrared camera 10, and outputs the acquired image data to the display 12 for display. The image processing device 11 also receives predetermined information (vehicle information) from the ECU 13, and operates according to the received vehicle information. For example, when the image processing device 11 receives vehicle information indicating that the headlights are turned on from the ECU 13, it starts imaging by the infrared camera 10 in conjunction with this information. That is, the image processing device 11 according to this embodiment opens the shutter 103 of the infrared camera 10 and acquires image data from the infrared sensor 104 in a dark place where the vehicle 90 turns on its headlights, such as at night or in a tunnel.

[0017] The display 12 is a display device including, for example, a liquid crystal panel or an organic electroluminescence panel, and is provided at a position visible to the driver of the vehicle 90. The display 12 displays an image captured by the infrared camera 10 via the image processing device 11.

[0018] The ECU 13 (Electronic Control Unit) has data related to a specific function of the vehicle and controls the function. For example, the ECU 13 controls turning on and off the headlights of the vehicle 90. In this case, the ECU 13 may supply the image processing device 11 with vehicle information indicating that the headlights are turned on. Note that the ECU 13 may supply the image processing device 11 with, for example, a signal from an illuminance sensor of the vehicle 90 instead of the above-mentioned information.

[0019] The ECU 13 may also control the operation of the wipers of the vehicle 90. In this case, the ECU 13 may provide the image processing device 11 with vehicle information indicating the operation of the wipers as information associated with the contrast of the thermal image captured by the infrared camera 10. The ECU 13 may provide the image processing device 11 with signals generated by a humidity sensor, a raindrop sensor, etc., of the vehicle 90, and information on rain or fog detected by these sensors as information associated with the contrast of the thermal image. In addition to the above-mentioned information, the ECU 13 may provide the image processing device 11 with predetermined information acquired from outside the vehicle 90 via a communication device of the vehicle 90 as the vehicle information.

[0020] Furthermore, the ECU 13 may receive predetermined information from the image processing device 11. Furthermore, for example, when the ECU 13 receives a warning signal from the image processing device 11, the ECU 13 may use a speaker or a display device of the vehicle 90 to emit a sound or display a warning to the driver in accordance with the received warning signal.

[0021] With the above-mentioned configuration, the image processing device 11 performs a predetermined process on the image captured by the infrared camera 10, and displays the image on the display 12 in a manner that is visible to the driver. In this way, the image processing device 11 allows the driver to recognize objects around the vehicle 90.

[0022] Next, the configuration of the infrared camera 10 will be described with reference to Fig. 2. Fig. 2 is a configuration diagram of the infrared camera. The infrared camera 10 shown in Fig. 2 has, as its main components, a housing 101, an objective lens 102, a shutter 103, an infrared sensor 104, and a camera control circuit 105.

[0023] The housing 101 houses each component of the infrared camera 10 and is fixed to the vehicle 90. The objective lens 102 receives infrared light incident from the range captured by the infrared camera 10 and projects the infrared light onto the infrared sensor 104. The shutter 103 includes a light-shielding plate material and is interposed between the objective lens 102 and the infrared sensor 104 so as to be able to open and close.

[0024] When the shutter 103 is closed, it blocks light entering the infrared sensor 104 from the objective lens 102. On the other hand, when the shutter 103 is open, it does not block light entering the infrared sensor 104 from the objective lens 102. In addition, the shutter 103 is a black body as viewed from the infrared sensor 104, and the infrared sensor 104 is calibrated with the shutter 103 closed.

[0025] The infrared sensor 104 is composed of thermally sensitive elements arranged in an array, receives infrared light incident through the objective lens 102, and generates image data based on changes in the resistance value of each of the thermally sensitive elements. The infrared sensor 104 is communicably connected to a camera control circuit 105, and operates upon receiving a predetermined control signal from the camera control circuit 105. When the infrared sensor 104 generates image data, it supplies the generated image data to the camera control circuit 105.

[0026] When the infrared sensor 104 receives sunlight with a strength greater than a predetermined value for a predetermined period of time or longer, the output of the elements in the area receiving the sunlight becomes saturated. Also, the temperature of the elements in the area receiving the sunlight rises above a specified value, causing abnormal conditions such as irreversible deformation of the elements or changes in characteristics. For this reason, the infrared camera 10 has a shutter 103 to protect the infrared sensor 104 from direct sunlight.

[0027] The dynamic range of the infrared sensor 104 is set so that the resolution for detecting objects such as pedestrians is relatively high. Therefore, the infrared sensor 104 is set so that the detection signal is saturated even in a temperature range where no abnormality occurs.

[0028] The camera control circuit 105 is a control circuit including an MCU (Micro Controller Unit) and controls the shutter 103 and the infrared sensor 104. The camera control circuit 105 is also communicably connected to the image processing device 11, receives a control signal from the image processing device 11, and controls each component of the infrared camera 10 in response to the received control signal. The camera control circuit 105 controls the shutter 103 to be kept closed when the infrared camera 10 is not capturing an image. The camera control circuit 105 controls the shutter 103 to be open when the infrared camera 10 is capturing an image. At this time, the camera control circuit 105 also supplies image data generated by the infrared sensor 104 to the image processing device 11.

[0029] Furthermore, the camera control circuit 105 temporarily closes the shutter 103 under a predetermined condition. The predetermined condition is, for example, when the shutter 103 is temporarily closed to protect the infrared sensor 104 or when calibration is performed. In this case, for example, the camera control circuit 105 receives an instruction from the image processing device 11 to temporarily close the shutter 103.

[0030] Next, the image processing device 11 will be described with reference to Fig. 3. Fig. 3 is a block diagram of the image processing device according to the first embodiment. The image processing device 11 has, as its main components, a communication IF 120, a ROM 130, a RAM 140, a system control circuit 150, an image data acquisition unit 160, an image processing unit 170, an image recognition unit 180, and an image data output unit 190. These components are connected via a bus 110 so as to be able to communicate with each other as appropriate.

[0031] The communication IF 120 is an interface of a communication line for controlling the infrared camera 10. The communication IF 120 supplies the infrared camera 10 with a control signal for the image processing device 11 to control the infrared camera 10.

[0032] The communication IF 120 also includes a vehicle information acquisition unit 121. The vehicle information acquisition unit 121 is an interface for acquiring vehicle information from the ECU 13. When the vehicle information acquisition unit 121 in the communication IF 120 acquires the vehicle information from the ECU 13, it appropriately supplies the acquired vehicle information to the configuration of the image processing device 11. The vehicle information acquisition unit 121 may acquire information related to humidity around the vehicle 90 or information related to rainfall as vehicle information associated with the contrast of the thermal image captured by the infrared camera 10. More specifically, the vehicle information may be, for example, information related to rain detected by a raindrop sensor in the vehicle 90 or information related to fog detected by a humidity sensor. The vehicle information may also be information related to the operation of the wipers. The vehicle information may also be a signal indicating the lighting of the fog lamps. The vehicle information acquisition unit 121 detects the presence of water vapor or water droplets in the atmosphere around the vehicle 90 that reduces the contrast of the thermal image captured by the infrared camera 10, or detects the operation of a device required for the water vapor or water droplets. When the vehicle information acquisition unit 121 acquires the vehicle information described above, the vehicle information acquisition unit 121 supplies the acquired vehicle information to the system control circuit 150 .

[0033] The communication IF 120 may also be an interface for outputting a predetermined signal to the ECU 13. For example, the communication IF 120 may supply the ECU 13 with a signal for issuing a predetermined warning to the driver.

[0034] The ROM 130 (Read Only Memory) is a non-volatile memory that stores preset information or data. The ROM 130 stores in advance, for example, a program for the image processing device 11 to realize the functions according to the present embodiment.

[0035] The RAM 140 (RAM (Random Access Memory)) is a volatile memory having a storage area in which the image processing device 11 can temporarily expand data. The RAM 140 may be, for example, a DRAM (Dynamic Random Access Memory), and may include a register associated with the system control circuit 150 or the like. The RAM 140 includes an area in which the programs stored in the ROM 130 are expanded and executed. The RAM 140 may also be used, for example, when processing image data supplied from the infrared camera 10.

[0036] The system control circuit 150 includes an arithmetic unit such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The system control circuit 150 executes a program for enabling the image processing device 11 to perform the functions according to this embodiment. The program may be configured as software stored in the ROM 130, for example. The program may also be configured as a combination of software and hardware. The system control circuit 150 receives a predetermined signal from each component of the image processing device 11, and outputs a predetermined instruction or the like to each component in response to the received signal. The system control circuit 150 includes, as functional blocks, a determination unit 151 and an instruction unit 152.

[0037] The judgment unit 151 makes a predetermined judgment on the thermal image related to the image data obtained from the image processing device 11. For example, the judgment unit 151 judges whether to issue a warning when an object such as a pedestrian is included in the thermal image. In addition, for example, the judgment unit 151 may judge whether the shutter 103 of the infrared camera 10 is opened or closed.

[0038] The determination unit 151 also receives image analysis data from the image processing unit 170. The determination unit 151 acquires information associated with the contrast of the thermal image included in the received image analysis data, and determines whether or not to perform enhancement processing to enhance the contrast of the thermal image based on the acquired information. The information associated with the contrast of the thermal image may be, for example, a difference in pixel values ​​(brightness values) of the thermal image calculated by the image processing unit 170. In the following description, the difference in pixel values ​​of the thermal image is referred to as a contrast value. The contrast value of the thermal image may be calculated from brightness information of the entire thermal image, or may be a value for a partial area extracted from the entire thermal image.

[0039] The "enhancement process" in this embodiment refers to a process of increasing the contrast of the entire thermal image, and further includes a process of sharpening the tones of the image. By carrying out the enhancement process, the thermal image data has less blurred tones, improving the accuracy of object detection. For example, when the vehicle 90 is traveling in rainy or foggy conditions, the thermal image captured by the infrared camera 10 has a blurred tones throughout the image. In such a case, the image processing device 11 generates a converted image by carrying out enhancement process on the thermal image data.

[0040] The determination unit 151 determines to perform emphasis processing when the contrast value of the thermal image is less than a preset threshold value. The determination unit 151 may also monitor a change in the contrast value of the thermal image, and determine to perform emphasis processing when the change in the contrast value exceeds a preset threshold value.

[0041] The determination unit 151 may also use vehicle information as information associated with the contrast of the thermal image. For example, the determination unit 151 may obtain information regarding the operation of the wipers of the vehicle 90 via the vehicle information acquisition unit 121, and determine to perform the highlighting process when the wipers are operating. For example, the determination unit 151 may obtain information regarding the humidity around the vehicle 90 from the vehicle information acquisition unit 121, and determine to perform the highlighting process when the determination unit 151 obtains information indicating the occurrence of rain or fog.

[0042] The determination unit 151 may also use a combination of the contrast value of the thermal image and vehicle information as information associated with the contrast of the thermal image. For example, the determination unit 151 may determine to perform highlighting processing when the contrast value of the thermal image is less than a threshold value and the vehicle 90 operates the wipers. The determination unit 151 supplies the results of these determinations to the instruction unit 152.

[0043] The instruction unit 152 issues various instructions to each component of the image processing device 11. The instruction unit 152 receives information on the result of the judgment from the judgment unit 151, and issues instructions to each component according to the received information. For example, when the instruction unit 152 receives a signal from the judgment unit 151 to judge whether to issue a warning, the instruction unit 152 may output a signal indicating that a warning should be issued to the ECU 13 via the communication IF 120. The instruction unit 152 may also output an instruction to the infrared camera 10 regarding opening and closing of the shutter 103 via the communication IF 120. When the judgment unit 151 judges that emphasis processing should be performed, the instruction unit 152 supplies a signal instructing the image processing unit 170 to perform emphasis processing.

[0044] The image data acquisition unit 160 is an interface that acquires thermal image data (input thermal image data), which is data related to a thermal image, from the infrared camera 10. The image data acquisition unit 160 acquires image data periodically, for example, from the infrared camera 10. For example, the image data acquisition unit 160 receives one frame of an image every 1 / 15th of a second. When the image data acquisition unit 160 receives the image data, it supplies the received image data to the image processing unit 170.

[0045] The image processing unit 170 is an image processing circuit including, for example, a GPU (Graphics Processing Unit). The image processing unit 170 may be a processing circuit including an FPGA (field-programmable gate array) or a DSP (Digital Signal Processor). The image processing unit 170 performs predetermined processing on the thermal image data in cooperation with the RAM 140 to generate converted image data. The converted image data includes image data related to an image that has been subjected to enhancement processing for enhancing the contrast of the thermal image. The image processing unit 170 will be described in detail later.

[0046] The image recognition unit 180 receives image data from the image processing unit 170, and detects an object image related to a predetermined object such as a vehicle or a person from the received image data. More specifically, for example, the image recognition unit 180 has a recognition dictionary for detecting a predetermined object image, and detects the object image by referring to data in the recognition dictionary. When the image recognition unit 180 detects a predetermined object image, it generates information related to the position and size of the detected object image, and supplies the generated information to the system control circuit 150.

[0047] The image data output unit 190 is an interface for outputting the image data (converted image data) processed by the image processing unit 170 to the display 12. The image data output by the image data output unit 190 is output according to a data format corresponding to the specifications of the display 12. This data format is, for example, HDMI (High-Definition Multimedia Interface) (registered trademark) or DVI (Digital Visual Interface).

[0048] Furthermore, when the image processing unit 170 generates an image in which the contrast of the thermal image is enhanced (contrast-enhanced image), the image data output unit 190 outputs image data relating to this image (contrast-enhanced image data). The image data output unit 190 may output image data in which gamma correction or the like has been further performed on the contrast-enhanced image data.

[0049] Next, the image processing unit 170 will be described with reference to Fig. 4. Fig. 4 is a block diagram of the image processing unit 170 according to the first embodiment. The image processing unit 170 mainly includes a defective pixel correction unit 171, a NUC unit 172, and an image conversion processing unit 173.

[0050] The defective pixel correction unit 171 stores defective pixels of the infrared sensor 104 in advance, and performs a process (interpolation process) of interpolating the pixel value of the stored defective pixel from the pixel values ​​of surrounding pixels. The defective pixel correction unit 171 receives input thermal image data from the infrared camera 10 via the image data acquisition unit 160, and performs the above-mentioned interpolation process on the received image data. The defective pixel correction unit 171 supplies the image data after the interpolation process to the NUC unit 172.

[0051] The NUC unit 172 performs NUC (Non-Uniformity Correction), which is a calibration process for suppressing variations in pixel values ​​output in response to input light. The NUC unit 172 has in advance settings of gain and offset values ​​corresponding to the characteristics of each pixel of the infrared sensor 104. The NUC unit 172 calibrates each pixel value of the image data received from the defective pixel correction unit 171 according to the in advance settings.

[0052] The image conversion processing unit 173 receives the calibrated image data from the NUC unit 172, and performs a predetermined process on the received image data to generate converted image data. The converted image data is the image data received from the NUC unit 172 with improved contrast or resolution. By improving the contrast or resolution of the image data, the image processing device 11 can suppress a decrease in the recognition rate of objects. After generating the converted image data, the image conversion processing unit 173 supplies the generated converted image data to the image recognition unit 180 and the image data output unit 190.

[0053] The image analysis unit 175 analyzes the image data received from the NUC unit 172 and outputs image analysis data that is the analysis result. More specifically, the image analysis unit 175 calculates, for example, the contrast of an image based on the received image data and supplies the calculation result to the system control circuit 150.

[0054] When image data is transferred between the above-mentioned components or when image processing is performed in each component, image processing unit 170 may exchange image data with RAM 140. Alternatively, image processing unit 170 may realize functions according to the above-mentioned processing flow for image data stored in RAM 140.

[0055] Next, the process of determining whether or not to perform contrast enhancement processing will be described with reference to Fig. 5. Fig. 5 is a first flowchart of the image processing method according to the first embodiment. The process shown in Fig. 5 is started, for example, when input thermal image data is supplied from the infrared camera 10 to the image processing device 11. This process is repeatedly executed during the period when the input thermal image data is being supplied from the infrared camera 10.

[0056] First, the image data acquisition unit 160 of the image processing device 11 acquires thermal image data from the infrared camera 10 (step S1). The image data acquisition unit 160 supplies the acquired thermal image data to the image processing unit 170.

[0057] Next, the system control circuit 150 acquires information associated with the contrast of the thermal imaging data (step S2). More specifically, the system control circuit 150 acquires information associated with the contrast by the following procedure. First, when the image processing unit 170 receives the thermal imaging data from the image data acquisition unit 160, the defective pixel correction unit 171 corrects defective pixels, and then the NUC unit 172 calibrates the variation in pixel values. Then, the image analysis unit 175 analyzes the image data calibrated by the NUC unit 172 and calculates the contrast value of the thermal image. Furthermore, the image analysis unit 175 supplies image analysis data including the contrast value to the system control circuit 150.

[0058] Next, the determination unit 151 of the system control circuit 150 determines whether or not to execute contrast enhancement processing based on the acquired information (step S3). Specifically, the determination unit 151 compares the contrast value with a preset threshold, and determines to execute enhancement processing when the contrast value is less than the threshold. Alternatively, the determination unit 151 determines whether or not it is a foggy or rainy state. In this case (step S3: YES), the instruction unit 152 instructs the image processing unit 170 to execute contrast enhancement processing (step S4). Then, after the instruction unit 152 supplies a signal instructing the image processing unit 170 to execute enhancement processing, the system control circuit 150 ends the processing related to the contrast enhancement processing. Note that the image processing unit 170 that has received the instruction to execute enhancement processing executes a process to enhance contrast, which will be described later, to generate converted image data.

[0059] On the other hand, the determination unit 151 compares the contrast value with a preset threshold value, and if the contrast value is equal to or greater than the threshold value, does not determine that enhancement processing should be executed. In this case (step S3: NO), the system control circuit 150 ends the process without issuing an instruction to execute contrast enhancement processing.

[0060] Next, the process of generating converted image data executed by the image processing device 11 will be described with reference to Fig. 6. Fig. 6 is a second flowchart of the image processing method according to the first embodiment. The flowchart shown in Fig. 6 is started, for example, when input thermal image data is supplied from the infrared camera 10 to the image processing device 11 and an instruction signal to execute emphasis processing is received from the instruction unit 152.

[0061] First, the image data acquisition unit 160 of the image processing device 11 acquires thermal image data from the infrared camera 10 (step S10). When the image data acquisition unit 160 acquires the thermal image data, it supplies the acquired thermal image data to the image processing unit 170.

[0062] Next, when the image processing unit 170 receives the thermal imaging data, it sets a local area including a plurality of first local areas and a plurality of second local areas that include the first local areas and are larger than the first local areas in the thermal image (step S11).

[0063] Next, the image processing unit 170 calculates a frequency distribution (histogram) regarding the pixel values ​​of the pixels that configure the first local region and the second local region (step S12).

[0064] Next, image processing unit 170 generates redistributed data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution to the frequency distribution (step S13). Note that image processing unit 170 sets a first threshold to be applied to the first local region and a second threshold to be applied to the second local region independently. In other words, the first threshold and the second threshold may be the same or different.

[0065] Next, the image processing unit 170 generates a local tone curve from the redistribution data (step S14). Specifically, the image processing unit 170 generates a local tone curve by cumulatively adding the frequencies of the redistribution data from low pixel values ​​toward successively higher pixel values.

[0066] Next, the image processing unit 170 generates a blend tone curve (step S15). Specifically, the image processing unit 170 synthesizes a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a preset weighting coefficient. The image processing unit 170 normalizes the generated blend tone curve by the width between the maximum and minimum pixel values ​​of the thermal imaging data.

[0067] Next, the image processing unit 170 generates converted image data by applying a blend tone curve to pixels corresponding to each local region (step S16). At this time, the image processing unit 170 performs processing, for example, according to the following formula (1).

number

[0068] In addition, in the enhancement process, the image processing unit 170 may set the weighting coefficient W2 to a negative value. By setting the weighting of the tone curve of the second local region, which is larger than the first local region, to a negative value, the image processing device 11 reduces the blurred tones that occur over a relatively wide range.

[0069] In addition, in the enhancement process, when the weighting coefficient W2 is set to a negative value, the image processing unit 170 may set the weighting coefficient W1 to a relatively large value. By setting the weighting of the tone curve of the first local region, which is smaller than the second local region, to a relatively large value, the image processing device 11 increases the contrast in a relatively narrow range. Through the above-mentioned process, the image processing device 11 converts the thermal image into a relatively sharp image.

[0070] Next, the image data output unit 190 outputs the converted image data (step S17). When the image data output unit 190 outputs the converted image data, the image processing device 11 ends the series of processes. When new thermal imaging data is supplied, the image processing device 11 executes the above-mentioned processes again from step S10.

[0071] The above describes the image processing method executed by the image processing device 11. By executing the above information, the image processing device 11 reduces the influence of radiant heat and suppresses a decrease in contrast of the entire screen.

[0072] Next, a specific example of a local region will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of a local region according to the first embodiment. Fig. 7 shows a thermal image D11. In addition, in the thermal image D11, the upper left corner is set as the origin, the H direction is set to the right, and the V direction is set to the lower side. In the following description, the coordinates of the H direction and the V direction in the thermal image D11 will be indicated in parentheses as (h, v).

[0073] The image processing unit 170 sets a plurality of first local regions R1 and second local regions R2 in the thermal image D11. For example, the image processing unit 170 sets the first local region R1(0,0) at a position corresponding to the coordinates (0,0) of the image D11. The image processing unit 170 also sets the first local regions R1(h,v) at arbitrary positions (h,v) so that they are adjacent to each other. That is, as shown by the two-dot chain line in the figure, the first local region R1(h+1,v) is set adjacent to the right side of the first local region R1(h,v), and the first local region R1(h,v+1) is set adjacent to the lower side of the first local region R1(h,v). In this way, the image processing unit 170 sets m first local regions R1 in the H direction and n first local regions R1 in the V direction up to the first local region R1(m-1,n-1) in the lower right of the thermal image D11.

[0074] The image processor 170 also sets second local regions R2(h,v) corresponding to the respective first local regions R1. The second local region R2(h,v) corresponding to the coordinates (h,v) is a rectangular region including the first local region R1(h,v) and larger than the first local region R1(h,v). Note that the second local region R2 may extend outside the thermal image D11 at the outer edge of the thermal image D11, as shown at the coordinates (0,0) and (m,n) in the figure. In this case, the size of the second local region R2 is cut according to the size of the thermal image D11.

[0075] In this way, the image processing unit 170 sets the first local region R1(h,v) and the second local region R2(h,v) as local regions corresponding to the coordinates (h,v). Then, the image processing unit 170 generates a local tone curve and a blend tone curve for each of the set local regions.

[0076] Next, a specific example of a local tone curve will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of processing for generating a local tone curve. In Fig. 8, data D12 to D15 are shown in order from the top.

[0077] Data D12 shown at the top of Fig. 8 is a frequency distribution in a predetermined local region. In the image processing device 11, the image conversion processing unit 173 of the image processing unit 170 sets a local region from the thermal imaging data received from the NUC unit 172, and calculates a frequency distribution as shown in data D12 for the set local region. In data D12, the horizontal axis indicates pixel value, and the vertical axis indicates frequency corresponding to the pixel value. The frequency distribution shown in data D12 is indicated by a curve having a frequency peak in the center.

[0078] 8 shows excess data D131 that exceeds a predetermined threshold value with respect to the frequency distribution, which is hatched. In order to redistribute the excess data D131 that exceeds a preset threshold value, the image conversion processing unit 173 extracts the excess data D131 and redistributes the frequency of the excess data D131 with respect to the frequency distribution.

[0079] The third data D14 from the top in Fig. 8 is redistributed data. The image conversion processing unit 173 distributes the frequency corresponding to the excess data D131 to pixel value data indicating the pixel value distribution between the minimum and maximum values ​​of the thermal imaging data within a range not exceeding a threshold. In the example shown in the figure, the image conversion processing unit 173 generates redistributed data by distributing the excess data D131 to data D141 on the left side and data D142 on the right side.

[0080] The data D15 shown in the fourth row (bottom) of Fig. 8 is data showing a local tone curve. The data D15 is a cumulative frequency distribution in which the redistribution data generated in the data D14 is sequentially accumulated from the minimum pixel value to the maximum pixel value. The image conversion processing unit 173 generates local tone curves in the first local region R1(h,v) and the second local region R2(h,v) by the above-mentioned procedure.

[0081] Next, the blend tone curve will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of a blend tone curve. Fig. 9 shows a first local tone curve D16, a second local tone curve D17, and a blend tone curve D18.

[0082] The first local tone curve D16 is a local tone curve of a first local region at a predetermined position of the thermal image data. The second local tone curve D17 is a second local tone curve corresponding to the first local tone curve D16. That is, the second local region of the second local tone curve D17 includes the first local region of the first local tone curve D16.

[0083] The blend tone curve D18 is generated by combining the first local tone curve D16 and the second local tone curve according to the above-mentioned formula (1). The blend tone curve D18 is generated by combining the first local tone curve D16 and the second local tone curve D17, and then normalizing the result using the maximum pixel value of the thermal imaging data. In other words, the blend tone curve D18 has a relationship in which the horizontal axis represents the input pixel value and the vertical axis represents the output pixel value. As a result, the image conversion processing unit 173 generates image conversion data at each pixel position corresponding to the blend tone curve D18.

[0084] The first embodiment has been described above. As described above, the image processing device 11 generates local tone curves for the first local region and the second local region, and combines them after setting weights. As a result, the image processing device 11 suppresses blurred tones that are noticeable in a relatively wide range in the thermal image and expands the contrast seen in a relatively narrow range. The image processing device 11 improves the contrast of the image by simple processing, and can suppress a decrease in the accuracy of detecting an object under conditions such as rainy weather and fog. Therefore, according to the first embodiment, it is possible to provide an image processing device, an image processing method, and a program that improve the contrast of an image by simple processing. Also, by improving the contrast of the entire screen, it is possible to improve the recognition rate of an object based on a thermal image.

[0085] Although the image processing device 11 described above sets two types of local areas, the first local area and the second local area, the setting of the local areas is not limited to this. That is, the image processing device 11 sets a plurality of local areas of different sizes. In this case, the image processing device 11 may set three or more types of local areas of different sizes, and when generating a blend tone curve, synthesize local tone curves corresponding to these three or more types of areas.

[0086] <Embodiment 2> Next, a second embodiment will be described. Fig. 10 is a flowchart of an image processing method according to the second embodiment. The flowchart shown in Fig. 10 shows the processing mainly performed by the image conversion processing unit 173. The flowchart shown in Fig. 10 is started, for example, when input thermal image data is supplied from the infrared camera 10 to the image processing device 11 and an instruction signal to execute emphasis processing is received from the instruction unit 152.

[0087] First, the image conversion processing unit 173 acquires the calibrated thermal imaging data from the NUC unit 172 (step S101).

[0088] Next, the image conversion processing unit 173 calculates the frequency distribution of the thermal imaging data (step S102). Here, when the pixel values ​​of the thermal imaging data are, for example, about 16 bits, the image conversion processing unit 173 according to this embodiment calculates the frequency distribution by grouping about the lowest 2 bits to 4 bits into one bin (also called a class). This allows the image processing device 11 to reduce the processing time required for image conversion while suppressing a substantial decrease in calculation accuracy.

[0089] Next, the image conversion processing unit 173 performs range calculation of the thermal imaging data (step S103). In the range calculation, the image conversion processing unit 173 sets a minimum pixel value on the low temperature side and a maximum pixel value on the high temperature side in the frequency distribution. Furthermore, the image conversion processing unit 173 sets a central pixel value. Then, the image conversion processing unit 173 calculates a first pixel range which is the difference between the maximum pixel value and the central pixel value. Similarly, the image conversion processing unit 173 calculates a second pixel range which is the difference between the central pixel value and the minimum pixel value. Then, the image conversion processing unit 173 compares the first pixel range with the second pixel range, and applies the larger pixel range to the smaller pixel range to set the pixel range of the entire image.

[0090] An example of range calculation will be described with reference to FIG. 11. FIG. 11 is a frequency distribution diagram of thermal imaging data. Data D22 shown in FIG. 11 shows an example of thermal imaging data. In the data D22, the image conversion processing unit 173 sets the bins after cutting off several percent (for example, 1%) above and below the frequency as the minimum pixel value (MIN) and the maximum pixel value (MAX), respectively. By performing such processing, the image processing device 11 can process data by excluding data that is significantly out of the range due to an error or the like when such data exists in the image data. The image conversion processing unit 173 also sets a central value. The median (MEDIAN) or average value of the frequency distribution can be selected as the central value. In the case of the example shown in the data D22, the image conversion processing unit 173 sets the median value as the central value.

[0091] In the image data shown in data D22, the first pixel range R11, which is the difference between MAX and MEDIAN, is larger than the second pixel range R12, which is the difference between MEDIAN and MIN. Therefore, the image conversion processing unit 173 sets the range of pixel values ​​on the low temperature side to MIN2, which is away from MEDIAN by the first pixel range R11. By setting the pixel range in this way, the image processing device 11 improves the contrast after conversion.

[0092] Returning to FIG. 10, the description will continue. Image conversion processing unit 173 performs range restriction (step S104). In the range restriction, image conversion processing unit 173 changes the setting of data that exists outside the range between the set minimum pixel value and maximum pixel value to a bin closer to either the minimum pixel value or the maximum pixel value. In the example shown in FIG. 11, if there is data that is below MIN2 in the frequency distribution, image conversion processing unit 173 places this data in the bin corresponding to MIN2. Similarly, if there is data that is above MAX in the frequency distribution, image conversion processing unit 173 places this data in the bin corresponding to MAX.

[0093] Next, the image conversion processing unit 173 divides the thermal image into unit areas (step S105). A unit area is an area that is an element for constituting a local area for generating a local tone curve. That is, the image conversion processing unit 173 sets each local area as a collection of unit areas. By dividing the thermal image into unit areas, the image processing device 11 can efficiently perform processing on a plurality of local areas of different sizes.

[0094] The unit area generated by the image conversion processing unit 173 will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of the unit area. Fig. 12 shows a thermal image D23 and a unit area U10.

[0095] The thermal image D23 is a thermal image based on the input thermal imaging data received by the image conversion processing unit 173. In the example shown in Fig. 12, the image conversion processing unit 173 divides the thermal image D23 into 64 parts horizontally and 48 parts vertically. The unit area U10 is one area when the thermal image D23 is divided as described above. In the thermal image D23 shown in Fig. 12, 64 unit areas U10 are set in the horizontal direction (H-axis direction) with the upper left corner as the origin, and 48 unit areas U10 are set in the vertical direction (V-axis direction). In Fig. 12, one unit area U10 is indicated by hatching in the upper left corner of the thermal image D23.

[0096] The image conversion processing unit 173 sets a unit area U10(h,v) at an arbitrary coordinate (h,v) when the upper left corner is the coordinate (0,0) of the origin and the lower right corner is the coordinate (63,47). The image conversion processing unit 173 divides the thermal image D23 into 64x48 in this way, thereby setting 3072 unit areas U10. For example, if the thermal image D23 is QVGA (Quarter Video Graphics Array) and has a pixel count of 320x240 pixels, the unit area is 5x5 pixels.

[0097] Returning to Fig. 10, the description will continue. Next, image conversion processing unit 173 calculates a unit frequency distribution, which is a frequency distribution of a unit area (step S106). That is, in the case of the example shown in Fig. 12, image conversion processing unit 173 calculates the frequency distribution for each of 3072 unit areas. At this time, image conversion processing unit 173 performs the calculation so that the unit frequency distribution falls within the range limit described above.

[0098] Next, the image conversion processing unit 173 sets a first local region (step S110). The image conversion processing unit 173 also sets a second local region (step S120). The image conversion processing unit 173 further sets a third local region (step S130).

[0099] The local region will be described with reference to Fig. 13. Fig. 13 is a diagram showing an example of a local region according to embodiment 2. Fig. 12 shows a 5×5 unit region U10 and a first local region U11, a second local region U12, and a third local region U13 superimposed thereon.

[0100] The first local area U11 is composed of one unit area U10. The first local area U11 shown in the figure corresponds to the unit area U10 of coordinates (h, v) located at the center of the unit area U10 arranged in a 5×5 matrix. The second local area U12 is composed of a 3×3 unit area U10. The second local area U12 corresponds to the unit area U10(h, v) at the center of the unit area U10 arranged in a 5×5 matrix, and includes the first local area U11(h, v). The third local area U13 is composed of a 5×5 unit area U10. The third local area U13 includes the first local area U11(h, v) and the second local area U12(h, v) at the center. That is, the third local area U13 corresponds to the coordinates (h, v).

[0101] In the process of setting the first local region, the second local region, and the third local region, the image conversion processing unit 173 calculates the frequency distribution for each of the local regions. At this time, the image conversion processing unit 173 uses the unit frequency distribution. That is, when calculating the frequency distribution for each of the local regions, the image conversion processing unit 173 adds up the unit frequency distributions in the unit regions that each of the local regions has. This allows the image conversion processing unit 173 to efficiently calculate the frequency distribution for each local region.

[0102] Returning to FIG. 10, the description will be continued. After setting the local regions, the image conversion processing unit 173 generates redistribution data for each local region (steps S112, S122, and S132). The threshold value for the excess data set when generating the redistribution data can be set individually for each local region. The redistribution process performed here is similar to a method generally known as CLAHE (Contrast Limited Adaptive Histogram Equalization). When generating the redistribution data, the image conversion processing unit 173 uses the center value set in step S103. That is, the image conversion processing unit 173 distributes the redistribution data equally to the side with pixel values ​​higher than the center value and the side with pixel values ​​lower than the center value, respectively, using the center value as a reference.

[0103] Next, the image conversion processing unit 173 generates a first local tone curve corresponding to the first local region (step S113). The image conversion processing unit 173 also generates a second local tone curve corresponding to the second local region (step S123). Similarly, the image conversion processing unit 173 generates a third local tone curve corresponding to the third local region (step S133). Note that the process of generating the redistribution data and the process of generating the local tone curves corresponding to each local region may be performed in parallel or sequentially. Note that the image conversion processing unit 173 normalizes each of the generated local tone curves by dividing them by the total frequency in each region.

[0104] Next, the image conversion processing unit 173 generates a blend tone curve by synthesizing the generated local tone curves (Step S140). The image conversion processing unit 173 generates the blend tone curve so that the following formula (2) is satisfied.

number

[0105] The concept of the above formula (2) will be described with reference to Fig. 14. Fig. 14 is a diagram showing the relationship between the local tone curve and the blend tone curve according to the second embodiment. Fig. 14 shows each term corresponding to formula (2) and the flow of calculation.

[0106] First, the first local tone curve LTa(i) is multiplied by the weighting coefficient Wa (step S201). Next, the second local tone curve LTb(i) is multiplied by the weighting coefficient Wb (step S202). Furthermore, the third local tone curve LTc(i) is multiplied by the weighting coefficient Wc (step S203). Then, these three terms and the offset value F2 are added together (step S204) to calculate the blend tone curve BT2(i).

[0107] In the above calculation, in order to effectively perform the enhancement process, it is preferable to subtract the tone characteristics of the third local tone curve for the larger area of ​​the local area. That is, it is preferable that the weighting coefficient Wc takes a negative value, and the weighting coefficients Wa and Wb are set to positive values ​​correspondingly, and the output pixel value of the blend tone curve BT2 is set to a suitable value. Through the above process, the image processing device 11 reduces the blurred tones of the thermal image and converts the thermal image into a relatively sharp image.

[0108] FIG. 15 shows examples of a local tone curve and a blend tone curve. FIG. 15 is a diagram showing examples of a local tone curve and a blend tone curve according to the second embodiment. In the graph shown in FIG. 15, a first local tone curve, a second local tone curve, a third local tone curve, and a blend tone curve are superimposed. As shown in the figure, each local tone curve has its own slope according to the size of the local area. A weighting coefficient and an offset are set for the blend tone curve so that the image processing device 11 can perform the desired function.

[0109] For example, in the example of Fig. 15, the slopes of the local tone curves in the low pixel value range are steep, and exist in different pixel value ranges (i.e., different temperature bands). On the other hand, the slope of the blend tone curve in the low pixel value range is a relatively gentle curve. In this way, the image processing device 11 suppresses the variation in contrast that occurs in local areas of different sizes, and expands the contrast of the temperature band of the object to be detected, by synthesizing the features of the tone curves of each local area.

[0110] Next, the image conversion processing unit 173 performs tone conversion of the thermal imaging data using the generated blend tone curve (step S141). As described above, when calculating the frequency distribution of the thermal imaging data, the image conversion processing unit 173 calculates and processes a frequency distribution in which the lower 2 to 4 bits are grouped together as one bin. In other words, the image conversion processing unit 173 performs processing similar to extracting designated pixels such that each pixel in the local region is spaced apart from each other by several bits, and applying the blend tone curve to the extracted designated pixels. Therefore, in the subsequent processing, processing is performed to accommodate all pixels in the input thermal imaging data.

[0111] Next, the image conversion processing unit 173 performs image data interpolation processing (step S142). Specifically, the image conversion processing unit 173 interpolates pixel values ​​of pixels existing between the designated pixels set in this step S141 using converted pixel values ​​of the surrounding designated pixels. An example of the interpolation method is bilinear interpolation. In this way, the image conversion processing unit 173 performs tone conversion on all pixels of the thermal imaging data. That is, the image conversion processing unit 173 generates converted image data by converting the input thermal imaging data.

[0112] Next, the image conversion processing unit 173 outputs the generated converted image data (step S150). The image processing unit 170 supplies the converted image data output by the image conversion processing unit 173 to, for example, the image recognition unit 180. As a result, the image processing device 11 detects an object using the converted image data. The image processing unit 170 also appropriately performs processing for displaying the converted image data output by the image conversion processing unit 173 on the display 12. For example, the image processing unit 170 performs gamma correction on the converted image data, and supplies the gamma-corrected image data to the display 12 via the image data output unit 190. When the image conversion processing unit 173 outputs the converted image data, it ends the series of processes.

[0113] The image processing method according to the second embodiment has been described above. With the above-mentioned method, the image processing device 11 can improve the contrast of an image by simple processing, and suppress a decrease in the accuracy of detecting an object under conditions such as rainy weather and fog. As described in steps S141 and S142, the image processing device 11 can improve the contrast of an image and increase the processing speed of image processing by extracting and processing the designated pixels.

[0114] Next, a modified example of the embodiment 2 will be described. Fig. 16 is a flowchart of an image processing method according to the modified example of the embodiment 2. The flowchart shown in Fig. 16 differs from the flowchart shown in Fig. 10 in the processing between step S140 and step S150.

[0115] After generating the blend tone curve in step S140 in FIG. 16, the image conversion processing unit 173 performs tone curve interpolation (step S143). As described above, when calculating the frequency distribution of the thermal imaging data, the image conversion processing unit 173 calculates and processes the frequency distribution in which the lower 2 to 4 bits are grouped into one bin. Therefore, the blend tone curve in step S140 does not correspond to all pixels of the thermal imaging data. Therefore, the image conversion processing unit 173 generates tone curves for pixels existing between designated pixels by interpolating from blend tone curves for surrounding designated pixels. A specific method of interpolation is, for example, bilinear interpolation. By performing tone curve interpolation, the image conversion processing unit 173 generates tone curves that correspond to all pixels of the thermal imaging data.

[0116] Next, the image conversion processing unit 173 performs tone conversion on the thermal imaging data using the generated tone curve (step S144). The image conversion processing unit 173 performs a process of converting input pixel values ​​and calculating output pixel values ​​for all pixels of the thermal imaging data. In this way, the image conversion processing unit 173 generates converted image data.

[0117] The second embodiment has been described above. According to the second embodiment, it is possible to provide an image processing device, an image processing method, and a program that improve the contrast of an image by simple processing. Also, by improving the contrast of the entire screen, it is possible to improve the recognition rate of an object based on a thermal image.

[0118] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0119] 10. Infrared Camera 11 Image processing device 12 Display 13 ECU 90 vehicles 101 Case 102 Objective Lens 103 Shutter 104 Infrared Sensor 105 Camera control circuit 110 Bus 120 Communication Interface 121 Vehicle Information Acquisition Department 130 ROM 140 RAM 150 System control circuit 151 Judgment Department 152 Instruction section 160 Image data acquisition unit 170 Image Processing Unit 171 Defective pixel correction section 172 NUC Department 173 Image conversion processing section 175 Image Analysis Unit 180 Image Recognition Unit 190 Image data output section

Claims

1. an image data acquisition unit that acquires thermal image data from an infrared camera that captures a thermal image of the periphery of the vehicle; a determination unit that determines whether or not to perform an enhancement process for enhancing the contrast of the thermal image based on information associated with the contrast of the thermal image; an image processing unit that generates contrast-enhanced image data from the thermal imaging data based on the determination; an output unit that outputs the contrast-enhanced image data, The image processing unit includes: setting a local region including a plurality of first local regions and a plurality of second local regions including the first local regions and larger than the first local regions on the thermal image; Calculating a frequency distribution regarding pixel values ​​of pixels constituting the first local region and the second local region; generating redistributed data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution into the frequency distribution; generating a local tone curve by accumulating the redistributed data from low pixel values ​​toward high pixel values; generating a blended tone curve by combining a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a weighting coefficient that is set in advance for each of the first local tone curve and the second local tone curve; generating the contrast-enhanced image data by applying the blend tone curve to pixels corresponding to the local region; Image processing device.

2. The determination unit makes the determination to perform the enhancement process when a difference in pixel values ​​of the thermal image as information associated with the contrast is less than a preset threshold value. The image processing device according to claim 1 .

3. A vehicle information acquisition unit that acquires vehicle information related to humidity or rainfall around the vehicle as information associated with the contrast, The determination unit makes the determination based on the vehicle information.

3. The image processing device according to claim 1 or 2.

4. The computer Acquire thermal image data from an infrared camera that captures thermal images of the area around the vehicle; determining whether or not to perform an enhancement process for enhancing the contrast based on information associated with the contrast of the thermal image; Based on the above judgment, setting a local region including a plurality of first local regions and a plurality of second local regions including the first local regions and larger than the first local regions on the thermal image; Calculating a frequency distribution regarding pixel values ​​of pixels constituting the first local region and the second local region; generating redistributed data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution into the frequency distribution; generating a local tone curve by accumulating the redistributed data from low pixel values ​​toward high pixel values; generating a blended tone curve by combining a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a weighting coefficient that is set in advance for each of the first local tone curve and the second local tone curve; generating contrast-enhanced image data by applying the blend tone curve to pixels corresponding to the local region; outputting the contrast-enhanced image data; Image processing methods.

5. Acquire thermal image data from an infrared camera that captures thermal images of the area around the vehicle; determining whether or not to perform an enhancement process for enhancing the contrast based on information associated with the contrast of the thermal image; Based on the above judgment, setting a local region including a plurality of first local regions and a plurality of second local regions including the first local regions and larger than the first local regions on the thermal image; Calculating a frequency distribution regarding pixel values ​​of pixels constituting the first local region and the second local region; generating redistributed data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution into the frequency distribution; generating a local tone curve by accumulating the redistributed data from low pixel values ​​toward high pixel values; generating a blended tone curve by combining a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a weighting coefficient that is set in advance for each of the first local tone curve and the second local tone curve; generating contrast-enhanced image data by applying the blend tone curve to pixels corresponding to the local region; outputting the contrast-enhanced image data; A method for causing a computer to execute an image processing method program.

Images