Image processing device, image processing method, and program

Abstract

To provide an image processing apparatus or the like, which reduces the influence of radiation heat and suppresses reduction in overall contrast of a screen.SOLUTION: In an image processing apparatus 11, an image data acquisition unit 160 acquires thermal image data from an infrared camera. An image processing unit 170 divides the thermal image into reference sections, sets a reference central pixel value for a predetermined pixel position, based on pixel values of the reference sections, sets local regions, and sets distribution ratio in the local regions based on the reference central pixel value. The image processing unit 170 generates redistribution data based on the distribution ratio and generates local tone curve. The image processing unit 170 generates blend tone curve by synthesizing local tone curves corresponding to the multiple local regions, using a predetermined weighting factor. The image processing unit 170 generates conversion image data by applying the blend tone curve. An image data output unit 190 outputs the conversion image data.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an image processing apparatus, an image processing method, and a program. [Background technology]

[0002] Many object detection systems have been developed that use infrared cameras to detect the heat of objects. It is desirable for such object detection systems to be able to effectively detect the heat of objects such as people and animals in a variety of environments.

[0003] Many object detection systems have been developed that use infrared cameras to detect the heat of objects. It is desirable for such object detection systems to be able to detect the heat of objects such as people and animals. For this reason, technologies to improve the dynamic range of images and the gradation of pixel values ​​have also been developed.

[0004] For example, a technique has been disclosed that converts the histogram of an image in accordance with environmental data acquired from a predetermined sensor (Patent Document 1).

[0005] For example, a technique has been disclosed that identifies an area in an image that may correspond to the sky based on the position of the horizon in the image, and analyzes the distribution of irradiance levels in the identified sky area to determine the dynamic range attributable to the sky area (Patent Document 2). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] US Patent No. 08077995 [Patent Document 2] US Patent No. 09819880 Summary of the Invention [Problem to be solved by the invention]

[0007] However, when an infrared camera has stray light due to the effect of radiant heat emitted from internal parts such as the lens barrel, the contrast of the object to be detected may be reduced, which may result in a reduction in the recognition rate of the object based on the thermal image.

[0008] The present invention has been made to solve such problems, and provides an image processing device and the like that reduces the effects of radiant heat and suppresses a decrease in the contrast of a detection object. [Means for solving the problem]

[0009] The image processing device according to the present invention includes an image data acquisition unit, an image processing unit, and an image data output unit. The image data acquisition unit acquires thermal image data from an infrared camera that captures a thermal image of the periphery of a vehicle. The image processing unit generates converted image data from the thermal image data. The image data output unit outputs the converted image data. The image processing unit divides the thermal image into predetermined reference sections, and sets a reference center pixel value for each pixel position based on statistical data of pixel values ​​of each of the reference sections. The image processing unit sets local areas including a plurality of first local areas and a plurality of second local areas that include the first local areas and are larger than the first local areas, for the thermal image. The image processing unit calculates a frequency distribution regarding pixel values ​​of pixels that constitute the first local area and the second local area. The image processing unit sets a distribution ratio in the local area based on the reference center pixel value. The image processing unit generates redistribution data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution to the frequency distribution based on the distribution ratio. The image processing unit cumulatively adds the redistribution data from low pixel values ​​toward high pixel values ​​to generate a local tone curve. The image processing unit generates a blended tone curve by combining a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a weighting coefficient set in advance for each of them. The image processing unit generates converted image data by applying the blended tone curve to pixels corresponding to the local regions.

[0010] In the image processing method according to the present invention, a computer executes the following steps. The computer acquires thermal image data from an infrared camera that captures a thermal image of the periphery of a vehicle. The computer divides the thermal image into predetermined reference sections. The computer sets a reference center pixel value for each pixel position based on statistical data of pixel values ​​of each of the reference sections. The computer sets local regions including a plurality of first local regions and a plurality of second local regions that include the first local regions and are larger than the first local regions for the thermal image. The computer calculates a frequency distribution regarding pixel values ​​of pixels that constitute the first local region and the second local region. The image processing unit sets a distribution ratio in the local region based on the reference center pixel value. The computer generates redistribution data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution to the frequency distribution based on the distribution ratio. The computer accumulates and adds the redistribution data from low pixel values ​​toward high pixel values ​​to generate a local tone curve. The computer generates a blended tone curve by combining a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a weighting coefficient set in advance for each of them. The computer generates transformed image data by applying the blended tone curve to pixels corresponding to the first local region. The computer outputs the transformed image data.

[0011] The program according to the present invention causes a computer to execute the following image processing method. In the image processing method, the computer executes the following steps. The computer acquires thermal image data from an infrared camera that captures a thermal image of the periphery of a vehicle. The computer divides the thermal image into predetermined reference sections. The computer sets a reference center pixel value for each pixel position based on statistical data of pixel values ​​of each of the reference sections. The computer sets local areas including a plurality of first local areas and a plurality of second local areas that include the first local areas and are larger than the first local areas for the thermal image. The computer calculates a frequency distribution regarding pixel values ​​of pixels that constitute the first local area and the second local area. The image processing unit sets a distribution ratio in the local area based on the reference center pixel value. The computer generates redistribution data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution to the frequency distribution based on the distribution ratio. The computer accumulates and adds the redistribution data from low pixel values ​​toward high pixel values ​​to generate a local tone curve. The computer generates a blended tone curve by combining a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a weighting coefficient set in advance for each of them. The computer generates transformed image data by applying the blended tone curve to pixels corresponding to the first local region. The computer outputs the transformed image data. Effect of the Invention

[0012] According to the present invention, it is possible to provide an image processing device, an image processing method, and a program that reduce the influence of radiant heat and suppress a decrease in the contrast of a detection object. [Brief description of the drawings]

[0013] [Figure 1] 1 is a configuration diagram of a vehicle equipped with an image processing device according to a first embodiment. [Diagram 2] FIG. 1 is a diagram illustrating the configuration of an infrared camera. [Diagram 3] 1 is a block diagram of an image processing device according to a first embodiment. [Figure 4] FIG. 2 is a block diagram of an image processing unit according to the first embodiment. [Diagram 5] 1 is a flowchart of an image processing method according to a first embodiment. [Figure 6] FIG. 13 is a diagram showing an example of a reference partition and a reference central pixel value. [Figure 7] FIG. 2 is a diagram illustrating an example of a local region according to the first embodiment. [Figure 8] FIG. 11 illustrates an example of a process for generating excess data. [Figure 9] FIG. 11 is a diagram showing a first example of a distribution ratio. [Figure 10] 11A to 11C are diagrams illustrating an example of a process for generating redistribution data and a local tone curve. [Figure 11] FIG. 13 is a diagram showing an example of a blend tone curve. [Figure 12] FIG. 13 is a diagram showing a second example of the distribution ratio. [Figure 13] 11 is a flowchart of an image processing method according to a second embodiment. [Figure 14] FIG. 11 is a frequency distribution diagram of thermal imaging data. [Figure 15] FIG. 13 is a diagram illustrating an example of a unit region. [Figure 16] FIG. 11 is a diagram illustrating an example of a local region according to the second embodiment. [Figure 17] 13A to 13C are diagrams illustrating examples of a local tone curve and a blend tone curve according to the second embodiment. [Figure 18] 13 is a flowchart of an image processing method according to a modified example of the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The present invention will be described below through the embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. In addition, not all of the configurations described in the embodiments are necessarily essential as means for solving the problems. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. In each drawing, the same elements are given the same reference numerals, and duplicate explanations are omitted as necessary.

[0015] <Embodiment 1> Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram of a vehicle equipped with an image processing device according to the first embodiment. The image processing device according to the present embodiment receives a thermal image from an infrared camera provided in the vehicle, performs a predetermined process, and displays the received thermal image on a display. The image processing device according to the present embodiment also has a function of detecting a predetermined object from the thermal image received from the infrared camera. For example, when the image processing device detects an animal or a person in front of the vehicle, it issues a warning to the driver. Therefore, the image processing device processes the thermal image in order to detect the predetermined object. The vehicle 90 shown in FIG. 1 includes an infrared camera 10, an image processing device 11, a display 12, and an ECU 13.

[0016] The infrared camera 10 is provided on the vehicle 90 so as to capture an image of the periphery of the vehicle 90. The infrared camera 10 is fixed, for example, to the front of the vehicle 90 and captures an image of the area in front of the vehicle 90. The infrared camera 10 is communicably connected to an image processing device 11, receives a predetermined instruction signal from the image processing device 11, and operates according to the received instruction signal. The infrared camera 10 also supplies the image processing device 11 with image data (also called thermal image data) related to the image captured by the infrared camera 10 (also called thermal image).

[0017] The image processing device 11 is fixed at an arbitrary location of the vehicle 90, and is communicatively connected to each of the infrared camera 10 and the display 12. The image processing device 11 acquires image data generated by the infrared camera 10, and outputs the acquired image data to the display 12 for display. The image processing device 11 also receives predetermined information (vehicle information) from the ECU 13, and operates according to the received vehicle information. For example, when the image processing device 11 receives vehicle information indicating that the headlights are turned on from the ECU 13, it starts imaging by the infrared camera 10 in conjunction with this information. That is, the image processing device 11 according to this embodiment opens the shutter 103 of the infrared camera 10 and acquires image data from the infrared sensor 104 in a dark place where the vehicle 90 turns on its headlights, such as at night or in a tunnel.

[0018] The display 12 is a display device including, for example, a liquid crystal panel or an organic electroluminescence panel, and is provided at a position visible to the driver of the vehicle 90. The display 12 displays an image captured by the infrared camera 10 via the image processing device 11.

[0019] An ECU 13 (Electronic Control Unit) has data related to a predetermined function of the vehicle and controls the function. For example, the ECU 13 controls turning on and off the headlights of the vehicle 90. In this case, the ECU 13 may supply vehicle information indicating that the headlights are turned on to the image processing device 11. Note that the ECU 13 may supply, for example, a signal from an illuminance sensor of the vehicle 90 to the image processing device 11 instead of the above-mentioned information. The ECU 13 may supply other preset information to the image processing device 11. The ECU 13 may also receive predetermined information from the image processing device 11. For example, when the ECU 13 receives a warning signal from the image processing device 11, the ECU 13 may use a speaker or a display device of the vehicle 90 to emit a sound or display to warn the driver in accordance with the received warning signal.

[0020] With the above-mentioned configuration, the image processing device 11 performs a predetermined process on the image captured by the infrared camera 10, and displays the image on the display 12 in a manner that is visible to the driver. In this way, the image processing device 11 allows the driver to recognize objects around the vehicle 90.

[0021] Next, the configuration of the infrared camera 10 will be described with reference to Fig. 2. Fig. 2 is a configuration diagram of the infrared camera. The infrared camera 10 shown in Fig. 2 has, as its main components, a housing 101, an objective lens 102, a shutter 103, an infrared sensor 104, and a camera control circuit 105.

[0022] The housing 101 houses each component of the infrared camera 10 and is fixed to the vehicle 90. The objective lens 102 receives infrared light incident from the range captured by the infrared camera 10 and projects the infrared light onto the infrared sensor 104. The shutter 103 includes a light-shielding plate material and is interposed between the objective lens 102 and the infrared sensor 104 so as to be able to open and close.

[0023] When the shutter 103 is closed, it blocks light entering the infrared sensor 104 from the objective lens 102. On the other hand, when the shutter 103 is open, it does not block light entering the infrared sensor 104 from the objective lens 102. In addition, the shutter 103 is a black body as viewed from the infrared sensor 104, and the infrared sensor 104 is calibrated with the shutter 103 closed.

[0024] The infrared sensor 104 is composed of thermally sensitive elements arranged in an array, receives infrared light incident through the objective lens 102, and generates image data based on changes in the resistance value of each of the thermally sensitive elements. The infrared sensor 104 is communicably connected to a camera control circuit 105, and operates upon receiving a predetermined control signal from the camera control circuit 105. When the infrared sensor 104 generates image data, it supplies the generated image data to the camera control circuit 105.

[0025] When the infrared sensor 104 receives sunlight with a strength greater than a predetermined value for a predetermined period of time or longer, the output of the elements in the area receiving the sunlight becomes saturated. Also, the temperature of the elements in the area receiving the sunlight rises above a specified value, causing abnormal conditions such as irreversible deformation of the elements or changes in characteristics. For this reason, the infrared camera 10 has a shutter 103 to protect the infrared sensor 104 from direct sunlight.

[0026] The dynamic range of the infrared sensor 104 is set so that the resolution for detecting objects such as pedestrians is relatively high. Therefore, the infrared sensor 104 is set so that the detection signal is saturated even in a temperature range where no abnormality occurs.

[0027] The camera control circuit 105 is a control circuit including an MCU (Micro Controller Unit) and controls the shutter 103 and the infrared sensor 104. The camera control circuit 105 is also communicably connected to the image processing device 11, receives a control signal from the image processing device 11, and controls each component of the infrared camera 10 in response to the received control signal. The camera control circuit 105 controls the shutter 103 to be kept closed when the infrared camera 10 is not capturing an image. The camera control circuit 105 controls the shutter 103 to be open when the infrared camera 10 is capturing an image. At this time, the camera control circuit 105 also supplies image data generated by the infrared sensor 104 to the image processing device 11.

[0028] Furthermore, the camera control circuit 105 temporarily closes the shutter 103 under a predetermined condition. The predetermined condition is, for example, when the shutter 103 is temporarily closed to protect the infrared sensor 104 or when calibration is performed. In this case, for example, the camera control circuit 105 receives an instruction from the image processing device 11 to temporarily close the shutter 103.

[0029] Next, the image processing device 11 will be described with reference to Fig. 3. Fig. 3 is a block diagram of the image processing device according to the first embodiment. The image processing device 11 has, as its main components, a communication IF 120, a ROM 130, a RAM 140, a system control circuit 150, an image data acquisition unit 160, an image processing unit 170, an image recognition unit 180, and an image data output unit 190. These components are connected via a bus 110 so as to be able to communicate with each other as appropriate.

[0030] The communication IF 120 is an interface of a communication line for controlling the infrared camera 10. The communication IF 120 supplies the infrared camera 10 with a control signal for the image processing device 11 to control the infrared camera 10. The communication IF 120 is also an interface for acquiring vehicle information from the ECU 13. When the communication IF 120 acquires vehicle information from the ECU 13, it appropriately supplies the acquired vehicle information to the components of the image processing device 11. The communication IF 120 may also be an interface for outputting a predetermined signal to the ECU 13. For example, the communication IF 120 may supply the ECU 13 with a signal for issuing a predetermined warning to the driver.

[0031] The ROM 130 (Read Only Memory) is a non-volatile memory that stores preset information or data. The ROM 130 stores in advance, for example, a program for the image processing device 11 to realize the functions according to the present embodiment.

[0032] The RAM 140 (RAM (Random Access Memory)) is a volatile memory having a storage area in which the image processing device 11 can temporarily expand data. The RAM 140 may be, for example, a DRAM (Dynamic Random Access Memory), and may include a register associated with the system control circuit 150 or the like. The RAM 140 includes an area in which the programs stored in the ROM 130 are expanded and executed. The RAM 140 may also be used, for example, when processing image data supplied from the infrared camera 10.

[0033] The system control circuit 150 includes an arithmetic unit such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The system control circuit 150 executes a program for enabling the image processing device 11 to perform the functions according to this embodiment. The program may be configured as software stored in the ROM 130, for example. The program may also be configured as a combination of software and hardware. The system control circuit 150 receives a predetermined signal from each component of the image processing device 11, and outputs a predetermined instruction or the like to each component in response to the received signal. The system control circuit 150 includes, as functional blocks, a determination unit 151 and an instruction unit 152.

[0034] The judgment unit 151 makes a predetermined judgment on the thermal image related to the image data obtained from the image processing device 11. For example, the judgment unit 151 judges whether to issue a warning when an object such as a pedestrian is included in the thermal image. In addition, for example, the judgment unit 151 may judge whether the shutter 103 of the infrared camera 10 is opened or closed. The judgment unit 151 supplies the results of these judgments to the instruction unit 152.

[0035] The instruction unit 152 issues various instructions to each component of the image processing device 11. The instruction unit 152 receives information related to the result of the judgment from the judgment unit 151, and issues instructions to each component according to the received information. For example, when the instruction unit 152 receives a signal from the judgment unit 151 to judge whether to issue a warning, the instruction unit 152 may output a signal indicating that a warning should be issued to the ECU 13 via the communication IF 120. The instruction unit 152 may also output an instruction related to opening and closing the shutter 103 to the infrared camera 10 via the communication IF 120.

[0036] The image data acquisition unit 160 is an interface that acquires thermal image data (input thermal image data), which is data related to a thermal image, from the infrared camera 10. The image data acquisition unit 160 acquires image data periodically, for example, from the infrared camera 10. For example, the image data acquisition unit 160 receives one frame of an image every 1 / 15th of a second. When the image data acquisition unit 160 receives the image data, it supplies the received image data to the image processing unit 170.

[0037] The image processing unit 170 is an image processing circuit including, for example, a GPU (Graphics Processing Unit). The image processing unit 170 may be a processing circuit including an FPGA (field-programmable gate array) or a DSP (Digital Signal Processor). The image processing unit 170 cooperates with the RAM 140 to perform a predetermined process on the thermal imaging data and generate converted image data. The image processing unit 170 will be described in detail later.

[0038] The image recognition unit 180 receives image data from the image processing unit 170, and detects an object image related to a predetermined object such as a vehicle or a person from the received image data. More specifically, for example, the image recognition unit 180 has a recognition dictionary for detecting a predetermined object image, and detects the object image by referring to data in the recognition dictionary. When the image recognition unit 180 detects a predetermined object image, it generates information related to the position and size of the detected object image, and supplies the generated information to the system control circuit 150.

[0039] Image recognition unit 180 may also calculate the contrast of the received image data and supply the calculation result to system control circuit 150. In this case, system control circuit 150 may instruct image processing unit 170 to perform a function for improving the contrast of the image when the contrast of the image data falls below a preset threshold value.

[0040] The image data output unit 190 is an interface for outputting the image data (converted image data) processed by the image processing unit 170 to the display 12. The image data output by the image data output unit 190 is output according to a data format corresponding to the specifications of the display 12. This data format is, for example, HDMI (High-Definition Multimedia Interface) (registered trademark) or DVI (Digital Visual Interface).

[0041] Next, the image processing unit 170 will be described with reference to Fig. 4. Fig. 4 is a block diagram of the image processing unit 170 according to the first embodiment. The image processing unit 170 has a defective pixel correction unit 171, a NUC unit 172, and an image conversion processing unit 173 as its main components.

[0042] The defective pixel correction unit 171 stores defective pixels of the infrared sensor 104 in advance, and performs a process (interpolation process) of interpolating the pixel value of the stored defective pixel from the pixel values ​​of surrounding pixels. The defective pixel correction unit 171 receives input thermal image data from the infrared camera 10 via the image data acquisition unit 160, and performs the above-mentioned interpolation process on the received image data. The defective pixel correction unit 171 supplies the image data after the interpolation process to the NUC unit 172.

[0043] The NUC unit 172 performs NUC (Non-Uniformity Correction), which is a calibration process for suppressing variations in pixel values ​​output in response to input light. The NUC unit 172 has in advance settings of gain and offset values ​​corresponding to the characteristics of each pixel of the infrared sensor 104. The NUC unit 172 calibrates each pixel value of the image data received from the defective pixel correction unit 171 according to the in advance settings.

[0044] The image conversion processing unit 173 receives the calibrated image data from the NUC unit 172, and performs a predetermined process on the received image data to generate converted image data. The converted image data is the image data received from the NUC unit 172 with improved contrast or resolution. By improving the contrast or resolution of the image data, the image processing device 11 can suppress a decrease in the recognition rate of objects. After generating the converted image data, the image conversion processing unit 173 supplies the generated converted image data to the image recognition unit 180 and the image data output unit 190.

[0045] Next, a process for generating converted image data executed by the image processing device 11 will be described with reference to Fig. 5. Fig. 5 is a flowchart of an image processing method according to the first embodiment. The flowchart shown in Fig. 5 is started, for example, when the image processing device 11 starts capturing an image using the infrared camera 10, and is repeatedly executed during the period when the image capturing using the infrared camera 10 is being performed.

[0046] First, the image data acquisition unit 160 of the image processing device 11 acquires thermal image data from the infrared camera 10 (step S10). When the image data acquisition unit 160 acquires the thermal image data, it supplies the acquired thermal image data to the image processing unit 170.

[0047] Next, the image processor 170 divides the received thermal imaging data into predetermined reference sections (step S11).

[0048] Next, the image processing unit 170 sets a reference center pixel value for each pixel position based on the statistical data of the pixel values ​​of each of the reference sections (step S12).

[0049] Next, when the image processing unit 170 receives the thermal imaging data, it sets a local area including a plurality of first local areas and a plurality of second local areas that include the first local areas and are larger than the first local areas in the thermal image (step S13).

[0050] Next, the image processing unit 170 calculates a frequency distribution (histogram) regarding the pixel values ​​of the pixels that configure the first local region and the second local region (step S14).

[0051] Next, the image processing unit 170 sets a distribution ratio in the local region based on the reference center pixel value (step S15). Specifically, for example, the image processing unit 170 uses the reference center pixel value of the local region to set a distribution ratio from the calculated reference center pixel value.

[0052] Note that image processing unit 170 sets the distribution ratio high for pixel value ranges where contrast is desired to be improved and low for pixel value ranges where contrast is desired to be suppressed. Therefore, the distribution ratio can be set so that image processing device 11 improves the resolution for temperatures of detection targets such as humans and animals, and suppresses the resolution for low temperatures such as the sky and high temperatures such as an automobile engine or muffler.

[0053] Next, image processing unit 170 generates redistributed data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution to the frequency distribution (step S16). Note that image processing unit 170 sets a first threshold value to be applied to the first local region and a second threshold value to be applied to the second local region independently. In other words, the first threshold value and the second threshold value may be the same or different.

[0054] Next, the image processing unit 170 generates a local tone curve from the redistribution data (step S17). Specifically, the image processing unit 170 generates a local tone curve by cumulatively adding the frequencies of the redistribution data from low pixel values ​​to successively higher pixel values.

[0055] Next, the image processing unit 170 generates a blend tone curve (step S18). Specifically, the image processing unit 170 synthesizes a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a preset weighting coefficient. The image processing unit 170 normalizes the generated blend tone curve by the width between the maximum and minimum pixel values ​​of the thermal imaging data.

[0056] Next, the image processing unit 170 generates converted image data by applying a blend tone curve to pixels corresponding to each local region (step S19). At this time, the image processing unit 170 performs processing, for example, according to the following formula (1).

number

[0057] Next, the image data output unit 190 outputs the converted image data (step S20). When the image data output unit 190 outputs the converted image data, the image processing device 11 ends the series of processes. When new thermal imaging data is supplied, the image processing device 11 executes the above-mentioned processes again from step S10.

[0058] The above describes the image processing method executed by the image processing device 11. By executing the above information, the image processing device 11 improves the resolution of the pixel value range related to the desired temperature.

[0059] Next, the reference section and the reference central pixel value will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the reference section and the reference central pixel value. Fig. 6 shows a three-axis orthogonal coordinate system consisting of an HV plane formed by orthogonal H and V axes, and an L axis orthogonal to the HV plane. The H axis indicates the axis parallel to the horizontal direction of the thermal imaging data, and the V axis indicates the axis parallel to the vertical direction of the thermal imaging data. The L axis indicates the pixel value (brightness value) of the thermal image.

[0060] In the three-axis orthogonal coordinate system shown in the upper part of FIG. 6, the thermal image is plotted so that the origin of the HV plane coincides with the upper left corner of the thermal image, and the thermal image is further divided into 4×4, i.e., 16, reference sections D100(x). Here, x is an integer between 0 and 15. Above each reference section D100(x), a reference center pixel value D101(x) is plotted with a black circle. The reference center pixel value D101(x) is a center value calculated from the frequency distribution of pixel values ​​in the reference section D100(x). The center value adopted here may be the median value of the frequency distribution or may be the average value. The reference center pixel value D101(x) is plotted above the center of the reference section D100(x) corresponding to each reference section D100(x).

[0061] The three-axis orthogonal coordinate system shown in the lower part of Fig. 6 shows distribution data D102 of the reference center pixel value generated using the above-mentioned reference center pixel value D101. The distribution data D102 is obtained by performing spline interpolation on the reference center pixel value D101 shown in the upper part of the figure to calculate the reference center pixel value at a predetermined pixel position. The predetermined pixel position is, for example, a pixel position corresponding to the center of a local region described later.

[0062] As described above, the image processing unit 170 divides the thermal image into reference sections and calculates a reference central pixel value, which is a central value in each of the divided reference sections. The image processing unit 170 then interpolates the reference central pixel values ​​to further calculate a reference central pixel value at a predetermined pixel position. Note that the spline interpolation performed by the image processing unit 170 in the above description may be bilinear interpolation or Lagrange interpolation.

[0063] Next, a specific example of a local region will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of a local region according to the first embodiment. Fig. 7 shows a thermal image D11. In addition, in the thermal image D11, the upper left corner is set as the origin, the H direction is set to the right, and the V direction is set to the lower side. In the following description, the coordinates of the H direction and the V direction in the thermal image D11 will be indicated in parentheses as (h, v).

[0064] The image processing unit 170 sets a plurality of first local regions R1 and second local regions R2 in the thermal image D11. For example, the image processing unit 170 sets the first local region R1(0,0) at a position corresponding to the coordinates (0,0) of the image D11. The image processing unit 170 also sets the first local regions R1(h,v) at arbitrary positions (h,v) so that they are adjacent to each other. That is, as shown by the two-dot chain line in the figure, the first local region R1(h+1,v) is set adjacent to the right side of the first local region R1(h,v), and the first local region R1(h,v+1) is set adjacent to the lower side of the first local region R1(h,v). In this way, the image processing unit 170 sets m first local regions R1 in the H direction and n first local regions R1 in the V direction up to the first local region R1(m-1,n-1) in the lower right of the thermal image D11.

[0065] The image processor 170 also sets second local regions R2(h,v) corresponding to the respective first local regions R1. The second local region R2(h,v) corresponding to the coordinates (h,v) is a rectangular region including the first local region R1(h,v) and larger than the first local region R1(h,v). Note that the second local region R2 may extend outside the thermal image D11 at the outer edge of the thermal image D11, as shown at the coordinates (0,0) and (m,n) in the figure. In this case, the size of the second local region R2 is cut according to the size of the thermal image D11.

[0066] In this way, the image processing unit 170 sets the first local region R1(h,v) and the second local region R2(h,v) as local regions corresponding to the coordinates (h,v). Then, the image processing unit 170 generates a local tone curve and a blend tone curve for each of the set local regions.

[0067] Next, a specific example of a local tone curve will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a process for generating excess data. In Fig. 8, data D12 is shown on the upper side, and data D13 is shown on the lower side.

[0068] Data D12 shown in the upper part of Fig. 8 is a frequency distribution in a predetermined local region. In the image processing device 11, the image conversion processing unit 173 of the image processing unit 170 sets a local region from the thermal imaging data received from the NUC unit 172, and calculates a frequency distribution as shown in data D12 for the set local region. In data D12, the horizontal axis indicates pixel value, and the vertical axis indicates frequency corresponding to the pixel value. The frequency distribution shown in data D12 is indicated by a curve having a frequency peak in the center.

[0069] 8 shows excess data D131 that exceeds a predetermined threshold value with respect to the frequency distribution, which is hatched. In order to redistribute the excess data D131 that exceeds a preset threshold value, the image conversion processing unit 173 extracts the excess data D131 and redistributes the frequency of the excess data D131 with respect to the frequency distribution according to a predetermined distribution ratio.

[0070] Next, the distribution ratio will be described with reference to FIG. 9. FIG. 9 is a diagram showing a first example of the distribution ratio. FIG. 9 shows a graph E10 showing the relationship between the pixel value and the distribution ratio. In the graph E10, the horizontal axis indicates the pixel value, and the vertical axis indicates the distribution ratio. A curve E101 is also plotted on the graph E10. The curve E101 is a curve in which the pixel value shown on the horizontal axis reaches a peak at the position of the reference center pixel value, and gradually decreases to the left and right. In this way, the distribution ratio is set so that the distribution ratio reaches a peak at the reference center pixel value, and decreases as it moves away from the reference center pixel value.

[0071] More specifically, for example, a first pixel value E121 in Fig. 8 has a first difference E111 between itself and the reference center pixel value. A second pixel value E122 in Fig. 8 has a second difference E112 between itself and the reference center pixel value that is greater than the first difference. At this time, a first distribution ratio E131 corresponding to the first pixel value E121 is set to be higher than a second distribution ratio E132 corresponding to the second pixel value E122.

[0072] With this configuration, the image processing device 11 allocates a relatively large amount of data to the reference center pixel value and its neighboring pixel values, and enriches the expression of the image at the reference center pixel value and its neighboring pixel values. As a result, the image processing device 11 improves the contrast of the image at the reference center pixel value and its neighboring pixel values.

[0073] Next, the redistribution data and local tone curve generated by the image processing unit 170 will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of a process for generating redistribution data and a local tone curve.

[0074] The data D14 shown in the upper part of Fig. 10 is redistributed data. The image conversion processing unit 173 distributes the frequency corresponding to the excess data D131 to pixel value data between the minimum and maximum values ​​of the thermal imaging data. In the example shown in the figure, the image conversion processing unit 173 distributes the excess data D131 along a curve with a peak in the center like the data D141 to generate redistributed data D142.

[0075] Data D15 shown at the bottom of Fig. 10 is data indicating a local tone curve. Data D15 is a cumulative frequency distribution in which the redistribution data generated in data D14 is sequentially accumulated from the minimum pixel value to the maximum pixel value. The image conversion processing unit 173 generates local tone curves in the first local region R1(h,v) and the second local region R2(h,v) by the above-mentioned procedure.

[0076] Next, the blend tone curve will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example of a blend tone curve. Fig. 11 shows a first local tone curve D16, a second local tone curve D17, and a blend tone curve D18.

[0077] The first local tone curve D16 is a local tone curve of a first local region at a predetermined position of the thermal image data. The second local tone curve D17 is a second local tone curve corresponding to the first local tone curve D16. That is, the second local region of the second local tone curve D17 includes the first local region of the first local tone curve D16.

[0078] The blend tone curve D18 is generated by combining the first local tone curve D16 and the second local tone curve according to the above-mentioned formula (1). The blend tone curve D18 is generated by combining the first local tone curve D16 and the second local tone curve D17, and then normalizing the result using the maximum pixel value of the thermal imaging data. In other words, the blend tone curve D18 has a relationship in which the horizontal axis represents the input pixel value and the vertical axis represents the output pixel value. As a result, the image conversion processing unit 173 generates image conversion data at each pixel position corresponding to the blend tone curve D18.

[0079] Next, another example of the distribution ratio will be described with reference to FIG. 12. FIG. 12 is a diagram showing a second example of the distribution ratio. FIG. 12 shows a graph E20 showing the relationship between the pixel value and the distribution ratio. In the graph E20, the horizontal axis indicates the pixel value, and the vertical axis indicates the distribution ratio. In addition, a broken line E201 is plotted in the graph E20. The broken line E201 is a broken line that bends left and right and descends from a pixel value near the center value as a peak. In this way, the distribution ratio is set so that the distribution ratio peaks at the center value and decreases as it moves away from the center value.

[0080] More specifically, for example, a first pixel value E221 in Fig. 12 has a difference from the central value that is a first difference E211. A second pixel value E222 in Fig. 12 has a difference from the central value that is a second difference E212 that is larger than the first difference. At this time, a first distribution ratio E231 corresponding to the first pixel value E221 is set to be higher than a second distribution ratio E232 corresponding to the second pixel value E222.

[0081] The first embodiment has been described above. As described above, the image processing device 11 generates local tone curves for the first local region and the second local region, and combines them after setting weighting. In addition, at that time, the image processing device 11 sets a distribution ratio with the reference center pixel value as a peak, and applies this to the frequency distribution. By redistributing the excess data with the reference center pixel value as the center in this way, the image processing device 11 suppresses luminance spots in the low frequency range that exist in the entire thermal image. Therefore, according to the first embodiment, it is possible to provide an image processing device, an image processing method, and a program that reduce the influence of radiant heat and suppress a decrease in the contrast of the detection target. In addition, by suppressing a decrease in the contrast of the detection target, it is possible to improve the recognition rate of an object based on a thermal image.

[0082] Although the image processing device 11 described above sets two types of local areas, the first local area and the second local area, the setting of the local areas is not limited to this. That is, the image processing device 11 sets a plurality of local areas of different sizes. In this case, the image processing device 11 may set three or more types of local areas of different sizes, and when generating a blend tone curve, synthesize local tone curves corresponding to these three or more types of areas.

[0083] Also, instead of peaking at the central value, the distribution ratio set by the image conversion processing unit 173 may peak at a pixel value that is a predetermined pixel value away from the central value. Also, the distribution ratios shown in the above-mentioned Fig. 8 and Fig. 11 are exemplified as being symmetrical with respect to the central value, but the distribution ratio is not limited to being symmetrical with respect to the central value.

[0084] <Embodiment 2> Next, a description will be given of embodiment 2. Fig. 13 is a flowchart of an image processing method according to embodiment 2. The flowchart shown in Fig. 13 illustrates the processing that is mainly performed by image conversion processing unit 173.

[0085] First, the image conversion processing unit 173 acquires the calibrated thermal imaging data from the NUC unit 172 (step S101).

[0086] Next, the image conversion processing unit 173 divides the thermal image received from the NUC unit 172 into reference regions (step S102). Furthermore, the image conversion processing unit 173 calculates a frequency distribution for each reference region in the thermal imaging data (step S103).

[0087] Here, when the pixel values ​​of the thermal imaging data are, for example, about 16 bits, the image conversion processing unit 173 according to this embodiment calculates a frequency distribution in which the lowest 2 to 4 bits are grouped together into one bin (also called a class). This allows the image processing device 11 to reduce the processing time required for image conversion while suppressing a substantial decrease in calculation accuracy.

[0088] Next, the image conversion processing unit 173 calculates a reference center pixel value of each reference area from the frequency distribution of each reference area (step S104). The reference center pixel value may be a median value or an average value.

[0089] Next, the image conversion processing unit 173 calculates the reference center pixel value of each unit area (step S105). The unit area is obtained by dividing the thermal image into a predetermined size. The unit area will be described in detail later. The image conversion processing unit 173 uses the reference center pixel value calculated here when generating redistribution data.

[0090] Next, the image conversion processing unit 173 performs range calculation of the thermal imaging data (step S106). In the range calculation, the image conversion processing unit 173 sets a minimum pixel value on the low temperature side and a maximum pixel value on the high temperature side in the frequency distribution. Furthermore, the image conversion processing unit 173 sets a central pixel value. Then, the image conversion processing unit 173 calculates a first pixel range which is the difference between the maximum pixel value and the central pixel value. Similarly, the image conversion processing unit 173 calculates a second pixel range which is the difference between the central pixel value and the minimum pixel value. Then, the image conversion processing unit 173 compares the first pixel range with the second pixel range, and applies the larger pixel range to the smaller pixel range to set the pixel range of the entire image.

[0091] An example of range calculation will be described with reference to FIG. 14. FIG. 14 is a frequency distribution diagram of thermal imaging data. Data D22 shown in FIG. 14 shows an example of thermal imaging data. In the data D22, the image conversion processing unit 173 sets the bins after cutting off several percent (for example, 1%) above and below the frequency as the minimum pixel value (MIN) and the maximum pixel value (MAX), respectively. By performing such processing, the image processing device 11 can process data by excluding data that is significantly out of the range due to an error or the like when such data exists in the image data. The image conversion processing unit 173 also sets a central value. The median (MEDIAN) or average value of the frequency distribution can be selected as the central value. In the case of the example shown in the data D22, the image conversion processing unit 173 sets the median value as the central value.

[0092] In the image data shown in data D22, the first pixel range R11, which is the difference between MAX and MEDIAN, is larger than the second pixel range R12, which is the difference between MEDIAN and MIN. Therefore, the image conversion processing unit 173 sets the range of pixel values ​​on the low temperature side to MIN2, which is away from MEDIAN by the first pixel range R11. By setting the pixel range in this way, the image processing device 11 improves the contrast after conversion.

[0093] As described above, the image processing device 11 detects the maximum pixel value and the minimum pixel value of the pixel value data. Then, the image processing device 11 applies the relatively larger difference between the first pixel range, which is the difference between the maximum pixel value and the central pixel value, and the second pixel range, which is the difference between the central pixel value and the minimum pixel value, to the relatively smaller difference. Furthermore, the image processing device 11 sets the pixel value range expanded in this way as an updated pixel range. The image processing device 11 also sets the distribution ratio using this updated pixel range.

[0094] The above-mentioned center pixel value may be a reference center pixel value. That is, the image processing device 11 may apply the relatively larger difference between a first reference pixel range, which is the difference between the maximum pixel value and the reference center pixel value, and a second reference pixel range, which is the difference between the reference center pixel value and the minimum pixel value, to the relatively smaller difference.

[0095] Returning to FIG. 13, the description will continue. Image conversion processing unit 173 performs range restriction (step S107). In the range restriction, image conversion processing unit 173 changes the setting of data that exists outside the range between the set minimum pixel value and maximum pixel value to a bin closer to either the minimum pixel value or the maximum pixel value. In the case of the example shown in FIG. 14, if there is data that is below MIN2 in the frequency distribution, image conversion processing unit 173 places this data in the bin corresponding to MIN2. Similarly, if there is data that is above MAX in the frequency distribution, image conversion processing unit 173 places this data in the bin corresponding to MAX.

[0096] Next, the image conversion processing unit 173 divides the thermal image into unit areas (step S108). A unit area is an area that is an element for constituting a local area for generating a local tone curve. That is, the image conversion processing unit 173 sets each local area as a collection of unit areas. By dividing the thermal image into unit areas, the image processing device 11 can efficiently perform processing on a plurality of local areas of different sizes.

[0097] The unit area generated by the image conversion processing unit 173 will be described with reference to Fig. 15. Fig. 15 is a diagram showing an example of the unit area. Fig. 15 shows a thermal image D23 and a unit area U10.

[0098] The thermal image D23 is a thermal image based on the input thermal imaging data received by the image conversion processing unit 173. In the example shown in Fig. 15, the image conversion processing unit 173 divides the thermal image D23 into 64 parts horizontally and 48 parts vertically. The unit area U10 is one area when the thermal image D23 is divided as described above. In the thermal image D23 shown in Fig. 15, 64 unit areas U10 are set in the horizontal direction (H-axis direction) with the upper left corner as the origin, and 48 unit areas U10 are set in the vertical direction (V-axis direction). In Fig. 15, one unit area U10 is indicated by hatching in the upper left corner of the thermal image D23.

[0099] The image conversion processing unit 173 sets a unit area U10(h,v) at an arbitrary coordinate (h,v) when the upper left corner is the coordinate (0,0) of the origin and the lower right corner is the coordinate (64,48). The image conversion processing unit 173 divides the thermal image D23 into 64x48 in this way, thereby setting 3072 unit areas U10. For example, if the thermal image D23 is QVGA (Quarter Video Graphics Array) and has a pixel count of 320x240 pixels, the unit area is 5x5 pixels.

[0100] Returning to Fig. 13, the description will continue. Next, image conversion processing unit 173 calculates a unit frequency distribution, which is a frequency distribution of a unit area (step S109). That is, in the case of the example shown in Fig. 15, image conversion processing unit 173 calculates the frequency distribution for each of 3072 unit areas. At this time, image conversion processing unit 173 performs the calculation so that the unit frequency distribution falls within the range limit described above.

[0101] Next, the image conversion processing unit 173 sets a first local region (step S110). The image conversion processing unit 173 also sets a second local region (step S120). The image conversion processing unit 173 further sets a third local region (step S130).

[0102] The local region will be described with reference to Fig. 16. Fig. 16 is a diagram showing an example of a local region according to the second embodiment. Fig. 16 shows a 5×5 unit region U10 and a first local region U11, a second local region U12, and a third local region U13 superimposed thereon.

[0103] The first local area U11 is composed of one unit area U10. The first local area U11 shown in the figure corresponds to the unit area U10 of coordinates (h, v) located at the center of the unit area U10 arranged in a 5×5 matrix. The second local area U12 is composed of a 3×3 unit area U10. The second local area U12 corresponds to the unit area U10(h, v) at the center of the unit area U10 arranged in a 5×5 matrix, and includes the first local area U11(h, v). The third local area U13 is composed of a 5×5 unit area U10. The third local area U13 includes the first local area U11(h, v) and the second local area U12(h, v) at the center. That is, the third local area U13 corresponds to the coordinates (h, v).

[0104] In the process of setting the first local region, the second local region, and the third local region, the image conversion processing unit 173 calculates the frequency distribution for each of the local regions. At this time, the image conversion processing unit 173 uses the unit frequency distribution. That is, when calculating the frequency distribution for each of the local regions, the image conversion processing unit 173 adds up the unit frequency distributions in the unit regions that each of the local regions has. This allows the image conversion processing unit 173 to efficiently calculate the frequency distribution for each local region.

[0105] Returning to Fig. 13, the description will be continued. The image conversion processing unit 173 sets a distribution ratio for each local region (steps S111, S121, and S131). More specifically, the image conversion processing unit 173 uses the reference center pixel value calculated in step S105 to set a distribution ratio that peaks at the reference center pixel value or a pixel value region including the reference center pixel value, and that decreases as the pixel value moves away from the reference center pixel value. Note that the distribution ratio set here is set individually for each local region.

[0106] Next, the image conversion processing unit 173 generates redistribution data for each local region (steps S112, S122, and S132). Note that the threshold value for the excess data set when generating the redistribution data can be set individually for each local region.

[0107] Next, the image conversion processing unit 173 generates a first local tone curve corresponding to the first local region (step S113). The image conversion processing unit 173 also generates a second local tone curve corresponding to the second local region (step S123). Similarly, the image conversion processing unit 173 generates a third local tone curve corresponding to the third local region (step S133). Note that the process of generating the redistribution data and the process of generating the local tone curves corresponding to each local region may be performed in parallel or sequentially. Note that the image conversion processing unit 173 normalizes each of the generated local tone curves by dividing them by the total frequency in each region.

[0108] Next, the image conversion processing unit 173 generates a blend tone curve by synthesizing the generated local tone curves (Step S140). The image conversion processing unit 173 generates the blend tone curve so that the following formula (2) is satisfied.

number

[0109] FIG. 17 shows examples of a local tone curve and a blend tone curve. FIG. 17 is a diagram showing examples of a local tone curve and a blend tone curve according to the second embodiment. In the graph shown in FIG. 17, a first local tone curve, a second local tone curve, a third local tone curve, and a blend tone curve are superimposed. As shown in the figure, each local tone curve has its own slope according to the size of the local area. A weighting coefficient and an offset are set for the blend tone curve so that the image processing device 11 can perform the desired function.

[0110] For example, in the example of Fig. 17, the slopes of the local tone curves in the low pixel value range are steep, and exist in different pixel value ranges (i.e., different temperature bands). On the other hand, the slope of the blend tone curve in the low pixel value range is a relatively gentle curve. In this way, the image processing device 11 suppresses the variation in contrast that occurs in local areas of different sizes, and expands the contrast of the temperature band of the object to be detected, by synthesizing the features of the tone curves of each local area.

[0111] Returning to FIG. 13, the description will be continued. The image conversion processing unit 173 performs tone conversion of the thermal imaging data using the generated blend tone curve (step S141). As described above, when calculating the frequency distribution of the thermal imaging data, the image conversion processing unit 173 calculates and processes a frequency distribution in which the lower 2 to 4 bits are grouped together as one bin. In other words, the image conversion processing unit 173 performs processing similar to extracting designated pixels such that each pixel in the local region is spaced apart from each other by several bits, and applying the blend tone curve to the extracted designated pixels. Therefore, in the subsequent processing, processing is performed to accommodate all pixels in the input thermal imaging data.

[0112] Next, the image conversion processing unit 173 performs image data interpolation processing (step S142). Specifically, the image conversion processing unit 173 interpolates pixel values ​​of pixels existing between the designated pixels set in this step S141 using converted pixel values ​​of the surrounding designated pixels. An example of the interpolation method is bilinear interpolation. In this way, the image conversion processing unit 173 performs tone conversion on all pixels of the thermal imaging data. That is, the image conversion processing unit 173 generates converted image data by converting the input thermal imaging data.

[0113] Next, the image conversion processing unit 173 outputs the generated converted image data (step S150). The image processing unit 170 supplies the converted image data output by the image conversion processing unit 173 to, for example, the image recognition unit 180. As a result, the image processing device 11 detects an object using the converted image data. The image processing unit 170 also appropriately performs processing for displaying the converted image data output by the image conversion processing unit 173 on the display 12. For example, the image processing unit 170 performs gamma correction on the converted image data, and supplies the gamma-corrected image data to the display 12 via the image data output unit 190. When the image conversion processing unit 173 outputs the converted image data, it ends the series of processes.

[0114] The image processing method according to the second embodiment has been described above. With the above-mentioned method, the image processing device 11 can detect an object using image data in which the influence of radiant heat is reduced. As described in steps S141 and S142, the image processing device 11 can improve the contrast of the image and increase the processing speed of the image processing by extracting and processing the designated pixels.

[0115] As described above, when setting the redistribution ratio, the image processing device 11 sets the peak of the redistribution ratio to the reference center pixel value, and performs the redistribution ratio for the set pixel value range (range limit). This allows the image processing device 11 to suppress low-frequency noise received from, for example, the lens barrel of the infrared camera 10. Therefore, according to the second embodiment, it is possible to provide an image processing device, an image processing method, and a program that reduce the influence of radiant heat and suppress a decrease in contrast of the entire screen.

[0116] Next, a modified example of the embodiment 2 will be described. Fig. 18 is a flowchart of an image processing method according to the modified example of the embodiment 2. The flowchart shown in Fig. 18 differs from the flowchart shown in Fig. 12 in the processing between step S140 and step S150.

[0117] After generating the blend tone curve in step S140 of FIG. 18, the image conversion processing unit 173 performs tone curve interpolation (step S143). As described above, when calculating the frequency distribution of the thermal imaging data, the image conversion processing unit 173 calculates and processes the frequency distribution in which the lower 2 to 4 bits are grouped into one bin. Therefore, the blend tone curve in step S140 does not correspond to all pixels of the thermal imaging data. Therefore, the image conversion processing unit 173 generates tone curves for pixels existing between designated pixels by interpolating from blend tone curves for surrounding designated pixels. A specific method of interpolation is, for example, bilinear interpolation. By performing tone curve interpolation, the image conversion processing unit 173 generates tone curves that correspond to all pixels of the thermal imaging data.

[0118] Next, the image conversion processing unit 173 performs tone conversion on the thermal imaging data using the generated tone curve (step S144). The image conversion processing unit 173 performs a process of converting input pixel values ​​and calculating output pixel values ​​for all pixels of the thermal imaging data. In this way, the image conversion processing unit 173 generates converted image data.

[0119] The second embodiment has been described above. According to the second embodiment, it is possible to provide an image processing device, an image processing method, and a program that reduce the influence of radiant heat and suppress the decrease in contrast of a detection target. In addition, by suppressing the decrease in contrast of a detection target, it is possible to improve the recognition rate of an object based on a thermal image.

[0120] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0121] 10. Infrared Camera 11 Image processing device 12 Display 13 ECU 90 vehicles 101 Case 102 Objective Lens 103 Shutter 104 Infrared Sensor 105 Camera control circuit 110 Bus 120 Communication Interface 130 ROM 140 RAM 150 System control circuit 151 Judgment Department 152 Instruction section 160 Image data acquisition unit 170 Image Processing Unit 171 Defective pixel correction section 172 NUC Department 173 Image conversion processing section 180 Image Recognition Unit 190 Image data output section

Claims

1. an image data acquisition unit that acquires thermal image data from an infrared camera that captures a thermal image of the periphery of the vehicle; an image processing unit that generates converted image data from the thermal image data; an output unit that outputs the converted image data, The image processing unit includes: Dividing the thermal image into predetermined reference segments; setting a reference center pixel value at a predetermined pixel position based on statistical data of each pixel value of the reference block; setting a local region including a plurality of first local regions and a plurality of second local regions including the first local regions and larger than the first local regions on the thermal image; Calculating a frequency distribution regarding pixel values ​​of pixels constituting the first local region and the second local region; A distribution ratio in the local region is set based on the reference central pixel value; generating redistributed data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution based on the distribution ratio to the frequency distribution; generating a local tone curve by accumulating the redistributed data from low pixel values ​​toward high pixel values; generating a blended tone curve by combining a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a weighting coefficient that is set in advance for each of the first local tone curve and the second local tone curve; generating the transformed image data by applying the blend tone curve to pixels corresponding to the first local region; Image processing device.

2. The image processing unit includes: setting a pixel value range of the thermal image based on statistical data of pixel values ​​of the thermal image; setting the distribution ratio such that a first distribution ratio for a first pixel value having a first difference from the reference center pixel value in the pixel value range is higher than a second distribution ratio for a second pixel value having a second difference larger than the first difference; The image processing device according to claim 1 .

3. The image processing unit includes: Detecting a maximum pixel value and a minimum pixel value of the thermal imaging data, a first reference pixel range which is a difference between the maximum pixel value and the reference center pixel value, and a second reference pixel range which is a difference between the reference center pixel value and the minimum pixel value, the difference being relatively larger than the first reference pixel range, is applied to the difference being relatively smaller than the second reference pixel range, thereby setting an updated reference pixel range for each pixel position; setting the distribution ratio for the update reference pixel range; The image processing device according to claim 2 .

4. The computer Acquire thermal image data from an infrared camera that captures thermal images of the area around the vehicle; Dividing the thermal image into predetermined reference segments; setting a reference center pixel value for each pixel position based on statistical data of each pixel value of the reference block; setting a local region including a plurality of first local regions and a plurality of second local regions including the first local regions and larger than the first local regions on the thermal image; Calculating a frequency distribution regarding pixel values ​​of pixels constituting the first local region and the second local region; A distribution ratio in the local region is set based on the reference central pixel value; generating redistributed data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution based on the distribution ratio to the frequency distribution; generating a local tone curve by accumulating the redistributed data from low pixel values ​​toward high pixel values; generating a blended tone curve by combining a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a weighting coefficient that is set in advance for each of the first local tone curve and the second local tone curve; generating transformed image data by applying the blend tone curve to pixels corresponding to the first local region; outputting the converted image data; Image processing methods.

5. Acquire thermal image data from an infrared camera that captures thermal images of the area around the vehicle; Dividing the thermal image into predetermined reference segments; setting a reference center pixel value for each pixel position based on statistical data of each pixel value of the reference block; setting a local region including a plurality of first local regions and a plurality of second local regions including the first local regions and larger than the first local regions on the thermal image; Calculating a frequency distribution regarding pixel values ​​of pixels constituting the first local region and the second local region; A distribution ratio in the local region is set based on the reference central pixel value; generating redistributed data by redistributing excess data that exceeds a preset frequency threshold in the frequency distribution based on the distribution ratio to the frequency distribution; generating a local tone curve by accumulating the redistributed data from low pixel values ​​toward high pixel values; generating a blended tone curve by combining a first local tone curve corresponding to the first local region and a second local tone curve corresponding to the second local region using a weighting coefficient that is set in advance for each of the first local tone curve and the second local tone curve; generating transformed image data by applying the blend tone curve to pixels corresponding to the first local region; outputting the converted image data; A method for causing a computer to execute an image processing method program.

Images