Image Processing Device

Abstract

To provide an image processing apparatus capable of improving the grasping accuracy of a luminance value ratio for each pixel.SOLUTION: In a storage unit, a filter of only one color included in a color filter is arranged as the color filter of an imaging unit, and correction data related to a luminance value ratio of one color of a captured image captured when light is made incident only on a specific portion in the filter is stored for each of multiple colors constituting the color filter. An image correction unit performs correction processing for grasping the luminance value ratio in the one color using a luminance value of the one color included in the captured image and correction data corresponding to the one color for a prescribed pixel, for each of the multiple colors constituting the color filter, when an illuminance determination unit determines that illuminance of the periphery of a mobile body is equal to or less than a predetermined threshold, and creating a correction image using the grasped luminance value ratio of each of the multiple colors in each pixel.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to an image processing device that is mounted on a moving object such as a vehicle. [Background technology]

[0002] Conventionally, for example, Patent Document 1 proposes an image processing device mounted on a moving body such as a vehicle. Specifically, this image processing device includes an imaging unit and an image correction unit that processes the captured image captured by the imaging unit. The imaging unit includes a photodiode and a color filter. The color filter is configured by regularly arranging red, green, and blue filters, for example, in a Bayer array. For this reason, the captured image is an image having red pixels that hold red luminance values ​​(hereinafter also simply referred to as red luminance values), green pixels that hold green luminance values ​​(hereinafter also simply referred to as green luminance values), and blue pixels that hold blue luminance values ​​(hereinafter also simply referred to as blue luminance values).

[0003] Then, in this image processing device, the image correction unit determines the red luminance value, green luminance value, and blue luminance value of each pixel from the captured image. For example, the red luminance value of a red pixel is determined as is. The green luminance value of a red pixel is determined from the average value of the green luminance values ​​of the green pixels surrounding the red pixel, and the blue luminance value of a red pixel is determined from the average value of the blue luminance values ​​of the blue pixels surrounding the red pixel. In other words, in this image processing device, the luminance value of a color different from that of a pixel is determined from the average luminance value of the surrounding pixels.

[0004] The image correction unit grasps the red luminance value, green luminance value, and blue luminance value for every pixel and derives the ratio of each luminance value.The image processing device then creates a corrected image based on the ratio of each luminance value, and outputs the created corrected image to an image display device, driving support device, or the like mounted on a moving body such as a vehicle.As a result, the image display device or driving support device recognizes colors based on the luminance value ratio of each pixel in the corrected image and performs predetermined processing. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2013-107476 A Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, currently, the performance of the imaging unit has improved, and it has become possible to capture images of subjects that are farther away. In this case, light does not enter each pixel evenly, and light may enter specific pixels. In particular, when the surroundings are at night, light from distant tail lights or reflectors is likely to enter only specific pixels (for example, 1 to 3 pixels).

[0007] As described above, the method of determining the luminance value of a color different from that of a pixel from the average luminance value of the surroundings is useful when light is incident on each pixel as a whole, but when light is incident only on a specific pixel, the brightness of each pixel is different, so the accuracy of the determined luminance value is likely to be low. In other words, the accuracy of determining the luminance value ratio of each pixel is likely to be low.

[0008] In view of the above, an object of the present invention is to provide an image processing device capable of improving the accuracy of grasping the luminance value ratio of each pixel. [Means for solving the problem]

[0009] Claim 1 for achieving the above object provides an image processing device, comprising: an image acquisition unit (101) mounted on a moving body, which acquires an image captured by an imaging unit (20) having color filters of multiple colors, and in which each pixel holds a luminance value for each color corresponding to each color filter; an illuminance determination unit (102) which determines the illuminance around the moving body; an image correction unit (103) which performs a predetermined correction process on the captured image acquired by the image acquisition unit to create a corrected image; an output unit (105) which outputs the corrected image processed by the image correction unit; and a memory unit (12) which stores predetermined data, and the memory unit stores data for the color filters as color filters of the imaging unit. A filter of only one of the colors contained therein is arranged, and correction data regarding the luminance value ratio of one color in an image captured when light is incident only on a specific point on the filter is stored for each of the multiple colors that make up the color filter.When the illuminance determination unit determines that the illuminance around the moving body is below a predetermined threshold, the image correction unit performs a correction process for each of the multiple colors that make up the color filter to determine the luminance value ratio of one color for a specific pixel using the luminance value of one color contained in the captured image and the correction data corresponding to that one color, and creates a corrected image using the determined luminance value ratio of each of the multiple colors at each pixel.

[0010] According to this, the image processing device uses the correction data to grasp the luminance value ratio of each color in each pixel of the captured image. Therefore, the grasped luminance value ratio of each color in each pixel is a ratio excluding the sensitivity difference of the color filter, and is a ratio that indicates the color difference in a state where the brightness is the same. Therefore, the accuracy of grasping the luminance value ratio of each pixel can be improved.

[0011] Note that the reference symbols in parentheses attached to each component indicate an example of the correspondence between the component and specific components described in the embodiments described below. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a configuration of an image processing system according to a first embodiment. [Diagram 2] FIG. 2 is a schematic diagram for explaining an arrangement of pixels in a captured image. [Diagram 3] FIG. 2 is a block diagram showing a configuration of the image processing device shown in FIG. [Figure 4] FIG. 13 is a schematic diagram showing a case where light is incident only on a specific pixel. [Diagram 5] FIG. 13 is a diagram showing pixel positions and luminance value ratios. [Figure 6] FIG. 2 is a schematic diagram of red and green pixels. [Figure 7A] FIG. 13 is a diagram showing pixel positions and luminance value ratios. [Figure 7B] FIG. 7B is a diagram showing pixel positions and luminance value ratios created using FIG. 7A. [Figure 8A] FIG. 13 is a diagram showing pixel positions and luminance value ratios. [Figure 8B] FIG. 8B is a diagram showing pixel positions and luminance value ratios created using FIG. 8A. [Figure 9A] FIG. 13 is a diagram showing pixel positions and luminance value ratios. [Figure 9B] FIG. 9B is a diagram showing pixel positions and luminance value ratios created using FIG. 9A. [Figure 10] 4 is a flowchart showing an operation performed by the image processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, parts that are the same or equivalent to each other will be denoted by the same reference numerals.

[0014] (First embodiment) The first embodiment will be described with reference to the drawings. The image processing device of this embodiment is preferably mounted on a moving body such as a vehicle and used to configure an image processing system together with an on-board camera, an image display device, a driving support device, and the like mounted on the moving body. In the following, an example in which the image processing device of this embodiment is mounted on a vehicle to configure an image processing system will be described.

[0015] The image processing system is a system mounted on a vehicle, and as shown in Fig. 1, includes an image processing device 10, an in-vehicle camera 20, an illuminance sensor 30, an image display device 40, and a driving support device 50. The in-vehicle camera 20, the illuminance sensor 30, the image display device 40, and the driving support device 50 are connected to the image processing device 10 via signal lines or the like.

[0016] The vehicle-mounted camera 20 is configured to have a CMOS (abbreviation of Complementary Metal Oxide Semiconductor) image sensor as an imaging element, and a color filter. The CMOS image sensor is configured with photodiodes that convert light into electric charges arranged in a lattice pattern. In this embodiment, the color filter used has red filters, green filters, and blue filters arranged in a Bayer array. However, the color filters may be arranged in other arrays. In this embodiment, the vehicle-mounted camera 20 corresponds to an imaging unit.

[0017] The vehicle-mounted camera 20 is installed near the windshield arranged in front of the vehicle so that a predetermined range in front of the vehicle is the imaging range. In this embodiment, two vehicle-mounted cameras 20 are provided in front of the vehicle so that they can capture the same field of view. In other words, the vehicle-mounted camera 20 in this embodiment is a stereo camera. The vehicle-mounted camera 20 in this embodiment repeatedly captures images at a preset time interval, for example, 60 times per second, and outputs the captured images to the image processing device 10. The mounting position of the vehicle-mounted camera 20, the imaging range of the vehicle-mounted camera 20, and the imaging cycle of the vehicle-mounted camera 20 may be determined arbitrarily.

[0018] Here, the vehicle-mounted camera 20 has red, green, and blue color filters as described above. Therefore, the captured image has red pixels that hold red luminance values, blue pixels that hold blue luminance values, and green pixels that hold green luminance values, as shown in Fig. 2. In Fig. 2, the red pixels are indicated by R, the blue pixels by B, and the green pixels by G, and each pixel has a luminance value within the range of, for example, 0 to 255.

[0019] The illuminance sensor 30 is configured using a photodiode, a transistor, and the like, and outputs a detection signal corresponding to the illuminance (that is, brightness) of the surroundings of the vehicle.

[0020] The image display device 40 is provided within the driver's field of vision, for example, near the front windshield, and displays the corrected image output from the image processing device 10.

[0021] The driving support device 50 is an ECU, and is configured with a microcomputer or the like equipped with a storage unit or the like configured with a CPU, a ROM, a RAM, a flash memory, and a HDD, etc. CPU is an abbreviation for Central Processing Unit, ROM is an abbreviation for Read Only Memory, RAM is an abbreviation for Random Access Memory, and HDD is an abbreviation for Hard Disk Drive. Storage media such as ROM are non-transient substantial storage media.

[0022] The driving support device 50 performs driving support such as, for example, warning output control and driving control based on the corrected image output from the image processing device 10. The warning output control is a control that outputs a warning when the vehicle is likely to depart from the lane. The driving control is a control of steering and braking of the vehicle that is performed so as to drive within the lane. The driving support device 50 may be configured with one microcomputer or may be configured with multiple microcomputers.

[0023] The image processing device 10 is composed of a microcomputer having a storage unit 12 composed of a CPU 11, a ROM, a RAM, a flash memory, and a HDD. The image processing device 10 has each unit as shown in FIG. 3 so that the CPU 11 executes a program stored in the storage unit. In this embodiment, the storage unit 12 corresponds to a non-transferable tangible recording medium that stores a computer program. The computer program is executed to execute a method corresponding to the computer program. The image processing device 10 may be composed of one microcomputer or multiple microcomputers.

[0024] Specifically, the image processing device 10 of this embodiment includes an image acquisition unit 101, an illuminance determination unit 102, an image correction unit 103, a determination unit 104, and an output unit 105, as shown in FIG.

[0025] The image acquisition unit 101 acquires images captured by the vehicle-mounted cameras 20. In this embodiment, since the two vehicle-mounted cameras 20 are provided as described above, the image acquisition unit 101 acquires images from each of the vehicle-mounted cameras 20.

[0026] The illuminance determination unit 102 acquires a detection signal from the illuminance sensor 30, and compares the acquired detection signal with a predetermined threshold to determine whether the detection signal is equal to or lower than the predetermined threshold. That is, the illuminance determination unit 102 determines whether the illuminance is equal to or lower than the predetermined threshold based on the detection signal from the illuminance sensor 30. The predetermined threshold is set based on a value when the surrounding situation is nighttime. That is, the illuminance determination unit 102 determines whether the surrounding situation is nighttime or other situation (for example, daytime).

[0027] The image correction unit 103 performs a correction process to create a corrected image based on the captured image. Specifically, when the illuminance determination unit 102 determines that it is nighttime and that there is a point light source, the image correction unit 103 refers to correction data described later, grasps the luminance value ratio of the point light source range described later, and creates a corrected image. When the illuminance determination unit 102 determines that it is not nighttime and that there is no point light source, the image correction unit 103 grasps the red luminance value, green luminance value, and blue luminance value of each pixel from the average luminance value, etc., as in the conventional case, and creates a corrected image based on the luminance value ratio of each pixel. Furthermore, for a range different from the point light source range described later, the image correction unit 103 grasps the red luminance value, green luminance value, and blue luminance value of each pixel from the average luminance value, etc., as in the conventional case, and creates a corrected image based on the luminance value ratio of each pixel. Note that the point light source in this embodiment is a light source that emits light only to specific pixels (for example, 1 to 3 pixels), and is, for example, a distant tail light or reflector at night.

[0028] Here, the red luminance value, green luminance value, and blue luminance value of each pixel are grasped from the average luminance value or the like, which can be simply explained as follows. For example, the red luminance value of a red pixel in a captured image is grasped as the red luminance value of the red pixel. The green luminance value of a red pixel is grasped from the average value of the green luminance values ​​of the green pixels surrounding the red pixel, and the blue luminance value of a red pixel is grasped from the average value of the blue luminance values ​​of the blue pixels surrounding the red pixel. Similarly, the green luminance value of a green pixel is grasped as the green luminance value of the green pixel. The red luminance value of a green pixel is grasped from the average value of the red luminance values ​​of the red pixels surrounding the green pixel, and the blue luminance value of a green pixel is grasped from the average value of the blue luminance values ​​of the blue pixels surrounding the green pixel. The blue luminance value of a blue pixel is grasped as the blue luminance value of the blue pixel. The red luminance value of a blue pixel is determined from the average value of the red luminance values ​​of the red pixels surrounding the blue pixel, and the green luminance value of a blue pixel is determined from the average value of the green luminance values ​​of the green pixels surrounding the blue pixel.

[0029] In this embodiment, when determining whether or not there is a point light source, the image correction unit 103 scans each pixel of the captured image in order to grasp the presence or absence of a point light source. In this embodiment, each pixel of the captured image is scanned in order to derive the average luminance value of each pixel, and if the luminance value protrudes by a predetermined value or more from the average luminance value and the protruding luminance portion extends only over a predetermined range (i.e., a predetermined pixel), it is determined that there is a point light source. In other words, in this embodiment, it is determined that there is a point light source when light is incident only on a specific pixel. In this embodiment, for example, when the luminance values ​​of about 1 to 3 pixels protrude, it is determined that there is a point light source in the protruding pixel. In other words, when the luminance values ​​of a large number of connected pixels (for example, 4 pixels or more) protrude by a predetermined value or more from the average luminance value, the image correction unit 103 determines that light is incident on each pixel as a whole and does not determine that there is a point light source.

[0030] Note that the method for determining whether or not there is a point light source may be different. For example, a part having a luminance value that is higher than the luminance values ​​of the surrounding pixels by a predetermined value or more may be considered to be a point light source, or a point light source may be derived using a method using standard deviation.

[0031] Then, the image correction unit 103 sets a predetermined range including the pixels that become the point light source as the point light source range. For example, the image correction unit 103 of this embodiment sets a 5×5 pixel range substantially centered on the point light source as the point light source range.

[0032] Next, the image correction unit 103 obtains the red luminance value ratio when only red pixels are included, the green luminance value ratio when only green pixels are included, and the blue luminance value ratio when only blue pixels are included, for the point light source area. That is, the image correction unit 103 obtains the luminance value ratio for each pixel excluding the sensitivity difference of the color filter. The following describes an example of obtaining the luminance value ratio of only green pixels.

[0033] First, for example, assume that the color filter is composed of only green filters, and green light is incident only on the central pixel C (i.e., a specific location) as shown in FIG. 4. In FIG. 4, the central pixel C is hatched. In this case, ideally, only the green luminance value of the central pixel C becomes high, but in reality, lens blur occurs, and the luminance value ratio along the arrow A in FIG. 4 becomes a luminance value ratio distribution as shown in FIG. 5. FIG. 5 shows a diagram normalized with the peak of the luminance value set to 1 (i.e., a diagram shown by ratio). The luminance value ratio of the x position and the y position in each two-dimensional pixel is expressed by the following formula 1. In addition, σ in formula 1 is a coefficient determined by lens dependency that defines the performance (e.g., resolution, etc.) of the vehicle-mounted camera 20.

[0034]

number

[0035] Then, the image correction unit 103 sequentially scans the point light source area and grasps the luminance value ratio of each pixel for each color. An example of scanning a part where green pixels and red pixels are alternately arranged in the order of arrow B as shown in FIG. 6 and grasping the green luminance value ratio of the scanned part will be described below with reference to FIG. 7A, FIG. 7B, FIG. 8A, FIG. 8B, FIG. 9A, FIG. 9B. In addition, as shown in FIG. 6, the pixel positions are P1 to P11 in order from the left side of the paper, and the green pixels are at P1, P3, P5, P7, P9, P11, and the red pixels are at P2, P4, P6, P8, P10. In addition, FIG. 7A, FIG. 7B, FIG. 8A, FIG. 8B, FIG. 9A, FIG. 9B show diagrams normalized with the peak of the luminance value set to 1 (i.e., diagrams shown by ratio). Furthermore, in the following, an example of using the luminance value ratio distribution of FIG. 5 as correction data will be described.

[0036] When the green luminance value ratio stands out at only one point among the scanned pixels as shown in Fig. 7A, image correction unit 103 refers to the luminance value ratio distribution in Fig. 5 as correction data and determines that the green luminance value ratios at each of pixel positions P1 to P11 are as shown in Fig. 7B. Therefore, image correction unit 103 determines that a point light source exists at pixel position P7 and can grasp the green luminance value ratios at each of pixel positions P1 to P11.

[0037] Furthermore, as shown in Fig. 8A, when two points with high green luminance value ratios are the same, the luminance value ratio distribution in Fig. 5 is referenced as correction data, and the luminance value ratios at each of pixel positions P1 to P11 are determined to be as shown in Fig. 8B. Therefore, image correction unit 103 determines that a point light source exists at pixel position P6, and can grasp the green luminance value ratios at each of pixel positions P1 to P11.

[0038] Furthermore, as shown in Fig. 9A, when the green luminance value ratio stands out at two points and the luminance value ratios of the two outstanding points are different, the luminance value ratio distribution of Fig. 5 is referenced as correction data, and it is determined that the luminance value ratios of each of the pixel positions P1 to P11 are as shown in Fig. 9B. Therefore, the image correction unit 103 determines that a point light source exists on the pixel position P5 side of the pixel position P6, and can grasp the luminance value ratios of each of the pixel positions P1 to P11.

[0039] Then, the image correction unit 103, although details are omitted, grasps the red luminance value ratio when only red pixels are used and the blue luminance value ratio when only blue pixels are used, as well as the green luminance value ratio in each pixel. However, in reality, the captured image is scanned two-dimensionally to grasp the red luminance value ratio, green luminance value ratio, and blue luminance value ratio in each pixel in the two dimensions. Note that the grasped red luminance value ratio is the luminance value ratio when only red pixels are used, the grasped green luminance value ratio is the luminance value ratio when only green pixels are used, and the grasped blue luminance value ratio is the luminance value ratio when only blue pixels are used. In other words, the ratio of each pixel is a ratio excluding the sensitivity difference of the color filter, and is a ratio indicating the color difference in a state where the brightness is the same. Then, the image correction unit 103 creates a corrected image based on the grasped red luminance value ratio, green luminance value ratio, and blue luminance value ratio.

[0040] Furthermore, in the range different from the range with point light source, the image correction unit 103 creates a corrected image by adding the luminance value ratio of each pixel using the average luminance value, etc., as described above. Then, the image correction unit 103 also performs general tone curve processing, etc., on the created corrected image as necessary.

[0041] The determination unit 104 compares the results of the corrected images for each captured image. In this embodiment, since two vehicle-mounted cameras 20 are provided, the corrected images for the two captured images are compared. Then, if the difference between the red luminance value ratio, the green luminance value ratio, and the blue luminance value ratio in each corrected image is within a predetermined threshold range, the determination unit 104 performs an abnormality determination to determine that the two created corrected images are normal. On the other hand, if the difference between the red luminance value ratio, the green luminance value ratio, and the blue luminance value ratio in each corrected image is outside the predetermined threshold range, the determination unit 104 performs an abnormality determination to determine that the vehicle-mounted camera 20 has malfunctioned or the like.

[0042] The output unit 105 outputs the corrected image. As a result, the corrected image is displayed on the image display device 40. Furthermore, the driving assistance device 50 executes driving assistance based on the corrected image.

[0043] Next, the process executed by the image processing device 10 will be described with reference to the flowchart of FIG.

[0044] First, in step S1, the image processing device 10 acquires a captured image from the vehicle-mounted camera 20 by the image acquisition unit 101. In this embodiment, since two vehicle-mounted cameras 20 are provided, two captured images are acquired. Also, as described above, the captured image is an image having a red luminance value of a red pixel, a green luminance value of a green pixel, and a blue luminance value of a blue pixel.

[0045] Next, in step S2, the image processing device 10 determines whether it is nighttime or not using the illuminance determination unit 102 based on the detection signal of the illuminance sensor 30. Then, when the image processing device 10 determines that it is nighttime using the illuminance determination unit 102 (i.e., step S2: YES), in step S3, the image correction unit 103 scans the captured image to grasp the luminance value of each pixel.

[0046] In step S4, the image processing device 10 determines whether or not there is a point light source in the image correction unit 103. As described above, the image correction unit 103 of this embodiment sequentially scans the captured image to derive the average luminance value, and determines that there is a point light source when the luminance value is equal to or greater than the average luminance value by a predetermined value and the portion with that luminance value exists only within a predetermined range (i.e., predetermined pixels).

[0047] Next, in step S5, the image processing device 10 defines a predetermined range including the point light source as a range with a point light source in the image correction unit 103. Although not particularly limited, in this embodiment, the range with a point light source is defined as a 5×5 pixel range centered on the point light source.

[0048] Then, in step S6, the image processing device 10 derives the luminance value ratio of the point light source area and creates a corrected image of the point light source area by the image correction unit 103. Specifically, as described above, the image correction unit 103 creates an image of only red pixels, an image of only blue pixels, and an image of only blue pixels in the point light source area, and creates the corrected image by grasping the luminance value ratio of each pixel based on the correction data.

[0049] In step S7 after the process of step S6, the image processing device 10 performs normal processing in the image correction unit 103. Specifically, in the normal processing after the process of step S6, the image correction unit 103 grasps the red luminance value, green luminance value, and blue luminance value of each pixel in a range different from the point light source range from the average luminance value, derives the luminance value ratio of each pixel, and creates a corrected image. Thereafter, the image correction unit 103 performs tone curve processing, etc. on the corrected image as necessary.

[0050] Furthermore, if the image processing device 10 determines that it is not nighttime (i.e., step S2: NO) or that there is no point light source (i.e., step S4: NO), it performs normal processing on the entire captured image in step S7. Specifically, for the entire captured image, the red luminance value, green luminance value, and blue luminance value of each pixel are found from the average luminance value, and a corrected image is created by deriving the luminance value ratio of each pixel. Thereafter, the image correction unit 103 performs tone curve processing or the like on the corrected image as necessary.

[0051] Next, in step S8, the image processing device 10 determines whether or not the corrected image is normal by the determination unit 104. In this embodiment, as described above, two vehicle-mounted cameras 20 are provided, and captured images are obtained from each vehicle-mounted camera 20. Then, a corrected image is created from each captured image. Therefore, the determination unit 104 determines whether or not the corrected image is normal by comparing the two corrected images.

[0052] When the determination unit 104 determines that the corrected image is not normal (i.e., step S8: NO), the image processing device 10 ends the process. At this time, the result of determining that the image is not normal may be output to the driving support device 50, and an alarm output control or the like may be controlled to notify the occupant that the image could not be detected appropriately.

[0053] When the determination unit 104 determines that the corrected image is normal (i.e., step S8: YES), the image processing device 10 outputs the corrected image from the output unit 105 in step S9. As a result, the corrected image is displayed on the image display device 40. In addition, the driving assistance device 50 executes driving assistance based on the corrected image.

[0054] According to the embodiment described above, the image processing device 10 grasps the luminance value ratio of each color in each pixel of the captured image using the correction data. Therefore, the grasped luminance value ratio of each color in each pixel is a ratio excluding the sensitivity difference of the color filter, and is a ratio indicating the color difference in a state where the brightness is the same. Therefore, the accuracy of grasping the luminance value ratio of each pixel can be improved.

[0055] (1) In this embodiment, two vehicle-mounted cameras 20 are provided for capturing an image of a common field of view, and an image acquisition unit 101 acquires captured images from each vehicle-mounted camera 20. Then, an image correction unit 103 creates a corrected image from each captured image. A determination unit 104 compares the created corrected images, and determines that the images are normal if the difference is within a predetermined threshold range, and determines that the images are abnormal if the difference is outside the predetermined threshold range. This makes it possible to perform abnormality determination while ensuring robustness.

[0056] (2) In this embodiment, the image correction unit 103 defines a point light source area and creates a corrected image using correction data for the point light source area. That is, the image correction unit 103 creates a corrected image using correction data for an area where light is incident only on specific pixels and the luminance value ratio is difficult to grasp. Therefore, compared to a case where a corrected image is created using correction data for the entire captured image, it is possible to reduce the processing load while improving the accuracy of grasping the luminance value ratio of each pixel in the point light source area.

[0057] (Other embodiments) Although the present disclosure has been described based on the embodiment, it is understood that the present disclosure is not limited to the embodiment or structure. The present disclosure also encompasses various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more than one element, or less than one element, are also within the scope and concept of the present disclosure.

[0058] For example, in the above-described first embodiment, a vehicle is taken as an example of a moving body, but the moving body may be a train or the like.

[0059] In the first embodiment, the example in which FIG. 5 is used as the correction data has been described, but the above-mentioned formula 1 may be used as the correction data.

[0060] Furthermore, in the above first embodiment, an example was described in which correction data was used for the area with a point light source, but the luminance value ratio of each pixel may be grasped by using correction data for the entire captured image. However, when applying correction data to the entire captured image, there is a concern that the load of processing may increase, so it is preferable to use correction data only for the area with a point light source.

[0061] In the above first embodiment, an example has been described in which the storage unit 12 is included as a component separate from the image correction unit 103. However, the storage unit 12 may be incorporated within the image correction unit 103.

[0062] The control unit and the method described in the present disclosure may be implemented by a special-purpose computer provided by configuring a processor and a memory programmed to execute one or more functions embodied in a computer program. Alternatively, the control unit and the method described in the present disclosure may be implemented by a special-purpose computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control unit and the method described in the present disclosure may be implemented by one or more special-purpose computers configured by a combination of a processor and a memory programmed to execute one or more functions and a processor configured with one or more hardware logic circuits. The computer program may also be stored in a computer-readable non-transitory tangible recording medium as instructions executed by the computer. [Explanation of symbols]

[0063] 10 Image processing device 12 Storage section 20 Imaging unit (vehicle camera) 101 Image acquisition unit 102 Illuminance judgment section 103 Image correction section 104 Output section

Claims

1. An image processing device, an image acquisition unit (101) mounted on a moving object, which is imaged by an image capture unit (20) having a plurality of color filters and acquires an image in which a luminance value for each color corresponding to each of the color filters is held in each pixel; An illuminance determination unit (102) for determining the illuminance around the moving object; an image correction unit (103) that performs a predetermined correction process on the captured image acquired by the image acquisition unit to create a corrected image; an output unit (105) that outputs the corrected image processed by the image correction unit; A storage unit (12) for storing predetermined data, In the storage unit, a filter of only one color included in the color filter is arranged as a color filter of the imaging unit, and correction data regarding a luminance value ratio of the one color of an image captured when light is incident only on a specific portion of the filter is stored for each of the multiple colors constituting the color filter, The image correction unit is an image processing device that, when the illuminance determination unit determines that the illuminance around the moving body is below a predetermined threshold, performs a correction process for each of the multiple colors that constitute the color filter to determine the luminance value ratio of one color for a specific pixel using the luminance value of that color contained in the captured image and correction data corresponding to the one color, and creates the corrected image using the determined luminance value ratios of each of the multiple colors at each of the pixels.

2. The image acquisition unit acquires images captured by different imaging units that capture a common field of view, the image correction unit performs a correction process to create the corrected image from each of the captured images acquired by the image acquisition unit; 2. The image processing apparatus according to claim 1, further comprising a determination unit (104) for comparing the created plurality of corrected images and determining that the corrected images have been created normally when a difference between brightness value ratios is within a predetermined threshold range.

3. 3. The image processing device according to claim 1, wherein the image correction unit scans the captured image to identify a point light source and defines a predetermined range including the point light source as a point light source range, and performs the correction process on the point light source range by defining the predetermined range as the point light source range.

Images