Image processing device

Abstract

The objective of the present invention is to provide an image processing device capable of accurately calculating the distance to an object serving as a light source, from an image captured by a stereo camera using a normal exposure amount. To this end, an image processing device 100 comprises a stereo camera 10 that captures a first image P1, a normal shutter image processing unit 102 that detects first object information from the first image P1, an object information acquiring device 10 that acquires second object information, a distance correcting unit 109 that corrects a first distance included in the first object information, and a distance correction value calculating unit 108 that calculates a correction value for the first distance, wherein: the distance correction value calculating unit 108 learns a correlation between the amount of flare light of the object in the first image P1 and the difference between the first distance and a second distance included in the second object information, and calculates the correction value on the basis of the correlation and the amount of flare light; and the distance correcting unit 109 corrects the first distance by subtracting the correction value from the first distance.

Description

Image processing device

[0001] The present invention relates to an image processing device that recognizes an environment outside a vehicle based on an image captured outside the vehicle.

[0002] In recent years, automobiles equipped with driving assistance functions that assist with acceleration, deceleration, and steering operations according to the external environment have become increasingly common, and there is also a growing need for driving assistance functions during nighttime driving. To provide appropriate driving assistance during nighttime driving, it is necessary to accurately estimate the distance to and speed of the preceding vehicle, regardless of the brightness of the preceding vehicle's brake lights. Therefore, recent image processing devices are required to have enhanced functionality to accurately detect light sources ranging from low to high brightness, and technologies such as HDR (High Dynamic Range), which expand the dynamic range during imaging, are being used.

[0003] However, when the dynamic range during imaging is widened, a flare (halation) phenomenon, in which the area around a high-intensity light source (e.g., a brake light during braking) becomes bright and blurred, is likely to occur when an image of the light source is captured. When flare occurs, there is a problem that a three-dimensional object cannot be correctly detected in the flare area. For example, Patent Document 1 discloses a prior art aimed at solving this problem.

[0004] Patent Document 1 describes an image processing device comprising: "a camera that captures a first image with a first exposure amount and captures a second image with a second exposure amount that is less than the first exposure amount; a three-dimensional object extraction unit that extracts a first area in which a three-dimensional object exists from the first image and extracts a second area in which the three-dimensional object exists from the second image; a three-dimensional object information detection unit that detects first three-dimensional object information from the first area and detects second three-dimensional object information from the second area; and a three-dimensional object information integration unit that calculates integrated three-dimensional object information that integrates the first three-dimensional object information and the second three-dimensional object information."

[0005] International Publication No. 2022 / 254795

[0006] According to the image processing device described in Patent Document 1, by integrating first three-dimensional object information obtained from a first image at a first exposure amount (normal shutter) in which flare occurs, and second three-dimensional object information obtained from a second image at a second exposure amount (low exposure shutter) in which flare does not occur, it is possible to suppress the influence of flare in the first image in the integrated three-dimensional object information.

[0007] However, because the detection results from the normal shutter and the low-exposure shutter are combined, the error in the combined result increases as the flare from the normal shutter increases. Here, when combining the two detection results, it is possible to eliminate the influence of flare in the combined result by setting the weight of the detection result from the normal shutter to zero, but this creates the problem of lengthening the observation cycle because distance observation is performed only with the low-exposure shutter.

[0008] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide an image processing device that can accurately calculate the distance to an object that serves as a light source from an image captured with a normal exposure amount using a stereo camera.

[0009] In order to achieve the above object, the present invention provides an image processing device including a stereo camera that captures a first image, which is an image with a first exposure amount, and a normal shutter image processing unit that detects information about an object included in the first image as first object information, the image processing device including: an object information acquisition device that acquires information about the object as second object information; a distance correction unit that corrects a first distance, which is a distance to the object, included in the first object information; and a distance correction value calculation unit that calculates a correction value for the first distance, wherein the distance correction value calculation unit learns a correlation between an amount of flare light of the object in the first image and a difference between the first distance and a second distance, which is the distance to the object, included in the second object information, and calculates the correction value based on the correlation and the amount of flare light, and the distance correction unit corrects the first distance by subtracting the correction value from the first distance.

[0010] According to the image processing device of the present invention, it is possible to accurately calculate the distance to an object that serves as a light source from an image captured with a stereo camera at a normal exposure level.

[0011] FIG. 1 is a hardware configuration diagram of an image processing device. FIG. 2 is a functional block diagram of an image processing device in the prior art. FIG. 3 is a diagram showing an example of a first image and a second image output from a stereo camera. FIG. 4 is a diagram showing a method for measuring a first distance. FIG. 5 is a diagram showing a mechanism by which parallax error occurs due to flare. FIG. 6 is a diagram showing an example of a method for calculating the amount of flare light. FIG. 7 is a functional block diagram of an image processing device in a first embodiment of the present invention. FIG. 8 is a diagram showing processing in a learning phase of a distance correction value calculation unit. FIG. 9 is a diagram showing processing of a distance correction unit. FIG. 10 is a functional block diagram of an image processing device in a second embodiment of the present invention.

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In each drawing, the same parts are designated by the same reference numerals, and duplicated explanations will be omitted.

[0013] 1 is a hardware configuration diagram of an image processing device 100 according to a first embodiment of the present invention. The image processing device 100 is an in-vehicle device that recognizes the external environment based on captured images of the vehicle, such as white lines on the road, pedestrians, other vehicles, other three-dimensional objects, traffic lights, signs, and lit lamps. The image processing device 100 then determines control policies, such as acceleration / deceleration assistance and steering assistance for the vehicle, based on the recognized external environment.

[0014] 1, the image processing device 100 includes a stereo camera 10 (a left camera 10L and a right camera 10R), an image input interface 11, an image processing unit 12, an arithmetic processing unit 13, a storage unit 14, and a CAN interface 15. The components from the image input interface 11 to the CAN interface 15 are configured as a single or multiple computer units interconnected via an internal bus, and various functions of the image processing unit 12, the arithmetic processing unit 13, etc. are realized by the arithmetic devices (CPUs, etc.) included in the computer units executing predetermined programs.

[0015] The stereo camera 10 consists of a left camera 10L and a right camera 10R mounted on the vehicle to capture images of the area ahead. The imaging element of the left camera 10L captures a first left image P1L with a normal exposure (hereinafter referred to as a normal shutter) and a second left image P2L with a lower exposure (hereinafter referred to as a low-exposure shutter), either simultaneously or alternately with a time lag. In parallel with the image capture by the left camera 10L, the imaging element of the right camera 10R captures a first right image P1R with a normal shutter and a second right image P2R with a low-exposure shutter, either simultaneously or alternately with a time lag. "Exposure" here refers to the brightness of the screen, determined by factors such as exposure time, aperture, and gain during capture. The normal shutter widens the dynamic range and prevents underexposure and overexposure from non-light-emitting objects to light-emitting objects. The low-exposure shutter reduces the exposure compared to the normal shutter, reducing the effects of flare. "Flare" is a phenomenon in which, when a bright object such as a light spot is photographed with a camera, the object appears blurred due to strong light being reflected inside the lens.

[0016] The image input interface 11 is an interface that controls the imaging of the stereo camera 10 and captures the captured images P1L, P2L, P1R, and P2R. The image P captured through this interface is sent to the image processing unit 12 and the like through an internal bus.

[0017] The image processing unit 12 compares the left images P1L, P2L captured by the left camera 10L with the right images P1R, P2R captured by the right camera 10R, and performs corrections such as correction of device-specific deviations caused by the imaging element and noise interpolation on each image, and then stores the corrected image P in the memory unit 14.

[0018] Furthermore, the image processing unit 12 calculates corresponding points between the left and right images with the same exposure amount, calculates parallax information (distance information for each point on the image), and stores the information in the storage unit 14. Specifically, the first parallax information is calculated by comparing the first left image P1L and the first right image P1R, which were captured at the same timing, and the second parallax information is calculated by comparing the second left image P2L and the second right image P2R, which were captured at the same timing.

[0019] The arithmetic processing unit 13 recognizes various objects necessary for perceiving the environment around the vehicle using the images P1L, P2L, P1R, and P2R stored in the memory unit 14 and the parallax information. The various objects recognized here include people, other vehicles, other obstacles, traffic lights, signs, taillights and headlights of other vehicles, etc. These recognition results and some of the intermediate calculation results are recorded in the memory unit 14. Furthermore, the arithmetic processing unit 13 determines a control policy for the vehicle using the object recognition results.

[0020] The memory unit 14 is a storage device such as a semiconductor memory, and stores the corrected images P1L, P2L, P1R, and P2R and disparity information that are the output of the image processing unit 12, as well as the object recognition results and vehicle control policies that are the output of the calculation processing unit 13.

[0021] The CAN interface 15 is an interface that transmits the object recognition results obtained by the arithmetic processing unit 13 and the vehicle's control policy to an in-vehicle network (Controller Area Network) 16. Note that the in-vehicle network 16 is connected to a control system (ECU, etc.) that controls the vehicle's drive system, braking system, steering system, etc., and the stereo camera 10 can perform driving assistance such as automatic braking and steering avoidance in accordance with the environment outside the vehicle by using the control system.

[0022] The image processing device 100 of this embodiment will be described below in comparison with an image processing device 200 of the prior art.

[0023] 2 is a functional block diagram of a conventional image processing device 200. In FIG. 2, the image processing device 200 includes an imaging unit 101, a normal shutter image processing unit 102, a low-exposure shutter image processing unit 103, a lamp pair detection unit 104, a lamp pair detection unit 105, a lamp detection result linking unit 106, a flare determination unit 107, an object information integration unit 201, a classification unit 110, and a detection result output unit 112.

[0024] The imaging unit 101 is composed of a stereo camera 10, which outputs a first image P1 and first parallax information captured with a normal shutter and a second image P2 and second parallax information captured with a low-exposure shutter simultaneously or alternately with a staggered timing. An example of the first image P1 and second image P2 output from the stereo camera 10 is shown in FIG. 3. In this example, the first image P1 and the second image P2 are output alternately with a staggered timing.

[0025] Returning to FIG. 2 , the normal shutter image processing unit 102 extracts and outputs first object information (shown in FIG. 3 ), which is information about the object, from the first image P1 input from the imaging unit 101. The low-exposure shutter image processing unit 103 extracts and outputs second object information (shown in FIG. 3 ), which is information about the object, from the second image P2 input from the imaging unit 101. The first object information includes the distance to the object detected in the first image P1 (first distance), and the second object information includes the distance to the object detected in the second image P2 (second distance). FIG. 4 shows a method for measuring the first distance. Due to the difference in the left and right installation positions of the left camera 10L and the right camera 10R, the positions of the object in the captured first left image P1L and first right image P1R also shift left and right. This shift amount becomes parallax, and the first distance is calculated by converting this parallax into distance using triangulation. The second distance can also be measured in a similar manner.

[0026] FIG. 5 illustrates the mechanism by which parallax error occurs due to flare. With a normal shutter, if the flare occurrence conditions differ between the first left image P1L and the first right image P1R when capturing a light source due to differences in the lens characteristics of the left camera 10L and the right camera 10R, the left and right images will appear different, resulting in parallax error. On the other hand, with a low-exposure shutter, the effect of flare is reduced by narrowing the exposure, thereby suppressing parallax error. As shown in FIG. 3, in the first image P1 captured with a normal shutter, the brightness of the preceding vehicle's body is high, so the accuracy of detecting the vehicle's height is high. However, the lamp pair appears to expand laterally due to flare, reducing the accuracy of detecting the vehicle's width. As a result, accurate parallax of the preceding vehicle cannot be obtained, and the accuracy of the distance to the preceding vehicle also decreases. On the other hand, in the second image P2 captured with a low-exposure shutter, the brightness of the vehicle body decreases, so the accuracy of detecting the vehicle's height decreases, but the accuracy of detecting the vehicle's width improves due to the suppression of flare. As a result, accurate parallax of the preceding vehicle can be obtained, thereby improving the accuracy of the distance to the preceding vehicle.

[0027] 2 , the lamp pair detection unit 104 performs lamp pair detection on the first image P1 based on the first object information from the normal shutter image processing unit 102. The lamp pair detection unit 105 performs lamp pair detection on the second image P2 based on the second object information from the low-exposure shutter image processing unit 103.

[0028] The lamp detection result linking unit 106 performs a linking process of the object information of each lamp pair detected by the lamp pair detection unit 104 and the lamp pair detection unit 105 .

[0029] The flare determination unit 107 calculates the flare light amount in the first image P1. The "flare light amount" is a physical quantity that represents the degree of flare. FIG. 6 shows an example of a method for calculating the flare light amount. First, a brightness threshold for extracting lamps for each exposure amount is determined in advance, and the pixel area exceeding the brightness threshold is extracted as the lamp. Next, lamp sizes W1 and W2 are calculated from the width of each lamp. Then, the difference between the lamp size W1 for the normal shutter (first image P1) and the lamp size W2 for the low-exposure shutter (second image P2) is calculated as the flare light amount. As an example of the difference between the lamp sizes W1 and W2, it is possible to calculate the area by subtracting the area of ​​the circular area for lamp size W2 from the area of ​​the circular area for lamp size W1, as shown in FIG. 6. Note that the flare light amount may also be calculated from only the image taken with the normal shutter. In this case, the amount of flare light can be calculated as the area of ​​the region where the brightness exceeds a predetermined threshold, the radius of the circle circumscribing the region where the brightness exceeds the predetermined threshold, the width of the region where the brightness exceeds the predetermined threshold, the average / maximum brightness of a predetermined region centered on the light point, etc.

[0030] 2 , if the amount of flare light calculated by flare determination unit 107 is equal to or greater than a predetermined threshold, object information integration unit 201 integrates, under a predetermined rule, the first object information from normal shutter image processing unit 102 and the second object information from low-exposure shutter image processing unit 103. On the other hand, if the amount of flare light calculated by flare determination unit 107 is less than the predetermined threshold, the first object information from normal shutter image processing unit 102 is output as is.

[0031] The identification unit 110 determines the type of the object based on the object information output by the object information integration unit 201 .

[0032] The detection result output unit 112 outputs the object information (position, speed, shape, etc.) output by the object information integration unit 201 and the object type output by the identification unit 110 to the control system (ECU, etc.) via the CAN interface 15 and the in-vehicle network 16.

[0033] In the image processing device 200 of the prior art, the detection results from the normal shutter and the low-exposure shutter are integrated, and as the amount of flare light from the normal shutter increases, the error in the integrated result increases. Here, when integrating the two detection results, it is possible to eliminate the influence of flare in the integrated result by setting the weight of the detection result from the normal shutter to zero, but this creates the problem of a longer observation cycle because distance observation is performed only with the low-exposure shutter. The image processing device 100 of this embodiment solves this problem.

[0034] Fig. 7 is a functional block diagram of an image processing device 100 according to this embodiment. In Fig. 7, the difference from the image processing device 200 (shown in Fig. 2) according to the prior art is that the image processing device 100 includes a distance correction value calculation unit 108, a distance correction unit 109, an identification unit 111, and a detection result output unit 113 instead of the object information integration unit 201 (shown in Fig. 2).

[0035] The distance correction unit 109 corrects the first distance included in the first object information from the normal shutter image processing unit 102 and outputs the first object information with the corrected first distance. The method of correcting the first distance will be described later.

[0036] The identification unit 110 determines the type of the three-dimensional object based on the corrected first object information output by the distance correction unit 109 .

[0037] The identification unit 111 determines the type of the object based on the second object information output by the low-exposure shutter image processing unit 103 .

[0038] The detection result output unit 113 outputs the second object information (position, speed, shape, etc.) output from the low-exposure shutter image processing unit 103 and the object type output from the identification unit 111 to the control system (ECU, etc.) via the CAN interface 15 and the in-vehicle network 16.

[0039] In a correction phase, which is a phase other than the learning phase described below, the distance correction value calculation unit 108 converts the amount of flare light input from the flare determination unit 107 into a correction value (distance correction value) for the first distance obtained with a normal shutter, based on the correlation between the amount of flare light and the parallax error learned in the learning phase, and outputs the converted value.

[0040] FIG. 8 is a diagram illustrating processing performed by the distance correction value calculation unit 108 in the learning phase. During the learning phase, the distance correction value calculation unit 108 performs regression analysis on the combined information of the flare light intensity of multiple objects and the parallax difference between normal and low-exposure shutters, obtained from multiple frames of the first image P1 and the second image P2 stored and accumulated in the storage unit 14, to determine the correlation (regression line or regression curve) between the flare light intensity and the parallax difference between normal and low-exposure shutters. For example, if the regression model representing the relationship between variables x and y is a linear equation (y = ax + b), the regression coefficients a and b are determined by regression analysis. Note that the regression analysis model is determined in advance in a calibration or other process. The timing of performing the regression analysis is not particularly limited, and it may be performed at any time as long as it does not interfere with other functions of the image processing device 100. When the parallax error with a low-exposure shutter is set to zero, the parallax difference is equal to the parallax error. Therefore, the parallax error can be determined from the flare light intensity using the above correlation. In the correction phase, the distance correction unit 109 converts the flare light amount input from the flare determination unit 107 into a parallax error based on the above-mentioned correlation, and calculates a distance correction value by converting the parallax error into a distance.

[0041] 9 is a diagram showing the processing of distance correction unit 109. Distance correction unit 109 corrects the first distance by subtracting the distance correction value input from distance correction value calculation unit 108 from the first distance included in the first object information input from normal shutter image processing unit 102. This makes it possible to perform highly accurate distance observation even with only a normal shutter when flare occurs.

[0042] (Summary) In this embodiment, an image processing device 100 includes a stereo camera 10 that captures a first image P1 that is an image with a first exposure amount, and a normal shutter image processing unit 102 that detects information about an object included in the first image P1 as first object information. The image processing device 100 also includes an object information acquisition device (stereo camera 10) that acquires information about the object as second object information, a distance correction unit 109 that corrects a first distance that is a distance to the object and is included in the first object information, and a distance correction value calculation unit 108 that calculates a correction value for the first distance. The distance correction value calculation unit 108 learns a correlation between an amount of flare light of the object in the first image P1 and a difference between the first distance and a second distance that is the distance to the object and is included in the second object information, and calculates the correction value based on the correlation and the amount of flare light. The distance correction unit 109 corrects the first distance by subtracting the correction value from the first distance.

[0043] According to the present embodiment configured as described above, the correlation (regression line or regression curve) between the amount of flare light from the light source contained in the image (first image P1) captured with a normal shutter by the stereo camera 10 and the difference between the distance to the light source (first distance) captured with a normal shutter and the distance to the light source (second distance) captured by the object information acquisition device 10 is learned, and the first distance is corrected based on this correlation. This makes it possible to accurately calculate the distance to the object serving as the light source from the image (first image P1) captured with a normal exposure by the stereo camera 10. As a result, the distance and speed estimation accuracy of the image processing device 100 and the success rate of detecting and tracking oncoming vehicles are improved. Furthermore, it becomes possible to use inexpensive lenses that are prone to generating flare in the stereo camera 10.

[0044] Furthermore, the object information acquisition device 10 in this embodiment is a stereo camera 10 that captures a second image P2, which is an image with a second exposure amount that is less than the first exposure amount, and the image processing device 100 includes a low-exposure shutter image processing unit 103 that calculates the distance to the object included in the second image P2 as the second distance. This eliminates the need to separately add an object information acquisition device for acquiring the distance to a high-brightness light source with higher accuracy than with a normal shutter, making it possible to prevent an increase in the cost of the image processing device 100.

[0045] Furthermore, the image processing device 100 in this embodiment includes object extraction units (a normal shutter image processing unit 102 and a low-exposure shutter image processing unit 103) that extract a first region where the light source exists from the first image P1 and a second region where the light source exists from the second image P2. This makes it possible to extract regions where the light source exists from each of the first image P1 and the second image P2 captured by the stereo camera 10.

[0046] Furthermore, the image processing device 100 in this embodiment includes an object information detection unit (a normal shutter image processing unit 102 and a low-exposure shutter image processing unit 103) that detects the first object information from the first region and detects the second object information from the second region. This makes it possible to detect object information from each of the first image P1 and the second image P2 captured by the stereo camera 10.

[0047] Furthermore, the image processing device 100 in this embodiment includes a flare determination unit 107 that calculates the amount of flare light based on luminance information (lamp size W1) of the object (lamp) in the first image P1 and luminance information (lamp size W2) of the object (lamp) in the second image P2. This makes it possible to calculate the amount of flare light from the first image P1 taken with a normal shutter and the second image P2 taken with a low-exposure shutter.

[0048] The second embodiment of the present invention will be described, focusing on the differences from the first embodiment.

[0049] 10 is a functional block diagram of the image processing device 100 according to this embodiment. The image processing device 100 according to this embodiment includes a distance measurement sensor 120 such as a LiDAR sensor, instead of the low-exposure shutter image processing unit 103, the lamp pair detection unit 105, and the lamp detection result linking unit 106 (shown in FIG. 7).

[0050] The flare determination unit 107 calculates the amount of flare light from the light source included in the first image P1 based only on the detection result of the lamp pair detection unit 104. In the learning phase, the distance correction value calculation unit 108 calculates a parallax difference (parallax error) by subtracting a parallax equivalent value of the distance to the light source measured by the distance measurement sensor 120 from the parallax of the light source acquired with the normal shutter, and determines the correlation with the amount of flare light (a regression line or a regression curve).

[0051] (Summary) The distance measuring sensor 120 in this embodiment constitutes an object information acquisition device that acquires the distance to an object included in the first image P1 captured by the stereo camera 10 as the second distance.

[0052] According to this embodiment configured as described above, it is possible to learn the correlation between the amount of flare light and the deviation of the first distance from the second distance measured by the distance measuring sensor 120, thereby making it possible to further improve the accuracy of distance observation using a normal shutter.

[0053] Furthermore, the image processing device 100 in this embodiment includes a flare determination unit 107 that calculates the amount of flare light based on luminance information (lamp size W1) of the object (lamp) in the first image P1, which makes it possible to calculate the amount of flare light from only the first image P1 captured with a normal shutter.

[0054] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to add part of the configuration of one embodiment to the configuration of another embodiment, or to delete part of the configuration of one embodiment or replace it with part of another embodiment.

[0055] 10...Stereo camera (object information acquisition device), 10L...Left camera, 10R...Right camera, 11...Image input interface, 12...Image processing unit, 13...Calculation processing unit, 14...Memory unit, 15...CAN interface, 16...In-vehicle network, 100...Image processing device, 101...Imaging unit, 102...Normal shutter image processing unit, 103...Low exposure shutter image processing unit, 104...Lamp pair detection unit, 105...Lamp pair detection unit output unit, 106...lamp detection result linking unit, 107...flare determination unit, 108...distance correction value calculation unit, 109...distance correction unit, 110, 111...identification unit, 112, 113...detection result output unit, 120...ranging sensor (object information acquisition device), 200...image processing device, 201...object information integration unit, P1...first image, P1L...first left image, P1R...first right image, P2...second image, P2L...second left image, P2R...second right image.

Claims

1. An image processing device comprising: a stereo camera which captures a first image which is an image with a first exposure amount; and a normal shutter image processing unit which detects information of an object included in the first image as first object information; an object information acquisition device which acquires information of the object as second object information; a distance correction unit which corrects a first distance which is the distance to the object included in the first object information; and a distance correction value calculation unit which calculates a correction value of the first distance, wherein the distance correction value calculation unit learns a correlation between an amount of flare light of the object in the first image and a difference of the first distance from a second distance which is the distance to the object included in the second object information, and calculates the correction value based on the correlation and the amount of flare light, and the distance correction unit corrects the first distance by subtracting the correction value from the first distance.

2. In the image processing device described in claim 1, the object information acquisition device is the stereo camera that captures a second image, which is an image with a second exposure amount that is less than the first exposure amount, and the image processing device is characterized in that it is equipped with a low exposure shutter image processing unit that calculates the distance to the object included in the second image as the second distance.

3. An image processing device as described in claim 2, characterized in that it comprises an object extraction unit that extracts a first area in which the object exists from the first image and extracts a second area in which the object exists from the second image.

4. An image processing device according to claim 3, further comprising an object information detection unit which detects the first object information from the first region and detects the second object information from the second region.

5. An image processing device according to claim 2, further comprising a flare determination unit that calculates the amount of flare light based on luminance information of the object in the first image and luminance information of the object in the second image.

6. An image processing device according to claim 1, characterized in that the object information acquisition device is a distance measuring sensor that measures the second distance.

7. An image processing device according to claim 6, further comprising a flare determination unit that calculates the flare light amount based on luminance information of the object in the first image.

Images