Image processing apparatus

Abstract

The purpose of the present invention is to provide an image processing device capable of accurately calculating the distance to an object serving as a light source on the basis of a normal exposure image captured by a stereo camera. To this end, an image processing device (100) is provided with: a stereo camera (10) that captures a first image (P1); a normal shutter image processing unit (102) that detects first object information from the first image (P1); an object information acquisition device (10) that acquires second object information; a distance correction unit (109) that corrects a first distance included in the first object information; and a distance correction value calculation unit (108) that calculates a correction value for the first distance, the distance correction value calculation unit (108) learning a correlation between a spot light amount of the object in the first image (P1) and a difference between the first distance and a second distance included in the second object information; the correction value is calculated on the basis of the correlation and the spot light amount, and a distance correction unit (109) corrects the first distance by subtracting the correction value from the first distance.

Description

Technical Field

[0001] This invention relates to an image processing apparatus for identifying the external environment of a vehicle based on images obtained by photographing the exterior of the vehicle. Background Technology

[0002] In recent years, cars equipped with driver assistance functions that aid in acceleration, deceleration, and steering based on the external environment have become increasingly common, leading to a greater demand for driver assistance features during nighttime driving. To provide appropriate driver assistance at night, it is necessary to accurately estimate the distance to and speed of the preceding vehicle, regardless of the brightness of its brake lights. Therefore, recent image processing devices have demanded enhanced capabilities in accurately detecting light sources ranging from low to high brightness, utilizing technologies such as HDR (High Dynamic Range) to expand the dynamic range during shooting.

[0003] However, if the dynamic range during shooting is increased, a bright but blurry halo phenomenon is easily generated around the light source when shooting high-brightness light sources (e.g., brake lights during braking). Moreover, if a halo occurs, there is a problem that the three-dimensional object cannot be accurately detected in the halo area. As a prior art disclosed with the aim of solving this problem, there is, for example, Patent Document 1.

[0004] Patent document 1 describes "an image processing apparatus, characterized in that it comprises: a camera that captures a first image with a first exposure and captures a second image with a second exposure less than the first exposure; a stereoscopic object extraction unit that extracts a first region containing a stereoscopic object from the first image and extracts a second region containing the stereoscopic object from the second image; a stereoscopic object information detection unit that detects first stereoscopic object information from the first region and second stereoscopic object information from the second region; and a stereoscopic object information synthesis unit that calculates and synthesizes a comprehensive stereoscopic object information that integrates the first stereoscopic object information and the second stereoscopic object information."

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: International Publication No. 2022 / 254795 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] According to the image processing apparatus described in Patent Document 1, by integrating first stereoscopic information obtained from a first image with a first exposure amount (normal shutter speed) that produces a light spot and second stereoscopic information obtained from a second image with a second exposure amount (low exposure shutter speed) that does not produce a light spot, the influence of the light spot in the first image can be suppressed in the integrated stereoscopic information.

[0010] However, since the detection results of both normal shutter speed and low exposure shutter speed are combined, the error of the combined result increases as the bokeh of the normal shutter speed increases. Here, when combining the two detection results, the influence of the bokeh in the combined result can be removed by setting the weight of the detection result of the normal shutter speed to zero. However, since this results in distance observation using only the low exposure shutter speed, the observation period becomes longer.

[0011] The present invention was made in view of the above-mentioned problems, and its object is to provide an image processing apparatus capable of accurately calculating the distance to an object that becomes a light source based on an image captured by a stereo camera with a normal exposure.

[0012] Methods for solving problems

[0013] To achieve the above objectives, the image processing apparatus of the present invention comprises: a stereo camera that captures an image with a first exposure, i.e., a first image; and a normal shutter image processing unit that detects information about an object contained in the first image as first object information. The image processing apparatus is characterized by comprising: an object information acquisition device that acquires the object information as second object information; a distance correction unit that corrects a first distance, i.e., a distance to the object contained in the first object information; and a distance correction value calculation unit that calculates a correction value for the first distance. The distance correction value calculation unit learns a correlation between the amount of light from the object in the first image and the difference between the first distance and the distance to the object contained in the second object information, i.e., a second distance, and calculates the correction value based on the correlation and the amount of light from the object. The distance correction unit corrects the first distance by subtracting the correction value from the first distance.

[0014] Invention Effects

[0015] According to the image processing apparatus of the present invention, the distance to the object that becomes the light source can be accurately calculated based on an image with normal exposure taken by a stereo camera. Attached Figure Description

[0016] Figure 1 This is a hardware structure diagram of an image processing device.

[0017] Figure 2This is a functional block diagram of an image processing device in the prior art.

[0018] Figure 3 This is a diagram showing an example of the first and second images output from a stereo camera.

[0019] Figure 4 This is a diagram illustrating the method for measuring the first distance.

[0020] Figure 5 This is a diagram illustrating the mechanism of parallax error caused by flare.

[0021] Figure 6 This is a diagram illustrating an example of a method for calculating the amount of light in a light spot.

[0022] Figure 7 This is a functional block diagram of the image processing apparatus according to the first embodiment of the present invention.

[0023] Figure 8 This is a diagram representing the processing during the learning phase of the distance correction value calculation unit.

[0024] Figure 9 This is a diagram showing the processing of the distance correction unit.

[0025] Figure 10 This is a functional block diagram of an image processing apparatus according to a second embodiment of the present invention. Detailed Implementation

[0026] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same parts are labeled with the same reference numerals, and repeated descriptions are omitted.

[0027] Example 1

[0028] Figure 1 This is a hardware structure diagram of the image processing device 100 in the first embodiment of the present invention. The image processing device 100 is an in-vehicle device that identifies the external environment based on images captured outside the vehicle, such as white lines on the road, pedestrians, other vehicles, other three-dimensional objects, signals, signs, lights, and other external environmental features. Moreover, the image processing device 100 determines the vehicle's acceleration / deceleration assist, steering assist, and other control strategies based on the identified external environment.

[0029] like Figure 1As shown, the image processing apparatus 100 includes a stereo camera 10 (left camera 10L, right camera 10R), an image input interface 11, an image processing unit 12, an arithmetic processing unit 13, a storage unit 14, and a CAN interface 15. Furthermore, the structure from the image input interface 11 to the CAN interface 15 consists of one or more computer units interconnected via an internal bus. The arithmetic processing unit (CPU, etc.) within each computer unit executes a predetermined program to realize various functions of the image processing unit 12, the arithmetic processing unit 13, etc.

[0030] The stereo camera 10 consists of a left camera 10L and a right camera 10R mounted on the vehicle to capture images of the front of the vehicle. The left camera 10L's imaging element simultaneously or at staggered intervals captures a first left image P1L with a normal exposure (hereinafter referred to as normal shutter speed) and a second left image P2L with a less than normal exposure (hereinafter referred to as low exposure shutter speed). Simultaneously or at staggered intervals, the right camera 10R's imaging element simultaneously or at staggered intervals captures a first right image P1R with a normal shutter speed and a second right image P2R with a low exposure shutter speed. Here, "exposure" refers to the brightness of the image determined by the exposure time, aperture, gain, etc., during shooting. The normal shutter speed expands the dynamic range and prevents luminous objects from appearing black or white compared to non-luminous objects. The low exposure shutter speed reduces the exposure compared to the normal shutter speed, reducing the effect of bokeh. "Bokeh" refers to the phenomenon where, when photographing a bright object such as a spot of light, strong light is reflected within the lens, resulting in a blurred image of the object.

[0031] The image input interface 11 is used to control the stereo camera 10 to capture and acquire the captured images P1L, P2L, P1R, and P2R. The image P acquired through this interface is sent to the image processing unit 12, etc., via the internal bus.

[0032] The image processing unit 12 compares the left images P1L and P2L captured by the left camera 10L with the right images P1R and P2R captured by the right camera 10R. After performing corrections such as corrections for device-inherent deviations caused by the imaging element and noise interpolation on each image, the corrected image P is stored in the storage unit 14.

[0033] Furthermore, the image processing unit 12 calculates the corresponding locations between the left and right images with the same exposure, calculates parallax information (distance information relative to each point on the image), and stores it in the storage unit 14. Specifically, the first parallax information is calculated by comparing the first left image P1L and the first right image P1R, which were captured at the same time, and the second parallax information is calculated by comparing the second left image P2L and the second right image P2R, which were captured at the same time.

[0034] The processing unit 13 uses the images P1L, P2L, P1R, P2R and parallax information stored in the storage unit 14 to identify various objects necessary for perceiving the environment surrounding the vehicle. These identified objects include people, other vehicles, other obstacles, traffic lights, signs, taillights and headlights of other vehicles, etc. A portion of these identification results and intermediate calculation results are recorded in the storage unit 14. Furthermore, the processing unit 13 uses the object recognition results to determine the vehicle's control strategy.

[0035] The storage unit 14 is a storage device such as a semiconductor memory, which stores the corrected images P1L, P2L, P1R, P2R and parallax information output by the image processing unit 12, as well as the object recognition results and the vehicle control strategy output by the arithmetic processing unit 13.

[0036] The CAN interface 15 is the interface that transmits the object recognition results obtained by the arithmetic processing unit 13 and the vehicle's control strategy to the Controller Area Network 16. In addition, the Controller Area Network 16 is connected to a control system (ECU, etc.) that controls the vehicle's drive system, braking system, steering system, etc. The stereo camera 10 can perform driving assistance such as automatic braking and steering avoidance according to the external environment of the vehicle through the control system.

[0037] The image processing apparatus 100 in this embodiment will be described below by comparing it with the image processing apparatus 200 in the prior art.

[0038] Figure 2 This is a functional block diagram of an image processing device 200 in the prior art. Figure 2 In this image processing device 200, there are a shooting unit 101, a normal shutter image processing unit 102, a low exposure shutter image processing unit 103, a lamp pair detection unit 104, a lamp pair detection unit 105, a lamp detection result association unit 106, a spot determination unit 107, an object information integration unit 201, a recognition unit 110, and a detection result output unit 112.

[0039] The imaging unit 101 is composed of a stereo camera 10, which simultaneously or at staggered intervals outputs a first image P1 and first parallax information at a normal shutter speed and a second image P2 and second parallax information at a low exposure shutter speed. Figure 3 An example is shown of a first image P1 and a second image P2 output from a stereo camera 10. In this example, the first image P1 and the second image P2 are output alternately at staggered timings.

[0040] return Figure 2 Normally, the shutter image processing unit 102 extracts and outputs first object information (which is the information of the object) from the first image P1 input by the shooting unit 101. Figure 3As shown). The low-exposure shutter image processing unit 103 extracts and outputs information about the object, namely the second object information, from the second image P2 input by the shooting unit 101. Figure 3 (As shown). The first object information includes the distance to the object detected in the first image P1 (first distance), and the second object information includes the distance to the object detected in the second image P2 (second distance). Figure 4 The method for measuring the first distance is shown. Due to the difference in the left and right positions of the left camera 10L and the right camera 10R, the positions of objects in the captured first left image P1L and first right image P1R are also shifted to the left and right. This shift is called parallax, and the first distance is calculated by converting this parallax into distance using triangulation. Furthermore, the method for measuring the second distance is the same.

[0041] Figure 5 This diagram illustrates the mechanism of parallax error caused by light spots. In normal shutter speeds, if the light spot formation conditions when capturing the light source differ in the first left image P1L and the first right image P1R due to the difference in lens characteristics between the left camera 10L and the right camera 10R, the images appear different to the left and right, resulting in parallax error. On the other hand, in low-exposure shutter speeds, the effect of light spots is reduced by decreasing the exposure, thus suppressing parallax error. Figure 3 As shown, in the first image P1 with a normal shutter speed, the detection accuracy of the leading vehicle's height is high due to the high brightness of the vehicle body. However, the detection accuracy of the leading vehicle's width decreases because the lights appear to expand laterally due to the light spots. Consequently, accurate parallax of the leading vehicle cannot be obtained, and the accuracy of the distance to the leading vehicle also decreases. On the other hand, in the second image P2 with a low exposure shutter speed, although the detection accuracy of the leading vehicle's height decreases due to the reduced brightness of the vehicle body, the detection accuracy of the leading vehicle's width improves because the light spots are suppressed. As a result, accurate parallax of the leading vehicle can be obtained, and therefore the accuracy of the distance to the leading vehicle also improves.

[0042] return Figure 2 The light pair detection unit 104 performs light pair detection on the first image P1 based on the first object information from the normal shutter image processing unit 102. The light pair detection unit 105 performs light pair detection on the second image P2 based on the second object information from the low exposure shutter image processing unit 103.

[0043] The lamp detection result association unit 106 performs association processing on the object information of each lamp pair detected by the lamp pair detection unit 104 and the lamp pair detection unit 105.

[0044] The spot determination unit 107 calculates the spot light intensity of the first image P1. "Spot light intensity" is a physical quantity that represents the degree of spot intensity. Figure 6Here's an example of a method for calculating spot light intensity. First, a brightness threshold for extracting lamps is predetermined for each exposure level, and the regions of pixels exceeding this threshold are extracted as lamps. Next, the lamp sizes W1 and W2 are calculated based on the width of each lamp. Then, the difference between the lamp size W1 at a normal shutter speed (first image P1) and the lamp size W2 at a low exposure shutter speed (second image P2) is calculated as the spot light intensity. The difference between lamp sizes W1 and W2 is expressed as follows: Figure 6 As illustrated, consider calculating the area obtained by subtracting the area of ​​the circular region of lamp size W2 from the area of ​​the circular region of lamp size W1. Alternatively, the spot light intensity can also be calculated based solely on an image taken at a normal shutter speed. In this case, consider calculating the area of ​​the region whose brightness exceeds a predetermined threshold, the radius of the circle circumscribed by the region whose brightness exceeds the predetermined threshold, the width of the region whose brightness exceeds the predetermined threshold, and the average / maximum brightness of a predetermined region centered on the spot as the spot light intensity.

[0045] return Figure 2 If the amount of light in the spot calculated by the spot determination unit 107 is above a predetermined threshold, the object information integration unit 201 integrates the first object information from the normal shutter image processing unit 102 and the second object information from the low exposure shutter image processing unit 103 under a predetermined rule. On the other hand, if the amount of light in the spot calculated by the spot determination unit 107 is less than the predetermined threshold, the first object information from the normal shutter image processing unit 102 is directly output.

[0046] The identification unit 110 determines the category of the object based on the object information output by the object information integration unit 201.

[0047] The detection result output unit 112 outputs the object information (position, speed, shape, etc.) from the object information integration unit 201 and the object category from the identification unit 110 to the control system (ECU, etc.) via the CAN interface 15 and the vehicle network 16.

[0048] In the existing image processing apparatus 200, since the detection results of both normal shutter speed and low exposure shutter speed are combined, the error of the combined result increases as the amount of light in the bokeh increases with the normal shutter speed. Here, when combining the two detection results, the influence of the bokeh in the combined result can be removed by setting the weight of the normal shutter speed detection result to zero. However, since this results in distance observation using only the low exposure shutter speed, the observation period becomes longer. The image processing apparatus 100 in this embodiment solves this problem.

[0049] Figure 7 This is a functional block diagram of the image processing device 100 in this embodiment. Figure 7 In the image processing device 200 of the prior art ( Figure 2The difference is that the image processing device 100 has a distance correction value calculation unit 108, a distance correction unit 109, a recognition unit 111, and a detection result output unit 113 instead of the object information integration unit 201. Figure 2 (As shown).

[0050] The distance correction unit 109 corrects the first distance contained in the first object information from the normal shutter image processing unit 102, and outputs first object information corrected for the first distance. The method for correcting the first distance will be described later.

[0051] The identification unit 110 determines the category of the three-dimensional object based on the corrected first object information output by the distance correction unit 109.

[0052] The recognition unit 111 determines the category of the object based on the second object information output by the low-exposure shutter image processing unit 103.

[0053] The detection result output unit 113 outputs the second object information (position, speed, shape, etc.) as output by the low exposure shutter image processing unit 103 and the object category as output by the recognition unit 111 to the control system (ECU, etc.) via the CAN interface 15 and the vehicle network 16.

[0054] In the correction phase, which is a phase other than the learning phase described later, the distance correction value calculation unit 108 converts the light spot quantity input from the light spot determination unit 107 into a correction value (distance correction value) of the first distance obtained by the normal shutter, based on the correlation between the light spot quantity and parallax error learned in the learning phase, and outputs it.

[0055] Figure 8This diagram illustrates the processing during the learning phase of the distance correction value calculation unit 108. During the learning phase, the distance correction value calculation unit 108 performs regression analysis on the combined information of the spot light intensity of multiple objects and the parallax difference of normal / low exposure shutter speeds, obtained from the first image P1 and the second image P2 of multiple frames stored / accumulated in the storage unit 14, to determine the correlation (regression line or regression curve) between the spot light intensity and the parallax difference of normal / low exposure shutter speeds. For example, if the regression model representing the relationship between variables x and y is a linear equation (y = ax + b), the regression coefficients a and b are determined through regression analysis. Furthermore, the regression analysis model is predetermined through processes such as calibration. The timing of performing the regression analysis is not particularly limited; it can be performed at any time as long as it does not interfere with other functions of the image processing apparatus 100. When the parallax error of the low exposure shutter speed is set to zero, the parallax difference is equal to the parallax error; therefore, by using the aforementioned correlation, the parallax error can be determined based on the spot light intensity. During the calibration phase, the distance calibration unit 109 converts the light intensity of the light spot input from the light spot determination unit 107 into parallax error based on the above-mentioned correlation, and converts the parallax error into distance, thereby obtaining the distance calibration value.

[0056] Figure 9 This diagram illustrates the processing of the distance correction unit 109. The distance correction unit 109 corrects the first distance by subtracting the distance correction value input by the distance correction value calculation unit 108 from the first object information input by the normal shutter image processing unit 102. Therefore, even when glare occurs, high-precision distance observation can be performed using only the normal shutter speed.

[0057] (Summarize)

[0058] In this embodiment, the image processing apparatus 100 includes: a stereo camera 10 that captures an image with a first exposure, i.e., a first image P1; a normal shutter image processing unit 102 that detects information about an object contained in the first image P1 as first object information; wherein the image processing apparatus 100 includes: an object information acquisition device (stereo camera 10) that acquires the object information as second object information; a distance correction unit 109 that corrects the distance to the object contained in the first object information, i.e., a first distance; and a distance correction value calculation unit 108 that calculates a correction value for the first distance. The distance correction value calculation unit 108 learns the correlation between the amount of light from the object in the first image P1 and the difference between the first distance and the distance to the object contained in the second object information, i.e., a second distance, and calculates the correction value based on the correlation and the amount of light from the object. The distance correction unit 109 corrects the first distance by subtracting the correction value from the first distance.

[0059] According to this embodiment configured as described above, the correlation (regression line or regression curve) between the light spot intensity of the light source contained in the image (first image P1) captured by the stereo camera 10 at a normal shutter speed and the difference between the distance to the light source obtained by the normal shutter speed (first distance) and the distance to the light source obtained by the object information acquisition device 10 (second distance) is learned. Based on this correlation, the first distance is corrected, thereby enabling accurate calculation of the distance to the object that becomes the light source from the image (first image P1) captured by the stereo camera 10 at a normal exposure speed. As a result, the distance / velocity estimation accuracy of the image processing device 100 and the success rate of detecting / tracking oncoming vehicles are improved. Furthermore, an inexpensive lens that easily produces light spots can be used in the stereo camera 10.

[0060] Furthermore, the object information acquisition device 10 in this embodiment is a stereo camera 10 that captures a second image P2 with a second exposure amount less than the first exposure amount, and the image processing device 100 has a low-exposure shutter image processing unit 103 that calculates the distance to the object contained in the second image P2 as the second distance. Therefore, it is not necessary to add an additional object information acquisition device for acquiring the distance to a high-brightness light source with higher accuracy than a normal shutter speed, thus preventing an increase in the cost of the image processing device 100.

[0061] Furthermore, the image processing apparatus 100 in this embodiment includes an object extraction unit (typically a shutter image processing unit 102 and a low-exposure shutter image processing unit 103), which extracts a first region where the light source is located from a first image P1 and a second region where the light source is located from a second image P2. Thus, it is possible to extract the region where the light source is located from both the first image P1 and the second image P2 captured by the stereo camera 10.

[0062] Furthermore, the image processing apparatus 100 in this embodiment includes an object information detection unit (typically a shutter image processing unit 102 and a low-exposure shutter image processing unit 103) that detects the first object information from the first region and the second object information from the second region. Thus, object information can be detected based on the first image P1 and the second image P2 captured by the stereo camera 10.

[0063] Furthermore, the image processing apparatus 100 in this embodiment includes a spot determination unit 107 that calculates the amount of light spot based on the brightness information (lamp size W1) of the object (lamp) in the first image P1 and the brightness information (lamp size W2) of the object (lamp) in the second image P2. Thus, the amount of light spot can be calculated based on the first image P1 at a normal shutter speed and the second image P2 at a low exposure shutter speed.

[0064] Example 2

[0065] The second embodiment of the present invention will be described focusing on the differences from the first embodiment.

[0066] Figure 10 This is a functional block diagram of the image processing apparatus 100 in this embodiment. In this embodiment, the image processing apparatus 100 replaces the low-exposure shutter image processing unit 103, the lamp pair detection unit 105, and the lamp detection result correlation unit 106. Figure 7 (As shown) it is equipped with a ranging sensor 120 consisting of LiDAR or the like.

[0067] The spot determination unit 107 calculates the spot light intensity of the light source contained in the first image P1 based solely on the detection results of the lamp pair detection unit 104. During the learning phase, the distance correction value calculation unit 108 calculates the disparity difference (disparity error) by subtracting the disparity conversion value of the distance to the light source measured by the range sensor 120 from the disparity of the light source obtained by the normal shutter, and thereby determines the correlation (regression line or regression curve) with the spot light intensity.

[0068] (Summarize)

[0069] In this embodiment, the ranging sensor 120 constitutes an object information acquisition device, which acquires the distance from the stereo camera 10 to the object contained in the first image P1 as a second distance.

[0070] According to this embodiment configured as described above, it is possible to learn the correlation between the amount of light in the spot and the deviation of the first distance measured by the range sensor 120 relative to the second distance, thereby further improving the accuracy of distance observation using a normal shutter speed.

[0071] Furthermore, the image processing apparatus 100 in this embodiment includes a spot determination unit 107 that calculates the amount of light spot based on the brightness information (lamp size W1) of the object (lamp) in the first image P1. Therefore, the amount of light spot can be calculated solely based on the first image P1 at a normal shutter speed.

[0072] The embodiments of the present invention have been described in detail above, but the present invention is not limited to the above embodiments and includes various modifications. For example, the above embodiments are examples described in detail for the purpose of easily understanding and illustrating the present invention, and are not limited to having all the structures described. In addition, a part of the structure of another embodiment may be added to the structure of a certain embodiment, a part of the structure of a certain embodiment may be deleted, or a part of the structure of another embodiment may be replaced.

[0073] Symbol Explanation

[0074] 10… Stereo camera (object information acquisition device), 10L… Left camera, 10R… Right camera, 11… Image input interface, 12… Image processing unit, 13… Processing unit, 14… Storage unit, 15… CAN interface, 16… Vehicle network, 100… Image processing device, 101… Imaging unit, 102… Normal shutter image processing unit, 103… Low exposure shutter image processing unit, 104… Lamp pair detection unit, 105… Lamp pair detection unit, 106… Lamp detection result 107…Spot determination unit, 108…Distance correction value calculation unit, 109…Distance correction unit, 110, 111…Identification unit, 112, 113…Detection result output unit, 120…Distance sensor (object information acquisition device), 200…Image processing device, 201…Object information integration unit, P1…First image, P1L…First left image, P1R…First right image, P2…Second image, P2L…Second left image, P2R…Second right image.

Claims

1. An image processing apparatus comprising: a stereo camera that captures an image with a first exposure, i.e., a first image; and a conventional shutter image processing unit that detects information about an object contained in the first image as first object information, characterized in that, The image processing device includes: An object information acquisition device acquires the object information as second object information; The distance correction unit corrects the distance to the object contained in the first object information, i.e., the first distance; as well as The distance correction value calculation unit calculates the correction value for the first distance. The distance correction value calculation unit learns the correlation between the spot light intensity of the object in the first image and the difference between the first distance and the second distance (the distance to the object contained in the second object information), and calculates the correction value based on the correlation and the spot light intensity. The distance correction unit corrects the first distance by subtracting the correction value from the first distance.

2. The image processing apparatus according to claim 1, characterized in that, The object information acquisition device is the stereo camera that captures a second image, which has a lower exposure than the first exposure. The image processing apparatus includes a low-exposure shutter image processing unit that calculates the distance to the object contained in the second image as the second distance.

3. The image processing apparatus according to claim 2, characterized in that, The image processing apparatus includes an object extraction unit that extracts a first region containing the object from the first image and extracts a second region containing the object from the second image.

4. The image processing apparatus according to claim 3, characterized in that, The image processing apparatus includes an object information detection unit that detects first object information from a first region and second object information from a second region.

5. The image processing apparatus according to claim 2, characterized in that, The image processing apparatus includes a light spot determination unit, which calculates the light spot intensity based on the brightness information of the object in the first image and the brightness information of the object in the second image.

6. The image processing apparatus according to claim 1, characterized in that, The object information acquisition device is a ranging sensor that measures the second distance.

7. The image processing apparatus according to claim 6, characterized in that, The image processing apparatus includes a light spot determination unit, which calculates the light spot intensity based on the brightness information of the object in the first image.

Images