Vehicle exterior imaging device

Abstract

To ensure reliability of an image pickup device to be provided in a vehicle for imaging the outside.SOLUTION: An outside imaging device 10 has: a plurality of image pickup devices 11 to 18; and a control unit 24 that generates information on a distance or direction as to an outside imaging object, using imaging images of the plurality of image pickup devices 11 to 18. At least two image pickup devices of the plurality of image pickup devices 11 to 18 are provided in a vehicle 1 so as to allow for overlappedly imaging a common imaging area. The control unit 24 is configured to generate correction information on the distance or direction as to the outside imaging object based on an imaging position to be used in monocular processing of the imaging image of the remaining image pickup device on the basis of the distance and direction in the imaging image of one image pickup device of the at least two image pickup devices allowing for overlappedly imaging the common imaging area.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to an exterior imaging device for a vehicle. [Background technology]

[0002] Vehicles are provided with imaging devices to capture images of the outside of the vehicle (Patent Documents 1 and 2). It becomes possible to generate (relative) distance or direction information about an object outside the vehicle using an image captured by an imaging device installed in the vehicle. In controlling autonomous driving and driving assistance, information on the distance or direction of such an object captured outside the vehicle is extremely important. For this reason, it is being considered to provide multiple imaging devices in various parts of the vehicle and capture images of the 360-degree surroundings in separate areas using the multiple imaging devices. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-194538 [Patent Document 2] International Publication No. WO2015 / 029934 Summary of the Invention [Problem to be solved by the invention]

[0004] However, it is important that the imaging range and angle of view of an imaging device installed in a vehicle do not change due to vibrations caused by the vehicle as it travels. In particular, when capturing images of a 360-degree surrounding area divided into sections using multiple imaging devices, even a slight change in the imaging range or angle of view of one imaging device is likely to affect processing based on images captured by the multiple imaging devices.

[0005] For this reason, conventionally, it has been considered to attach each imaging device directly to the vehicle body itself, which has high rigidity. However, when an imaging device is directly attached to the vehicle body in this way, the location where the imaging device can be attached is extremely limited. Moreover, even if the imaging device is directly attached to the vehicle body to prevent it from shifting, the position and orientation of the imaging device may still change over time even if the vehicle body is not subjected to any impact. It is also possible to do away with multiple imaging devices and install a 360-degree camera on the vehicle's roof, for example. However, even if such an imaging device is used, the possibility that its position and orientation may change over time cannot be eliminated.

[0006] As described above, it is required to ensure or maintain the reliability of the imaging device provided in the vehicle for capturing images of the area outside the vehicle. [Means for solving the problem]

[0007] The vehicle exterior image capture device of the present invention includes a plurality of image capture devices provided on the vehicle for capturing images of the exterior of the vehicle, and a control unit that can generate distance or direction information about an image capture target outside the vehicle using images captured by the plurality of image capture devices, the plurality of imaging devices provided on the vehicle for capturing images of the outside of the vehicle include a stereo camera formed by two of the imaging devices whose mutual arrangements are defined, and a plurality of imaging devices for monocular processing that can capture images of an imaging area common to the two imaging devices of the stereo camera in an overlapping manner; Multiple For monocular processing the imaging device is provided so as to be able to capture an image of an image area common to the other image pickup devices for monocular processing in an overlapping manner, and at least one of the image pickup devices for monocular processing is provided so as to be able to capture an image of an image area common to the other image pickup devices for monocular processing in an overlapping manner with the stereo camera. The control unit is provided in the vehicle so as to be able to capture images of a common imaging area in an overlapping manner. In the stereo camera using the two imaging devices whose mutual arrangements are specified, distance and direction information is generated based on the parallax of an imaged object outside the vehicle imaged in a common imaging area, and distance and direction information is generated based on the parallax Distance and direction The information is used in monocular processing of the captured image of the at least one imaging device for monocular processing. Generates correction information about the distance or direction of an imaged object outside the vehicle based on the image capture position When generating correction information for the imaging devices for a plurality of monocular processing, the stereo camera formed by the two imaging devices whose mutual arrangement is specified is used as a reference, and correction information is generated in order from the imaging device for monocular processing that can capture an imaging area common to the stereo camera in an overlapping manner, according to the order of overlapping of the imaging areas. do.

[0009] Preferably, before The control unit may use the captured images of the two imaging devices of the stereo camera to generate distance and direction information based on the parallax of an object outside the vehicle that is captured in an imaging area common to the imaging device for monocular processing, and may use the distance and direction information of the stereo cameras to generate correction information for the imaging device for monocular processing.

[0011] Preferably, the imaging devices for multiple monocular processing are provided on the vehicle so as to capture images of the surroundings of the vehicle in a divided manner, and each imaging device for monocular processing is provided on the vehicle so as to be able to capture an overlapping imaging area common to other imaging devices for monocular processing that are adjacent to the imaging device in the direction of division of the surroundings of the vehicle.

[0013] Preferably, the plurality of monocular processing imaging devices are provided on the vehicle so as to capture images of the entire periphery of the vehicle, or the entire periphery excluding the imaging range of the stereo camera, in a divided manner, and the control unit generates correction information in order from the imaging device for monocular processing on one end side which is capable of capturing overlapping images of a common imaging area at one end side of the imaging range of the stereo camera, to the imaging device for monocular processing on the other end side which is capable of capturing overlapping images of a common imaging area at the other end side of the imaging range of the stereo camera, and for the imaging device for monocular processing to be processed last, generates correction information by processing in the order of the one end side and correction information by processing in the order of the other end side, and compares the two generated correction information to evaluate the series of corrections.

[0014] Preferably, the imaging device for monocular processing that is processed last is configured to capture an image of the rear of the vehicle, and the control unit evaluates the series of corrections based on road information contained in the corrected captured image of the imaging device for monocular processing that is processed last.

[0015] Preferably, the control unit is a good series of corrections. and If the evaluation is not possible, an alarm or a maintenance request should be output.

[0016] Preferably, the correction information for the distance or direction of an object to be imaged outside the vehicle based on the imaging position used in monocular processing of the image captured by the imaging device is distance or direction correction information for the distance or direction information for each imaging position in the image captured by the imaging device that is set for monocular processing.

[0017] Preferably, the control unit uses the correction information to correct the distance or direction of an object outside the vehicle imaged by the imaging device in monocular processing, and obtains the corrected distance or direction based on the attitude of the vehicle. [Effects of the Invention]

[0018] In the present invention, a vehicle is provided with a plurality of imaging devices for capturing images of the outside of the vehicle, and at least three of the imaging devices are arranged in the vehicle so as to be able to capture an overlapping image of a common imaging area, thereby enabling the at least three imaging devices to capture an image of the common imaging area in the captured image of the outside of the vehicle. The control unit of the vehicle's exterior image capture device then uses the captured images from at least two of the three imaging devices to generate distance and direction information based on the parallax of the captured object outside the vehicle that is captured in the common imaging area. The control unit uses the parallax that can be expected based on the arrangement of the two imaging devices on the vehicle to more accurately generate distance or direction information about the captured object outside the vehicle that is captured in the common imaging area, without being affected by changes in the imaging range or angle of view due to changes in the arrangement of the other imaging devices. As a result, the control unit can use the distance and direction information based on the parallax to generate reliable correction information about the distance or direction of the captured object outside the vehicle that is based on the imaging position, which is used in monocular processing of the captured image from at least one remaining imaging device out of the at least three imaging devices. In this way, the present invention makes good use of the fact that a vehicle is provided with multiple imaging devices for capturing images outside the vehicle, and by having at least three imaging devices capture images of overlapping common imaging areas, it is possible to accurately correct the distance or direction of an object captured outside the vehicle using monocular processing. As a result, even if the images captured by the imaging devices provided in the vehicle are processed using monocular processing, it is possible to obtain the distance or direction of an object captured outside the vehicle with the same degree of accuracy as with compound-eye processing. The present invention makes it possible to ensure or maintain the reliability of an imaging device provided in a vehicle for capturing images of the outside of the vehicle. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is an explanatory diagram of a vehicle to which an outside-vehicle image capturing device according to an embodiment of the present invention is applied. [Figure 2] FIG. 2 is an explanatory diagram of an imaging situation and a captured image in monocular processing of one imaging device in FIG. [Figure 3] FIG. 3 is an explanatory diagram of the influence of misalignment that may occur in an imaging device installed in a car. [Figure 4] FIG. 4 is an explanatory diagram of the influence of misalignment that may occur between multiple imaging devices installed in an automobile. [Figure 5] FIG. 5 is an explanatory diagram of a control system including an outside-vehicle image capturing device in the automobile of FIG. [Figure 6] FIG. 6 is an explanatory diagram of the imaging ranges of the multiple imaging devices in FIG. 5 and the order of deviation correction control. [Figure 7] FIG. 7 is a flowchart of the 360-degree calibration control by the CPU of FIG. [Figure 8] FIG. 8 is a detailed flowchart of the stereo-based monocular misalignment correction control. [Figure 9] FIG. 9 is a detailed flowchart of the monocular deviation correction control based on the adjacent monocular. [Figure 10] FIG. 10 is a block diagram for imaging control using the deviation correction information generated by the control of FIGS. [Figure 11] FIG. 11 is a diagram for explaining the relative movement of the imaging target in the images captured by two adjacent imaging devices through the process of FIG. [Figure 12] FIG. 12 is an explanatory diagram of an example of a method for evaluating deviation correction. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0021] FIG. 1 is an explanatory diagram of an automobile 1 to which an outside-vehicle image capturing device 10 according to an embodiment of the present invention is applied. The automobile 1 in Fig. 1 has a vehicle body 2. The vehicle body 2 is provided with a vehicle compartment 3 in which passengers ride.

[0022] An example of a vehicle is an automobile 1. The automobile 1 in Fig. 1 is traveling forward along a lane between a pair of lane boundary lines on a straight road. Meanwhile, in the field of automobiles 1, research and development is progressing on driving control using automated driving and driving assistance, for example. When the automobile 1 is driven automatically or by driving assistance, the automobile 1 needs information about the driving environment outside the automobile 1. For this reason, the automobile 1 is provided with a plurality of imaging devices 11 to 18 as shown in Fig. 1, and is provided with a sensor that uses output waves, such as a lidar. For example, the automobile 1 in Figure 1 is provided with multiple imaging devices in the exterior imaging device 10 of the automobile 1, including a front center imaging device 13, a front right imaging device 11, a front left imaging device 12, a right front imaging device 14, a right rear imaging device 15, a left front imaging device 16, a left rear imaging device 17, and a rear center imaging device 18.

[0023] The front center imaging device 13 is provided, for example, at the front center of the vehicle interior 3. The front center imaging device 13 captures images of the area in front of the automobile 1 within a field angle range Rfc of, for example, 120 degrees.

[0024] The front right imaging device 11 and the front left imaging device 12 are fixedly supported relative to each other by, for example, a common stay (not shown) and are installed in the front center of the vehicle interior 3. The front right imaging device 11 and the front left imaging device 12 capture images of the area in front of the vehicle 1 over a viewing angle range Rfr, Rfl of, for example, 120 degrees. The front right imaging device 11 and the front left imaging device 12 are installed so that they can capture images of the area in front of the vehicle 1 over substantially overlapping imaging ranges. In this case, the images captured by the front right imaging device 11 and the front left imaging device 12 can be processed by the compound eye processing of the stereo camera 19 to generate relative distance and direction information based on the parallax of a surrounding object outside the vehicle, such as a pedestrian, captured in an imaging area common to the two images. The relative distance and direction to surrounding objects determined by the compound eye processing of the stereo camera 19 are unlikely to be affected by, for example, the posture or vibration of the vehicle body 2 while traveling, and a high degree of accuracy can be expected.

[0025] The right front imaging device 14 is provided on the right A-pillar of the vehicle body 2 or in the passenger compartment 3 near the right A-pillar. The right front imaging device 14 captures images of the right front of the automobile 1 in a field angle range Rrf of, for example, 120 degrees.

[0026] The right rear imaging device 15 is provided on the right side of the vehicle body 2, for example, on the right door mirror. The right rear imaging device 15 captures images of the right rear of the automobile 1 within a viewing angle range Rrr of, for example, 120 degrees.

[0027] The left front imaging device 16 is provided on the left A-pillar of the vehicle body 2 or in the passenger compartment 3 near the left A-pillar. The left front imaging device 16 captures an image of the right front of the automobile 1 in a field angle range Rlf of, for example, 120 degrees.

[0028] The left rear imaging device 17 is provided on the left side of the vehicle body 2, for example, on the left door mirror. The left rear imaging device 17 captures images of the left rear of the automobile 1 within a viewing angle range Rlr of, for example, 120 degrees.

[0029] The rear center imaging device 18 is provided, for example, at the rear center of the vehicle interior 3. The rear center imaging device 18 captures images of the area behind the automobile 1 within a viewing angle range Rrc of, for example, 120 degrees.

[0030] In this way, the front central imaging device 13, the right front imaging device 14, the right rear imaging device 15, the left front imaging device 16, the left rear imaging device 17, and the rear central imaging device 18 can capture images of the outside of the vehicle in a 360-degree range around the body 2 of the automobile 1.

[0031] FIG. 2 is an explanatory diagram of an imaging situation and a captured image in monocular processing by one imaging device 41 in FIG.

[0032] Fig. 2(A) is an explanatory diagram of the imaging situation of the imaging device 41. The imaging device 41 of Fig. 2(A) is provided in the automobile 1. The imaging device 41 of Fig. 2(A) captures an image of the outside of the vehicle within a vertical angle of view range 42. 2A shows a road surface on which the automobile 1 is traveling and a pedestrian standing on the road surface. A normal line perpendicular to the road surface can be defined.

[0033] Fig. 2(B) is an explanatory diagram of an image 51 captured by the imaging device 41 in Fig. 2(A). The image 51 in Fig. 2(B) is an image of the outside of the vehicle captured using the vertical and horizontal angles of view of the imaging device 41. An image of a pedestrian is included in the center of the captured image 51. 2B, a captured image 51 is schematically shown with multiple concentric circles having the imaging device 41 as the imaging center. The concentric circles are lines connecting equal distances from the imaging center, with the imaging device 41 as the imaging center. The captured image 51 also shows multiple straight lines extending radially from the imaging center. In this way, the captured image 51 of the imaging device 41 can be associated with a relative distance and direction based on the imaging center where the imaging device 41 is located, for each imaging position of the captured image 51. In monocular processing of the captured image 51, the relative distance and direction based on such association can be obtained based on the imaging position in the captured image.

[0034] 2(C) is an explanatory diagram of a road surface model 55 for monocular processing, corresponding to the captured image 51 of the imaging device 41. The captured image 51 is composed of information such as the luminance values ​​of a plurality of pixels that divide the imaging range in a matrix. In the road surface model 55 of FIG. 2(C), each pixel of the captured image 51 is associated with a relative distance and direction on the road surface of FIG. 2(A) with the imaging center where the imaging device 41 is located as the reference. In the captured image 51 of FIG. 2(B), a pixel 57 corresponding to the center of the bottom edge of a rectangular frame 52 set around the image of the pedestrian can be identified as indicating the relative distance and direction of the pedestrian. By identifying pixel 57 in road surface model 55 of FIG. 2(C), it is possible to obtain the distance and direction on the road surface to the pedestrian in FIG. 2(A). In this way, the distance and direction on the road surface to the surrounding objects around the automobile 1 can be obtained by monocular processing based on one captured image 51. In controlling automatic driving or driving assistance, the driving of the automobile 1 can be controlled using information on the relative distance or direction of such an object captured outside the vehicle. By dividing and capturing images of the 360-degree surroundings of the automobile 1 using multiple imaging devices 11-18, or by using a single imaging device capable of capturing 360 degrees, the automobile 1 can obtain information about surrounding objects that may affect the automobile's driving, thereby improving the reliability of autonomous driving and driving assistance control.

[0035] FIG. 3 is an explanatory diagram of the influence of deviation that may occur in the imaging device provided in the automobile 1. In FIG.

[0036] FIG. 3A is an explanatory diagram of a deviation that may occur in an image 61 captured by an imaging device. 3A, the captured image 61 may be subject to rotational deviation around the center of the captured image 61, vertical deviation in the up and down direction of the captured image 61, and horizontal deviation in the left and right direction of the captured image 61 due to, for example, changes in the attachment state of the imaging device to the vehicle body 2 or changes in the attitude of the vehicle body 2 itself. Even if the imaging device is directly attached to the vehicle body 2 itself, the position and orientation of the imaging device may change, for example, due to an impact being applied to the vehicle body 2. In particular, when the imaging device is not directly attached to the vehicle body 2 itself and therefore high attachment rigidity cannot be expected, the attachment state of the imaging device is likely to deviate over time from the reference position in the design or from the attachment position at the time of manufacture.

[0037] 3B shows a captured image 62 when the imaging device is displaced downward and to the left from the reference position. A road surface model is drawn and superimposed on the captured image 62. A captured image 62 in FIG. 3(B) is an image of the pedestrian in FIG. 2(A) similar to the captured image 51 in FIG. 2(B). The orientation of the imaging device that captures the captured image 62 in FIG. 3(B) is shifted to the lower left on the paper surface. In this case, the imaging position of the image of the pedestrian included in the captured image 62 is shifted to a position in the upper right direction of the captured image 62. As a result, pixel 63 identified as indicating the relative distance and direction of the pedestrian is the pixel to the upper right of pixel 57 identified in FIG. 2(C). The relative distance or direction of the pedestrian identified based on the position of pixel 63 and the road surface model will be less accurate than the captured image 51 in FIG. 2(B) where no deviation occurs.

[0038] 3(C) shows a captured image 64 when the imaging device is displaced laterally to the left from the reference position. A road surface model is drawn and superimposed on the captured image 64. A captured image 64 in FIG. 3(C) is an image of the pedestrian in FIG. 2(A) similar to the captured image 51 in FIG. 2(B). The orientation of the imaging device that captures the captured image 64 in FIG. 3(C) is shifted to the left on the paper surface. In this case, the imaging position of the image of the pedestrian included in the captured image 64 is shifted to a position to the right of the captured image 64. As a result, pixel 65 identified as indicating the relative distance and direction of the pedestrian is the pixel to the right of pixel 57 identified in FIG. 2(C). The relative distance or direction of the pedestrian identified based on the position of pixel 65 and the road surface model will be less accurate than the captured image 51 in FIG. 2(B) where no deviation occurs.

[0039] FIG. 4 is an explanatory diagram of the influence of misalignment that may occur between multiple imaging devices provided on the automobile 1. In FIG. 4(A) to 4(C) show images 71 and 72 captured by two imaging devices, a left front imaging device 16 and a left rear imaging device 17, which are provided on the left side of a car 1 traveling forward. Time flows from FIG. 4(A) to FIG. 4(C). 4, the left front imaging device 16 is provided at a reference position, whereas the left rear imaging device 17 is provided at a position shifted downward and to the right from the reference position.

[0040] FIG. 4A is an explanatory diagram of a situation in which a pedestrian, which is the subject of the image capture, is captured in the central region of the left-front captured image 71 while the left-rear captured image 72 remains misaligned. 4A, the left front imaging device 16, out of the left front imaging device 16 and the left rear imaging device 17, captures an image of a pedestrian. The pedestrian is captured at position 73 in the central region of the image captured by the left front imaging device 16.

[0041] FIG. 4B is an explanatory diagram of a situation in which a pedestrian, which is the subject of imaging, is being imaged in the rear area of ​​the left front captured image 71 immediately after FIG. 4A. In FIG. 4B, which is a timing after FIG. 4A, the automobile 1 is traveling forward, so the pedestrian is imaged at position 74 in the left area of ​​the image 71 captured by the left front imaging device 16.

[0042] FIG. 4C is an explanatory diagram of a situation in which a pedestrian, who is the subject of imaging, is imaged in the front area of ​​the shifted left rear captured image 72 immediately after FIG. 4B. 4(C), which is a timing after that of FIG. 4(B), the pedestrian is imaged by the left rear imaging device 17, not the left front imaging device 16. The pedestrian is imaged at position 75 near the center of the image 72 captured by the left rear imaging device 17. In Figure 4(C), the position 75 of the image of the pedestrian included in the image 72 captured by the left rear imaging device 17 is shifted to the upper left compared to the position 76 that would be present if there was no shift due to the shift occurring in the left rear imaging device 17. As a result, the distance and direction of the pedestrian obtained by monocular processing based on the image captured by the left rear imaging device 17 in Figure 4(C) will be less accurate than position 76 if there was no deviation. 4(A) and 4(B), the pedestrian is imaged moving horizontally from right to left in image 71 captured by the left front imaging device 16, and then is imaged as if moving instantaneously in image 72 captured by the left rear imaging device 17 in Fig. 4(C). The distance and direction of the pedestrian change discontinuously as they move between the multiple captured images 71 and 72.

[0043] In this embodiment, we propose a technology that can make the various problems described above less likely to occur even if the imaging ranges and angles of view of multiple imaging devices 11 to 18 installed on a moving automobile 1 shift due to changes in their mounting state or vibrations. In this embodiment, a technique is proposed that allows the multiple imaging devices 11-18 to withstand use even if they are not directly attached to the highly rigid vehicle body 2. By not directly attaching the multiple imaging devices 11-18 to the highly rigid vehicle body 2, it becomes possible to attach them in any position suitable for imaging, etc. As a result, in this embodiment, the reliability of the multiple imaging devices 11 to 18 provided in the automobile 1 for capturing images of the outside of the automobile 1 can be ensured or maintained.

[0044] FIG. 5 is an explanatory diagram of the control system 9 including the outside-vehicle image capturing device 10 in the automobile 1 of FIG. The control system 9 in Figure 5 constitutes the control system of the automobile 1, and includes an exterior imaging device 10, a driving assistance device 32, an occupant protection device 33, an exterior protection device 34, an exterior communication device 35, a sensor device 36, and a vehicle network 31 to which these are connected.

[0045] The vehicle network 31 may be, for example, a controller area network (CAN), a local interconnect network (LIN), a local area network (LAN), FlexRay, or a clock extension peripheral interface (CXPI) that is used in the automobile 1. The vehicle network 31 may be configured, for example, with multiple bus cables connected to multiple control devices and a central gateway (CGW) to which the multiple bus cables are bus-connected. The multiple control devices can send and receive messages to each other through the vehicle network 31. This allows the multiple control devices to cooperate to control the automobile 1.

[0046] The driving assistance device 32 controls, for example, the steering device, drive device, braking device, etc. (not shown) of the automobile 1, and controls the traveling of the automobile 1. The drive device may be equipped with an engine that generates driving force by burning fuel such as gasoline, diesel, or hydrogen gas, or may be equipped with a motor that generates driving force by storing power in a battery or by generating electricity, or may be equipped with both. The driving assistance device 32 may control the traveling of the automobile 1 according to the operation of the passenger of the automobile 1, may control the traveling of the automobile 1 by assisting the operation of the passenger, or may control the traveling of the automobile 1 by controlling autonomous automatic driving.

[0047] The occupant protection device 33 controls an airbag device, a seat belt device, etc. (not shown) for protecting the occupants of the automobile 1, and executes control for protecting the occupants of the automobile 1. The occupant protection device 33 may execute control for protecting the occupants of the automobile 1 based on the detection or prediction of a collision of the automobile 1.

[0048] The vehicle exterior protection device 34 controls an airbag device (not shown) or the like for protecting pedestrians and cyclists around the vehicle 1, and executes control to protect pedestrians and cyclists around the vehicle 1. The vehicle exterior protection device 34 may execute control to protect pedestrians and cyclists, for example, based on detection or prediction of a collision between the pedestrian or cyclist and the vehicle 1.

[0049] The exterior communication device 35 establishes a communication path with, for example, a base station (not shown) installed along a road, and transmits and receives data via wireless communication between the control system 9 and a server device (not shown) connected to the base station. This allows the exterior communication device 35 to transmit information such as a collision generated by the control system 9 of the automobile 1 to the server device outside the vehicle. In this case, the control system 9 of the automobile 1 can transmit information such as a collision generated by the control system 9 of the automobile 1 to the server device outside the vehicle. The exterior communication device 35 may also establish a communication path with other moving bodies, such as other vehicles traveling around the automobile 1. In this case, the control system 9 of the automobile 1 can transmit information such as collision information generated by the control system 9 of the automobile 1 to the other moving bodies, and can receive information such as driving information from the other moving bodies.

[0050] The sensor device 36 is connected to various sensors provided in the automobile 1, such as an acceleration sensor 37. The acceleration sensor 37 detects the acceleration accompanying the traveling of the automobile 1. The sensor device 36 may generate information such as the traveling direction, traveling speed, and attitude of the automobile 1 based on the detected value of the acceleration sensor 37.

[0051] The vehicle exterior imaging device 10 includes a plurality of imaging devices 11 to 18, an input / output port 22, an interior input / output unit 21, a timer 25, a memory 23, a CPU 24, and a system bus 26 to which these are connected. The components of the outside-vehicle image capture device 10 can input and output data to and from each other via a system bus 26 .

[0052] The plurality of imaging devices 11 to 18 may be any devices provided on the automobile 1 for capturing images of the outside of the automobile 1, and may be the devices shown in FIG. In this embodiment, the front center imaging device 13 in FIG. 1 is arranged so that at least a part of the imaging area of ​​its angle of view range Rfc overlaps with the angle of view range Rfr of the front right imaging device 11 and the angle of view range Rfl of the front left imaging device 12. The angle of view range Rrf of the right front imaging device 14 is set so that a part of the imaging area on the left side thereof overlaps with the angle of view range Rfr of the front right imaging device 11 and the angle of view range Rfl of the front left imaging device 12. In addition, the angle of view range Rrf of the right front imaging device 14 is set so that a part of the imaging area on the left side thereof overlaps with the angle of view range Rfc of the front center imaging device 13. The angle of view range Rrr of the right rear imaging device 15 is set so that a part of the imaging area on the left side thereof overlaps with the imaging area of ​​the angle of view range Rrf of the right front imaging device 14. The angle of view range Rlf of the left front imaging device 16 is set so that a part of the imaging area on the right side thereof overlaps with the angle of view range Rfr of the front right imaging device 11 and the angle of view range Rfl of the front left imaging device 12. In addition, the angle of view range Rlf of the left front imaging device 16 is set so that a part of the imaging area on the right side thereof overlaps with the angle of view range Rfc of the front center imaging device 13. The angle of view range Rlr of the left rear imaging device 17 is set so that a part of the imaging area on the right side thereof overlaps with the imaging area of ​​the angle of view range Rlf of the left front imaging device 16. The angle of view range Rrc of the rear center imaging device 18 is set so that a part of the imaging area on the left side thereof overlaps with the angle of view range Rrr of the right rear imaging device 15. In addition, the angle of view range Rrc of the rear center imaging device 18 is set so that a part of the imaging area on the right side thereof overlaps with the angle of view range Rlr of the left rear imaging device 17.

[0053] A plurality of imaging devices 11 to 18 are connected to the input / output port 22. The plurality of imaging devices 11 to 18 output to the input / output port 22 the images that they each periodically capture.

[0054] The in-vehicle input / output unit 21 is connected to a vehicle network 31. The in-vehicle input / output unit 21 inputs and outputs messages including data to and from other devices constituting the control system 9, such as a driving assistance device 32, via the vehicle network 31.

[0055] The timer 25 measures time. The time of the timer 25 may be calibrated by radio waves from a GNSS satellite (not shown). A GNSS receiver that receives signals from the GNSS satellite may be connected to the vehicle network 31 or the sensor device 36.

[0056] The memory 23 may be configured with, for example, a HDD, SD, RAM, ROM, etc. The memory 23 stores, for example, programs executed by the CPU 24 and data such as tables used in executing the programs. The memory 23 may store, for example, a table of distances and directions for multiple pixels of the road surface model 55 in FIG. 2(C).

[0057] The CPU 24 reads and executes the program recorded in the memory 23. In this way, the CPU 24 functions as a control unit of the outside-of-vehicle image capturing device 10. The CPU 24 as a control unit of the outside-of-vehicle image capturing device 10 controls the overall operation of the outside-of-vehicle image capturing device 10 . When the CPU 24 acquires captured images from each of the plurality of imaging devices 11 to 18 via the input / output port 22, for example, it recognizes and identifies surrounding objects such as pedestrians based on the acquired captured images. The CPU 24 uses the images captured by the multiple imaging devices 11-18 to generate a relative distance or direction to the surrounding object identified as the imaging target outside the vehicle. The CPU 24 outputs the information generated about the identified surrounding objects to other devices constituting the control system 9 via the in-vehicle input / output unit 21 and the vehicle network 31. The CPU 24 may appropriately execute calibration control to correct misalignment of each of the plurality of imaging devices 11 to 18 of the outside-of-vehicle imaging device 10 as necessary.

[0058] FIG. 6 is an explanatory diagram of the imaging ranges of the multiple imaging devices 11 to 18 in FIG. 5 and the order of deviation correction control in 360-degree calibration.

[0059] FIG. 6 shows a plurality of images captured by the plurality of imaging devices 11 to 18 shown in FIG. The plurality of captured images are arranged with their imaging ranges overlapping each other, with the center front image (IRfc) of the automobile 1 at the center. In the central part of Figure 6, the captured image IRfc of the front center imaging device 13, the captured image IRfr of the front right imaging device 11, and the captured image IRlr of the front left imaging device 12 are shown with their imaging areas almost entirely overlapping. On the right side of FIG. 6, an image IRrf captured by the right front imaging device 14, an image IRrr captured by the right rear imaging device 15, and an image IRrc1 captured by the rear central imaging device 18 are shown overlapping in part of their imaging areas in that order from the center. On the left side of FIG. 6, an image IRlf captured by the left front imaging device 16, an image IRlr captured by the left rear imaging device 17, and an image IRrc2 captured by the rear center imaging device 18 are shown overlapping in part of their imaging areas in that order from the center.

[0060] In this way, the multiple imaging devices 11 to 18 shown in FIG. 1 capture images of the automobile 1 from the front center to the rear center in a clockwise direction, and from the front center to the rear center in a counterclockwise direction. 1 can capture images of a 360-degree area around the automobile 1. Peripheral objects such as pedestrians around the automobile 1 can be captured by at least one of the imaging devices 11 to 18. 6, the vertical offset of the multiple captured images is for ease of viewing and explanation. The capturing ranges of the multiple captured images may basically be aligned along the periphery of the automobile 1 without any vertical offset of the automobile 1.

[0061] FIG. 7 is a flowchart of 360-degree calibration control by the CPU 24 of the outside-vehicle image capture device 10 of FIG. 7 uses the captured image IRfr of the front right imaging device 11 and the captured image IRlr of the front left imaging device 12, which can be used for compound eye processing of the stereo camera 19, as initial reference images to generate deviation correction information for single eye processing of the other imaging devices 13 to 18. Then, in the 360 ​​degree calibration control of Fig. 7, deviation correction information for each captured image is generated in clockwise and counterclockwise order, starting from the central captured image IRfc in Fig. 6. The misalignment correction information generated here may be used in monocular processing of the captured images of each imaging device to obtain the relative distance or direction of the imaged object outside the vehicle (such as a surrounding object) based on the imaging position within the imaging range. In this case, the deviation correction information is distance or direction correction information for the relative distance or direction information for each pixel (imaging position) in the captured image, which is set in advance, for example, at the time of design, for monocular processing of the captured image of each imaging device.

[0062] The CPU 24 of the outside-vehicle image capture device 10 may repeatedly execute the 360-degree calibration control shown in FIG. The CPU 24 may execute all of the multiple processes of the 360-degree calibration control in FIG. 7 at once, or may execute the processes for the image captured by one imaging device and then execute the processes for the image captured by the next imaging device at intervals after a time has elapsed. The CPU 24 may not only periodically execute the calibration control of FIG. 7, but also execute it appropriately, for example, when an impact is applied to the vehicle body 2, when the vehicle starts to travel, at the time of manufacturing and shipping, or during vehicle inspection. 7, the CPU 24 may generate misalignment correction information based on only one imaging target of the latest captured image at the time of each processing (current time), but it is preferable to generate average misalignment correction information based on multiple imaging targets or multiple imaging targets captured multiple times. This is expected to suppress the influence of recognition errors depending on the type of imaging target and the influence of errors due to imaging timing, and generate more reliable misalignment correction information.

[0063] In step ST1, the CPU 24 generates deviation correction information to be used for the monocular processing of the front central imaging device 13, based on information on the relative distance and direction of an object to be imaged outside the vehicle by compound eye processing of the stereo camera 19 of the front right imaging device 11 and the front left imaging device 12, whose relative positions are specified. As shown in Fig. 6, the imaging area of ​​the front central imaging device 13 overlaps with the imaging area of ​​both the front right imaging device 11 and the front left imaging device 12, which form the stereo camera 19. At least three of the multiple imaging devices are provided on the automobile 1 so as to be able to capture images of their common imaging areas overlapping each other. The CPU 24 recognizes an object outside the vehicle that is commonly captured in the images captured by these three imaging devices, and generates information on the relative distance and direction based on the parallax of the object captured by the stereo camera 19. The CPU 24 generates deviation correction information for the image captured by the front central imaging device 13 related to processing, based on the difference between the relative distance and direction based on the parallax of the object being imaged by the stereo camera 19 and the relative distance and direction based on the road pattern at the front central imaging device 13.

[0064] In step ST2, similarly to step ST1, the CPU 24 generates information for correcting deviation of the image captured by the right front imaging device 14 for monocular processing, based on the relative distance and direction based on the parallax of the object being imaged by the stereo camera 19. A part of the imaging area of ​​the right front imaging device 14 overlaps with the imaging areas of the front right imaging device 11 and the front left imaging device 12 that constitute the stereo camera 19, and a common object can be imaged overlapping in the imaging area common to the three. Similarly to step ST1, the CPU 24 generates information for correcting deviation of the image captured by the left front imaging device 16 for monocular processing, based on the relative distance and direction based on the parallax of the object being imaged by the stereo camera 19. A part of the imaging area of ​​the left front imaging device 16 overlaps with the imaging areas of the front right imaging device 11 and the front left imaging device 12 that constitute the stereo camera 19, and a common object can be imaged overlapping in the imaging area common to the three.

[0065] In step ST3, the CPU 24 utilizes the fact that a portion of the image captured by the right rear imaging device 15 for monocular processing overlaps with the image captured by the right front imaging device 14 for monocular processing, capturing a common image of the same object, and generates misalignment correction information for the image captured by the right rear imaging device 15, based on information about the relative distance or direction of the object outside the vehicle in the image captured by the right front imaging device 14 for monocular processing after misalignment correction. The common object can be captured in an overlapping image in the overlapping common imaging area in the two captured images. Furthermore, the CPU 24 utilizes the fact that a portion of the image captured by the left rear imaging device 17 for monocular processing overlaps with the image captured by the left front imaging device 16 for monocular processing, capturing a common image of the same object, and generates misalignment correction information for the image captured by the left rear imaging device 17, based on information about the relative distance or direction of the object outside the vehicle in the image captured by the left front imaging device 16 for monocular processing after misalignment correction. The common object can be captured in an overlapping image in the overlapping common imaging area in the two captured images.

[0066] In step ST4, the CPU 24 utilizes the fact that a portion of the image captured by the rear center imaging device 18 for monocular processing overlaps with the image captured by the right rear imaging device 15 for monocular processing, capturing a common object in an overlapping manner, to generate misalignment correction information for the image captured by the rear center imaging device 18, based on information about the relative distance or direction of the object outside the vehicle in the image captured by the right rear imaging device 15 for monocular processing after misalignment correction. The common object can be captured in an overlapping manner in the overlapping common imaging area in the two captured images. Furthermore, the CPU 24 utilizes the fact that a portion of the image captured by the rear central imaging device 18 for monocular processing overlaps with an image captured by the left rear imaging device 17 for monocular processing, capturing a common image of the same object, and generates misalignment correction information for the image captured by the rear central imaging device 18, based on information about the relative distance or direction of the object outside the vehicle in the image captured by the left rear imaging device 17 for monocular processing after misalignment correction. The common object can be captured in an overlapping image in the overlapping common imaging area in the two captured images. When only obtaining information on misalignment correction for the rear central imaging device 18 for monocular processing, the CPU 24 may generate only one of the two pieces of misalignment correction information described above.

[0067] In this way, when generating misalignment correction information for multiple single-eye processing imaging devices 13 to 18, the CPU 24 as a control unit uses the captured images of two imaging devices 11 and 12 that can be used for compound-eye processing of the stereo camera 19, whose relative positions are specified, as a reference and generates misalignment correction information in order from the single-eye processing imaging devices 13 to 15 that can capture images by overlapping an imaging area common to the stereo camera 19. Furthermore, the multiple monocular processing imaging devices 14-18 are provided on the automobile 1 so as to capture images of the 360-degree surroundings of the automobile 1 in a divided manner, and are provided on the automobile 1 so as to be able to capture images of overlapping imaging areas common to other monocular processing imaging devices adjacent in the direction of division of the 360-degree surroundings of the automobile 1, and generate deviation correction information based on the imaging positions of the other monocular processing imaging devices in the captured images after deviation correction. The CPU 24 generates deviation correction information for the relative distance and direction of the imaging target outside the vehicle based on the imaging positions in the imaging ranges used in monocular processing for the multiple monocular processing imaging devices 13-18 in order according to the order of overlap of the imaging areas. The CPU 24 can generate deviation correction information for a plurality of monocular processing imaging devices 14-18 that capture images of the 360-degree surroundings of the automobile 1 in a divided manner, starting with the monocular processing imaging device 14 at one end of the imaging range of the stereo camera 19, which can capture overlapping images of a common imaging area at one end, and the monocular processing imaging device 15 at the other end of the imaging range of the stereo camera 19, which can capture overlapping images of a common imaging area at the other end. The rear central imaging device 18, which is subjected to the last deviation correction process, is arranged to capture images of the rear side of the automobile 1, opposite to the stereo camera 19. For the rear central imaging device 18 for monocular processing, which is processed last, the CPU 24 generates information on deviation correction in the clockwise direction and information on deviation correction in the counterclockwise direction.

[0068] In step ST5, the CPU 24 evaluates whether the multiple pieces of deviation correction information for the multiple monocular processing imaging devices 13 to 15 generated by the above processing are capable of reliably obtaining the relative distance and direction of surrounding objects outside the vehicle. The CPU 24 may, for example, compare information on misalignment correction in the clockwise direction (one end side) with information on misalignment correction in the counterclockwise direction (the other end side) for the rear central imaging device 18, which is the last to undergo misalignment correction processing, to evaluate the series of corrections. If the series of corrections is good enough to reliably obtain relative distance and direction through monocular processing, the CPU 24 ends this control. If the series of corrections cannot be evaluated as good, the CPU 24 advances the process to step ST6.

[0069] In step ST6, the CPU 24 outputs a warning to the effect that it may not be possible to obtain highly accurate distances or orientations of external objects around the automobile 1 based on the images captured by the multiple imaging devices 13 to 18. The warning is notified to the occupants through a user interface such as a speaker or a liquid crystal device (not shown) provided in the automobile 1. Furthermore, the CPU 24 outputs a maintenance request for the installation of the multiple imaging devices 11 to 18. Information on the maintenance request may be output to the external communication device 35 via the internal input / output unit 21 and the vehicle network 31. The external communication device 35 may transmit the information to a server device (not shown) of a company that maintains and inspects the automobile 1. Thereafter, the CPU 24 ends this control.

[0070] FIG. 8 is a detailed flowchart of the stereo-based monocular misalignment correction control. The CPU 24 may execute the stereo-based monocular displacement correction control of FIG. 8 in each of the following cases: when generating displacement correction for the front central imaging device 13 in step ST1 of FIG. 7; when generating displacement correction for the right front imaging device 14 in step ST2 of FIG. 7; and when generating displacement correction for the left front imaging device 16 in step ST2 of FIG. 7.

[0071] In step ST11, the CPU 24 acquires two captured images by stereo imaging using the front right imaging device 11 and the front left imaging device 12, the positions of which are defined relative to each other.

[0072] In step ST12, the CPU 24 acquires the captured image by monocular imaging using the imaging devices 13, 14, and 16 involved in the misalignment correction process.

[0073] In step ST13, the CPU 24 identifies an imaging target, such as a peripheral object, in the triple common imaging area of ​​the three captured images acquired in step ST11 and step ST12. The CPU 24 may, for example, extract image components that are commonly included in the triple common imaging area to identify an imaging target, such as a peripheral object. Note that the imaging devices 13, 14, and 16 involved in the deviation correction process may be misaligned or tilted in their mounting positions relative to the vehicle body 2. The CPU 24 may identify an imaging target, such as a peripheral object, in an area smaller than the lane imaging area shown in FIG. 6, taking into account such deviations, for example, deviation correction performed in the previous process.

[0074] In step ST14, the CPU 24 acquires, as stereo position information, the relative distance and direction of the identified imaging target determined by compound eye processing of the stereo camera 19. The CPU 24 may acquire the relative distance and direction of the identified imaging target based on the parallax of the identified imaging target by trigonometric calculations or the like, based on two images captured by the stereo camera 19 that have not undergone misalignment correction.

[0075] In step ST15, the CPU 24 acquires, as monocular position information, the relative distance and direction of the identified imaging target determined by the monocular processing of the imaging device involved in the misalignment correction process. The CPU 24 may acquire the relative distance and direction of the identified imaging target based on a road pattern that does not include misalignment correction for the images captured by the imaging devices 13, 14, and 16 involved in the misalignment correction process.

[0076] In step ST16, the CPU 24 generates misalignment correction information to be used in the monocular processing of the imaging devices 13, 14, and 16 involved in the misalignment correction process. The CPU 24 may generate, as the misalignment correction information, the difference between the distance and direction of the imaging target in the stereo position information and the distance and direction of the imaging target in the monocular position information. As shown in FIG. 3A, the imaging devices 13, 14, and 16 involved in the misalignment correction process may experience rotational misalignment in addition to horizontal and vertical misalignment. If at least rotational misalignment exists, the CPU 24 may generate, as the misalignment correction information, misalignment correction information for all pixels of the imaging devices 13, 14, and 16 involved in the misalignment correction process, individually for each pixel.

[0077] In step ST17, the CPU 24 stores the generated misalignment correction information in the memory 23 in association with the imaging devices 13, 14, and 16 involved in the misalignment correction process.

[0078] In this way, the CPU 24 identifies the imaging position of the imaging target that is imaged overlapping the triple common imaging area in each of the images captured by the three imaging devices, i.e., the two imaging devices 11 and 12 of the stereo camera 19 and the imaging devices 13, 14, and 16 related to the misalignment correction processing, and can generate misalignment correction information that can be used in the monocular processing of the imaging devices 13, 14, and 16 related to the misalignment correction processing, based on the difference in imaging positions in the three captured images. The CPU 24 can use the information on the relative distance and direction of the stereo camera 19 to generate misalignment correction information about the relative distance and direction of the imaging target outside the vehicle based on the imaging positions used in the monocular processing of the imaging devices 13, 14, and 16 for the monocular processing.

[0079] FIG. 9 is a detailed flowchart of the monocular deviation correction control based on the adjacent monocular. The CPU 24 may execute the monocular displacement correction control based on the adjacent monocular of FIG. 9 in each of the following cases: when generating displacement correction for the right rear imaging device 15 in step ST3 of FIG. 7; when generating displacement correction for the left rear imaging device 17 in step ST3 of FIG. 7; and when generating displacement correction for the rear central imaging device 18 in step ST3 of FIG. 7.

[0080] In step ST21, the CPU 24 determines whether or not the misalignment correction process has been completed for the imaging devices 14, 16, 15 (17) adjacent to the imaging devices 15, 17, 18 involved in the process. If the misalignment correction process has not been completed for the adjacent imaging devices 14, 16, 15 (17), the CPU 24 repeats this process. Once the misalignment correction process has been completed for the adjacent imaging devices 14, 16, 15 (17), the CPU 24 proceeds to step ST22.

[0081] In step ST22, the CPU 24 acquires one reference captured image by monocular imaging using the adjacent imaging devices 14, 16, 15 (17) for which the deviation correction process has been completed.

[0082] In step ST23, the CPU 24 acquires the captured image by monocular imaging using the imaging devices 15, 17, and 18 involved in the misalignment correction process.

[0083] In step ST24, the CPU 24 identifies an imaging target, such as a peripheral object, in the double common imaging area of ​​the two captured images acquired in step ST22 and step ST23. The CPU 24 may, for example, extract image components that are commonly included in the double common imaging area to identify an imaging target, such as a peripheral object. Note that the imaging devices 15, 17, and 18 involved in the misalignment correction process may be misaligned or tilted in their mounting positions relative to the vehicle body 2. The CPU 24 may identify an imaging target, such as a peripheral object, in an area smaller than the imaging area of ​​the lane shown in FIG. 6, taking into account such misalignment, for example, the misalignment correction performed in the previous process.

[0084] In step ST25, the CPU 24 acquires the relative distance and direction of the identified imaging target in the reference image as position information in the reference image. The CPU 24 may acquire the relative distance and direction of the identified imaging target based on a road pattern obtained by correcting the misalignment of the reference images of the adjacent imaging devices 14, 16, 15 (17).

[0085] In step ST26, the CPU 24 acquires, as monocular position information, the relative distance and direction of the identified imaging target determined by the monocular processing of the imaging devices 15, 17, and 18 involved in the misalignment correction process. The CPU 24 may acquire the relative distance and direction of the identified imaging target based on a road pattern that does not include misalignment correction for the images captured by the imaging devices 15, 17, and 18 involved in the misalignment correction process.

[0086] In step ST27, the CPU 24 generates misalignment correction information to be used in the monocular processing of the imaging devices 15, 17, and 18 involved in the misalignment correction process. The CPU 24 may generate, as the misalignment correction information, the difference between the distance and direction of the imaging target in the reference image and the distance and direction of the imaging target in the single image. As shown in FIG. 3A, the imaging devices 15, 17, and 18 involved in the misalignment correction process may experience rotational misalignment in addition to horizontal and vertical misalignment. If rotational misalignment exists, the CPU 24 may generate, as the misalignment correction information, misalignment correction information for all pixels of the imaging devices 15, 17, and 18 involved in the misalignment correction process, individually for each pixel.

[0087] In step ST28, the CPU 24 stores the generated misalignment correction information in the memory 23 in association with the imaging devices 15, 17, and 18 involved in the misalignment correction process.

[0088] In this way, the CPU 24 identifies the imaging position of the imaging target that is imaged overlappingly in the doubly common imaging area in each of the images captured by the two imaging devices, i.e., the imaging devices 15, 17, 18 related to the misalignment correction processing and the adjacent imaging devices 14, 16, 15(17), and can generate misalignment correction information that can be used in the monocular processing of the imaging devices 15, 17, 18 related to the misalignment correction processing, based on the difference in imaging position in the two captured images.The CPU 24 can generate misalignment correction information about the relative distance and direction of the imaging target outside the vehicle based on the imaging position to be used in the monocular processing of the imaging devices 15, 17, 18 for monocular processing, using information about the relative distance or direction of the imaging devices 14, 16, 15(17) for which misalignment correction information has been generated.

[0089] FIG. 10 is a block diagram for imaging control using the deviation correction information generated by the control of FIGS. The CPU 24 executes the program recorded in the memory 23 to implement a peripheral object recognition processing unit 82, a peripheral object position information generation unit 83, a quantitative deviation correction unit 84, a real-time deviation correction unit 85, and a peripheral object information output unit 86 in the outside-vehicle image capture device 10, as shown in FIG. 10 . 10 also illustrates one imaging device 81 used for monocular processing in the vehicle exterior image capture device 10, and a vehicle control execution unit 87 that executes control based on peripheral object information generated by the vehicle exterior image capture device 10. The imaging device 81 may be one selected from, for example, the front center imaging device 13, the right front imaging device 14, the right rear imaging device 15, the left front imaging device 16, the left rear imaging device 17, and the rear center imaging device 18. The memory 23 of the outside-vehicle imaging device 10 stores road surface model data 91 that can be associated with the captured image in the monocular processing of the imaging device 81, and deviation correction information 92 for the imaging device 81 generated by the above-mentioned processing. The CPU 24 may implement the imaging control block of FIG. 10 for each of the imaging devices 13 to 18 for monocular processing provided in the outside-of-vehicle imaging device 10.

[0090] The imaging device 81 captures an image of the outside of the automobile 1 and outputs the captured image to a surrounding object recognition processing unit 82.

[0091] The peripheral object recognition processing unit 82 executes a process for recognizing peripheral objects captured in an image captured by the imaging device 81. The peripheral object recognition processing unit 82 may recognize other vehicles, pedestrians, cyclists, etc. that exist in the vicinity outside the captured image of the vehicle through peripheral object recognition processing. The peripheral object recognition processing unit 82 may also recognize other objects, such as roads, roadside installations, roadside structures, and lane boundary lines drawn on roads. The peripheral object recognition processing unit 82 may recognize the captured peripheral objects using, for example, feature information of the peripheral objects pre-recorded in the memory 23 for each peripheral object, or may recognize the captured peripheral objects by deep learning network processing that has learned the peripheral objects, or may recognize the captured peripheral objects by combining these. The peripheral object recognition processing unit 82 may perform the peripheral object recognition processing by converting the captured image by resolution conversion, size conversion, color component conversion, or differential processing, or may perform the peripheral object recognition processing by cutting out a portion of the captured image. The peripheral object recognition processing unit 82 may perform the peripheral object recognition processing by different processing for each region of the captured image.

[0092] The peripheral object position information generating unit 83 generates position information including relative distance and direction for each of one or more peripheral objects recognized by the peripheral object recognition processing unit 82. The surrounding object position information generation unit 83 may generate position information including relative distance and direction for the pixel at the imaging position of the surrounding object using road surface model data 91 recorded in memory 23 for monocular processing.

[0093] The quantitative deviation correction unit 84 executes a correction process to reduce the quantitative deviation of the imaging device 81 that is included in the relative distance and direction of the surrounding objects. The quantitative deviation correction unit 84 acquires deviation correction information 92 recorded in the memory 23 in association with the imaging device 81, and corrects the relative distance and direction of surrounding objects using the acquired deviation correction information 92. This allows the CPU 24 to obtain the distance and direction of the surrounding objects in a manner that suppresses the influence of the deviation in the mounting position of the imaging device 81.

[0094] The real-time deviation correction unit 85 acquires the latest attitude information 93 of the automobile 1, calculates the deviation of the imaging device 81 caused by changes in the attitude of the automobile 1, and executes a correction process to reduce the calculated deviation. The real-time deviation correction unit 85 may acquire the latest attitude information 93 of the automobile 1 based on the detection of the acceleration sensor 37 from the sensor device 36, for example. The real-time deviation correction unit 85 calculates the deviation of the imaging device 81 using the acquired attitude information 93, information about the body 2 of the automobile 1, and information about the mounting position of the automobile 1 relative to the imaging device 81. The information about the body 2 of the automobile 1 and the information about the mounting position of the imaging device 81 may be recorded as data in the memory 23 or as parameters in a program. The road surface model basically indicates the relative distance and direction corresponding to each pixel in the imaging state of FIG. 2(A). When the attitude of the automobile 1 changes, the height of the imaging device 81 from the road surface, the orientation of the imaging device 81, and so on change. The real-time deviation correction unit 85 may calculate the amount of change in the attitude and distance of the imaging device 81 relative to the road surface of FIG. 2(A) under the changed attitude of the automobile 1 as the real-time deviation of the imaging device 81. The real-time deviation correction unit 85 corrects the relative distance and direction of the surrounding objects using the calculated amount of change in distance and direction. This allows the CPU 24 to obtain the distance and direction of the surrounding objects relative to each other while suppressing the influence of the shift in the imaging device 81 caused by the change in the attitude of the automobile 1.

[0095] The surrounding object information output unit 86 outputs the generated information about the recognized surrounding objects to each device of the automobile 1. In the information on surrounding objects that the surrounding object information output unit 86 outputs to each device of the automobile 1, the relative distance and direction of the surrounding objects have been corrected by the correction process described above, and are highly accurate as they are free from the influence of misalignment of the mounting position of the imaging device 81 and the influence of misalignment of the imaging device 81 due to changes in the attitude of the automobile 1.

[0096] The vehicle control execution unit 87 uses the information on the surrounding objects output by the surrounding object information output unit 86 to execute processing according to the driving environment of the automobile 1. The vehicle control execution unit 87 may be, for example, the driving assistance device 32, the passenger protection device 33, the vehicle exterior protection device 34, or the vehicle exterior communication device 35 shown in FIG. For example, the driving assistance device 32 uses the information on the distance and direction of the surrounding objects output by the surrounding object information output unit 86 to control the driving of the automobile 1 so that the automobile 1 avoids the surrounding objects present around the automobile 1. The occupant protection device 33 uses the information on the distance and direction of the surrounding object output by the surrounding object information output unit 86 to predict or detect that the traveling automobile 1 will collide with a surrounding object present around the automobile 1. When a collision with a surrounding object is predicted or detected, the occupant protection device 33 executes occupant protection control by deploying an airbag or applying tension to a seat belt to absorb the impact of the collision with the surrounding object. The vehicle exterior protection device 34 uses the distance and direction information of surrounding objects output by the surrounding object information output unit 86 to predict or detect a collision between the moving vehicle 1 and a pedestrian or cyclist present in the vicinity of the vehicle 1. When a collision with a pedestrian or cyclist is predicted or detected, the vehicle exterior protection device 34 executes vehicle exterior protection control to deploy an airbag around the pedestrian or cyclist. The deployed airbag prevents the pedestrian or cyclist from being directly hit by the body 2 of the colliding vehicle 1. The exterior-vehicle communication device 35 uses the information about the distance and direction of the surrounding object output by the surrounding object information output unit 86 to predict or detect that the traveling automobile 1 will collide with a surrounding object present around the automobile 1. When a collision with a surrounding object is predicted or detected, the exterior-vehicle communication device 35 transmits information about the collision to a server device of an emergency response organization. This allows members of the emergency response organization to rush to the collision site and begin rescue operations early.

[0097] FIG. 11 is a diagram for explaining the relative movement of the imaging target in the images captured by two adjacent imaging devices through the process of FIG. 11, like Fig. 4, shows images captured by two imaging devices, a left front imaging device 16 and a left rear imaging device 17, which are provided on the left side of an automobile 1 traveling forward, after misalignment has been corrected. Time flows from Fig. 11(A) to Fig. 11(C). 4, the left front imaging device 16 is provided at a reference position, whereas the left rear imaging device 17 is provided at a position shifted downward and to the right from the reference position.

[0098] FIG. 11(A) is an explanatory diagram of a situation in which the shift in the left rear captured image has been corrected and the pedestrian, which is the image capture target, is captured in the central region of the left front captured image. 11(A), the left front imaging device 16, out of the left front imaging device 16 and the left rear imaging device 17, captures an image of a pedestrian. The pedestrian is captured in the central region of the image captured by the left front imaging device 16.

[0099] FIG. 11B is an explanatory diagram of a situation in which a pedestrian, which is the subject of imaging, is being imaged in the rear area of ​​the left front captured image immediately after FIG. 11A. In FIG. 11B, which is a timing after FIG. 11A, the automobile 1 is traveling forward, so the pedestrian is captured in the left area of ​​the image captured by the left front imaging device 16.

[0100] FIG. 11C is an explanatory diagram of a situation in which a pedestrian, which is the image capture target, is captured in the front area of ​​the left rear captured image in which the deviation has been corrected immediately after FIG. 11B. 11(C), which is a timing after that of FIG. 11(B), the pedestrian is imaged by the left rear imaging device 17, not the left front imaging device 16. An image 101 of the pedestrian is captured at a position in the front region on the left side of the image captured by the left rear imaging device 17. In Figure 11(C), the position of the image of the pedestrian included in the image captured by the left rear imaging device 17 is the same as when there is no misalignment in the left rear imaging device 17, because the misalignment of the left rear imaging device 17 has been corrected. As a result, the distance and direction of the pedestrian obtained by monocular processing based on the image captured by the left rear imaging device 17 in FIG. 11(C) has high accuracy. 11(A) and 11(B), the pedestrian who is imaged as moving horizontally from right to left in the image captured by the left front imaging device 16 will also be imaged as moving horizontally from right to left at the same speed in the image captured by the left rear imaging device 17 in Fig. 11(C). The distance and direction of the pedestrian's image in the multiple captured images will be good, maintaining the same moving speed and direction as in those captured images, even when the pedestrian moves beyond those captured images.

[0101] FIG. 12 is an explanatory diagram of an example of a method for evaluating deviation correction. The CPU 24 evaluates the multiple deviation corrections generated for the multiple imaging devices 13 to 18, for example, in step ST5 of FIG. In this case, the CPU 24 may perform, for example, an evaluation based on the imaging target in the captured image after the deviation correction shown in Figures 12(A) and 12(B), a comparative evaluation of the two rear captured images after the deviation correction shown in Figure 12(C), a comparative evaluation of the multiple captured images after the deviation correction related to the multiple monocular processes shown in Figure 12(D), etc. When performing multiple types of comparative evaluations, the CPU 24 may determine the final evaluation as good if all the comparative evaluations are good, and may determine the final evaluation as not good in other cases.

[0102] FIG. 12A is an explanatory diagram of a schematic captured image 111 after the deviation of the rear center imaging device 18 has been corrected and an example of a method for evaluating the image. FIG. 12B is an explanatory diagram of a captured image 112 after displacement correction that may be judged to be unsatisfactory by the same evaluation as in FIG. 12A. The misalignment-corrected captured image 111 of FIG. 12(A) and the misalignment-corrected captured image 112 of FIG. 12(B) include a pair of straight lane boundary lines on the left and right of the lane in which the automobile 1 is traveling.

[0103] In the evaluation process of the captured images 111, 112, the CPU 24 performs image analysis processing such as peripheral object recognition processing to recognize and identify image components of a pair of left and right linear lane boundary lines contained in the captured images 111, 112 after the deviation correction. The CPU 24 evaluates the image components of the recognized pair of left and right lane boundary lines in the captured images 111 and 112 after deviation correction, such as the image capturing positions and left and right balance. The CPU 24 may evaluate a pair of image components contained in the misalignment-corrected captured images 111 and 112 by referring to information about the driving state of the automobile 1 at the time of capturing images, for example, information about the driving state, such as whether the automobile 1 is traveling straight. Then, as shown in FIG. 12(A), if the image components of a pair of left and right lane boundary lines are included symmetrically in the imaging range of the image 111 after the deviation correction, the CPU 24 may evaluate the deviation correction information as good. 12(B), when the image components of a pair of left and right lane boundary lines are included in the imaging range of the post-correction captured image 112 at positions shifted from symmetry by a predetermined amount or more, the CPU 24 may not evaluate the deviation correction information as good. If the deviation correction information that is not evaluated as good is for, for example, a clockwise direction, the CPU 24 may select the deviation correction information for a counterclockwise direction as the information to be used in FIG. Image components that may be included as a pair in the captured image include curbs on the left and right sides of the road and guardrails. Furthermore, a line of signposts, utility poles, traffic lights, and the like erected along the road can be recognized as a track extending along the road based on the image components. Furthermore, bridges and the like on the road on which the automobile 1 is traveling may also be included in the captured images 111, 112 as essentially symmetrical image components. The CPU 24 may evaluate the imaging positions, left and right balance, and the like in the captured images 111, 112 for these road image components included in the corrected captured images 111, 112 captured by the rear center imaging device 18.

[0104] FIG. 12(C) is an explanatory diagram of an example of an evaluation method based on a schematic captured image 113 of the rear central imaging device 18 after misalignment correction in a clockwise direction and a schematic captured image 114 of the rear central imaging device 18 after misalignment correction in a counterclockwise direction.

[0105] In the evaluation process of the captured image, the CPU 24 performs image analysis processing such as surrounding object recognition processing, and recognizes and identifies the image components of a pair of left and right straight lane boundary lines contained in the schematic captured image 113 after correcting the deviation in the clockwise direction. In addition, the CPU 24 performs image analysis processing as a surrounding object recognition processing, and recognizes and identifies image components of a pair of straight line lane boundary lines on the left and right contained in the schematic captured image 114 after the deviation correction in the left direction. The CPU 24 evaluates the amount of error in the imaging positions of the image components of the pair of left and right lane boundary lines recognized in the two captured images 113 and 114 after the deviation correction. As shown in FIG. 12(C), if the error in the imaging positions after the shift correction in the captured images 113, 114 for the image components of a pair of left and right lane boundary lines recognized in the two shift-corrected captured images 113, 114 is equal to or greater than a predetermined amount, the information on these shift corrections may not be evaluated as good.

[0106] FIG. 12D is an explanatory diagram of an example of an evaluation method based on variations in the normal directions of schematic captured images after displacement correction of the multiple imaging devices 13 to 18 used in monocular processing. In Fig. 12(D), seven images captured by multiple imaging devices 13-18 that constitute a 360-degree camera and are used for monocular processing are arranged horizontally after displacement correction. The image IRfc at the center of the arrangement is an image captured in front of the automobile 1 by the front central imaging device 13 after displacement correction. The image IRrc1 at the right end of the arrangement is an image captured in rear of the automobile 1 by the rear central imaging device 18 after displacement correction. The image IRrc2 at the left end of the arrangement is an image captured in rear of the automobile 1 by the rear central imaging device 18 after displacement correction.

[0107] The CPU 24 identifies vectors 116 to 122 in the normal direction of the road surface included in each of the seven images captured by the imaging devices 13 to 18 after the deviation correction. For example, if a pair of lane lines is included in the image after the deviation correction, the CPU 24 may identify a normal vector for a plane between the pair of lane lines, using the image capturing positions of the pair of lane lines as a reference. The CPU 24 evaluates the amount of variation in the directions of the normal vectors 116-122 of the road surfaces identified for the images captured by the imaging devices 13-18 after the deviation correction. If the tips of all of the normal vectors 116 to 122 of the road surface fall within the range of a predetermined error circle, for example, the CPU 24 may evaluate the deviation correction information as good. If any one of the normal vectors 116 to 122 of the plurality of road surfaces does not fall within the range of a predetermined error circle, the CPU 24 may not evaluate the deviation correction information as good. In addition, the CPU 24 may identify the normal vector 115 of the road surface recognized by the stereo camera 19 based on the captured image IRfr of the front right imaging device 11 and the captured image IRlr of the front left imaging device 12 that constitute the stereo camera 19. In this case, the CPU 24 may use the range of a predetermined error circle centered on the tip of the normal vector 115 in the stereo camera 19 as a reference and evaluate whether the variations in the directions of the normal vectors 116 to 122 of the multiple road surfaces fall within the range of the error circle.

[0108] In this way, the CPU 24 lastly generates misalignment correction information for the rear central imaging device 18 that is provided to capture images of the rear side of the automobile 1. The CPU 24 then evaluates the misalignment correction information based on one or more captured images including at least the captured image that is last to be processed. The CPU 24 can perform evaluation of a series of misalignment correction information based on the image components of the image captured by the rear central imaging device 18 for monocular processing that is last to be processed, even while the automobile 1 is traveling straight ahead.

[0109] As described above, in this embodiment, the automobile 1 is provided with a plurality of imaging devices 11-18 for capturing images of the outside of the vehicle, and at least three of the imaging devices are provided on the automobile 1 so as to be able to capture images of a common imaging area in an overlapping manner. This allows the at least three imaging devices to capture an image of the common imaging area in the captured image of the outside of the vehicle. The CPU 24, which serves as the control unit of the exterior image capture device 10 of the automobile 1, uses the captured images of at least two of the three imaging devices 11 and 12 to generate relative distance and direction information based on the parallax of the captured object outside the vehicle captured in the common imaging area. Based on the parallax that can be estimated based on the arrangement of the two imaging devices 11 and 12 on the automobile 1, the CPU 24 can generate more accurate relative distance or direction information about the captured object outside the vehicle captured in the common imaging area without being affected by changes in the imaging range or angle of view due to changes in the arrangement of the other imaging devices 13-18. As a result, the CPU 24 can use the parallax-based relative distance and direction information to generate accurate displacement correction information for the relative distance or direction obtained by monocular processing of at least one of the remaining imaging devices 13, 14, and 15. Furthermore, by using the displacement correction information in the monocular processing of the imaging devices 13, 14, and 15, the CPU 24 can obtain more accurate relative distance or direction.

[0110] Furthermore, the CPU 24 generates, in order, misalignment correction information for the imaging devices 16, 17, and 18 to be used for other monocular processing, using the misalignment-corrected captured images of the imaging devices 14, 15, 16, and 17. By sequentially using the misalignment-corrected captured images of the imaging devices 14, 15, 16, and 17 as references, the CPU 24 can generate misalignment correction information to be used for monocular processing for all of the imaging devices 13 to 18 that capture images in 360 degrees, and can obtain relative distances or directions of imaging targets, such as surrounding objects, captured by these devices with the same degree of certainty as that of the stereo camera 19.

[0111] In this way, in this embodiment, by suitably utilizing the fact that multiple imaging devices 11-18 are provided for imaging the outside of the vehicle, at least three imaging devices are caused to capture images of overlapping common imaging areas, and the relative distance or direction of an object imaged outside the vehicle in monocular processing can be accurately corrected. As a result, in this embodiment, even though the images captured by the imaging devices 13-18 provided on the vehicle 1 are processed monocularly, it is possible to obtain a relative distance or direction of an object imaged outside the vehicle that can be expected to have the same high degree of accuracy as with compound eye processing. In this embodiment, it is possible to ensure or maintain the reliability of the multiple imaging devices 11 to 18 provided in the automobile 1 for capturing images of the outside of the automobile 1.

[0112] The above-described embodiment is an example of a preferred embodiment of the present invention, but the present invention is not limited to this, and various modifications and changes are possible within the scope of the gist of the invention.

[0113] In the above-described embodiment, the front right imaging device 11 and the front left imaging device 12 that constitute the stereo camera 19 capable of compound eye processing are used as masters to generate information for correcting deviations in the imaging ranges of the multiple imaging devices 13 to 18 used for single eye processing. The imaging device used as a reference for misalignment correction may not be provided in the automobile 1 so as to constitute the stereo camera 19. If at least three or more imaging devices are installed in automobile 1 so that they can capture images of overlapping common imaging areas, it is possible to generate misalignment correction information to be used in the monocular processing of the remaining imaging devices based on the parallax of the common object outside the vehicle in the images captured by two of the imaging devices. Then, in the monocular processing of the remaining imaging devices, it is possible to generate more accurate relative distance and direction information using the misalignment correction information.

[0114] In the above-described embodiment, the automobile 1 is provided with a plurality of monocular imaging devices 13-18 that capture images of the entire periphery of the automobile 1 by dividing 360 degrees into a plurality of parts. Then, deviation correction information is generated for the plurality of imaging devices 13-18. Alternatively, for example, an imaging device capable of capturing 360-degree images may be provided in the automobile 1 in order to capture images of the entire circumference of the automobile 1. For an imaging device capable of capturing 360-degree images, it is also possible to generate deviation correction information by the same process as described above.

[0115] In the above-described embodiment, the distance and direction based on the parallax between the two front right imaging devices 11 and 12 of the stereo camera 19 are used as a reference, and information for correcting the deviation in distance and direction is generated for other imaging devices for monocular processing whose imaging ranges overlap with the reference, and for further other imaging devices. In this way, if there is an overlap in the imaging range between the image captured by the reference imaging device and the image captured by the imaging device to be corrected, the CPU 24 can use the imaging target in the overlapping imaging range to generate information for correcting the deviation in distance or direction so that the distance or direction in the image captured by the imaging device to be corrected matches the distance or direction in the image captured by the reference imaging device. Furthermore, the CPU 24 can also generate information for correcting the deviation in distance or direction for other imaging devices whose imaging ranges overlap with the imaging device whose deviation has been corrected in this way, using the imaging device whose deviation has been corrected as a reference so that the distance or direction matches the distance or direction in the image captured by the reference imaging device.

[0116] In the above-described embodiment, the distance and direction based on the parallax between the two front right imaging devices 11 and 12 of the stereo camera 19 are used as the reference. Additionally, for example, automobile 1 may be equipped with a lidar that inputs and outputs millimeter waves, a radar that inputs and outputs radio waves such as infrared, or the like, in order to detect external objects around automobile 1. By scanning and detecting the surroundings of automobile 1 with the lidar or radar, control system 9 can obtain a three-dimensional distribution map of surrounding objects around automobile 1. CPU 24 may use this three-dimensional distribution map as a reference to generate distance or direction deviation correction information for a first imaging device whose imaging range overlaps with the distribution map. Furthermore, CPU 24 may use this first imaging device as a reference to generate distance or direction deviation correction information for a next imaging device whose imaging range overlaps with the first imaging device, using an imaging target in the overlapping imaging range to align the distance or direction in the image captured by the next imaging device with the distance or direction in the image captured by the first imaging device. Alternatively, for example, the automobile 1 may use a GNSS receiver or highly accurate three-dimensional map data to detect the current position and orientation of the automobile 1. In this case, the control system 9 can obtain with high accuracy the relative distance and direction between the vehicle and surrounding objects outside the vehicle included in the three-dimensional map data. The CPU 24 may generate distance or direction error correction information for the first imaging device based on the imaged positions of the surrounding objects in the three-dimensional map data captured by the first imaging device. Furthermore, the CPU 24 may use the first imaging device as a reference and, for a second imaging device whose imaging range overlaps with the first imaging device, generate distance or direction error correction information using the imaging target in the overlapping imaging range so that the distance or direction in the image captured by the second imaging device matches the distance or direction in the image captured by the first imaging device. In this way, even if there are not two imaging devices 11 and 12 that constitute the stereo camera 19, the CPU 24 as a control unit can generate correction information about the distance or direction of an object to be imaged outside the vehicle based on the imaging position, based on the distance and direction in the image captured by one of the at least two imaging devices that can capture the common imaging area in an overlapping manner, to be used in monocular processing of the image captured by the remaining imaging device, provided that at least two of the multiple imaging devices are installed in the automobile 1 so that they can capture an overlapping common imaging area. However, because there are two imaging devices 11 and 12 that make up the stereo camera 19, the CPU 24 as a control unit can obtain distance and direction as a highly accurate reference based on the parallax of the stereo camera 19, and can use the distance and direction information based on the parallax to generate, with high accuracy, correction information about the distance or direction of an imaged object outside the vehicle based on the imaging position, which is used in monocular processing of the image captured by at least one remaining imaging device. Moreover, in this case, there is no need to specify or restrict the placement of components other than the imaging devices, such as a lidar, radar, or GNSS receiver, on the automobile 1 so as to obtain high positional accuracy relative to the imaging devices. [Explanation of symbols]

[0117] 1...Automobile (vehicle), 2...Vehicle body, 3...Vehicle interior, 9...Control system, 10...Vehicle exterior imaging device, 11...Imaging device, 11...Front right imaging device, 12...Front left imaging device, 13...Front center imaging device, 14...Front right imaging device, 15...Rear right imaging device, 16...Front left imaging device, 17...Rear left imaging device, 18...Rear center imaging device, 19...Stereo camera, 21...In-vehicle input / output unit, 22...Input / output port, 23...Memory, 24...CPU, 25...Timer, 26...System bus, 31...Vehicle network, 32...Driving assistance device, 33...Occupant protection device, 34...Vehicle exterior protection device positioning, 35...exterior vehicle communication device, 36...sensor device, 37...acceleration sensor, 41...imaging device, 51, 61, 62, 64, 71, 111-114...captured image, 52...rectangular frame, 55...road surface model, 57, 63, 65...pixels, 71...left front captured image, 72...left rear captured image, 81...imaging device, 82...peripheral object recognition processing unit, 83...peripheral object position information generation unit, 84...quantitative deviation correction unit, 85...real-time deviation correction unit, 86...peripheral object information output unit, 87...vehicle control execution unit, 91...road surface model data, 92...deviation correction information, 93...attitude information, 115-122...normal vector

Claims

1. a plurality of imaging devices provided on the vehicle for capturing images of the exterior of the vehicle; a control unit that can generate distance or direction information about an object outside the vehicle using images captured by the plurality of imaging devices; and a plurality of imaging devices provided in the vehicle for capturing images of the outside of the vehicle; a stereo camera using the two imaging devices whose mutual arrangement is defined; a plurality of imaging devices for monocular processing capable of capturing an image of an imaging area common to the two imaging devices of the stereo camera in an overlapping manner; the plurality of imaging devices for monocular processing are provided so as to be able to capture an imaging area common to the imaging devices for other monocular processing in an overlapping manner, and at least one imaging device for monocular processing is provided on the vehicle so as to be able to capture an imaging area common to the stereo camera in an overlapping manner; The control unit In the stereo camera using the two imaging devices whose mutual arrangements are defined, distance and direction information is generated based on a parallax of an image of an object outside the vehicle imaged in a common imaging area, generating correction information for a distance or a direction of an image capture target outside the vehicle based on an image capture position, the correction information being used in monocular processing of an image captured by the at least one imaging device for monocular processing, using distance and direction information based on the parallax; When generating correction information for the imaging devices for a plurality of monocular processing, the stereo camera formed by the two imaging devices whose mutual arrangements are specified is used as a reference, and correction information is generated in order from the imaging device for monocular processing that can capture an imaging area common to the stereo camera in an overlapping manner, according to the order of overlapping of the imaging areas. An exterior imaging device for a vehicle.

2. The control unit generating distance and direction information based on a parallax of an object outside the vehicle captured in an imaging area common to the imaging device for monocular processing using images captured by the two imaging devices of the stereo camera; generating correction information for the imaging device for monocular processing using distance and direction information of the stereo camera; The vehicle exterior imaging device according to claim 1.

3. The imaging devices for a plurality of monocular processing are provided on the vehicle so as to capture images of the surroundings of the vehicle in a divided manner; each of the imaging devices for monocular processing is provided on the vehicle so as to be capable of capturing an image of an imaging area common to other imaging devices for monocular processing adjacent to the imaging device in a division direction around the vehicle in an overlapping manner; 3. The vehicle exterior image capturing device according to claim 1.

4. The imaging devices for a plurality of monocular processing are provided on the vehicle so as to capture an image of the entire periphery of the vehicle or the entire periphery excluding the imaging range of the stereo camera in a divided manner, The control unit generating correction information in order from the imaging device for monocular processing on one end side of the imaging range of the stereo camera, which is capable of capturing an overlapping image of a common imaging area at one end side of the imaging range of the stereo camera, and the imaging device for monocular processing on the other end side of the imaging range of the stereo camera, which is capable of capturing an overlapping image of a common imaging area at the other end side of the imaging range of the stereo camera; For the imaging device for monocular processing that is processed last, correction information based on processing in the order of one end side and correction information based on processing in the order of the other end side are generated, and the two generated correction information are compared to evaluate the series of corrections. The vehicle exterior image capturing device according to any one of claims 1 to 3.

5. The imaging device for monocular processing that is last processed is configured to capture an image of the rear side of the vehicle, The control unit and evaluating a series of corrections based on road information included in the corrected captured image of the monocular imaging device to be processed last.

5. The vehicle exterior image capturing device according to claim 4.

6. The control unit If the series of corrections cannot be evaluated as satisfactory, an alarm or maintenance request is output.

6. The vehicle exterior image capturing device according to claim 4 or 5.

7. Correction information about the distance or direction of an image capture target outside the vehicle based on the image capture position, used in monocular processing of the image captured by the image capture device, Distance or direction correction information for distance or direction information for each imaging position in a captured image set for monocular processing for the imaging device, The outside image capturing device for a vehicle according to any one of claims 1 to 6.

8. The control unit, in monocular processing of a distance or direction of an object outside the vehicle imaged by the imaging device, Correction is performed using the correction information; obtaining a corrected distance or direction based on the attitude of the vehicle; The outside image capturing device for a vehicle according to any one of claims 1 to 7.

Images