In-vehicle camera system and method for determining exposure conditions for an in-vehicle camera

The in-vehicle camera system with multiple cameras and time-division exposure control addresses the issue of blurred images at high speeds, ensuring accurate three-dimensional and target recognition for vehicle controls.

JP7739205B2Active Publication Date: 2025-09-16ASTEMO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2022034138
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2022-03-07
Publication Date
2025-09-16
Estimated Expiration
2042-03-07

AI Technical Summary

Technical Problem

Existing exposure time control methods for onboard cameras in vehicles result in blurred images at high speeds, impairing accurate target recognition and vehicle control functions such as ACC, AEBS, and LKAS.

Method used

An in-vehicle camera system with multiple cameras capturing images in a stereoscopic viewing area, switching between exposure conditions for distance measurement and target recognition in a time-division manner, using first and second exposure conditions based on vehicle speed and external light, to obtain accurate three-dimensional and target information.

Benefits of technology

Enables accurate three-dimensional information and target recognition for vehicle controls like ACC, AEBS, and LKAS, independent of vehicle speed, by optimizing exposure times for distance measurement and target recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007739205000001
    Figure 0007739205000001
  • Figure 0007739205000002
    Figure 0007739205000002
  • Figure 0007739205000003
    Figure 0007739205000003
Patent Text Reader

Abstract

To provide an on-vehicle camera system capable of acquiring accurate three-dimensional information and target recognition information without any influence of a vehicle speed.SOLUTION: An on-board camera system comprises: a plurality of cameras which is arranged on an own vehicle to have a stereoscopic region where visual field regions overlap at least in part; a movement quantity computation part which finds a movement quantity of a feature point in the stereoscopic region that the plurality of cameras images on the basis of behavior of the own vehicle; a first exposure condition determination part which determines first exposure conditions of the plurality of cameras so that the movement quantity is equal to or less than a threshold; a second exposure condition determination part which determines second exposure conditions of the plurality of cameras on the basis of, external light conditions outside the vehicle; a three-dimensional information acquisition part which uses images captured under the first exposure conditions to acquire three-dimensional information on the stereoscopic region; a target recognition part which uses images captured under the second exposure conditions to recognize a target in a circumference of the own vehicle; and an exposure control part which switches respective exposure conditions of the plurality of cameras between the first exposure conditions and second exposure conditions.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present invention relates to an in-vehicle camera system that uses a plurality of cameras in combination to recognize the outside world of the vehicle, and a method for determining exposure conditions for the in-vehicle camera. [Background technology]

[0002] Known vehicle control technologies such as adaptive cruise control (ACC), advanced emergency braking system (AEBS), and lane keeping assist system (LKAS) are technical elements of driver assistance systems and autonomous driving systems. Specific configurations that realize these vehicle control technologies include constantly recognizing and tracking objects around the vehicle (e.g., other vehicles, pedestrians, cyclists, traffic signals, traffic signs, white lines, and obstacles) based on images captured by an onboard camera, allowing the vehicle to follow the vehicle ahead, activate emergency braking, and control steering to prevent the vehicle from leaving its lane. Furthermore, vehicles that enable automatic parking by arranging multiple cameras to monitor not only the front, rear, and sides of the vehicle, are becoming more common.

[0003] Here, Patent Document 1 is known as a conventional technique for controlling the exposure time of an on-board camera. The abstract of this document states that the problem is to "capture and display images of the surroundings of a vehicle, taking into consideration the vehicle's driving state," and as a solution, states that "the on-board camera control device includes a vehicle speed acquisition means for acquiring the speed of the vehicle on which the on-board camera is mounted, and a camera control means for changing the exposure time of the on-board camera in accordance with the vehicle speed acquired by the vehicle speed acquisition means." Paragraph 0036 of this document also states that "When the vehicle is moving at high speed, it is possible to display images of the surroundings of the vehicle in real time, although the images may not be clear." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-174078 Summary of the Invention [Problem to be solved by the invention]

[0005] As described above, the exposure time control of Patent Document 1 uniformly changes the exposure time of the onboard camera according to the vehicle speed in order to display images of the surroundings of the vehicle in real time, but this exposure time control has the problem that it is not possible to obtain clear images when the vehicle is moving at high speed. Therefore, if the blurred images captured by the technology of Patent Document 1 are used, it may be impossible to accurately recognize targets around the vehicle, and vehicle control such as ACC, AEBS, and LKAS may not be able to be properly implemented.

[0006] Therefore, the present invention aims to provide an in-vehicle camera system and a method for determining exposure conditions for an in-vehicle camera that can acquire accurate three-dimensional information and target recognition information required for vehicle controls such as ACC, AEBS, and LKAS without being affected by vehicle speed by capturing images while switching between exposure control for distance measurement and exposure control for target recognition in a time-division manner. [Means for solving the problem]

[0007] In order to solve the above problem, the in-vehicle camera system of the present invention is an in-vehicle camera system comprising: a plurality of cameras arranged on the host vehicle so as to have a stereoscopic viewing area in which at least a portion of the field of view overlaps; a movement amount calculation unit that calculates the movement amount of feature points in the stereoscopic viewing area captured by the plurality of cameras based on the behavior of the host vehicle; a first exposure condition determination unit that determines first exposure conditions for the plurality of cameras so that the movement amount is equal to or less than a threshold; a second exposure condition determination unit that determines second exposure conditions for the plurality of cameras based on the external light conditions outside the vehicle; a three-dimensional information acquisition unit that acquires three-dimensional information of the stereoscopic viewing area using images captured under the first exposure conditions; a target recognition unit that recognizes targets around the host vehicle using images captured under the second exposure conditions; and an exposure control unit that switches the exposure conditions of each of the plurality of cameras between the first exposure condition and the second exposure condition. [Effects of the Invention]

[0008] According to the in-vehicle camera system and the method for determining exposure conditions for an in-vehicle camera of the present invention, by capturing images by switching between exposure control for distance measurement and exposure control for target recognition in a time-division manner, accurate three-dimensional information and target recognition information required for vehicle controls such as ACC, AEBS, and LKAS can be obtained without being affected by vehicle speed. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a functional block diagram of an in-vehicle camera system according to an embodiment. [Figure 2] FIG. 2 is a top view showing the relationship between the viewing area of ​​each camera and the stereo viewing area. [Figure 3] FIG. 2 is a functional block diagram of the camera control unit in FIG. 1. [Figure 4] 3 is a processing flowchart of the vehicle-mounted camera system according to the embodiment. [Figure 5] Specific examples of calculation lines for 3D information and calculation areas for the amount of movement of feature points. DETAILED DESCRIPTION OF THE INVENTION

[0010] An in-vehicle camera system 100 according to an embodiment of the present invention will be described below with reference to the drawings.

[0011] 1 is a functional block diagram of an in-vehicle camera system 100. This in-vehicle camera system 100 is a system mounted on a host vehicle 1, and includes a processing device 10, a camera 20 (21 to 26), a vehicle control device 30, and a floodlight 40. Below, an overview of the camera 20, the vehicle control device 30, and the floodlight 40 will be given, followed by a detailed description of the processing device 10.

[0012] <Camera 20> The camera 20 is a sensor that captures images of the surroundings of the vehicle and outputs image data P. In this embodiment, the vehicle 1 is equipped with a plurality of cameras 20 (21-26) so that they can capture images of the entire periphery of the vehicle. Each camera is installed so that at least a portion of its imaging range overlaps with the imaging range of the other cameras.

[0013] 2 is a top view of the vehicle 1, illustrating the relationship between the visual field area C of each camera and the stereo vision area V. The vehicle 1 of this embodiment has a front visual field area C indicated by a solid line. 21 and a front camera 21 for capturing an image of the right front view area C shown by a dashed line. 22 and a right rear view area C shown by a dashed line. 23 and a rear view area C shown by a solid line. 24 and a rear camera 24 for capturing an image of the left rear view area C shown by a dashed line. 25 and a left rear camera 25 for capturing an image of the left front field of view C shown by a dashed line. 26 A left front camera 26 is installed to capture images of the left and right sides of the vehicle, and these six cameras 20 can capture images of the entire periphery outside the vehicle.

[0014] In the area where multiple viewing areas C overlap, the same object can be captured from multiple viewing directions (stereo imaging), and by using well-known stereo matching technology, 3D information of the captured object (moving objects, stationary objects, road surface, etc. around the vehicle) can be generated. Therefore, the area where viewing areas C overlap is called the stereo viewing area V.

[0015] Note that while FIG. 2 illustrates front, rear, left, and right stereoscopic viewing areas V1 to V4, the number and orientation of the stereoscopic viewing areas V are not limited to this example. Also, FIG. 2 illustrates a simplified view field C of each camera, and the actual imaging distance of each camera does not necessarily have to be the size relationship shown. To give an example, the actual imaging distance of each camera is approximately 200 m for the front camera 21 and rear camera 24, and approximately 50 m for the other cameras. In this case, the imaging distance of the stereoscopic viewing area V is approximately 50 m.

[0016] <Vehicle control device 30> The vehicle control device 30 is a control device that controls the steering system, drive system, and braking system (not shown) based on the results of target ranging described below, thereby executing vehicle control such as ACC, AEBS, LKAS, automatic parking, etc. The vehicle control device 30 also transmits vehicle speed information of the host vehicle 1 to the camera control unit 12 described below, and requests the camera control unit 12 to transmit desired information (3D map, host vehicle attitude, target recognition, target ranging) with priority.

[0017] <Floodlight 40> The floodlight 40 is a device that projects a desired amount of visible light or near-infrared light within the stereo vision area V so that each camera can capture clearer image data P. Note that if the outside world of the vehicle 1 is sufficiently bright, the floodlight 40 does not need to be used. Furthermore, even when the floodlight 40 is used, it does not need to be constantly projecting light, and it is sufficient to project light in synchronization with the image capturing timing of each camera.

[0018] <Processing device 10> The arithmetic processing device 10 is a device for acquiring three-dimensional information about the surroundings of the vehicle based on the output (image data P) of the camera 20, estimating the vehicle's attitude, recognizing targets around the vehicle (e.g., other vehicles, pedestrians, cyclists, traffic signals, traffic signs, white lines, obstacles, etc.), and measuring the distance to the targets. Specifically, the arithmetic processing device 10 is a computer equipped with hardware such as an arithmetic device such as a CPU, a storage device such as a semiconductor memory, and a communication device. The arithmetic device executes a predetermined program to realize various functional units such as a camera control unit 12, which will be described later. However, the following description will omit such well-known techniques as appropriate.

[0019] 1, the arithmetic processing device 10 of this embodiment includes a sensor interface 11, a camera control unit 12, a projector control unit 13, an image distribution unit 14, a feature point movement amount calculation unit 15, a three-dimensional information acquisition unit 16, a three-dimensional map storage unit 17, a vehicle attitude estimation unit 18, a target recognition unit 19, and a target distance measurement unit 1a. The functions of each unit will be explained in order below.

[0020] <<Sensor Interface 11>> The sensor interface 11 is a functional unit that transmits commands from the camera control unit 12 to the cameras 20 (21 to 26), receives image data P from the cameras 20 (21 to 26), and transmits the image data P to the image distribution unit 14. This allows each camera to capture the image data P under the exposure conditions set by the camera control unit 12. Details of the exposure conditions will be described later.

[0021] <<Camera control unit 12>> The camera control unit 12 is a functional unit that controls the cameras 20 (21 to 26) via the sensor interface 11 and also controls the projector 40 via the projector control unit 13. As shown in Fig. 3, the camera control unit 12 has a reference exposure condition storage unit 12a, a first exposure condition determination unit 12b, an external light condition judgment unit 12c, a second exposure condition determination unit 12d, an exposure control unit 12e, an imaging control unit 12f, and a projecting condition determination unit 12g. Each unit will be described in detail below.

[0022] The reference exposure condition storage unit 12a is a functional unit that stores the reference exposure conditions (reference exposure time) to be set for each camera when the arithmetic processing device 10 is started up or when the vehicle speed or the environment outside the vehicle changes significantly.

[0023] The first exposure condition determination unit 12b is a functional unit that determines the first exposure condition based on the feature point movement amount received from the feature point movement amount calculation unit 15 described below or vehicle speed information received from the vehicle control device 30. This first exposure condition specifies the exposure time when capturing image data P1 for acquiring three-dimensional information, and an exposure time is set such that the feature point movement amount is equal to or less than a predetermined threshold (for example, 2 pixels) or an exposure time that is inversely proportional to the vehicle speed.

[0024] The external light condition determination unit 12c is a functional unit that determines the external light conditions outside the vehicle based on the luminance information of the image data P received from the image distribution unit 14, which will be described later. When determining the external light conditions, for example, the gain set for each camera and the average luminance in the image data P are taken into consideration. Therefore, even if the average luminance of the image data P is the same, if the gain is high, the external world is determined to be dark, and if the gain is low, the external world is determined to be bright.

[0025] The second exposure condition determination unit 12d is a functional unit that determines the second exposure condition based on the external light condition determined by the external light condition determination unit 12c. The second exposure condition specifies the exposure time when capturing image data P2 for recognizing a target, and the exposure time is set to be approximately inversely proportional to the amount of external light. The relationship between the amount of external light and the second exposure condition may be determined using a predetermined calculation formula or by referring to a table prepared in advance.

[0026] Note that, when comparing the exposure time under the first exposure condition with the exposure time under the second exposure condition, the former is basically shorter and the latter is basically longer. Therefore, image data P1 captured under the first exposure condition has the disadvantage that the image is dark and therefore not suitable for target recognition, but has the advantage that the sampling period is short and therefore it can quickly detect environmental changes around the vehicle. On the other hand, image data P2 captured under the second exposure condition has the disadvantage that the sampling period is long and therefore not suitable for quickly detecting environmental changes around the vehicle, but has the advantage that the image is bright and therefore it can accurately recognize targets.

[0027] The exposure control unit 12e is a functional unit that selects one of the reference exposure conditions, the first exposure conditions, and the second exposure conditions depending on the processing procedure, the vehicle speed, the type, distance, and collision probability of the recognized target, and transmits the selected exposure conditions to the imaging control unit 12f. In this embodiment, the exposure conditions of each camera are set in a time-division manner while taking into account the priority use of each camera at that time. Therefore, each camera outputs image data P1 for acquiring 3D information and image data P2 for target recognition at a predetermined ratio. For example, if the priority use of the camera 20 is to acquire 3D information, such as when the vehicle 1 is traveling at high speed or when a target with a potential collision is detected, the proportion of use of the first exposure condition can be increased to capture a large number of image data P1 and a small number of image data P2. If the priority use of the camera 20 is to recognize a target, the proportion of use of the second exposure condition can be increased to capture a small number of image data P1 and a large number of image data P2.

[0028] Furthermore, when the exposure conditions are set taking into consideration the vehicle speed, the exposure conditions are set as follows. For example, when the host vehicle 1 is moving at a certain speed (e.g., 10 km / h) or above, the first exposure conditions for acquiring 3D information are set preferentially for each camera. Specifically, the proportion (period or number of times) for which the first exposure conditions are set for each camera is increased, and the proportion (period or number of times) for which the second exposure conditions are set is reduced. Conversely, when the host vehicle 1 is moving at a speed less than the certain speed, the second exposure conditions for target recognition are set preferentially for each camera.

[0029] Here, the same exposure conditions must be set for the cameras corresponding to the same stereoscopic viewing area V, but different exposure conditions may be set during synchronization between cameras corresponding to different stereoscopic viewing areas V. For example, when the host vehicle 1 is moving forward, a first exposure condition for acquiring three-dimensional information may be preferentially set for the cameras corresponding to the front stereoscopic viewing area V1 (front camera 21, right front camera 22, left front camera 26), and a second exposure condition for target recognition may be preferentially set for the cameras corresponding to the rear stereoscopic viewing area V3 (right rear camera 23, rear camera 24, left rear camera 25).

[0030] The imaging control unit 12f is a functional unit that controls the imaging timing of each camera using the exposure conditions set by the exposure control unit 12e. Here, cameras corresponding to the same stereoscopic viewing area V must capture images synchronously, but the imaging timing may be different between cameras corresponding to different stereoscopic viewing areas V. Therefore, for example, if the same imaging period (e.g., 50 ms) is set for the camera group (21, 22, 26) corresponding to the front stereoscopic viewing area V1 and the camera group (23, 24, 25) corresponding to the rear stereoscopic viewing area V3, a time difference of, for example, half a period (e.g., 25 ms) may be set between the imaging timings of the former and latter cameras. This effectively halves the sampling period for capturing images outside the vehicle.

[0031] The light-projection condition determination unit 12g is a functional unit that determines the timing and amount of light projection by the projector 40 based on the output of the imaging control unit 12f, and transmits the results to the projector control unit 13. As described above, it is sufficient for the projector 40 to project light while each camera is capturing an image, and therefore, if the projector 40 is controlled not to project light while each camera is not capturing an image, the power consumption of the projector 40 can be reduced.

[0032] <<Projector control unit 13>> The light projector control unit 13 is a functional unit that controls the light projection of the light projector 40 in accordance with the light projection timing and light projection amount determined by the light projection condition determination unit 12g.

[0033] <<Image distribution unit 14>> The image distribution unit 14 is a functional unit that distributes the image data P received via the sensor interface 11 to the camera control unit 12, the feature point movement amount calculation unit 15, the three-dimensional information acquisition unit 16, or the target recognition unit 19, depending on the control procedure, the traveling status of the vehicle 1, the request content of the vehicle control device 30, etc. Note that information indicating the exposure conditions (or exposure time) is added to the image data P so that the image distribution unit 14 can distinguish the type of image data P.

[0034] <<Feature point movement amount calculation unit 15>> The feature point movement amount calculation unit 15 is a functional unit that calculates the amount of movement within the image data P for any feature point within the image data P received from the image distribution unit 14.

[0035] FIG. 4(a) shows an example of the movement of a feature point within image data P. As shown in the figure, if the far end of a white line drawn on the road surface is taken as the feature point, if the host vehicle 1 is stopped, the boundary between the far end of the white line and the road surface behind it is clear, and therefore the amount of movement of the feature point can be determined to be small. However, in the image data P of FIG. 4 captured from the host vehicle 1 traveling at high speed, the boundary between the far end of the white line and the road surface becomes unclear, and therefore the amount of movement of the feature point can be determined to be large. Therefore, the feature point movement amount calculation unit 15 calculates the amount of movement of the feature point based on the clarity of the image near the feature point, etc. Note that the amount of movement of the feature point is proportional to the vehicle speed, and therefore the calculation of the amount of movement of the feature point can also be regarded as an estimate of the vehicle speed.

[0036] <<3D information acquisition unit 16>> The three-dimensional information acquisition unit 16 is a functional unit that acquires three-dimensional information for each pixel on an arbitrary calculation line on a set of image data P obtained by synchronously capturing a stereo viewing area V using well-known stereo matching technology.

[0037] FIG. 4(b) shows an example of a calculation line for acquiring three-dimensional information, shown in image data P. In the image data P shown in FIG. 4, the sky is captured in approximately the upper half and the road surface is captured in approximately the lower half. Therefore, if a calculation line is set in the vertical direction of the image data P and stereo matching is performed on that calculation line, the closest distance information is calculated for the road surface pixels at the bottom of the image data P, increasingly distant distance information is calculated for the road surface pixels from the bottom to approximately the center of the image data P, and distance information indicating invalidity (distance measurement impossible) is calculated for the sky pixels from approximately the center to the top. Note that while FIG. 4 shows only one calculation line, if there is sufficient computing capacity, multiple calculation lines at any angle may be set on the image data P and three-dimensional information may be calculated for each of them.

[0038] <<3D map storage unit 17>> The 3D map storage unit 17 is a functional unit that generates and stores a 3D map that shows the road surface gradient, targets, etc. around the vehicle by chronologically accumulating the 3D information acquired by the 3D information acquisition unit 16. If past 3D information of a target that is thought to be a moving object is stored in the 3D map storage unit 17, it is desirable to prioritize the first exposure condition for each camera and update the current 3D information of the moving object, thereby enabling tracking of the moving object.

[0039] <<Vehicle attitude estimation unit 18>> The vehicle attitude estimation unit 18 is a functional unit that estimates the attitude of the vehicle 1 with respect to the road surface based on the three-dimensional map stored in the three-dimensional map storage unit 17.

[0040] <<Target recognition unit 19>> The target recognition unit 19 is a functional unit that uses a well-known pattern matching technique to recognize a target captured in the image data P2.

[0041] 4(c) shows an example of targets recognized by the target recognition unit 19 from the image data P. In this example, a preceding vehicle traveling in the host vehicle's driving lane and a preceding vehicle traveling in the lane to the right of the host vehicle's driving lane are recognized. In this case, the former is recognized as a target (other vehicle) by pattern matching with an image pattern captured from behind the vehicle, and the latter is recognized as a target (other vehicle) by pattern matching with an image pattern captured from behind the vehicle.

[0042] <<Target distance measuring unit 1a>> The target distance measuring unit 1a is a functional unit that estimates the distance to a target recognized by the target recognition unit 19 based on the overall width, overall height, etc. of the target in the image data P. If the target is within the stereo vision area V, the distance to the target may be calculated using stereo matching technology, or if the distance to the target is registered in advance as a 3D map, the distance indicated on the 3D map may be used as the distance to the target. The information on the distance to the target calculated here is transmitted to the vehicle control device 30, so that the vehicle control device 30 can execute various vehicle controls such as ACC, AEBS, and automatic parking depending on the distance to the target.

[0043] <Flowchart> Next, the processing of each part of the above-described vehicle-mounted camera system 100 will be explained in order using the flowchart in Fig. 5. For the sake of simplicity, it is assumed here that the brightness outside the vehicle is constant and that the vehicle speed is also constant.

[0044] First, in step S1, the camera control unit 12 (12a, 12e, 12f) sets the reference exposure conditions for each camera 20, and causes the camera 20 to capture image data P.

[0045] In step S2, the feature point movement amount calculation unit 15 calculates the feature point movement amount based on the image data P captured in step S1.

[0046] In step S3, the camera control unit 12 (12b) determines whether the calculated amount of movement of the feature point is equal to or less than a predetermined threshold value. If the amount of movement of the feature point is equal to or less than the predetermined threshold value, the process proceeds to step S5; if not, the process proceeds to step S4.

[0047] In step S4, the camera control unit 12 (12b, 12e, 12f) sets a shorter exposure time in the camera 20 and causes it to capture image data P. The processes of steps S2 and S3 described above are also performed on the newly captured image data P, so that an exposure time is ultimately determined that makes the amount of movement of the feature points equal to or less than a predetermined threshold.

[0048] In step S5, the camera control unit 12 (12b) determines the exposure time in which the amount of movement of the feature point is equal to or less than a predetermined threshold as the first exposure condition.

[0049] In step S6, the camera control unit 12 (12c) acquires the external light conditions outside the vehicle based on the image data P.

[0050] In step S7, the camera control unit 12 (12d) determines the second exposure condition based on the external light condition.

[0051] In step S8, the camera control unit 12 (12e) sets the purpose of the next imaging in consideration of the exposure conditions preferentially set for each camera, etc. If the purpose of the next imaging is to obtain distance information (three-dimensional information, vehicle attitude information), the process proceeds to step S9, and if the purpose of the next imaging is to obtain target information (target recognition information, target ranging information), the process proceeds to step S12.

[0052] In step S9, the camera control unit 12 (12b, 12e, 12f, 12g) sets the first exposure condition in the camera 20 and causes the camera 20 to capture image data P1.

[0053] In step S10, the three-dimensional information acquisition unit 16 acquires three-dimensional information based on a plurality of image data P1 obtained by synchronously capturing images of the same stereoscopic viewing area V. The acquired three-dimensional information is stored in the three-dimensional map storage unit 17 as a three-dimensional map.

[0054] In step S11, the host vehicle attitude estimation unit 18 estimates the attitude of the host vehicle 1. The estimated host vehicle attitude is transmitted to the vehicle control device 30 and used for vehicle control.

[0055] On the other hand, in step S12, the camera control unit 12 (12d, 12e, 12f, 12g) sets the second exposure condition in the camera 20 and causes the camera 20 to capture image data P2.

[0056] In step S13, the target recognition unit 19 recognizes the target in the image data P2.

[0057] In step S14, the target distance measuring unit 1a measures the distance to the recognized target. The measured distance to the target is transmitted to the vehicle control device 30 and used for desired vehicle control.

[0058] In step S15, the arithmetic processing device 10 determines whether driving has ended. If driving has not ended, the process returns to step S8 and continues capturing images. This allows each camera to capture image data P1 and P2 at a predetermined ratio. On the other hand, if driving has ended, the process of FIG. 5 ends. Note that if the brightness outside the vehicle or the vehicle speed has changed significantly, the process can be repeated from step S1.

[0059] According to the in-vehicle camera system of this embodiment described above, by capturing images by switching between exposure control for distance measurement and exposure control for target recognition in a time-division manner, it is possible to obtain accurate 3D information and target recognition information required for vehicle controls such as ACC, AEBS, and LKAS without being affected by vehicle speed. [Explanation of symbols]

[0060] 1. Your vehicle, 100 in-vehicle camera systems, 10 processing unit, 11 Sensor Interface, 12 camera control unit, 12a reference exposure condition storage unit, 12b first exposure condition determination unit, 12c External light condition determination unit, 12d second exposure condition determination unit; 12e exposure control unit; 12f Imaging control unit, 12g Light projection condition determination unit, 13 Emitter control unit, 14 Image distribution unit, 15. Feature point movement amount calculation unit, 16 3D information acquisition unit, 17 3D map memory section, 18 Vehicle attitude estimation unit, 19 Target recognition unit, 1a Target distance measurement section, 20(21~26) Camera, 30 Vehicle control device, 40 Floodlight C field of view V Stereo Viewing Area P Image data

Claims

1. a plurality of cameras arranged on the vehicle so as to form a stereoscopic viewing area in which at least a portion of the viewing area overlaps; a movement amount calculation unit that calculates a movement amount of a feature point in the stereo vision area captured by the plurality of cameras based on a behavior of the host vehicle; a first exposure condition determination unit that determines first exposure conditions for the plurality of cameras so that the movement amount is equal to or less than a threshold; a second exposure condition determination unit that determines second exposure conditions for the plurality of cameras based on external light conditions outside the vehicle; a three-dimensional information acquisition unit that acquires three-dimensional information of the stereoscopic viewing area using the image captured under the first exposure condition; a target recognition unit that recognizes targets around the host vehicle using the image captured under the second exposure condition; an exposure control unit that switches the exposure condition of each of the plurality of cameras between the first exposure condition and the second exposure condition; An in-vehicle camera system comprising:

2. The vehicle-mounted camera system according to claim 1, The vehicle-mounted camera system is characterized in that the exposure control unit prioritizes the first exposure condition over the second exposure condition when the vehicle speed is equal to or greater than a predetermined value.

3. The vehicle-mounted camera system according to claim 1, a storage unit that stores the three-dimensional information of the stereoscopic vision area previously acquired by the three-dimensional information acquisition unit in association with a target included in the stereoscopic vision area, The in-vehicle camera system is characterized in that the exposure control unit prioritizes the first exposure condition over the second exposure condition when three-dimensional information of a target currently included in the stereo vision area is stored in the memory unit.

4. The vehicle-mounted camera system according to claim 1, the plurality of cameras are arranged to have a plurality of the stereoscopic vision areas, The vehicle-mounted camera system is characterized in that the exposure control unit switches between the first exposure condition and the second exposure condition for each camera group that captures an image of each stereoscopic viewing area.

5. The vehicle-mounted camera system according to claim 4, and an exposure control unit that, when a target that may collide with the host vehicle is present in any of a plurality of stereo viewing areas, prioritizes the first exposure condition over the second exposure condition for a camera that captures an image of the stereo viewing area in which the target is detected.

6. The vehicle-mounted camera system according to claim 4, an imaging timing control unit that controls imaging timings of a camera that images the first stereoscopic vision area and a camera that images the second stereoscopic vision area among the plurality of cameras so that imaging timings are shifted by a predetermined time; The in-vehicle camera system is characterized in that the three-dimensional information acquisition unit acquires the three-dimensional information based on an image captured in the first stereo viewing area and an image captured in the second stereo viewing area.

7. The vehicle-mounted camera system according to claim 1, The vehicle-mounted camera system further comprises a floodlight that projects light in synchronization with the image capture by the camera.

8. an imaging step of capturing an image of a stereoscopic viewing area where the viewing areas of the plurality of cameras overlap; a movement amount calculation step of calculating a movement amount of a feature point in the stereo vision area captured by the plurality of cameras based on a behavior of the host vehicle; a first exposure condition determination step of determining first exposure conditions for the plurality of cameras so that the movement amount is equal to or less than a threshold; a second exposure condition determination step of determining second exposure conditions for the plurality of cameras based on external light conditions outside the vehicle; a three-dimensional information acquisition step of acquiring three-dimensional information of the stereoscopic viewing area using the image captured under the first exposure condition; a target recognition step of recognizing targets around the host vehicle using the image captured under the second exposure condition; an exposure control step of switching the exposure condition of each of the plurality of cameras to the first exposure condition or the second exposure condition; 1. A method for determining exposure conditions for an in-vehicle camera, comprising:

Citation Information

Patent Citations

  • Device and method for controlling in-vehicle camera

    JP2008174078A

  • Object recognition device, object recognition method, and object recognition program

    JP2014153167A

  • Camera device

    JP2015170901A

  • Object recognition device

    JP2020191010A

  • Object detection method and object detection device

    JP2022006844A