In-vehicle sensing systems and gating cameras

Abstract

A sensing system 10 is utilized for drive assistance or self-driving. A gating camera 20 divides a view field into a plurality of ranges with respect to a depth direction, and generates a plurality of slice images corresponding to the plurality of ranges. A main controller 60 processes the output of a main sensor group 50 and the output of the gating camera 20. The gating camera 20 can be switched between a first imaging mode in which performance is relatively high and power consumption is relatively large, and a second imaging mode in which performance is relatively low and power consumption is relatively small.

Description

[Technical Field]

[0001] The present disclosure relates to a sensing system for a vehicle. [Background technology]

[0002] For driver assistance and autonomous driving, object identification systems are used to sense the location and type of objects around the vehicle. Object identification systems include sensors and a processing unit that analyzes the sensor output. Sensors are selected from cameras, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), millimeter-wave radar, ultrasonic sonar, active sensors, etc., taking into account the application, required accuracy, and cost. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-257983 [Patent Document 2] International Publication WO2017 / 110413A1 Summary of the Invention [Problem to be solved by the invention]

[0004] A gating camera (also called a gated camera) has been proposed as an active sensor to replace the TOF camera (Patent Documents 1 and 2). A gating camera divides the imaging range into multiple ranges and captures images by changing the exposure timing and exposure time for each range. This allows a slice image to be obtained for each target range, and each slice image contains only the objects included in the corresponding range.

[0005] A gating camera is an active sensor, and both the image sensor and the lighting device consume power, so it consumes more power than a typical monocular camera or stereo camera.

[0006] The present disclosure has been made in light of such a situation, and one exemplary purpose of an embodiment thereof is to provide a gating camera with reduced power consumption. [Means for solving the problem]

[0007] 1. A gating camera according to one embodiment divides a field of view into a plurality of ranges in the depth direction and generates a plurality of slice images corresponding to the plurality of ranges. The gating camera includes an illumination device that emits pulsed illumination light, an image sensor, and a camera controller that controls the light emission timing of the illumination device and the exposure timing of the image sensor. The camera controller determines the operating / stopped state of the gating camera and / or the imaging mode of the gating camera.

[0008] A sensing system according to one embodiment is used for driving assistance or autonomous driving. The sensing system includes a main sensor, a gating camera that can switch between a first imaging mode with relatively high performance and relatively high power consumption and a second imaging mode with relatively low performance and relatively low power consumption, and that can divide a field of view into multiple ranges in the depth direction and generate multiple slice images corresponding to the multiple ranges, and a main controller that processes the output of the main sensor and the gating camera. The gating camera determines its operating / stopped state and / or imaging mode based on at least one of the slice images captured by the gating camera and at least one vehicle signal.

[0009] 2. A gating camera according to one embodiment divides a field of view into multiple ranges in the depth direction and generates multiple slice images corresponding to the multiple ranges. The gating camera includes an illumination device that irradiates the field of view with pulsed illumination light, an image sensor, and a camera controller that controls the light emission timing of the illumination device and the exposure timing of the image sensor. The camera controller is switchable between a first imaging mode, which offers relatively high performance and relatively high power consumption, and a second imaging mode, which offers relatively low performance and relatively low power consumption.

[0010] A sensing system according to one embodiment is used for driving assistance or autonomous driving. The sensing system includes a main sensor, a gating camera that can switch between a first imaging mode with relatively high performance and relatively high power consumption and a second imaging mode with relatively low performance and relatively low power consumption, and that can divide a field of view into multiple ranges in the depth direction and generate multiple slice images corresponding to the multiple ranges, and a main controller that processes the output of the main sensor and the output of the gating camera. [Effects of the Invention]

[0011] According to the present disclosure, the power consumption of a gating camera can be reduced. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram of a sensing system according to a first embodiment. [Figure 2] FIG. 10 is a diagram illustrating the operation of a gating camera. [Figure 3] 3(a) and (b) are diagrams illustrating images obtained by the gating camera. [Figure 4] 4(a) to 4(c) are diagrams illustrating the advantages of the gating camera in bad weather. [Figure 5] 4 is a time chart illustrating the operation of the sensing system. [Figure 6]4A to 4C are diagrams illustrating the operation of the gating camera according to the first embodiment. [Figure 7] 10A and 10B are diagrams illustrating the operation of the gating camera according to the second embodiment. [Figure 8] 10A to 10C are diagrams illustrating a photographing mode according to the third embodiment. [Figure 9] FIG. 10 is a block diagram of a sensing system according to a second embodiment. [Figure 10] FIG. 10 is a block diagram of a sensing system according to a third embodiment. [Figure 11] FIG. 10 is a block diagram of a sensing system according to a fourth embodiment. [Figure 12] FIG. 10 is a block diagram of a sensing system according to a fifth embodiment. [Figure 13] FIG. 10 is a block diagram of a sensing system according to a sixth embodiment. [Figure 14] 14(a) and 14(b) are diagrams showing an automobile equipped with a sensing system according to the embodiment. [Figure 15] 1 is a block diagram showing a vehicle lamp according to an embodiment of the present invention; [Figure 16] FIG. 13 is a block diagram of a sensing system including a gating camera according to a seventh embodiment. [Figure 17] 17(a) and (b) are diagrams showing slice images captured by a gating camera. [Figure 18] FIG. 18(a) is a diagram showing a histogram of the slice image of FIG. 17(a), and FIG. 18(b) is a diagram showing a histogram of the slice image of FIG. 17(b). [Figure 19] FIG. 19(a) is image data of the space above the slice image of FIG. 17(b), and FIG. 19(b) is a diagram showing a histogram of the image data of FIG. 19(a). [Figure 20] FIG. 10 is a diagram illustrating a shooting range in a weather detection mode. [Figure 21] 21(a) to 21(c) are diagrams for explaining generation of a difference image in the third embodiment. [Figure 22]22(a) and 22(b) are diagrams for explaining weather estimation based on differential images. DETAILED DESCRIPTION OF THE INVENTION

[0013] A summary of some exemplary embodiments of the present disclosure is provided. This summary is intended to provide a simplified overview of some concepts of one or more embodiments in order to provide a basic understanding of the embodiments as a prelude to the more detailed description that follows. It is not intended to limit the scope of the invention or disclosure. Furthermore, this summary is not an exhaustive overview of all possible embodiments, nor does it limit essential elements of the embodiments. For convenience, the term "one embodiment" may refer to one embodiment (example or variant) or multiple embodiments (examples or variants) disclosed herein.

[0014] A gating camera according to one embodiment divides a field of view into a plurality of ranges in the depth direction and generates a plurality of slice images corresponding to the plurality of ranges. The gating camera includes an illumination device that irradiates the field of view with pulsed illumination light, an image sensor, and a camera controller that controls the light emission timing of the illumination device and the exposure timing of the image sensor. The camera controller is switchable between a first imaging mode, which has relatively high performance and relatively high power consumption, and a second imaging mode, which has relatively low performance and relatively low power consumption.

[0015] The gating camera is configured to be switchable between a high-performance but high-power consumption mode and a lower-performance mode to reduce power consumption, and the mode is dynamically selected according to the driving environment. Rather than always operating at high performance, the system adaptively lowers performance, thereby reducing power consumption.

[0016] In one embodiment, the first and second imaging modes may have different frame rates, and lowering the frame rate can reduce power consumption.

[0017] In one embodiment, the first imaging mode and the second imaging mode may have different numbers of measurement ranges. Reducing the number of ranges can reduce power consumption.

[0018] In one embodiment, the first and second imaging modes may have different distances to the farthest boundary of the farthest range (farthest object distance). In this case, the first and second imaging modes may have different intensities of pulsed illumination light. When the farthest object distance is shortened, power consumption can be reduced by reducing the intensity of the pulsed illumination light. In addition, the amount of heat generated by the illumination device can be reduced.

[0019] In one embodiment, the gating camera may be capable of switching to a standby mode in which it can immediately take pictures in response to a command from the main controller.

[0020] In one embodiment, the camera controller may select a capture mode based on at least one vehicle signal, such as a rain sensor output, a fog sensor output, vehicle speed information, a fail signal indicating a malfunction of the main sensor, a fail signal indicating a malfunction or a decrease in recognition accuracy of the main controller, a wiper control signal, or a fog lamp control signal.

[0021] In one embodiment, the camera controller may select a shooting mode based on the slice images, or may determine the weather, i.e., visibility, based on the slice images, and reflect the determination result in the mode selection.

[0022] In one embodiment, the shooting mode of the camera controller may be selected by the main controller or may be selected depending on the state of the main controller. For example, the shooting mode may be switched depending on the accuracy of the main sensor input to the main controller, the recognition accuracy when the main controller performs object recognition, fail information of the main controller, etc.

[0023] A gating camera according to one embodiment divides a field of view into a plurality of ranges in the depth direction and generates a plurality of slice images corresponding to the plurality of ranges. The gating camera includes an illumination device that irradiates pulsed illumination light, an image sensor, and a camera controller that controls the light emission timing of the illumination device and the exposure timing of the image sensor. The camera controller determines the operating / stopped state of the gating camera and / or the imaging mode of the gating camera.

[0024] The gating camera is configured to be able to switch between an operating state and a stopped state, or to switch between shooting modes. The gating camera autonomously controls its own operating state. This allows it to reduce power consumption by stopping shooting or selecting a lower-performance shooting mode depending on the driving environment at the time, without placing an unnecessary load on the vehicle's higher-level controller.

[0025] In one embodiment, the camera controller may determine the operating / stopped state of the gating camera and / or the shooting mode of the gating camera based on the slice image. The slice image generated by the gating camera captures an object within a specific range in the depth direction of the field of view. By utilizing this property, visibility and weather conditions can be estimated and used to control the gating camera.

[0026] In one embodiment, the camera controller may be capable of switching between a first imaging mode, which has relatively high performance and relatively high power consumption, and a second imaging mode, which has relatively low performance and relatively low power consumption. The gating camera is configured to be capable of switching between a mode with high performance but high power consumption and a mode that reduces performance to reduce power consumption, and the mode is dynamically selected according to the driving environment. Rather than always operating at high performance, the performance can be adaptively reduced to reduce power consumption.

[0027] In one embodiment, the operating / stopping state of the gating camera may be controlled externally, and the camera controller may control the imaging mode based on the slice images.

[0028] In one embodiment, the camera controller may control the operating / stopping state and imaging mode of the gating camera based on the slice images.

[0029] In one embodiment, the gating camera may be constantly operating, and the camera controller may control the imaging mode of the gating camera based on the slice images.

[0030] In one embodiment, the gating camera does not need to support switching of the imaging mode, and the camera controller may control the operating / stopping state of the gating camera based on the slice images.

[0031] In one embodiment, the camera controller may intermittently perform photography to control its own state (i.e., operating state / stop state and / or the photography mode of the gating camera) apart from normal photography.

[0032] A sensing system according to one embodiment is used for driving assistance or autonomous driving. The sensing system includes a main sensor, a gating camera that can switch between a first imaging mode with relatively high performance and relatively high power consumption and a second imaging mode with relatively low performance and relatively low power consumption, and that can divide a field of view into multiple ranges in the depth direction and generate multiple slice images corresponding to the multiple ranges, and a main controller that processes the output of the main sensor and the gating camera. The gating camera determines its operating / stopped state and / or imaging mode based on at least one of the slice images captured by the gating camera and at least one vehicle signal.

[0033] In one embodiment, the gating camera may determine its operating / stopped state based on the slice images it captures.

[0034] In one embodiment, the gating camera may determine its operating / stopped state based on vehicle signals.

[0035] In one embodiment, the operating / stopping state of the gating camera may be controlled by the main controller.

[0036] (Embodiment) Preferred embodiments will be described below with reference to the drawings. The same or equivalent components, parts, and processes shown in each drawing will be given the same reference numerals, and redundant explanations will be omitted where appropriate. Furthermore, the embodiments are examples and do not limit the disclosure and invention, and all features and combinations thereof described in the embodiments are not necessarily essential to the disclosure and invention.

[0037] (Embodiment 1) 1 is a block diagram of a sensing system 10A according to embodiment 1. This sensing system 10A is installed in a vehicle such as an automobile or a motorcycle for the purpose of driving assistance or autonomous driving, and detects an object OBJ present around the vehicle.

[0038] The sensing system 10A includes a main sensor group 50, a main controller 60, and a gating camera 20. The main sensor group 50 may include one or more sensors. For example, the main sensor group 50 may include a camera 52 and a millimeter-wave radar 54. Alternatively, the main sensor group 50 may include a stereo camera. Alternatively, the main sensor group 50 may include a LiDAR or the like.

[0039] The main controller 60 detects the positions and types of objects around the vehicle based on the output of the main sensor group 50, and outputs the detection result RESULT. For example, the main controller 60 may be equipped with a classifier (a classifier), and the detection result RESULT may include information on the type (category, class) and position of the target object.

[0040] The gating camera 20 has a field of view divided into a plurality of ranges RNG1 to RNG2 in the depth direction. N Divide into multiple ranges RNG1~RNG N Multiple slice images IMGs1 to IMGs corresponding to N Adjacent ranges may overlap in depth at their boundaries.

[0041] The gating camera 20 includes an illumination device 22, an image sensor 24, a camera controller 26, and a processing unit 28.

[0042] The lighting device (floodlight) 22 irradiates the front of the vehicle with pulsed illumination light L1 in synchronization with a light emission timing signal S1 provided by the camera controller 26. The pulsed illumination light L1 is preferably infrared light, but is not limited to this and may be visible light having a predetermined wavelength. The lighting device 22 may use, for example, a laser diode (LD) or an LED. In a system in which the gating camera 20 is used only at night, the wavelength of the pulsed illumination light L1 may be near infrared, around 800 nm. In a system in which the gating camera 20 is used day and night, it is preferable that the pulsed illumination light L1 have a wavelength range longer than 1 μm.

[0043] The image sensor 24 includes a plurality of pixels, and is capable of controlling exposure in synchronization with an exposure timing signal S2 provided by the camera controller 26, thereby generating a slice image IMGr consisting of a plurality of pixels. The image sensor 24 is sensitive to the same wavelength as the pulsed illumination light L1, and captures the reflected light (return light) L2 reflected by the object OBJ. The i-th range RNG iThe slice image IMGr generated by the image sensor 24 with respect to the above is referred to as a raw image or a primary image as necessary to distinguish it from the slice image IMGs that is the final output of the gating camera 20. Furthermore, the raw image IMGr and the slice image IMGs are also collectively referred to simply as slice image IMG.

[0044] The camera controller 26 changes the light emission timing signal S1 and the exposure timing signal S2 for each range RNG to change the time difference between the light emission by the lighting device 22 and the exposure of the image sensor 24. The light emission timing signal S1 defines the timing to start light emission and the light emission duration. The exposure timing signal S2 defines the timing to start exposure (the time difference from light emission) and the exposure duration.

[0045] The arithmetic processing device 28 can be implemented as a combination of a processor (hardware) such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), microcomputer, or GPU (Graphics Processing Unit) and a software program executed by the processor (hardware). The arithmetic processing device 28 may be configured solely as hardware. The arithmetic processing device 28 processes raw image data IMGr generated by the image sensor 24 and outputs final slice images IMGs. Note that if the output IMGr of the image sensor 24 is used directly as the slice images IMGs, the arithmetic processing device 28 can be omitted.

[0046] 2 is a diagram illustrating the operation of the gating camera 20. In FIG. 2, the i-th range RNG i The figure shows how the range RNG is measured as a range of interest (ROI). The lighting device 22 emits light for a light emission period τ1 between times t0 and t1 in synchronization with the light emission timing signal S1. The top row shows a diagram of light rays with time on the horizontal axis and distance on the vertical axis. The range RNG is measured from the gating camera 20. i The distance to the boundary in front of MINi、 Range RNG iThe distance to the inner boundary of MAXi Let's say.

[0047] Light that leaves the lighting device 22 at a certain time travels a distance d MINi The round trip time T MINi teeth, T MINi =2×d MINi / c where c is the speed of light.

[0048] Similarly, light that leaves the lighting device 22 at a certain time travels a distance d MAXi The round trip time T MAXi teeth, T MAXi =2×d MAXi / c is.

[0049] Range RNG i When it is desired to photograph only the object OBJ included in the time t2=t0+T MINi Exposure begins at time t3 = t1 + T MAXi An exposure timing signal S2 is generated so that exposure is completed at this timing. This is one exposure operation.

[0050] i-th range RNG i When capturing an image, multiple exposures may be performed. In this case, the camera controller 26 simply repeats the above-described set of illumination and exposure operations multiple times at a predetermined cycle τ2. In this case, the raw image data IMGr generated by the image sensor 24 is the cumulative result of multiple exposures.

[0051] In this embodiment, in order to prevent the exposure (brightness value of the object image in the slice image) from varying for each range, the gating camera 20 optimizes the shutter speed (exposure time), number of exposures, sensitivity, irradiation intensity of the pulsed illumination light, etc. (shooting parameters) for each range.

[0052] 3(a) and (b) are diagrams illustrating images obtained by the gating camera 20. In the example of FIG. 3(a), an object (pedestrian) OBJ2 is present in range RNG2, and an object (vehicle) OBJ3 is present in range RNG3. FIG. 3(b) shows multiple slice images IMG1 to IMG3 obtained in the situation of FIG. 3(a). When capturing slice image IMG1, the image sensor 24 is exposed only to reflected light from range RNG1, and therefore no object image is captured in slice image IMG1.

[0053] When capturing slice image IMG2, image sensor 24 is exposed only to light reflected from range RNG2, so only object image OBJ2 appears in slice image IMG2. Similarly, when capturing slice image IMG3, image sensor 24 is exposed only to light reflected from range RNG3, so only object image OBJ3 appears in slice image IMG3. In this way, gating camera 20 allows objects to be captured separately for each range.

[0054] The gating camera 20 is advantageous for capturing images in bad weather. The reason for this is explained below. FIGS. 4(a) to 4(c) are diagrams illustrating the advantages of the gating camera 20 in bad weather. FIG. 4(a) shows an example of a driving scene in bad weather. An object (vehicle) OBJ3 is present in range RNG3. The dots shown in the figure also represent schematic representations of obstacles such as raindrops, snowflakes, or fog. FIG. 4(b) shows a slice image IMG3 of the third range obtained in the situation of FIG. 4(a). When capturing slice image IMG3, the image sensor 24 is exposed only to reflected light from range RNG3. Therefore, obstacles (raindrops, snowflakes, or fog) in ranges RNG1 and RNG2 are not captured in slice image IMG3. In other words, rain, snow, and fog in ranges other than the range being measured can be removed.

[0055] Figure 4(c) shows an image of the same field of view captured by a general camera. When captured by a general camera, the reflected light from all objects in range RNG3 is captured, resulting in many obstacles blocking object OBJ3.

[0056] A comparison of FIGS. 4(b) and (c) reveals that the slice image IMG generated by the gating camera 20 contains more information than a general camera in bad weather.

[0057] Returning to Fig. 1, in this embodiment, the gating camera 20 is utilized as an auxiliary sensor that assists the main sensor group 50. Therefore, the gating camera 20 does not operate all the time, but adaptively selects between an operating state (enabled state EN) and a stopped state (disabled state DIS) depending on the driving environment.

[0058] When the gating camera 20 is in an operating state, the camera controller 26 generates a light emission timing signal S1 and an exposure timing signal S2, thereby generating slice images in a plurality of ranges. When the gating camera 20 is in a stopped state, the camera controller 26 does not generate the light emission timing signal S1 or the exposure timing signal S2, and therefore no slice images are generated.

[0059] In bad weather, the slice image IMG generated by the gating camera 20 is supplied to the main controller 60. Then, the output of the gating camera 20 is used to control driving assistance or automatic driving.

[0060] The above is the configuration of the sensing system 10A. Next, the operation of the sensing system 10A will be described. Figure 5 is a time chart illustrating the operation of the sensing system 10A.

[0061] Under good weather (visibility) conditions, the reliability of the main sensor group 50 is high. In this case, the gating camera 20 is stopped, and the main controller 60 detects targets based on the output of the main sensor group 50.

[0062] In poor weather (visibility) conditions, the reliability of the main sensor group 50 decreases. In this case, the gating camera 20 becomes operational, and the main controller 60 detects targets based on the detection results of the gating camera 20 instead of or in addition to the output of the main sensor group 50.

[0063] This completes the operation of the sensing system 10 A. This sensing system 10 A can suppress an increase in power consumption due to the gating camera 20, while also suppressing a decrease in performance of the sensing system 10 A during bad weather.

[0064] Returning to Figure 1, gating camera 20 is configured to be able to switch between multiple shooting modes in an operating state. The multiple shooting modes include a first shooting mode with relatively high performance and high power consumption, and a second shooting mode with relatively low performance and low power consumption.

[0065] Several examples of multiple shooting modes will now be described.

[0066] Example 1 Each shooting mode may have a different frame rate. The "frame rate" is the frequency at which all ranges to be measured are measured, and is the reciprocal of the frame period. A shooting mode with a high frame rate enables high-speed sensing, but increases power consumption. A shooting mode with a low frame rate reduces power consumption, but decreases sensing speed.

[0067] 6 is a diagram illustrating the operation of the gating camera 20 according to the first embodiment. During the first shooting mode MODE1, the first frame period T F1 Then, shooting is performed in multiple ranges RNG1 to RNG3. In this example, there are three ranges. Note that light emission and exposure may be repeated multiple times during the shooting period for one range.

[0068] During the second shooting mode MODE2, the second frame period T F2 (T F2 >T F1 ) and shooting is performed in a plurality of ranges RNG1 to RNG3. By reducing the frame rate, the power consumption in the second shooting mode MODE2 becomes lower than the power consumption in the first shooting mode MODE1.

[0069] Even in situations where the gating camera 20 is needed, such as in bad weather, the highest performance may not be required depending on the situation. For example, even in bad weather, if the accuracy (reliability) of sensing based on the output of the main sensor group 50 is relatively high, sensing will not be hindered even if the frame rate of the gating camera 20 is reduced. Therefore, in such cases, power consumption can be reduced by selecting the second shooting mode MODE2.

[0070] For example, the shooting mode may be selected depending on the amount of rain or snow, the thickness of fog, etc. For example, when there is a relatively large amount of rain or snow or when the fog is relatively thick, the first shooting mode MODE1 may be selected, and when there is a relatively small amount of rain or snow or when the fog is relatively thin, the second shooting mode MODE2 may be selected.

[0071] Alternatively, the mode may be selected depending on the cause of the current bad weather (rain, snow, or fog). For example, the first shooting mode may be selected when fog occurs or snow falls, and the second shooting mode MODE2 may be selected when rain falls. For example, the presence or absence of rain, snow, or fog may be obtained based on information provided by wireless communication from outside the vehicle, such as VICS (registered trademark) (Vehicle Information and Communication System) information.

[0072] Alternatively, the first shooting mode MODE1 and the second shooting mode MODE2 may be switched depending on the driving scene. For example, the first shooting mode MODE1 may be selected when the driving speed is relatively high, and the second shooting mode MODE2 may be selected when the driving speed is relatively low.

[0073] Example 2 In the second embodiment, the number of ranges to be measured differs for each shooting mode. For example, in the first shooting mode MODE1, three ranges RNG1 to RNG3 are measured, and in the second shooting mode MODE2, two ranges RNG1 to RNG2 are measured.

[0074] 7 is a diagram illustrating the operation of the gating camera 20 according to the second embodiment. For example, assume that the first range RNG1 is 0 to 25 m, the second range RNG2 is 25 to 50 m, and the third range RNG3 is 50 to 100 m. Depending on the weather, a situation may arise in which sensing beyond 50 m is not possible even with the performance of the gating camera 20. In such a case, power consumption can be reduced by selecting the second shooting mode MODE2.

[0075] Furthermore, the visibility required for autonomous driving control or driving assistance control depends on the vehicle speed. Specifically, the higher the speed, the more distant target information is required. Therefore, when the vehicle speed is higher than a certain threshold, the first imaging mode MODE1 may be selected, and when the vehicle speed is lower than the threshold, the second imaging mode MODE2 may be selected.

[0076] In FIG. 7, the frame rate of the second shooting mode MODE2 is lower than the frame rate of the first shooting mode MODE1, but the frame rates may be the same.

[0077] The distance to the farthest boundary of the farthest range in the first shooting mode MODE1 (farthest shooting distance) is d MAX3 The farthest object distance in the second shooting mode MODE2 is d MAX2 and differs between the first shooting mode MODE1 and the second shooting mode MODE2.

[0078] In such a case, the intensity of the pulsed illumination light in the second imaging mode MODE2 may be lower than that in the first imaging mode MODE1, as shown in Fig. 7. Note that the intensity of the pulsed illumination light in Fig. 7 is not intended to imply continuous emission. In the second imaging mode MODE2, the pulsed illumination light does not need to be projected over a long distance, so the intensity can be lowered. Furthermore, the amount of heat generated by the illumination device 22 can be reduced.

[0079] Example 3 In Example 3, the depths of the multiple ranges differ for each shooting mode. Fig. 8 is a diagram illustrating the shooting modes according to Example 3. For example, in the first shooting mode MODE1, the first range RNG1 is 0 to 25 m, the second range RNG2 is 25 to 50 m, and the third range RNG3 is 50 to 100 m, whereas in the second shooting mode MODE2, the first range RNG1 is 0 to 25 m, the second range RNG2 is 25 to 50 m, and the third range RNG3 is 50 to 75 m. In other words, the farthest object distance in the first shooting mode MODE1 is 100 m, whereas the farthest object distance in the second shooting mode MODE2 is shorter at 75 m.

[0080] In the third embodiment, as in the second embodiment, the intensity can be reduced because the pulsed illumination light does not need to be projected over a long distance in the second imaging mode MODE2, and the amount of heat generated by the illumination device 22 can also be reduced.

[0081] The selection of the photographing mode in the third embodiment may be based on the weather and visibility, as in the second embodiment, or may be based on the vehicle speed.

[0082] Example 4 In the fourth embodiment, the gating camera 20 can be switched to a standby mode in addition to a plurality of shooting modes. In the standby mode, the gating camera 20 is ready to shoot at any time, but does not shoot spontaneously and remains in standby. In the standby mode, the camera controller 26 can communicate with the main controller 60, and immediately shoots when it receives a shooting instruction command from the main controller 60. By shooting only when instructed by the main controller 60, unnecessary power consumption can be reduced.

[0083] Next, the control of the operating state / stop state of the gating camera 20 and the control of the shooting mode will be described.

[0084] In this embodiment, the gating camera 20 autonomously controls the operating state / stop state and the shooting mode of the gating camera 20.

[0085] (Control method 1) In control method 1, the gating camera 20 switches between the operating state and the stopped state and selects the imaging mode based on the slice image IMG that it generates itself.

[0086] The slice image IMG captures a subject within a specific range in the depth direction of the field of view. This property can be used to estimate the visibility and weather conditions, and to control the gating camera. Weather estimation and visibility estimation based on the slice image IMG will be described later.

[0087] Based on the slice image IMG, the camera controller 26 determines whether to operate or stop the gating camera 20. The camera controller 26 switches the gating camera 20 to an operating state in a situation where a decrease in the reliability of the main sensor group 50 is estimated.

[0088] Furthermore, the camera controller 26 selects the first imaging mode MODE1 or the second imaging mode MODE2 based on the slice image IMG. The selection conditions may be determined based on the first to fourth embodiments relating to the imaging modes.

[0089] In this control method 1, gating camera 20, in a stopped state, intermittently (for example, every few tens of seconds to every few minutes) takes photographs for controlling the photographing mode and operation / stop.

[0090] (Control method 2) In control method 2, the camera controller 26 of the gating camera 20 switches between the operating state and the stopped state based on at least one vehicle signal. Examples of the vehicle signal include the output of the rain sensor, the output of the fog sensor, vehicle speed information, a fail signal indicating a malfunction of the main sensor, a fail signal indicating a malfunction of the main controller or a decrease in recognition accuracy, a wiper control signal, a fog lamp control signal, etc. If the camera controller 26 determines that the weather is bad based on the vehicle signal, it switches the gating camera 20 to the operating state and starts capturing images.

[0091] The camera controller 26 executes the imaging mode based on the slice image IMG, similarly to the control method 1.

[0092] In control method 2, unlike control method 1, it is not necessary to take pictures for controlling operation / stop while the camera is stopped.

[0093] (Control method 3) In control method 3, the camera controller 26 performs both the switching between the operating state and the stopped state and the control of the shooting mode based on at least one vehicle signal.

[0094] (Control Method 4) In the control method 4, the camera controller 26 switches between the moving state and the stopped state based on the slice images IMG, and controls the imaging mode based on at least one vehicle signal.

[0095] The control of the operating state / stop state and the control of the imaging mode may be performed based on both the vehicle signal and the analysis result of the slice image.

[0096] The imaging mode of the gating camera 20 may be controlled by the main controller 60. In this case, the main controller 60 may determine the imaging mode of the gating camera 20 based on a vehicle signal.

[0097] (Embodiment 2) 9 is a block diagram of a sensing system 10B according to embodiment 2. In embodiment 2, the operating state / stopped state of the gating camera 20 is controlled based on an external enable signal EN, and the imaging mode of the gating camera 20 is controlled autonomously by the gating camera 20 itself.

[0098] 9, for example, the operating state / stopped state of the gating camera 20 is controlled by the main controller 60. As described in the first embodiment, the main controller 60 may determine the operating state / stopped state of the gating camera 20 based on at least one piece of vehicle information and generate the enable signal EN. The main controller 60 may also control the operating state / stopped state of the gating camera 20 based on the slice image IMG generated by the gating camera 20.

[0099] Alternatively, the main controller 60 may control the operating / stopped state of the gating camera 20 based on both the vehicle signal and the slice image.

[0100] On the other hand, in the second embodiment, the camera controller 26 may select the imaging mode of the gating camera 20 based on the slice image IMG. Alternatively, the camera controller 26 may select the imaging mode of the gating camera 20 based on a vehicle signal. Alternatively, the camera controller 26 may select the imaging mode of the gating camera 20 based on both the vehicle signal and the slice image.

[0101] (Embodiment 3) 10 is a block diagram of a sensing system 10C according to embodiment 3. In embodiment 3, a gating camera 20 is mounted on a vehicle lamp 200. The vehicle lamp 200 includes a lamp ECU (Electronic Control Unit) 210 that comprehensively controls low beams and high beams (not shown).

[0102] The operating state / stopped state of the gating camera 20 is controlled based on an enable signal EN generated by the lighting fixture ECU 210, and the imaging mode of the gating camera 20 is autonomously controlled by the gating camera 20 itself. The lighting fixture ECU 210 controls the operating state / stopped state of the gating camera 20 based on at least one of a vehicle signal and a slice image.

[0103] (Embodiment 4) 11 is a block diagram of a sensing system 10D according to embodiment 4. In embodiment 4, the gating camera 20 basically operates constantly while the vehicle is running, and only the imaging mode can be switched.

[0104] The gating camera 20 (camera controller 26) selects a shooting mode based on at least one of the slice image IMG and the vehicle signal.

[0105] (Embodiment 5) 12 is a block diagram of a sensing system 10E according to embodiment 5. In embodiment 5, the gating camera 20 does not support switching between multiple shooting modes, and is only capable of switching between an operating state (enabled state) and a stopped state (disabled state).

[0106] The gating camera 20 (camera controller 26) switches the gating camera 20 between an operating state and a stopped state based on at least one of the slice image IMG and the vehicle signal.

[0107] (Embodiment 6) 13 is a block diagram of a sensing system 10F according to embodiment 6. In embodiment 6, some (in this example, the camera 52) or all of the main sensor group 50 is omitted.

[0108] As described in the first to fifth embodiments, the gating camera 20 can control both or either the switching between the operating state and the stopped state and the switching of the shooting mode. The switching between the operating state and the stopped state and the switching of the shooting mode are as described in the first to fifth embodiments.

[0109] 14(a) and (b) are diagrams showing an automobile 300 equipped with the sensing system 10 according to the embodiment. Referring to Fig. 14(a), the automobile 300 is equipped with headlamps (lighting fixtures) 302L and 302R.

[0110] The camera 52 and millimeter-wave radar 54 of the main sensor group 50 are disposed in locations suitable for vehicle sensing. For example, the camera 52 is provided behind the rearview mirror, and the millimeter-wave radar 54 is disposed in the front of the vehicle. The main controller 60 is disposed inside the vehicle cabin or in the engine room.

[0111] The lighting device 22 of the gating camera 20 is built into at least one of the left and right headlamps 302L, 302R. The image sensor 24 can be attached to a part of the vehicle, for example, behind the rearview mirror. Alternatively, the image sensor 24 may be mounted on the front grille or front bumper. The camera controller 26 may be mounted inside the vehicle cabin, in the engine compartment, or built into the headlamp.

[0112] See Fig. 14(b). As shown in Fig. 14(b), the image sensor 24 may be built into either the left or right headlamp 302L, 302R.

[0113] 15 is a block diagram showing a vehicle lamp 200 according to an embodiment. The vehicle lamp 200 corresponds to the headlamp 302 in FIG. 14(b), and includes a low beam unit 202, a high beam unit 204, a lamp ECU 210, and a gating camera 20.

[0114] The lamp ECU 210 controls the on / off or light distribution of the low beam unit 202 and the high beam unit 204 based on a control command from the vehicle-side ECU 310. In addition, in some embodiments and examples, the lamp ECU 210 may switch the gating camera 20 between an operating state and a stopped state.

[0115] The gating camera 20 is built into the housing of the vehicle lamp 200. At least one of the image sensor 24, the camera controller 26, and the processing unit 28 may be provided outside the housing of the vehicle lamp 200.

[0116] (Visibility estimation and weather estimation based on slice images) The gating camera can freely set a depth range of interest and capture only objects within that range of interest. This section describes a technique that utilizes this characteristic to allow the gating camera 20 to acquire weather information based on slice images IMG that it has captured.

[0117] 16 is a block diagram of a sensing system 10G including a gating camera 20G according to embodiment 7. The gating camera 20G includes an illumination device 22, an image sensor 24, a camera controller 26G, and a processing unit 28G.

[0118] In addition to the normal shooting mode, the camera controller 26G can select a weather detection mode. In the weather detection mode, shooting is performed within a predetermined range determined for weather detection. The calculation processing device 28G acquires weather-related information INFO_FCST based on image data (slice image) IMGf obtained for the predetermined range in the weather detection mode.

[0119] Fog, rain, and snow exist randomly, or in other words, uniformly, within the plane of the field of view and in the depth direction. On the other hand, objects other than fog, rain, and snow (vehicles, pedestrians, traffic signs, delineators, etc.) exist locally and non-randomly. In other words, in bad weather, the slice image obtained for a specified range RNGf will contain light diffused by the uniformly distributed fog, rain, and snow, which exhibits properties similar to random noise. Based on this property, information about the weather can be obtained by analyzing the image data IMGf.

[0120] Specific examples of weather detection will be described below based on Examples 1 to 3.

[0121] Example 1 In weather detection mode, camera controller 26G captures images using wide range RNGw, which is longer in the depth direction than normal shooting mode, as the predetermined range RNGf. The wide range RNGw may be a range in normal shooting mode extended toward the front. For example, in normal shooting mode, images are captured using four ranges RNG1 to RNG4, with RNG1 set to 0 to 25 m, RNG2 set to 25 to 50 m, RNG3 set to 50 to 75 m, and RNG4 set to 75 to 100 m. The wide range RNGw for weather detection can be a range obtained by extending the second range RNG2 toward the front, and may be set to, for example, 10 to 50 m. To extend the range, camera controller 26G increases the exposure time of image sensor 24.

[0122] Figures 17(a) and (b) are diagrams showing slice images captured by a gating camera. Figure 17(a) is a slice image obtained in normal shooting mode at a certain range (25 to 100 m) under bad weather conditions (in fog with a visibility of about 50 m based on infrared light energy). Figure 17(b) is a slice image IMGf obtained under the same conditions at a wide range IMGw (6 to 100 m), which is an expanded view of the range in Figure 17(a) toward the front.

[0123] The slice image in FIG. 17(a) shows only the light reflected from the object in the background, whereas the slice image IMGf in FIG. 17(b) shows not only the light reflected from the object in the background but also the light reflected from the fog (rain, snow) located in the area extending toward the front. Therefore, the object in the background is hidden by the fog, rain, or snow in the foreground. In other words, the slice image IMGf, which is captured widely in the foreground, contains a lot of information about the fog (rain, snow). Capturing such a slice image IMGf in weather detection mode makes it possible to estimate the weather.

[0124] FIG. 18(a) shows a histogram of the slice image of FIG. 17(a), and FIG. 18(b) shows a histogram of the slice image of FIG. 17(b). The horizontal axis of each histogram represents pixel value (gradation value), and the vertical axis represents frequency (occurrence frequency). As shown in FIG. 18(a), the histogram of reflected light from a target in the background reflects the size and reflectance of the target, resulting in sharp edges and multiple peaks, deviating from a normal distribution. On the other hand, as shown in FIG. 18(b), in the slice image obtained for a wide range RNGw, the background object is obscured by the fog, rain, or snow in the foreground, so the histogram of the slice image approaches the histogram of an image containing random noise. As a result, the histogram has a single peak and a gentle shoulder.

[0125] Thus, in bad weather, the histogram of the slice image IMGf obtained for a predetermined range RNGf tends to approach that shown in Figure 18(b). Conversely, in good weather without fog, rain, or snow, the histogram of the slice image IMGf approaches that shown in Figure 18(a). Therefore, the calculation processing device 28g can determine whether the weather is good or bad by obtaining the histogram of the slice image IMGf.

[0126] In the weather detection mode, it is not essential to increase the depth of the predetermined range RNGf, and the depth may be the same as in the normal shooting mode.

[0127] The calculation processing device 28g may create a histogram for the entire slice image IMGf, or may extract only a portion thereof (a predetermined region) to generate a histogram.

[0128] Detection accuracy can be improved by defining the predetermined area so that it contains only fog, rain, and snow in bad weather and no objects in good weather. Specifically, the predetermined area may be defined as the sky. In this case, the predetermined area should be positioned in the upper center of the image data.

[0129] Fig. 19(a) is image data of the space above slice image IMGf in Fig. 17(b), and Fig. 19(b) is a diagram showing a histogram of the image data in Fig. 19(a). In this way, when a histogram of the part corresponding to the sky is generated in bad weather, a histogram close to a normal distribution is obtained, whereas in good weather, there is no reflection, so the distribution of the histogram is concentrated in the low gradation region.

[0130] The calculation processing device 28g may generate a histogram of a portion (or the entirety) of the slice image IMGf and determine the weather based on characteristics that appear in the histogram under bad weather conditions. For example, the calculation processing device 28g may determine the weather based on the shape of the histogram, and may determine bad weather if it determines that the histogram is close to a normal distribution. A known algorithm known as a normality test may be used to determine whether the distribution is close to a normal distribution. Alternatively, in bad weather, the shoulders of the histogram become gentler, as shown in FIG. 18(b), and the variance σ becomes larger. Therefore, bad weather may be determined when the variance σ exceeds a predetermined threshold. Alternatively, the larger the variance σ, the worse the weather (thick fog, heavy rain, or heavy snow) may be determined.

[0131] Example 2 In Example 1, the weather was estimated based on one slice image IMGf obtained for one predetermined range RNGf, but in Example 2, images are taken for two predetermined ranges RNGfx and RNGfy, and the weather is estimated based on two slice images IMGfx and IMGfx obtained for each range.

[0132] Specifically, one of the two predetermined ranges (also called the narrow range) RNGfx is relatively narrow in the depth direction, and the other (also called the wide range) RNGfy is relatively wide in the depth direction. The wide range RNGy may be in a relationship where the narrow range RNGx is extended toward the front. Figure 20 is a diagram explaining the shooting range in weather detection mode. The dashed line indicates the field of view of the image sensor 24.

[0133] When narrow-range RNGfx and wide-range RNGy are photographed using a gating camera in bad weather, two slice images (referred to as the first slice image and the second slice image) as shown in Figures 18(a) and 18(b) are obtained.

[0134] The calculation processing device 28G can obtain information about the weather based on the first image data IMGfx obtained for the narrow range RNGfx and the second image data IMGfy obtained for the wide range RNGfy.

[0135] Specifically, the calculation processing device 28G calculates histograms for each of the first image data IMGfx and the second image data IMGfy, and obtains weather information based on the two histograms, which correspond to the two histograms shown in Figures 19(a) and 19(b).

[0136] The histogram obtained for the first image data IMGfx has a shape, variance, and mean that correspond to the targets included in the narrow range RNGfx. On the other hand, the histogram obtained for the second image data IMGfy approaches that of the first image data IMGfx in good weather. On the other hand, in bad weather, the histogram of the second image data IMGfy approaches that of an image containing a lot of random noise. Specifically, the histogram obtained in bad weather exhibits the following characteristics: (i) a shape close to a normal distribution, (ii) a gentle shoulder, and (iii) a large variance.

[0137] Therefore, the calculation processing device 28G may calculate the average of each of the two histograms, and determine that the weather is bad when the difference between the averages exceeds a predetermined threshold value (condition 1). Alternatively, the calculation processing device 28G may calculate the variance σ of each of the two histograms, and determine that the weather is bad when the difference between the variances exceeds a predetermined threshold value (condition 2).

[0138] The calculation processing device 28G may determine that the weather is bad when both conditions 1 and 2 are true, or may determine that the weather is bad when either one of the conditions is true.

[0139] The calculation processing device 28G may determine whether the shapes of the two histograms are similar to each other, and if they are not similar, determine that the weather is bad.

[0140] Example 3 In the third embodiment, slice images IMGfx and IMGfy are generated for the narrow-range RNGfx and wide-range RNGfy, and weather-related information is acquired based on the two slice images IMGfx and IMGfy, as in the second embodiment. More specifically, the calculation processing device 28G generates a difference image IMGd representing the difference between the first image data IMGfx and the second image data IMGfy, and acquires weather-related information based on the difference image IMGd.

[0141] 21(a) to 21(c) are diagrams illustrating generation of a difference image in Example 3. The pixel value IMGd(i) of each pixel in the difference image IMGd is the difference between the pixel values ​​IMGfx(i) and IMGfy(i) of corresponding pixels in the first image data IMGfx and second image data IMGfy. IMGd(i) = IMGfy(i) - IMGfx(i) IMG#(i) indicates the pixel value of the i-th pixel in image IMG#. In the difference image, most of the reflected light from objects in the background is canceled out, and a large amount of noise components (fog, rain, snow) from areas closer to the camera are included.

[0142] The arithmetic processing device 28G may estimate the weather based on the entire differential image IMGd, or may estimate the weather based on a portion of it. As shown in FIG. 20, within the hatched portion of the camera's field of view FOV, objects present in the farther range RNGfx are not captured. Therefore, the arithmetic processing device 28G may exclude the portion of the second image data IMGfy corresponding to the hatched field of view and estimate the weather based on the remaining portion (effective area). The effective area can also be understood as the range within which objects present in the narrow range RMGfx are captured.

[0143] 22(a) and (b) are diagrams illustrating weather estimation based on the difference image IMGd. A rectangular area is shown in the difference image IMGd of FIG. 22(a). This rectangular area is set in the valid area described above. FIG. 22(b) shows a histogram within the rectangular area of ​​FIG. 22(a). The calculation processing device 28G acquires information about the weather based on this histogram.

[0144] The calculation processing device 28G can estimate the weather based on the histogram, similar to the example 1. Specifically, it determines whether the histogram is caused by noise or an object based on the shape of the histogram, the number of peaks, the slope of the shoulders, the variance, etc.

[0145] The weather estimation described in the seventh embodiment can be used in combination with the techniques of the first to sixth embodiments to control the imaging mode of the gating camera 20 and to switch it on / off, but is not limited to this.

[0146] The method of weather estimation is not limited to that described in the seventh embodiment. For example, weather may be estimated by utilizing the fact that infrared transmittance decreases in bad weather (fog or rain). In this method, a gating camera captures images over a predetermined range. The amount of light emitted by the gating camera is known, and the distance to the predetermined range is also known. When the reflectance of an object is constant, the pixel value of the object changes depending on the transmittance between the gating camera and the object, in other words, the surrounding environment (weather). In other words, in good weather, the pixel value is relatively large, and in bad weather, the pixel value is relatively small. Therefore, the light transmittance and therefore the surrounding environment can be estimated based on the pixel values ​​of the slice image.

[0147] For example, the surrounding environment may be estimated based on pixel values ​​of specific objects having a predetermined reflectance included in the slice images. The specific objects may include at least one of signs, road surfaces, white lines, guardrails, and reflectors, each of which has a known reflectance.

[0148] The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of the components and the treatment processes, and that such modifications are also within the scope of the present invention. Such modifications will be described below.

[0149] (Variation 1) In the embodiment, the slice image IMG is output from the gating camera 20 to the main controller 60, but this is not limitative. For example, a classifier (classifier) ​​may be implemented in the arithmetic processing device 28 of the gating camera 20, and the classification result, i.e., the type (category) and position of the target object, may be output to the main controller 60.

[0150] (Variation 2) In the embodiment, the output of the gating camera 20 is used for driving assistance or autonomous driving control, but this is not a limitation. For example, in bad weather, the gating camera 20 may be activated and the slice image IMG generated by the gating camera 20 may be displayed on a display device such as a HUD (Head Up Display) to assist the user's field of view.

[0151] (Variation 3) In the embodiment, the case where two shooting modes are switched is described, but the number of switchable shooting modes may be three or more.

[0152] (Variation 4) In the above explanation, the gating camera 20 is enabled and operated only when necessary, and the shooting mode is switched during operation, but this is not the only option. The gating camera 20 may be configured to always take pictures while driving, and the shooting mode may be switchable.

[0153] The embodiments are merely examples, and it will be understood by those skilled in the art that there are various variations in the combination of each component and each treatment process, and that such variations are also included within the scope of this disclosure or the present invention. [Industrial Applicability]

[0154] The present disclosure can be used in a sensing system for a vehicle. [Explanation of symbols]

[0155] S1 Light emission timing signal S2 Exposure timing signal 10 Sensing System 20 Gating Camera 22 Lighting equipment 24 image sensors 26 Camera Controller 28 Processing Unit 50 Main sensors 52 Camera 54 Millimeter wave radar 60 Main Controller 200 Vehicle lighting fixtures 202 Low beam unit 204 High beam unit 210 Lighting ECU 300 cars 302 Headlamp 310 Vehicle ECU

Claims

1. A gating camera that divides a field of view into a plurality of ranges in the depth direction and generates a plurality of slice images corresponding to the plurality of ranges, an illumination device that irradiates a field of view with pulsed illumination light; An image sensor; a camera controller that controls the timing of light emission of the lighting device and the timing of exposure of the image sensor; Equipped with the camera controller is capable of switching between a first imaging mode having relatively high performance and relatively high power consumption and a second imaging mode having relatively low performance and relatively low power consumption; the gating camera is switchable between an operating state in which the first photographing mode and the second photographing mode can be selected, and a stopped state in which no photographing is performed, the camera controller determines switching between the operating state and the stopped state based on the plurality of slice images generated by the gating camera itself; The gating camera is characterized in that, in the stopped state, the gating camera intermittently takes photographs only for controlling operation / stop.

2. 2. The gating camera according to claim 1, wherein the second shooting mode has a frame rate lower than that of the first shooting mode.

3. 3. The gating camera according to claim 1, wherein the number of ranges to be measured in the first photographing mode is greater than that in the second photographing mode.

4. 4. The gating camera according to claim 1, wherein the distance to the innermost boundary of the farthest range in the first shooting mode is greater than that in the second shooting mode.

5. 5. The gating camera according to claim 4, wherein the intensity of the pulsed illumination light in the first photographing mode is higher than that in the second photographing mode.

6. 6. The gating camera according to claim 1, wherein the gating camera can be switched to a standby mode in which it can immediately perform photographing in response to an instruction from a main controller.

7. 7. The gating camera according to claim 1, wherein the camera controller selects a shooting mode based on at least one vehicle signal.

8. 8. The gating camera according to claim 1, wherein the camera controller selects a shooting mode based on the slice image.

9. 7. The gating camera according to claim 1, wherein the shooting mode of said camera controller is selected by a main controller or is selected according to the state of said main controller.

10. A sensing system for driving assistance or automated driving, A main sensor; a gating camera that can switch between a first imaging mode with relatively high performance and relatively high power consumption and a second imaging mode with relatively low performance and relatively low power consumption, and that can divide a field of view into a plurality of ranges in the depth direction and generate a plurality of slice images corresponding to the plurality of ranges; a main controller that processes the output of the main sensor and the output of the gating camera; Equipped with the gating camera is switchable between an operating state in which the first photographing mode and the second photographing mode can be selected, and a stopped state in which no photographing is performed, the main controller determines switching between the operating state and the stopped state based on the plurality of slice images generated by the gating camera itself; The sensing system is characterized in that, in the stopped state, the gating camera intermittently takes photographs only for the purpose of controlling operation / stop.

11. The sensing system according to claim 10 , wherein the second imaging mode has a frame rate lower than that of the first imaging mode.

12. The sensing system according to claim 10 or 11, wherein the first imaging mode has a greater number of ranges to be measured than the second imaging mode.

13. 13. The sensing system according to claim 10, wherein the distance to the innermost boundary of the farthest range in the first imaging mode is greater than that in the second imaging mode.

14. The sensing system according to claim 13 , wherein the intensity of the pulsed illumination light in the first imaging mode is higher than that in the second imaging mode.

15. 15. The sensing system according to claim 10, wherein the gating camera can be switched to a standby mode in which it can immediately perform imaging in response to an instruction from the main controller.

16. 16. The sensing system according to claim 10, wherein the imaging mode of the gating camera is switched in accordance with at least one of the density of fog, the amount of rainfall, and the amount of snowfall.

17. 17. The sensing system according to claim 10, wherein the imaging mode of the gating camera is switched in accordance with at least one of the output of a fog sensor and the output of a rain sensor.

18. 18. The sensing system according to claim 10, wherein the imaging mode of the gating camera is switched in accordance with at least one of the state of a fog lamp and the operating speed of a wiper.

19. 19. The sensing system according to claim 10, wherein the imaging mode of the gating camera is switched in accordance with the vehicle speed.

20. 20. The sensing system according to claim 10, wherein the imaging mode of the gating camera is determined by the gating camera.

21. 20. The sensing system according to claim 10, wherein the imaging mode of the gating camera is controlled by the main controller or in response to a state of sensing by the main controller.

22. At least a part of the gating camera is built into a vehicle lamp, 20. The sensing system according to claim 10, wherein the imaging mode of the gating camera is controlled by a control unit of the vehicle lamp.

23. A gating camera that divides a field of view into a plurality of ranges in the depth direction and generates a plurality of slice images corresponding to the plurality of ranges, an illumination device that irradiates a field of view with pulsed illumination light; An image sensor; a camera controller that controls the timing of light emission of the lighting device and the timing of exposure of the image sensor; Equipped with the camera controller determines switching between an operating state of the gating camera and a stopped state of the gating camera based on the plurality of slice images generated by the gating camera itself; The gating camera is characterized in that, in the stopped state, the gating camera intermittently takes photographs only for controlling operation / stop.

24. 24. The gating camera according to claim 23, wherein the camera controller determines a shooting mode of the gating camera based on at least one of the slice images and at least one vehicle signal.

25. the photographing modes include a first photographing mode and a second photographing mode, 25. The gating camera of claim 24, wherein the first shooting mode is a shooting mode with relatively high performance and relatively high power consumption, the second shooting mode is a shooting mode with relatively low performance and relatively low power consumption, and the second shooting mode has a frame rate lower than that of the first shooting mode.

26. A sensing system for driving assistance or automated driving, A main sensor; a gating camera that can switch between a first imaging mode with relatively high performance and relatively high power consumption and a second imaging mode with relatively low performance and relatively low power consumption, and that can divide a field of view into a plurality of ranges in the depth direction and generate a plurality of slice images corresponding to the plurality of ranges; a main controller that processes an output of the main sensor and an output of the gating camera; Equipped with A sensing system characterized in that the gating camera determines its own operating / stopped state based on the slice images it has captured, and in the stopped state, it intermittently captures images only for the purpose of controlling operation / stop.

Images