Gating cameras, vehicle sensing systems, vehicle lighting fixtures

Abstract

This gating camera 20 divides a view field into a plurality of ranges with respect to a depth direction, and generates a plurality of slice images corresponding to the plurality of ranges. A controller 26 is capable of generating a light emission control signal S1 and an exposure control signal S2. An illumination device 22 radiates probe light L1 in accordance with the light emission control signal S1 during normal imaging. An image sensor 24 carries out exposure in accordance with the exposure control signal S2. A calibrating light source 30 irradiates the image sensor 24 with calibration light in accordance with the light emission control signal S1 during calibration. The controller 26 sweeps a time difference τ between the light emission control signal S1 and the exposure control signal S2 and monitors a change in pixel values of the image sensor 24 during calibration.

Description

[Technical Field]

[0001] The present disclosure relates to a gating camera. [Background technology]

[0002] For autonomous driving and automatic control of headlamp light distribution, object identification systems are used to sense the position and type of objects around the vehicle. Object identification systems include sensors and a processing unit that analyzes the sensor output. Sensors are selected from cameras, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), millimeter-wave radar, ultrasonic sonar, etc., taking into account the application, required accuracy, and cost.

[0003] A typical monocular camera cannot obtain depth information, making it difficult to separate overlapping objects located at different distances.

[0004] A TOF (Time Of Flight) camera is known as a camera that can obtain depth information. A TOF camera projects infrared light using a light-emitting device, measures the time of flight until the reflected light returns to an image sensor, and obtains a TOF image by converting the time of flight into distance information.

[0005] A gating camera (also called a gated camera) has been proposed as an active sensor to replace the TOF camera (Patent Documents 1 and 2). A gating camera divides the imaging range into multiple ranges and captures images by changing the exposure timing and exposure time for each range. This allows a slice image to be obtained for each target range, and each slice image contains only the objects included in the corresponding range.

[0006] In active sensors such as distance measuring sensors and gating cameras, the time difference between the light emission timing of a light emitting device and the exposure timing of a light receiving device needs to be accurately calibrated. Patent Documents 1 to 3 disclose techniques related to calibration. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2020-60433 [Patent Document 2] Japanese Patent Application Publication No. 2020-85477 [Patent Document 3] Japanese Patent Publication No. 2020-148512 Summary of the Invention [Problem to be solved by the invention]

[0008] The technologies of Patent Documents 1 to 3 are based on ToF sensors and are premised on the existence of hardware for measuring the time of flight, and therefore cannot be applied to gating cameras.

[0009] Patent Document 1 discloses a method for calibrating a distance measurement system mounted on a small electronic device. Specifically, the electronic device is placed on a desk or the like, and the surface of the desk is used as a reflector. The application of this technique is limited to small electronic devices, and it cannot be used for sensors for vehicles, where reflectors are not always present at the same distance.

[0010] In Patent Document 2, an optical distance measuring device is equipped with a built-in reflector that reflects light emitted from the light-emitting section back to the light-receiving section. With this technology, part of the light emitted from the light-emitting section is blocked by the reflector, or only light reflected by the reflector is incident on part of the light-receiving section. In other words, part of the hardware is allocated to calibration, and cannot be used during normal shooting, resulting in part of the hardware (or part of the energy) being wasted.

[0011] Patent Document 3 discloses a technology in which a portion of the light emitted from a light source is made incident on an image sensor by a light guide and an optical fiber. As with Patent Document 2, a portion of the image sensor is allocated for calibration, which means that it cannot be used during normal shooting, resulting in a waste of hardware.

[0012] The present disclosure has been made in this context, and one exemplary purpose of an embodiment thereof is to provide a gating camera that can be calibrated. [Means for solving the problem]

[0013] One aspect of the present disclosure relates to a gating camera that divides a field of view into multiple ranges in the depth direction and generates multiple slice images corresponding to the multiple ranges. The gating camera includes a controller capable of generating a light emission control signal and a first exposure control signal, an illumination device that irradiates probe light in response to the light emission control signal during normal imaging, an image sensor that performs exposure in response to the first exposure control signal, and a calibration light source that irradiates calibration light onto the image sensor in response to the light emission control signal during calibration. During calibration, the controller sweeps the time difference between the light emission control signal and the first exposure control signal to obtain the time difference at which the pixel value of the image sensor increases. [Effects of the Invention]

[0014] According to certain aspects of the present disclosure, calibration of a gating camera is possible. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram of a sensing system according to an embodiment. [Figure 2] 10A and 10B are diagrams illustrating the operation of normal photography by a gating camera. [Figure 3] 3(a) and (b) are diagrams illustrating slice images obtained by a gating camera. [Figure 4] FIG. 2 is a diagram illustrating calibration according to the first embodiment. [Figure 5] FIG. 10 is a diagram showing the relationship between the time difference τ and the pixel value Pa of the pixel of interest. [Figure 6] FIG. 10 is a diagram illustrating calibration according to the second embodiment. [Figure 7] FIG. 2 is a diagram illustrating a first exposure and a second exposure. [Figure 8] 8(a) to 8(c) are diagrams illustrating the first exposure and the second exposure. [Figure 9] 9(a) and 9(b) are block diagrams of the illumination device and the calibration light source. [Figure 10] FIG. 1 is a block diagram of a sensing system. [Figure 11] 11(a) and (b) are diagrams showing a car equipped with a gating camera. [Figure 12] 1 is a block diagram showing a vehicle lamp equipped with a sensing system. DETAILED DESCRIPTION OF THE INVENTION

[0016] A summary of some exemplary embodiments of the present disclosure is provided. This summary is intended to provide a simplified overview of some concepts of one or more embodiments in order to provide a basic understanding of the embodiments as a prelude to the more detailed description that follows. It is not intended to limit the scope of the invention or disclosure. Furthermore, this summary is not an exhaustive overview of all possible embodiments, nor does it limit essential elements of the embodiments. For convenience, the term "one embodiment" may refer to one embodiment (example or variant) or multiple embodiments (examples or variants) disclosed herein.

[0017] A gating camera according to one embodiment divides a field of view into a plurality of ranges in the depth direction and generates a plurality of slice images corresponding to the plurality of ranges. The gating camera includes a controller capable of generating an emission control signal and a first exposure control signal, an illumination device that irradiates probe light in response to the emission control signal during normal imaging, an image sensor that performs exposure in response to the first exposure control signal, and a calibration light source that irradiates calibration light onto the image sensor in response to the emission control signal during calibration. During calibration, the controller sweeps the time difference between the emission control signal and the first exposure control signal and monitors changes in pixel values ​​of the image sensor at each time difference.

[0018] This configuration allows for timing error calibration in a gating camera that does not have hardware for measuring time-of-flight. Furthermore, by providing a light source for calibration in addition to the light source used during normal imaging, it becomes possible to capture images using all pixels of the image sensor during normal imaging, and the probe light generated by the illumination device is not blocked, reducing hardware waste.

[0019] In one embodiment, the controller may obtain the value of the time difference when the pixel value increases relatively.

[0020] In one embodiment, the controller may generate a second exposure control signal during a period when the image sensor cannot detect calibration light during calibration. The controller may obtain a time difference during which a pixel value obtained in response to the first exposure control signal is corrected by a pixel value obtained in response to the second exposure control signal. The second exposure control signal allows ambient light to be detected and the influence of ambient light to be reduced, thereby improving calibration accuracy. This configuration is particularly effective for vehicle sensors, where ambient light cannot be blocked during calibration.

[0021] In one embodiment, the second exposure control signal may be generated each time the time difference is changed, which can improve calibration accuracy when the ambient light fluctuates over time.

[0022] In one embodiment, the second exposure control signal may be generated together with the first exposure control signal, which means that the influence of ambient light can be further reduced by capturing ambient light each time an exposure for capturing calibration light is performed.

[0023] In one embodiment, the image sensor is a multi-tap image sensor, and may capture an image using a first tap in response to a first exposure control signal, and capture an image using a second tap in response to a second exposure control signal.

[0024] In one embodiment, the illumination device may include a laser diode and the calibration light source may include a light emitting diode. By using a light emitting diode instead of a laser diode as the calibration light source, costs can be reduced.

[0025] In one embodiment, the illumination device and the calibration light source may share a driving circuit.

[0026] In one embodiment, the controller may monitor multiple pixel values ​​of the image sensor and obtain a time difference for each pixel value, and if there is a timing error for each pixel of the image sensor, the time difference for each pixel can be calibrated.

[0027] In one embodiment, the controller may monitor pixel values ​​in a predetermined region of the image sensor and obtain the time difference at which the pixel values ​​increase, which is suitable when the timing error from pixel to pixel is negligible.

[0028] In one embodiment, the controller may monitor a plurality of pixel values ​​of the image sensor and obtain a time difference at which a representative value based on the plurality of pixel values ​​increases.

[0029] (Embodiment) Preferred embodiments will be described below with reference to the drawings. The same or equivalent components, parts, and processes shown in each drawing will be given the same reference numerals, and redundant explanations will be omitted as appropriate. Furthermore, the embodiments are examples and do not limit the disclosure and invention, and all features and combinations thereof described in the embodiments are not necessarily essential to the disclosure and invention.

[0030] (Embodiment) 1 is a block diagram of a sensing system 10 according to an embodiment. The sensing system 10 is mounted on a vehicle such as an automobile or a motorcycle, and detects an object OBJ that exists around the vehicle.

[0031] The sensing system 10 mainly includes a gating camera 20. The gating camera 20 includes an illumination device 22, an image sensor 24, a controller 26, a processing device 28, and a calibration light source 30. The imaging by the gating camera 20 is performed by scanning a field of view in the depth direction at a plurality of N (N≧2) ranges RNG1 to RNG2. N Adjacent ranges may overlap in the depth direction at their boundaries.

[0032] In addition to normal photography, the sensing system 10 is capable of calibration. First, the hardware and functions related to normal photography will be described.

[0033] The lighting device 22 is used for normal photography, and irradiates the area ahead of the vehicle with probe light L1 in synchronization with a light emission control signal S1 provided by the controller 26. The probe light L1 is preferably infrared light, but is not limited to this and may be visible light or ultraviolet light having a predetermined wavelength.

[0034] The image sensor 24 includes a plurality of pixels, and is capable of exposure control in synchronization with an exposure control signal S2 given from the controller 26, generating a raw image including a plurality of pixels. This image sensor 24 is used for both normal photography and calibration. The image sensor 24 is sensitive to the same wavelength as the probe light L1, and captures the reflected light (return light) L2 reflected by the object OBJ. The i-th range RNG i The slice image IMG_RAW generated by the image sensor 24 i The slice images IMGs, which are the final output of the gating camera 20, are referred to as raw images or primary images as needed. i Distinguish from.

[0035] The controller 26 generates a light emission control signal S1 and an exposure control signal S2, and controls the irradiation timing (light emission timing) of the probe light L1 by the illumination device 22 and the exposure timing by the image sensor 24. The controller 26 can be implemented as a combination of a processor (hardware) such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or microcomputer, and a software program executed by the processor (hardware).

[0036] The image sensor 24 and the processing device 28 are connected via a serial interface, etc. The processing device 28 receives the raw image IMG_RAW from the image sensor 24. i Receive sliced ​​images (IMGs) i Generate.

[0037] Since the gating camera 20 captures the reflected light from a distant object, a sufficient image may not be obtained with a single capture (set of light emission and exposure). i For each range, shooting may be repeated N times (N≧2). In this case, one range RNG i For N raw images IMG_RAW i1 ~IMG_RAW iNThe processing unit 28 generates one range RNG. i For N raw images IMG_RAW i1 ~IMG_RAW iN The images are combined into a single slice image (IMGs) i may be generated.

[0038] The controller 26 and the processing unit 28 may be configured as the same hardware, or may be realized by a combination of a microcontroller and a software program, for example.

[0039] The above is the configuration and functions for normal photography. Next, normal photography using the gating camera 20 will be described.

[0040] 2 is a diagram illustrating the normal shooting operation of the gating camera 20. In FIG. 2, the i-th range RNG i The figure shows how the range RNG is sensed from the gating camera 20. The lighting device 22 emits light for a light emission period τ1 between times t0 and t1 in synchronization with the light emission control signal S1. The top row shows a diagram of light rays with time on the horizontal axis and distance on the vertical axis. i The distance to the boundary in front of MINi、 Range RNG i The distance to the inner boundary of MAXi Let's say.

[0041] Light that leaves the lighting device 22 at a certain time travels a distance d MINi The round trip time T MINi teeth, T MINi =2×d MINi / c where c is the speed of light.

[0042] Similarly, light that leaves the lighting device 22 at a certain time travels a distance d MAXi The round trip time TMAXi teeth, T MAXi =2×d MAXi / c is.

[0043] Range RNG i When it is desired to photograph only the object OBJ included in the time t2=t0+T MINi Exposure begins at time t3 = t1 + T MAXi An exposure control signal S2 is generated so as to terminate the exposure at this time. This is one exposure operation.

[0044] i-th range RNG i When capturing an image, light emission and exposure may be performed multiple times, N. In this case, the controller 26 may repeat the above-described exposure operation multiple times at a predetermined cycle τ2.

[0045] 3(a) and (b) are diagrams illustrating slice images obtained by the gating camera 20. In the example of FIG. 3(a), an object (pedestrian) OBJ2 exists in range RNG2, and an object (vehicle) OBJ3 exists in range RNG3. FIG. 3(b) shows multiple slice images IMG1 to IMG3 obtained in the situation of FIG. 3(a). When capturing slice image IMG1, the image sensor is exposed only by reflected light from range RNG1, and therefore no object image is captured in slice image IMG1.

[0046] When capturing slice image IMG2, the image sensor is exposed only to the light reflected from range RNG2, so only object image OBJ2 appears in slice image IMG2. Similarly, when capturing slice image IMG3, the image sensor is exposed only to the light reflected from range RNG3, so only object image OBJ3 appears in slice image IMG3. In this way, with gating camera 20, it is possible to capture separate images of objects for each range.

[0047] The above is normal photography using the gating camera 20. Next, the configuration and functions related to the calibration of the gating camera 20 will be described. Returning to FIG. 1, the calibration may be performed using the ignition on as a trigger, or may be performed at any timing while driving.

[0048] The calibration light source 30 is activated during calibration and irradiates the image sensor 24 with calibration light L3 in response to a light emission control signal S1 generated by the controller 26.

[0049] The difference ΔT between the delay time Ta from the assertion of the light emission control signal S1 to the lighting device 22 emitting light during normal shooting and the delay time Tb from the assertion of the light emission control signal S1 to the lighting device 22 emitting light during calibration is assumed to be known.

[0050] During calibration, the controller 26 sweeps the time difference τ between the light emission control signal S1 and the exposure control signal S2, and monitors changes in the pixel values ​​of one or more pixels (referred to as target pixels) of the image sensor 24. For example, the controller 26 determines the time difference τ when a relatively large pixel value is obtained. CAL Obtain the time difference τ CAL The determination can be made in the controller 26 or the processing unit 28.

[0051] Next, the calibration operation will be explained based on several examples.

[0052] Example 1 4 is a diagram illustrating calibration according to the first embodiment. In the first embodiment, attention is paid to only one pixel (referred to as a pixel of interest) in the raw image IMG_RAW generated by the image sensor 24. The position of the pixel of interest is not limited, but may be the center of the image sensor 24.

[0053] For simplicity, the time difference τ between the light emission control signal S1 and the exposure control signal S2 is set to five levels (τ-2 ,τ -1 ,τ0,τ1,τ2). In practice, the time difference τ can be varied in finer steps and by a larger number of steps.

[0054] L3a represents the departure time of the calibration light L3 from the calibration light source 30, and L3b represents the arrival time of the calibration light L3 at the image sensor 24. There is a delay time Tb between the assertion of the light emission control signal S1 and the emission timing (departure time) of the calibration light source 30.

[0055] Furthermore, there is a propagation delay Tc of the calibration light L3 between the departure time (L3a) and the arrival time (L3b) of the calibration light L3. This propagation delay Tc is determined according to the distance between the calibration light source 30 and the image sensor 24.

[0056] Furthermore, IS represents the exposure period of the calibration light source 30. There is also a delay time Td between the assertion of the exposure control signal S2 and the actual start of exposure of the image sensor 24.

[0057] Ignoring the effects of noise and ambient light, the pixel value Pa of the pixel of interest will be zero when the arrival time L3b of the calibration light L3 is outside the exposure period IS of the image sensor 24. On the other hand, when the arrival time L3b of the calibration light L3 is included in the exposure period IS of the image sensor 24, the pixel value of the pixel of interest Pa will increase.

[0058] 5 is a diagram showing the relationship between the time difference τ and the pixel value Pa of the pixel of interest. The horizontal axis represents the time difference τ, and the vertical axis represents the pixel value. When the time difference τ is swept, CAL (In the example of Figure 4, τ = τ -1 ), the pixel value Pa of interest increases. The controller 26 calculates the time difference τ CAL Get.

[0059] Time difference τ CALThere is no particular limitation on how to determine τ. For example, the time difference τ when the pixel value Pa is at its maximum value is determined as τ CAL Alternatively, the time difference when the value obtained by differentiating the pixel value Pa with respect to the time difference τ on the horizontal axis exceeds a predetermined value may be expressed as τ CAL It may also be possible to use the following.

[0060] The controller 26 calculates this time difference τ CAL By using this, the time difference between the light emission control signal S1 and the exposure control signal S2 during normal photography can be corrected.

[0061] In this way, timing errors can be calibrated in a gating camera that does not have hardware for measuring time-of-flight. Furthermore, by providing a light source for calibration in addition to the light source used during normal imaging, it becomes possible to capture images using all pixels of the image sensor 24 during normal imaging, and the probe light L1 generated by the illumination device 22 is not blocked, thereby reducing wasted hardware.

[0062] Example 2 6 is a diagram illustrating calibration according to the second embodiment. In the second embodiment, the same time difference τ j For (j=-2,-1,0,1,2), light emission and exposure are repeated multiple times M times. As a result, one time difference τ j Regarding M pixel values ​​Pa j The controller 26 calculates the M pixel values ​​Pa j The pixel value P obtained by adding or averaging j The time difference τ when increases j Get.

[0063] Example 3 During calibration, if ambient light of a magnitude that cannot be ignored relative to the calibration light L3 enters the image sensor 24, the accuracy of the calibration will decrease.

[0064] Therefore, in the second embodiment, in addition to the exposure (first exposure) for detecting the calibration light L3, an exposure (second exposure) for measuring only the ambient light is performed.

[0065] FIG. 7 is a diagram illustrating the first exposure and the second exposure. The first exposure is performed during a period in which the image sensor 24 can detect the calibration light L3, and the exposure control signal S2 for the first exposure is referred to as the first exposure control signal S2a. The first exposure control signal S2a is the exposure control signal S2 in the first embodiment. The second exposure is performed during a period in which the image sensor 24 cannot detect the calibration light L3. The exposure control signal S2 for the second exposure is referred to as the second exposure control signal S2b. In other words, the second exposure control signal S2b is asserted at a timing sufficiently separated from the light emission control signal S1.

[0066] The controller 26 (or the processing device 28) corrects the pixel value Pa obtained in response to the first exposure control signal S2a with the pixel value Pb obtained in response to the second exposure control signal S2b. The controller 26 calculates the time difference τ CAL The simplest way is to subtract Pb from Pa to generate the corrected pixel value Pa' (=Pa-Pb).

[0067] 8(a) to 8(c) are diagrams for explaining the first exposure and the second exposure. As shown in FIG. 8(a), if the intensity of the ambient light does not change over time, the second exposure may be performed once during one calibration. The pixel value Pa obtained in the first exposure -2 ~Pa2 can be corrected using the pixel value Pb obtained in the second exposure. For example, Pa j Subtract Pb from the corrected Pa j ' may be calculated.

[0068] In reality, the intensity of ambient light often fluctuates over time. Therefore, in order to measure the ever-changing ambient light, the second exposure may be performed multiple times during one calibration, as shown in FIG. 8(b). For example, the second exposure may be performed at intervals of a time difference τ. In this case, the time difference τ-2 ,τ -1 , τ0, τ1, τ2, the pixel value Pa based on the first exposure -2 ,Pa -1 , Pa0, Pa1, Pa2 are obtained, and further, pixel values ​​Pb -2 ,Pb -1 , Pb0, Pb1, Pb2 are obtained. When j=-2,-1,0,1,2, the correction is i to Pb i This can be done by subtracting

[0069] In FIG. 8(c), as explained in FIG. 6 (Example 2), one time difference τ j In FIG. 8(c), the first exposure is performed multiple times M times. j The operation of is shown.

[0070] In this case, it is advisable to perform the second exposure every time the first exposure is performed. j and Pb j Each pixel value Pa j the corresponding pixel value Pb j By correcting using j ' is generated. M Pa j By processing the pixel value P j The controller 26 generates the pixel value P j The time difference τ when increases j Get.

[0071] The image sensor 24 may be a multi-tap CMOS sensor having multiple floating diffusions for each pixel. In this case, the image sensor 24 is a multi-tap image sensor, and may capture an image using a first tap in response to the first exposure control signal S2a and a second tap in response to the second exposure control signal S2b.

[0072] Next, specific configuration examples of the illumination device 22 and the calibration light source 30 will be described.

[0073] 9(a) and (b) are block diagrams of the illumination device 22 and the calibration light source 30. In FIG. 9(a), the illumination device 22 includes a semiconductor light-emitting element 22a and its drive circuit 22b. Since the semiconductor light-emitting element 22a needs to irradiate light far away from the vehicle, a high-intensity, highly directional laser diode is suitable. The drive circuit 22b supplies a drive current I to the semiconductor light-emitting element 22a in response to a light emission control signal S1. LD The driving circuit 22b is not particularly limited in configuration, and a known laser driver can be used.

[0074] Similarly, the calibration light source 30 includes a semiconductor light emitting element 30a and its drive circuit 30b. The semiconductor light emitting element 30a only needs to illuminate the nearby image sensor 24, so high output and directionality are not required, and therefore a light emitting diode is suitable. Note that a laser diode may also be used as the semiconductor light emitting element 30a.

[0075] In response to the light emission control signal S1, the drive circuit 30b supplies a drive current I LED The drive circuit 30b is not particularly limited in configuration, and a known LED driver can be used.

[0076] 9(b), the illumination device 22 and the calibration light source 30 share the drive circuits 22b and 30b. In this case, switches SW1 and SW2 may be inserted in series with the semiconductor light emitting elements 22a and 30a, and the illumination device 22 and the calibration light source 30 may be switched by turning on one of the switches SW1 and SW2.

[0077] The present invention has been described above based on an embodiment. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each treatment process, and that such modifications are also within the scope of the present invention. These modifications will be described below.

[0078] (Variation 1) In the first modification, a plurality of adjacent pixels of the image sensor 24 are set as pixels of interest. The gating camera 20 generates a representative value from the values ​​of the plurality of pixels, and calculates the time difference τ CAL The representative value may be an average value, a sum value, a maximum value, or the like of a plurality of pixel values.

[0079] (Variation 2) The exposure timing of the image sensor 24 may vary within the plane. In this case, a plurality of target pixels are determined at separate positions on the image sensor 24, and the time difference τ CAL This allows calibration of in-plane variations in timing errors of the image sensor 24.

[0080] (Application) 10 is a block diagram of a sensing system 10. The sensing system 10 includes a processing unit 40 in addition to the gating camera 20 described above. The sensing system 10 is an object detection system that is mounted on a vehicle such as an automobile or motorcycle and determines the type (also called category or class) of an object OBJ that exists around the vehicle.

[0081] The gating camera 20 detects multiple ranges RNG1 to RNG N Multiple slice images IMGs1 to IMGs corresponding to N The i-th slice image IMGs is generated. i has a corresponding range RNG i Only objects contained within the image will be captured.

[0082] The calculation processing unit 40 calculates the multiple ranges RNG1 to RNG2 obtained by the gating camera 20. N Multiple slice images IMGs1 to IMGs corresponding to NThe computing device 40 is configured to be able to identify the type of object based on the above. The computing device 40 includes a classifier 42 implemented based on a trained model generated by machine learning. The computing device 40 may include multiple classifiers 42 optimized for each range. The algorithm of the classifier 42 is not particularly limited, but may be, for example, You Only Look Once (YOLO), Single Shot Multibox Detector (SSD), R-CNN (Region-based Convolutional Neural Network), Spatial Pyramid Pooling (SPPnet), Faster R-CNN, Deconvolution-SSD (DSSD), or Mask R-CNN, or may be an algorithm developed in the future.

[0083] The arithmetic processing device 40 can be implemented as a combination of a processor (hardware) such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or microcomputer, and a software program executed by the processor (hardware). The arithmetic processing device 40 may be a combination of multiple processors. Alternatively, the arithmetic processing device 40 may be configured solely from hardware. The functions of the arithmetic processing device 40 and the functions of the processing device 28 may be implemented in the same processor.

[0084] 11(a) and (b) are diagrams showing an automobile 300 equipped with a gating camera 20. Referring to Fig. 11(a), the automobile 300 is equipped with headlamps (lamp fixtures) 302L and 302R.

[0085] As shown in FIG. 11(a), the lighting device 22 of the gating camera 20 may be built into at least one of the left and right headlamps 302L, 302R. The image sensor 24 may be attached to a part of the vehicle, for example, behind the rearview mirror. Alternatively, the image sensor 24 may be attached to the front grille or front bumper. The controller 26 may be installed inside the vehicle cabin, in the engine compartment, or built into the headlamps 302L, 302R. Furthermore, the lighting device 22 may also be attached to a location other than inside the headlamps, for example, inside the vehicle cabin, the front bumper, or the front grille.

[0086] As shown in FIG. 11(b), the image sensor 24 may be built in together with the lighting device 22 in either the left or right headlamp 302L or 302R.

[0087] 12 is a block diagram showing a vehicle lamp 200 equipped with a sensing system 10. The vehicle lamp 200, together with a vehicle-side ECU 310, constitutes a lamp system 304. The vehicle lamp 200 is equipped with the lamp-side ECU 210 and a lamp unit 220. The lamp unit 220 is a low beam or high beam lamp, and is equipped with a light source 222, a lighting circuit 224, and an optical system 226. The vehicle lamp 200 is further provided with a sensing system 10.

[0088] Information about the object OBJ detected by the sensing system 10 may be used for light distribution control of the vehicle lamp 200. Specifically, the lamp-side ECU 210 generates an appropriate light distribution pattern based on information about the type and position of the object OBJ generated by the sensing system 10. The lighting circuit 224 and the optical system 226 operate to obtain the light distribution pattern generated by the lamp-side ECU 210. The arithmetic processing device 40 of the sensing system 10 may be provided outside the vehicle lamp 200, i.e., on the vehicle side.

[0089] Furthermore, information about the object OBJ detected by the sensing system 10 may be transmitted to the vehicle-side ECU 310. The vehicle-side ECU 310 may use this information for automatic driving or driving assistance.

[0090] The embodiments merely illustrate one aspect of the principles and applications of the present invention, and many modifications and changes in arrangement are permitted to the embodiments as long as they do not deviate from the spirit of the present invention as defined in the claims. [Industrial Applicability]

[0091] The present disclosure can be used in sensing technology. [Explanation of symbols]

[0092] L1...probe light, S1...light emission control signal, L2...reflected light, S2...exposure control signal, S2a...first exposure control signal, S2b...second exposure control signal, L3...calibration light, 10...sensing system, 20...gating camera, 22...illumination device, 24...image sensor, 26...controller, 28...processing device, 30...calibration light source, 40...processing device, 42...classifier, 200...vehicle lamp, 210...lamp-side ECU, 220...lamp unit, 222...light source, 224...lighting circuit, 226...optical system, 300...automobile, 302L...headlamp, 304...lamp system, 310...vehicle-side ECU

Claims

1. A gating camera that divides a field of view into a plurality of ranges in a depth direction and generates a plurality of slice images corresponding to the plurality of ranges, a controller capable of generating a light emission control signal and a first exposure control signal; an illumination device that irradiates probe light in response to the light emission control signal during normal photography; an image sensor that is exposed in response to the first exposure control signal; a calibration light source that irradiates the image sensor with calibration light in response to the light emission control signal during calibration; Equipped with The gating camera is characterized in that, during the calibration, the controller sweeps the time difference between the light emission control signal and the first exposure control signal, monitors multiple pixel values ​​of the image sensor at each time difference, and obtains, for each pixel value, the time difference when the pixel value increases.

2. the controller generates a second exposure control signal during a period in which the image sensor cannot detect the calibration light during the calibration; 2. The gating camera according to claim 1, wherein the controller acquires the time difference when the value obtained by correcting the pixel value obtained in response to the first exposure control signal with the pixel value obtained in response to the second exposure control signal increases.

3. 3. The gating camera according to claim 2, wherein the second exposure control signal is generated each time the time difference is changed.

4. 3. The gating camera according to claim 2, wherein the second exposure control signal is generated together with the first exposure control signal.

5. 5. The gating camera according to claim 2, wherein the image sensor is a multi-tap image sensor, and captures an image using a first tap in response to the first exposure control signal, and captures an image using a second tap in response to the second exposure control signal.

6. the illumination device includes a laser diode; 6. The gating camera according to claim 1, wherein the calibration light source includes a light emitting diode.

7. 7. The gating camera according to claim 1, wherein the illumination device and the calibration light source share a driving circuit.

8. 8. The gating camera according to claim 1, wherein the controller monitors pixel values ​​within a predetermined region of the image sensor and obtains the time difference when the pixel values ​​increase.

9. 8. A gating camera according to claim 1, wherein the controller monitors a plurality of pixel values ​​of the image sensor and acquires the time difference when a representative value based on the plurality of pixel values ​​increases.

10. 10. The gating camera according to claim 1, which is mounted on a vehicle.

11. A gating camera according to any one of claims 1 to 9; a processor for processing the plurality of slice images captured by the gating camera; A vehicle sensing system comprising:

12. A vehicle lamp comprising the gating camera according to any one of claims 1 to 9.

Images