Gating cameras, vehicle sensing systems, vehicle lighting fixtures

Abstract

An illumination device 110 projects pulse illumination light L1 to a visual field. In a multitap image sensor 120, one pixel includes a plurality of floating diffusion (FD) regions. A camera controller 130 controls the light emission timing of the illumination device 110, and the exposure timing of the image sensor 120. A pulse exposure region fdp, which is one of the plurality of FD regions, is allocated for generation of a slice image SIMG, and a continuous exposure region fdc, which is another one of the FD regions, is allocated for generation of a normal image NIMG. The image sensor 120 executes, with reflection light of the pulse illumination light L1 from the visual field, multiple exposures by using the pulse exposure region fdp to thereby generate the slice image SIMG, and exposure by using the continuous exposure region fdc in a segment where the pulse exposure region fdp is not used to thereby generate the normal image NIMG.

Description

[Technical Field]

[0001] The present invention relates to a gating camera. [Background technology]

[0002] For autonomous driving and automatic control of headlamp light distribution, object identification systems are used to sense the position and type of objects around the vehicle. Object identification systems include sensors and a processing unit that analyzes the sensor output. Sensors are selected from cameras, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), millimeter-wave radar, ultrasonic sonar, etc., taking into account the application, required accuracy, and cost.

[0003] A typical monocular camera cannot obtain depth information, making it difficult to separate overlapping objects located at different distances.

[0004] A TOF (Time Of Flight) camera is known as a camera that can obtain depth information. A TOF camera projects infrared light using a light-emitting device, measures the time of flight until the reflected light returns to an image sensor, and obtains a TOF image by converting the time of flight into distance information.

[0005] A gating camera (also called a gated camera) has been proposed as an active sensor to replace the TOF camera (Patent Documents 1 and 2). A gating camera divides the imaging range into multiple ranges and captures images by changing the exposure timing and exposure time for each range. This allows a slice image to be obtained for each target range, and each slice image contains only the objects included in the corresponding range. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-257981 [Patent Document 2] International Publication WO2017 / 110417A1 Summary of the Invention [Problem to be solved by the invention]

[0007] In order to obtain an image (called a normal image) that is not divided into ranges like a normal camera using a gating camera, it is necessary to capture slice images in multiple ranges and then combine them.

[0008] In order to improve the depth resolution of a gating camera, in other words, to shorten the depth length (depth) of the range, it is necessary to shorten the pulse width and exposure time of the illumination light. In that case, to obtain a sufficiently bright slice image, the number of irradiation / exposure sets must be increased, which increases the time it takes to generate one slice image. Furthermore, shortening the depth of one range means that the number of ranges increases, which further increases the shooting time for all ranges.

[0009] An aspect of the present disclosure has been made in light of such a situation, and one exemplary purpose thereof is to provide a gating camera that can reduce the time required to generate a normal image. [Means for solving the problem]

[0010] One aspect of the present disclosure relates to a gating camera that divides a field of view into multiple ranges in the depth direction and generates multiple slice images corresponding to the multiple ranges. The gating camera includes an illumination device that irradiates the field of view with pulsed illumination light, a multi-tap image sensor in which each pixel has multiple FD (Floating Diffusion) regions, and a camera controller that controls the light emission timing of the illumination device and the exposure timing of the image sensor. One of the multiple FD regions is assigned as a pulsed exposure region for generating slice images, and another of the multiple FD regions is assigned as a continuous exposure region for generating normal images. The image sensor generates slice images by performing multiple exposures of pulsed illumination light reflected from the field of view using the pulsed exposure region, and generates normal images by performing exposure using the continuous exposure region in sections that do not use the pulsed exposure region. [Effects of the Invention]

[0011] According to an aspect of the present disclosure, the time required to generate a normal image can be reduced. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram of a sensing system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the basic operation of a gating camera. [Figure 3] 3(a) and (b) are diagrams illustrating slice images obtained by a gating camera. [Figure 4] 10 is a time chart illustrating generation of a slice image SIMG and a normal image NIMG by a gating camera. [Figure 5] 10 is a timing chart of sensing according to Modification 1. [Figure 6] 10 is a timing chart of sensing according to Modification 2. [Figure 7] FIG. 10 is a block diagram of a sensing system including a gating camera according to a second embodiment. [Figure 8]8 is a time chart illustrating the operation of the gating camera of FIG. 7. [Figure 9] 10A and 10B are diagrams illustrating the operation of the gating camera according to the third embodiment. [Figure 10] FIG. 10 is a circuit diagram of an image sensor used in a gating camera according to a fourth embodiment. [Figure 11] FIG. 1 is a block diagram of a sensing system. [Figure 12] 12(a) and 12(b) are diagrams showing a car equipped with a gating camera. [Figure 13] 1 is a block diagram showing a vehicle lamp equipped with a sensing system. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Outline of the embodiment) A summary of some exemplary embodiments of the present disclosure is provided. This summary is intended to provide a simplified overview of some concepts of one or more embodiments in order to provide a basic understanding of the embodiments as a prelude to the more detailed description that follows. It is not intended to limit the scope of the invention or disclosure. Furthermore, this summary is not an exhaustive overview of all possible embodiments, nor does it limit essential elements of the embodiments. For convenience, the term "one embodiment" may refer to one embodiment (example or variant) or multiple embodiments (examples or variants) disclosed herein.

[0014] A gating camera according to one embodiment divides a field of view into multiple ranges in the depth direction and generates multiple slice images corresponding to the multiple ranges. The gating camera includes an illumination device that irradiates the field of view with pulsed illumination light, a multi-tap image sensor in which each pixel has multiple FD (Floating Diffusion) regions, and a camera controller that controls the light emission timing of the illumination device and the exposure timing of the image sensor. One of the multiple FD regions is assigned as a pulsed exposure region for generating slice images, and another of the multiple FD regions is assigned as a continuous exposure region for generating normal images. The image sensor generates slice images by performing multiple exposures of pulsed illumination light reflected from the field of view using the pulsed exposure region, and generates normal images by performing exposure using the continuous exposure region in sections that do not use the pulsed exposure region.

[0015] To generate a slice image, it is necessary to repeatedly apply pulsed illumination light and synchronized short-term exposure. By accumulating electric charge in the continuous exposure area during periods when slice image exposure is not being performed, a normal image that is not divided into multiple ranges can be captured in a short time.

[0016] In one embodiment, the image sensor may be configured to read out each FD region at an independent timing, which can further reduce the time required to capture a normal image.

[0017] In one embodiment, the illumination device may be capable of irradiating the field of view with continuous illumination light in addition to pulsed illumination light. By exposing the field of view with continuous illumination light using a continuous exposure area, normal images across multiple ranges can be captured in an even shorter time.

[0018] In one embodiment, the lighting device may illuminate the field of view with continuous illumination during nighttime photography, and may turn off the continuous illumination during the daytime when sunlight is present, thereby reducing power consumption.

[0019] In one embodiment, there may be multiple continuous exposure areas. The image sensor may perform exposure using multiple continuous exposure areas in a time-division manner during a continuous exposure period in which a pulsed exposure area is not used, to generate a normal image. The multiple continuous exposure areas may have different exposure times. This allows images with a wide dynamic range to be captured when there is a large difference in brightness in the field of view by using images captured using multiple continuous exposure areas.

[0020] In one embodiment, the image sensor may generate slice images by pixel binning and generate normal images by dot-by-dot readout, thereby increasing the generation rate of slice images at the expense of reduced resolution, while high-resolution images can be obtained by dot-by-dot readout of normal images.

[0021] In one embodiment, the image sensor may be capable of binning two rows and two columns of pixels into a virtual pixel. Each pixel may include m (m≧1) pulsed exposure regions and n (n≧1) continuous exposure regions. The image sensor may include m first readout circuits and n second readout circuits. The m first readout circuits may correspond to the m pulsed exposure regions, and the n second readout circuits may correspond to the n continuous exposure regions. The i-th (1≦i≦m) first readout circuit may be capable of adding and reading out signals from the i-th pulsed exposure region of each of four pixels included in the virtual pixel, and the j-th (1≦j≦n) second readout circuit may be capable of reading out signals from the j-th continuous exposure region included in the corresponding pixel. With this configuration, a slice image can be generated by pixel binning, and a normal image can be generated dot-by-dot.

[0022] (Embodiment) Preferred embodiments will be described below with reference to the drawings. The same or equivalent components, parts, and processes shown in each drawing will be designated by the same reference numerals, and redundant descriptions will be omitted where appropriate. Furthermore, the embodiments are merely examples and do not limit the invention, and all features and combinations thereof described in the embodiments are not necessarily essential to the invention.

[0023] 1 is a block diagram of a sensing system 10 according to embodiment 1. This sensing system 10 is mounted on a vehicle such as an automobile or motorcycle, and detects an object OBJ that exists around the vehicle (within the field of view of the sensor).

[0024] The sensing system 10 mainly includes a gating camera 100. The gating camera 100 includes an illumination device 110, an image sensor 120, a camera controller 130, and a processing unit 140. The imaging by the gating camera 100 is performed by capturing an image in a field of view in the depth direction at a plurality of N (N≧2) ranges RNG1 to RNG2. N Adjacent ranges may overlap in the depth direction at their boundaries.

[0025] The lighting device 110 irradiates the area ahead of the vehicle with pulsed illumination light L1 in synchronization with a light emission timing signal S1 provided by the camera controller 130. The pulsed illumination light L1 is preferably infrared light, but is not limited to this and may be visible light or ultraviolet light having a predetermined wavelength.

[0026] The image sensor 120 includes a plurality of pixels px, and is capable of controlling exposure in synchronization with an exposure timing signal S2 provided by the camera controller 130, thereby generating an image made up of a plurality of pixels. The image sensor 120 is sensitive to the same wavelength as the pulsed illumination light L1, and captures reflected light (return light) L2 reflected by the object OBJ.

[0027] The camera controller 130 controls the irradiation timing (light emission timing) of the pulsed illumination light L1 by the illumination device 110 and the exposure timing by the image sensor 120. The functions of the camera controller 130 may be realized by software processing, hardware processing, or a combination of software processing and hardware processing. Specifically, the software processing is implemented by a combination of a processor (hardware) such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or microcomputer, and a software program executed by the processor (hardware). Note that the camera controller 130 may be a combination of multiple processors and software programs. Specifically, the hardware processing is implemented by hardware such as an ASIC (Application Specific Integrated Circuit), a controller IC, or an FPGA (Field Programmable Gate Array).

[0028] The image (slice image) SIMG generated by the image sensor 120 i are input to the arithmetic processing device 140. The arithmetic processing device 140 calculates a plurality of ranges RNG1 to RNG N Multiple slice images SIMG1 to SIMG N For example, the output data CAMERAOUT is a set of multiple slice images SIMG1 to SIMG N Furthermore, in this embodiment, the output data CAMERAOUT includes a normal image NIMG.

[0029] The arithmetic processing unit 140 may be implemented in the same hardware as the camera controller 130, or may be configured as separate hardware. Alternatively, some or all of the functions of the arithmetic processing unit 140 may be implemented as a processor or digital circuit built into the same module as the image sensor 120.

[0030] The above is the basic configuration of the gating camera 100. Next, the operation will be explained.

[0031] 2 is a diagram illustrating the basic operation of the gating camera 100. In FIG. 2, the i-th range RNG i The illumination device 110 emits light for a light emission period τ1 between times t0 and t1 in synchronization with the light emission timing signal S1. The top row shows a diagram of light rays with time on the horizontal axis and distance on the vertical axis. The gating camera 100 senses the range RNG i The distance to the boundary in front of MINi , Range RNG i The distance to the inner boundary of MAXi Let's say.

[0032] Light that leaves the lighting device 110 at a certain time travels a distance d MINi The round trip time T MINi teeth, T MINi =2×d MINi / c where c is the speed of light.

[0033] Similarly, light that leaves the lighting device 110 at a certain time travels a distance d MAXi The round trip time T MAXi teeth, T MAXi =2×d MAXi / c is.

[0034] Range RNG i When it is desired to photograph only the object OBJ included in the time t2=t0+T MINi Exposure begins at time t3 = t1 + T MAXi An exposure timing signal S2 is generated so that exposure is ended at this timing. This is one sensing operation.

[0035] i-th range RNG iThe sensing operation includes multiple sets of light emission and exposure. The camera controller 130 repeats the sensing operation multiple times at a predetermined cycle τ2.

[0036] As will be described in more detail later, the image sensor 120 is capable of multiple exposures, and multiple reflected light resulting from multiple pulse emissions is multiple-exposed onto the FD region (charge accumulation region) of each pixel px to generate a single slice image SIMG.

[0037] 3(a) and (b) are diagrams illustrating slice images obtained by the gating camera 100. In the example of FIG. 3(a), an object (pedestrian) OBJ2 exists in range RNG2, and an object (vehicle) OBJ3 exists in range RNG3. FIG. 3(b) shows multiple slice images SIMG1 to SIMG3 obtained in the situation of FIG. 3(a). When capturing slice image SIMG1, the image sensor is exposed only by reflected light from range RNG1, and therefore no object image is captured in slice image SIMG1.

[0038] When capturing slice image SIMG2, the image sensor is exposed only to the light reflected from range RNG2, so only object image OBJ2 appears in slice image SIMG2. Similarly, when capturing slice image SIMG3, the image sensor is exposed only to the light reflected from range RNG3, so only object image OBJ3 appears in slice image SIMG3. In this way, gating camera 100 allows objects to be captured separately for each range.

[0039] All ranges RNG1~RNG N After the sensing is completed, a plurality of slice images SIMG1 to SIMG N By combining these images, it is possible to generate an image similar to that taken with a normal camera (a normal image). However, in this case, it takes an extremely long time to generate one normal image.

[0040] The gating camera 100 according to this embodiment captures slice images SIMG1 to SIMG N In parallel with the generation of the normal image NIMG, a normal image NIMG that is not divided into a plurality of ranges can be generated. The generation of the normal image will be described below.

[0041] Returning to Figure 1, the image sensor 120 is a multi-tap type, and each pixel has multiple FD regions fd. Each pixel px of the image sensor 120 includes multiple FD regions, at least one of which is assigned as a pulse exposure region fdp for generating a slice image SIMG, and at least another of which is assigned as a continuous exposure region fdc for generating a normal image NIMG.

[0042] Under the control of the camera controller 130, the illumination device 110 repeatedly irradiates the field of view with pulsed illumination light L1, and the image sensor 120 performs multiple exposures of the reflected light L2 from the field of view using a pulsed exposure region fdp to generate a slice image SIMG. The exposure period using the pulsed exposure region fdp is called a pulsed exposure period Tp. The i-th range RNG i When sensing, the i-th range RNG i The reflected light L2x from the object OBJx present in the range RNG is detected by the pulse exposure area fdp, but the reflected light L2y from the object OBJy present in the other range RNG is not incident on the image sensor 120 during the pulse exposure period Tp, and is therefore not detected by the pulse exposure area fdp.

[0043] Furthermore, under the control of the camera controller 130, the image sensor 120 performs exposure using the continuous exposure region fdc in a section where the pulsed exposure region fdp is not used, and generates a normal image NIMG. The exposure period using the continuous exposure region fdc is called the continuous exposure period Tc. The continuous exposure region fdc detects reflected light L3x and L3y from objects OBJx and OBJy across the entire range. The reflected light L3 may include reflected light from the pulsed illumination light L1 and reflected light from sunlight. However, the range RNG of the detection target is iThe reflected light of the pulsed illumination light L1 from the target object is not incident on the image sensor 120 during the continuous exposure period Tc, and is therefore not detected by the continuous exposure area fdc. In other words, when capturing the normal image NIMG, the target object is within the range RNG i The object OBJ in the image is photographed using the reflected light of sunlight, not the reflected light of the pulsed illumination light L1.

[0044] The above is the configuration of the gating camera 100. Next, the operation will be explained.

[0045] 4 is a time chart for explaining the generation of a slice image SIMG and a normal image NIMG by the gating camera 100. In this example, the pixel includes three FD regions fd, two of which are pulse exposure regions fdp1 and fdp2, and two adjacent ranges RNG i ,RNG i+1 Two ranges RNG are assigned to generate the slice image SIMG. i ,RNG i+1 A set of these is called a zone. t0 represents the exposure start time of one zone, and t1 represents the exposure time.

[0046] In this time chart, a high on L1 indicates the emission of pulsed illumination light, and highs on fdp1, fdp2, and fdc indicate the exposure of each FD region. The exposure periods for pulsed exposure regions fdp1 and fdp2 are called pulse exposure periods Tp1 and Tp2, and the exposure period for continuous exposure region fdc is called continuous exposure period Tc. In addition to multiple FD regions, a pixel includes a light-receiving element such as a photodiode. Each FD region is exclusively connected to the light-receiving element during that exposure period. Furthermore, Qp1, Qp2, and Qc represent the charge amounts of FD regions fdp1, fdp2, and fdc, respectively.

[0047] The exposure timing of the pulse exposure regions fdp1 and fdp2 is determined according to the position of the range to be photographed. The calculation processing device 140 may acquire detailed distance information within the range by the indirect ToF method using the pixel values ​​of the two pulse exposure regions fdp1 and fdp2.

[0048] Range RNG i ,RNG i+1 In order to obtain a sufficiently bright slice image, that is, to accumulate a sufficient amount of charge in the pulsed exposure areas fdp1 and fdp2, it is necessary to repeat the emission of pulsed illumination light L1 and exposure by the pulsed exposure areas fdp1 and fdp2 several hundred to several hundred thousand times.

[0049] The remaining one of the three FD regions fd is a continuous exposure region fdc, which is allocated to generating the normal image NIMG. The continuous exposure region fdc is used during a period when both the pulse exposure regions fdp1 and fdp2 are unused (continuous exposure period).

[0050] During the continuous exposure period, the pixels are exposed to the reflected light L2 of the pulsed illumination light L1 by objects in the entire range of the field of view. The pixels are also exposed to the reflected light of sunlight by objects in the entire range of the field of view. As a result, the amount of charge accumulated in the continuous exposure region fdc during the continuous exposure period forms a normal image NIMG that captures the entire range of the field of view.

[0051] In this example, it is assumed that the image sensor 120 can only read out multiple FD regions at the same time. In other words, it is assumed that the charges in multiple FD regions are reset after one readout. In this case, when sensing of one zone is completed at time t1, the pulse exposure regions fdp1 and fdp2 and the continuous exposure region fdc are read out, and two slice images SIMG and one normal image NIMG can be obtained.

[0052] The operation of the gating camera 100 is as described above.

[0053] According to this gating camera 100, by using the FD region not used for generating slice images and continuously exposing to constant light that is not dependent on the pulsed illumination light L1, it is possible to generate normal images NIMG in parallel with the generation of slice images SIMG. NSince the normal image NIMG can be obtained without waiting for the completion of the imaging, the time required to generate the normal image NIMG can be shortened.

[0054] When the gating camera 100 is used during the day, sunlight acts as noise in generating slice images. Therefore, the wavelength of the pulsed illumination light L1 and the wavelength sensitivity of the image sensor 120 must be selected from a range where the spectral intensity of sunlight is weak. In other words, the image sensor 120 has low sensitivity to stationary light, which is dominant in capturing normal images NIMG. In FIG. 4, the pulsed exposure periods Tp1 and Tp2 are on the order of approximately 1 / 100 to 1 / 1000 of the emission interval of the pulsed illumination light L1. Therefore, the continuous exposure period Tc is several tens to several hundreds of times longer than the pulsed exposure periods Tp1 and Tp2. Therefore, even if the image sensor 120 has low sensitivity to sunlight, a sufficiently bright normal image can be generated.

[0055] Below, modified examples of the configuration and sensing of the gating camera 100 will be described.

[0056] 5 is a timing chart of sensing according to Modification 1. When the amount of charge sufficient to obtain a normal image with the required brightness has accumulated in the continuous exposure region fdc, without waiting for the completion of sensing for one zone, exposure of the continuous exposure region fdc may be stopped at that point. This prevents pixels from becoming saturated and causing the normal image to become overexposed.

[0057] Fig. 6 is a timing chart of sensing according to Modification 2. In Fig. 4, it is assumed that multiple FD regions of the image sensor 120 can only be read out at the same timing, but this is not limited to this. In Modification 2, the image sensor 120 is configured so that each FD region can be read out at an independent timing. In Modification 2, each time exposure of the continuous exposure region fdc is completed, the pulse exposure region fdp is read out to generate a normal image NIMG without waiting for exposure of the pulse exposure region fdp to be completed, and a new continuous exposure for generating the next normal image NIMG is started.

[0058] According to the second modification, the frame rate of the normal image NIMG can be increased compared to the case of FIG. 4 or FIG.

[0059] (Embodiment 2) As explained in the first embodiment, the range RNG to be detected i The reflected light of the pulsed illumination light L1 from the target object is not incident on the image sensor 120 during the continuous exposure period Tc, and is therefore not detected by the continuous exposure area fdc. i The object OBJ in the image is photographed using the reflected light of sunlight, not the reflected light of the pulsed illumination light L1.

[0060] 1, in order to obtain a normal image NIMG at night when there is no sunlight, sensing across two zones is required, which increases the sensing time for the normal image NIMG. In the second embodiment, a technology that solves this problem will be described.

[0061] 7 is a block diagram of a sensing system 10A including a gating camera 100A according to embodiment 2. Differences between the gating camera 100A and the gating camera 100 according to embodiment 1 will be described.

[0062] The illumination device 110A irradiates the field of view with continuous illumination light L4 in addition to pulsed illumination light L1. The rest is the same as in embodiment 1. Next, the operation of the gating camera 100A will be described.

[0063] FIG. 8 is a time chart illustrating the operation of the gating camera 100A of FIG.

[0064] Under the control of the camera controller 130, the illumination device 110A repeatedly irradiates the field of view with pulsed illumination light L1, and the image sensor 120 performs multiple exposures of the reflected light L2 from the field of view using the pulsed exposure region fdp to generate a slice image SIMG. The generation of the slice image SIMG is the same as in the first embodiment.

[0065] Furthermore, the illumination device 110A irradiates the field of view with continuous illumination light L4 under the control of the camera controller 130. The intensity of the continuous illumination light L4 is lower than the peak intensity of the pulsed illumination light L1.

[0066] Under the control of the camera controller 130, the image sensor 120 performs exposure using the continuous exposure region fdc during the continuous exposure period Tc in which the pulsed exposure region fdp is not used, and generates a normal image NIMG. The continuous exposure region fdc detects reflected light L3x and L3y from objects OBJx and OBJy across the entire range. The reflected light L3 may include reflected light of the pulsed illumination light L1 and reflected light of the continuous illumination light L4. However, within the range RNG of the detection target, i The reflected light of the pulsed illumination light L1 from the target object is not incident on the image sensor 120 during the continuous exposure period Tc, and is therefore not detected by the continuous exposure area fdc. In other words, when capturing the normal image NIMG, the target object is within the range RNG i The object OBJ in the image is photographed using the reflected light of the continuous illumination light L4, not the reflected light of the pulsed illumination light L1.

[0067] The gating camera 100A according to the second embodiment makes it possible to generate a normal image NIMG in a short time even at night when there is no sunlight.

[0068] During the daytime when sunlight is present, normal images can be captured using reflected sunlight, so irradiation of the continuous illumination light L4 can be turned off during the daytime, thereby suppressing increases in power consumption.

[0069] (Embodiment 3) In the above description, each pixel px has one continuous exposure area fdc, but this is not limited thereto, and each pixel px may have two or more continuous exposure areas fdc.

[0070] 9 is a diagram illustrating the operation of the gating camera according to embodiment 3. Under the control of the camera controller 130, the image sensor 120 performs exposure using two continuous exposure regions fdc1 and fdc2 in a time-division manner during a continuous exposure period Tc in which the pulsed exposure region fdp is not used, to generate a normal image NIMG. The exposure times of the two continuous exposure regions fdc1 and fdc2 are different, and two normal images NIMG are captured with different exposures.

[0071] According to the third embodiment, when there is a large difference in brightness in the field of view, it is easier to detect bright objects (objects with high reflectivity) and dark objects (objects with low reflectivity) by using images captured using multiple continuous exposure areas fdc1 and fdc2. Alternatively, by combining images captured using two continuous exposure areas fdc1 and fdc2, it is possible to capture a high dynamic range (HDR) image with a wide dynamic range that suppresses blown-out highlights and crushed shadows.

[0072] (Embodiment 4) 10 is a circuit diagram of an image sensor 120C used in a gating camera according to a fourth embodiment. In this example, each pixel px includes m (m≧1) pulsed exposure regions fdp and n (n≧1) continuous exposure regions fdc, for a total of m+n FD regions. In this example, m=4, n=2. Each FD region is provided with a tap, allowing signals to be read out.

[0073] That is, of the six FD regions, four FD regions are pulse exposure regions fdp1 to fdp4, and four taps TP1 to TP4 are provided therein. The remaining two FD regions are continuous exposure regions fdc1 and fdc2, and two taps TP5 and TP6 are provided therein.

[0074] Since each pixel includes four pulse exposure areas fdp1 to fdp4, four slice images corresponding to the four ranges are generated simultaneously. Also, as described in the third embodiment, two normal images NIMG are captured with different exposures using two continuous exposure areas fdc1 and fdc2.

[0075] In this embodiment, the four slice images are generated by pixel binning. Specifically, adjacent pixels (four pixels in this example) are combined to generate a virtual pixel pxbin. That is, the resolution of the slice images is lower than that of the normal image.

[0076] In FIG. 10, the i-th and (i+1)-th columns and the j-th and (j+1)-th rows are shown, and four pixels px across two rows and two columns are shown. i,j ,px i+1,j ,px i,j+1 ,px i+1,j+1 are combined by pixel binning.

[0077] The image sensor includes m (m=4) first readout circuits RO_BIN1 to RO_BIN4 for every two columns, and n (n=2) second readout circuits RO_DBD1 to RO_DBD2 for every column. The four first readout circuits RO_BIN1 to RO_BIN4 correspond to four pulse exposure regions fdp1 to fpd4 for each of the corresponding two columns of pixels. The two second readout circuits RO_DBD1 to RO_DBD2 correspond to two continuous exposure regions fdc1 to fdc2 for the corresponding column. Adjacent readout circuits RO_DBD and ROBIN can be shared, and can be used by switching between them.

[0078] The i-th (1≦i≦m) first readout circuit RO_BINi can add and read out the signals of the i-th pulse exposure region fdpi of each of the four pixels included in the virtual pixel pxbin.

[0079] The j-th (1≦j≦n) second readout circuit RO_DBDj can read out the signal of the j-th continuous exposure field fdcj included in the pixels of the corresponding column.

[0080] Furthermore, this image sensor 120 can generate a high-resolution normal image NIMG by dot-by-dot readout. On the other hand, the image sensor 120 can generate a slice image SIMG by pixel binning processing, thereby shortening the generation time at the expense of a reduction in resolution.

[0081] (Application) 11 is a block diagram of the sensing system 10. The sensing system 10 includes a processing unit 40 in addition to the gating camera 100 described above. The sensing system 10 is an object detection system that is mounted on a vehicle such as an automobile or motorcycle and determines the type (also called category or class) of an object OBJ that exists around the vehicle.

[0082] The gating camera 100 detects a plurality of ranges RNG1 to RNG N Multiple slice images IMG1 to IMG corresponding to N The output data CAMERAOUT of the gating camera 100 is generated as a plurality of slice images SIMG1 to SIMG N and includes the normal image NIMG.

[0083] The arithmetic processing device 40 is configured to be able to identify the type of object based on the output data CAMERAOUT from the gating camera 100. The arithmetic processing device 40 includes a classifier 42 implemented based on a trained model generated by machine learning. The arithmetic processing device 40 may include multiple classifiers 42 optimized for each range. The algorithm of the classifier 42 is not particularly limited, but may be, for example, You Only Look Once (YOLO), Single Shot Multibox Detector (SSD), Region-based Convolutional Neural Network (R-CNN), Spatial Pyramid Pooling (SPPnet), Faster R-CNN, Deconvolution-SSD (DSSD), or Mask R-CNN, or may employ an algorithm developed in the future.

[0084] The functions of the arithmetic processing device 40 may be realized by software processing, by hardware processing, or by a combination of software processing and hardware processing. Specifically, software processing is implemented by a combination of a processor (hardware) such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or microcomputer, and a software program executed by the processor (hardware). The arithmetic processing device 40 may also be a combination of multiple processors and software programs. Specifically, hardware processing is implemented by hardware such as an ASIC (Application Specific Integrated Circuit), a controller IC, or an FPGA (Field Programmable Gate Array). The functions of the arithmetic processing device 40 and the functions of the image processing device 140 may be implemented in the same processor.

[0085] 12(a) and (b) are diagrams showing an automobile 300 equipped with a gating camera 100. Referring to Fig. 12(a), the automobile 300 is equipped with headlamps (lamp fittings) 302L and 302R.

[0086] As shown in FIG. 12(a), the lighting device 110 of the gating camera 100 may be built into at least one of the left and right headlamps 302L, 302R. The image sensor 120 may be attached to a part of the vehicle, for example, behind the rearview mirror. Alternatively, the image sensor 120 may be provided in the front grille or front bumper. The camera controller 130 may be provided in the vehicle cabin, in the engine compartment, or built into the headlamps 302L, 302R.

[0087] As shown in FIG. 12(b), the image sensor 120 may be built in together with the lighting device 110 in either the left or right headlamp 302L or 302R.

[0088] The lighting device 110 may be provided in a part of the vehicle, for example, behind the rearview mirror, on the front grille, or on the front bumper.

[0089] 13 is a block diagram showing a vehicle lamp 200 equipped with a sensing system 10. The vehicle lamp 200, together with a vehicle-side ECU 310, constitutes a lamp system 304. The vehicle lamp 200 is equipped with the lamp-side ECU 210 and a lamp unit 220. The lamp unit 220 is a low beam or high beam lamp, and is equipped with a light source 222, a lighting circuit 224, and an optical system 226. The vehicle lamp 200 is further provided with a sensing system 10.

[0090] Information about the object OBJ detected by the sensing system 10 may be used for light distribution control of the vehicle lamp 200. Specifically, the lamp-side ECU 210 generates an appropriate light distribution pattern based on information about the type and position of the object OBJ generated by the sensing system 10. The lighting circuit 224 and the optical system 226 operate to obtain the light distribution pattern generated by the lamp-side ECU 210. The arithmetic processing device 40 of the sensing system 10 may be provided outside the vehicle lamp 200, i.e., on the vehicle side.

[0091] Furthermore, information about the object OBJ detected by the sensing system 10 may be transmitted to the vehicle-side ECU 310. The vehicle-side ECU 310 may use this information for automatic driving or driving assistance.

[0092] The embodiments are merely examples, and it will be understood by those skilled in the art that there are various variations in the combination of each component and each treatment process, and that such variations are also included within the scope of this disclosure or the present invention. [Industrial Applicability]

[0093] The present invention relates to a gating camera. [Explanation of symbols]

[0094] L1...pulsed illumination light, L2, L3...reflected light, L4...continuous illumination light, S1...light emission timing signal, S2...exposure timing signal, 10...sensing system, 40...processing device, 42...classifier, 100...gating camera, 110...illumination device, 120...image sensor, 130...camera controller, 140...processing device, 200...vehicle lamp, 210...lamp side ECU, 220...lamp unit, 222...light source, 224...lighting circuit, 226...optical system, 300...automobile, 302L...headlamp, 304...lamp system, 310...vehicle side ECU, fdp...pulsed exposure area, fdc...continuous exposure area.

Claims

1. A gating camera that divides a field of view into a plurality of ranges in a depth direction and generates a plurality of slice images corresponding to the plurality of ranges, an illumination device that irradiates the field of view with pulsed illumination light; a multi-tap image sensor in which one pixel has multiple FD (Floating Diffusion) regions; a camera controller that controls the light emission timing of the lighting device and the exposure timing of the image sensor; Equipped with one of the plurality of FD regions is assigned as a pulse exposure region for generating the slice image, and another of the plurality of FD regions is assigned as a continuous exposure region for generating a normal image; The image sensor generates a slice image by performing multiple exposures using the pulsed exposure area to the reflected light of the pulsed illumination light from the field of view, and generates the normal image by performing exposure using the continuous exposure area in a section where the pulsed exposure area is not used.

2. 2. The gating camera according to claim 1, wherein the image sensor is configured so that each of the FD regions can be read out at an independent timing.

3. 3. The gating camera according to claim 1, wherein the illumination device is capable of irradiating the field of view with continuous illumination light in addition to the pulsed illumination light.

4. 4. The gating camera according to claim 3, wherein the lighting device irradiates the field of view with the continuous lighting light during nighttime photography.

5. The gating camera of claim 1 or 2, characterized in that there are multiple continuous exposure areas, and the image sensor performs exposure using the multiple continuous exposure areas in a time-division manner during a section in which the pulse exposure area is not used, to generate the normal image, and the multiple continuous exposure areas have different exposure times.

6. the image sensor generates the slice images by pixel binning; 3. The gating camera according to claim 1, wherein the normal image is generated dot-by-dot.

7. The image sensor includes: Two rows and two columns of pixels can be binned as virtual pixels, Each pixel includes m (m≧1) pulse exposure areas and n (n≧1) continuous exposure areas, The image sensor includes: m first readout circuits; n second readout circuits; Equipped with the m first readout circuits are associated with the m pulse exposure areas, the n second readout circuits are associated with the n continuous exposure areas; the i-th (1≦i≦m) first readout circuit is capable of adding and reading out signals from the i-th pulse exposure region of each of the four pixels included in the virtual pixel; 7. The gating camera according to claim 6, wherein the jth (1≦j≦n) second readout circuit is capable of reading out a signal from the jth continuous exposure region included in the corresponding pixel.

8. 8. The gating camera according to claim 7, wherein m=4 and n=2.

9. 3. The gating camera according to claim 1, wherein the gating camera is mounted on a vehicle.

10. a gating camera according to claim 1 or 2; a processor for processing the plurality of slice images captured by the gating camera; A vehicle sensing system comprising:

11. A vehicle lamp comprising the gating camera according to claim 1 or 2.