ToF camera, vehicle sensing system, and vehicle lamp

Abstract

In a multi-tap image sensor (120), each pixel has n (n > = 2) FD (Floating Diffusion) regions, and exposure can be performed at different timings for each FD region. A camera controller (130, i) performs sensing in order from the near front to the inside over m (m > = 2) partitions, with one partition including n consecutive ranges as a unit. Furthermore, (ii) during sensing of each partition, the i-th (1 < = i < = n) FD region of the image sensor (120) is exposed by reflected light from the i-th range of the partition. The arithmetic processing device (140) corrects the pixel value of the i-th (1 < = i < = n) FD region of the j-th (2 < = j < = m) partition using information obtained in the (j-1)-th partition, and generates a range image of the i-th range of the j-th partition.

Description

Technical Field

[0001] The present disclosure relates to a ToF camera, a vehicle sensing system, and a vehicle lamp. Background Art

[0002] For autonomous driving or automatic control of headlight distribution, an object recognition system is used to sense the position and type of objects around the vehicle. The object recognition system includes a sensor and a processing unit that analyzes the output of the sensor. The sensor is selected from cameras, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), millimeter wave radar, ultrasonic sonar, etc., taking into account the purpose, required accuracy, and cost.

[0003] A general monocular camera cannot obtain depth information, so when multiple objects at different distances overlap, it is difficult to separate them.

[0004] A ToF camera is a camera that can obtain depth information. A ToF (Time of Flight) camera projects infrared light through a light-emitting device, measures the flight time until the reflected light returns to the image sensor, and obtains a ToF image that converts the flight time into distance information.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2009-257981

[0008] Patent Document 2: International Publication WO2017 / 110417A1 Summary of the invention

[0009] Problems to be solved by the invention

[0010] In the presence of fog in the field of view, the image sensor of the ToF camera receives not only reflected light from the object, but also reflected light from the fog. Figure 1 This is a diagram for explaining sensing in fog. Figure 1 The horizontal axis represents the distance from the ToF camera. Figure 1 , in order from top to bottom, the reflection intensity from fog, the reflection intensity from an object, and the pixel value of one pixel of a range image are shown.

[0011] The pixel value of the image sensor is determined by the sum of the reflected light from the fog and the reflected light from the object. Since the longer the propagation distance, the more the light is attenuated, the closer to the ToF camera, the higher the intensity of the reflected light from the fog, and the farther from the ToF camera, the lower the intensity of the reflected light from the fog.

[0012] If we look at the range where the object is located, RNG i , then the corresponding range image SIMG i The pixel value contains fog as a noise component, and the contrast is reduced.

[0013] In addition, in the case of thick fog, range image SIMG may occur at a range closer to the ToF camera. 1 The pixel value is greater than the range where the object is located RNG i Range image SIMG i Now, consider synthesizing multiple range images to generate one image (normal image). In this case, if the pixel value of the closer range is larger than the pixel value of the range where the object exists, the object will be buried in the fog and become invisible.

[0014] One of the exemplary objects of the present disclosure is to provide a ToF camera, a vehicle sensing system, and a vehicle lamp capable of performing photography with reduced influence of fog.

[0015] Means for solving problems

[0016] A certain mode of the present disclosure relates to a ToF camera, which divides the field of view into multiple ranges in the depth direction and generates multiple range images corresponding to the multiple ranges. The ToF camera includes an illumination device, a multi-tap image sensor, a camera controller, and an operation processing device. The illumination device irradiates pulse illumination light to the field of view. Regarding the multi-tap image sensor, 1 pixel has n (n≧2) FD (Floating Diffusion) areas, and can be exposed at different timings for each FD area. The camera controller controls the light emission timing of the illumination device and the exposure timing of the image sensor. The operation processing device generates multiple range images based on the output of the image sensor. The camera controller controls the illumination device and the image sensor to (i) perform sequential sensing from the front to the inside of m (m≧2) zones in a unit of a zone containing n continuous ranges, and (ii) in the sensing of each zone, expose the i-th (1≦i≦n) FD area of ​​the image sensor by the reflected light from the i-th range of the zone. The processing device corrects the pixel values ​​of the i-th (1≦i≦n) FD region of the j-th (2≦j≦m) partition using the information obtained in the j-1-th partition, and generates a range image of the i-th range of the j-th partition.

[0017] Effects of the Invention

[0018] According to one aspect of the present disclosure, the influence of fog can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a diagram for explaining sensing in fog.

[0020] Figure 2 is a block diagram of a sensing system according to an embodiment.

[0021] Figure 3 This figure explains the basic operation of a ToF camera.

[0022] Figure 4A This is a diagram for explaining a range image obtained by a ToF camera.

[0023] Figure 4B This is a diagram for explaining a range image obtained by a ToF camera.

[0024] Figure 5 This is a diagram for explaining the sensing of one partition.

[0025] Figure 6 This is a diagram for explaining the generation of a range image by the calculation processing device.

[0026] Figure 7 This is a diagram for explaining a range image generated by the calculation processing device.

[0027] Figure 8 is the block diagram of the sensing system.

[0028] Fig. 9A This is a diagram showing a car equipped with a ToF camera.

[0029] Fig. 9B This is a diagram showing a car equipped with a ToF camera.

[0030] Fig.10 : is a block diagram showing a vehicle lamp including a sensing system. DETAILED DESCRIPTION

[0031] (Overview of Embodiments)

[0032] The summary of several exemplary embodiments of the present disclosure is described. This summary serves as a preface to the detailed description to be described later. For the purpose of basic understanding of the embodiments, several concepts of one or more embodiments are simplified and described, and the breadth of the invention or disclosure is not limited. In addition, this summary is not a general summary of all conceivable embodiments, and does not limit the indispensable components of the embodiments. For convenience, "one embodiment" is sometimes used to refer to one or more embodiments disclosed in this specification. The embodiment includes an embodiment or a modified example. The ToF camera involved in one embodiment divides the field of view into multiple ranges in the depth direction and generates multiple range images corresponding to the multiple ranges. The ToF camera includes an illumination device, a multi-tap image sensor, a camera controller, and an operation processing device. The illumination device irradiates pulsed illumination light to the field of view. With respect to the multi-tap image sensor, 1 pixel has n (n≧2) FD (Floating Diffusion) areas, which can be exposed at different timings for each FD area. The camera controller controls the light emission timing of the illumination device and the exposure timing of the image sensor. The operation processing device generates multiple range images based on the output of the image sensor. The camera controller controls the lighting device and the image sensor to (i) perform sensing in order from the front to the inside over m (m≧2) partitions, taking one partition including n continuous ranges as a unit, and (ii) expose the i-th (1≦i≦n) FD area of ​​the image sensor based on the reflected light from the i-th range of the partition in the sensing of each partition. The processing device corrects the pixel value of the i-th (1≦i≦n) FD area of ​​the j-th (2≦j≦m) partition using the information obtained in the j-1-th partition, and generates a range image of the i-th range of the j-th partition.

[0033] In the presence of fog, when observing each partition, the partition closer to the ToF camera contains more reflected light from the fog than the partition farther away. Since the reflected light from the fog continuously decays according to the distance, it can be said that the influence of the fog on the partition of the current sensing object is close to the influence of the fog on the partition where the previous sensing was completed. In the above structure, by utilizing this property, the influence of the fog on the current partition is estimated using the sensing result of the previous partition, and by performing correction using the estimated influence of the fog, the influence of the fog can be reduced.

[0034] In one embodiment, the processing device may generate the range image of the i-th range of the j-th partition by subtracting the pixel value of the n-th FD region of the j-1-th partition from the pixel value of the i-th (1≦i≦n) FD region of the j-th partition.

[0035] The pixel value of the nth FD region of the previous partition contains the fog information of the range closest to the current partition in the partition. Therefore, by subtracting the pixel value of the nth FD region of the previous partition from the pixel value of each of the n FD regions of the current partition, the influence of fog can be reduced through simple calculation.

[0036] In one embodiment, the processing device may synthesize n×m range images to generate one synthesized image.

[0037] In one embodiment, each pixel of the composite image may be generated based on the maximum value among the pixels corresponding to the n×m range images.

[0038] In one embodiment, when there is no fog, the calculation processing device does not need to perform correction.

[0039] (Implementation Method)

[0040] Hereinafter, preferred embodiments will be described with reference to the accompanying drawings. The same or equivalent components, members, and processes shown in each of the drawings are given the same reference numerals, and repeated descriptions are appropriately omitted. In addition, the embodiments do not limit the invention but are exemplified, and all features or combinations thereof described in the embodiments are not necessarily essential features of the invention.

[0041] Figure 2 1 is a block diagram of a sensing system 10 according to the embodiment. The sensing system 10 is mounted on a vehicle such as an automobile or a motorcycle, and detects an object OBJ existing around the vehicle (within the field of view of the sensor).

[0042] The sensing system 10 mainly includes a ToF camera 100. The ToF camera 100 includes an illumination device 110, an image sensor 120, a camera controller 130, and a processing device 140. Based on the imaging of the ToF camera 100, the field of view is divided into a plurality of N (N≧2) ranges RNG in the depth direction. 1 ~RNG N Adjacent ranges may overlap in the depth direction at their boundaries.

[0043] The lighting device 110 irradiates the pulse illumination light L1 in front of the vehicle in synchronization with the emission timing signal S1 supplied from the camera controller 130. The pulse illumination light L1 is preferably infrared light, but may also be visible light or ultraviolet light having a predetermined wavelength.

[0044] The image sensor 120 includes a plurality of pixels px, and can control exposure in synchronization with an exposure timing signal S2 supplied from the camera controller 130, thereby generating an image composed of a plurality of pixels. The image sensor 120 has sensitivity to the same wavelength as the pulse illumination light L1, and captures reflected light (optical feedback) (reflected light) L2 reflected from the object OBJ.

[0045] The camera controller 130 controls the irradiation timing (light emission timing) of the pulsed illumination light L1 based on the illumination device 110 and the exposure timing based on the image sensor 120. The functions of the camera controller 130 can be implemented by software processing, hardware processing, or a combination of software processing and hardware processing. Software processing is specifically implemented by a combination of a processor (hardware) such as a CPU (Central Processing Unit) or an MPU (MicroProcessing Unit), a microcomputer, and a software program executed by the processor (hardware). In addition, the camera controller 130 can also be a combination of multiple processors and software programs. Hardware processing is specifically implemented by hardware such as an ASIC (Application Specific Integrated Circuit) or a controller IC, an FPGA (Field Programmable Gate Array), and the like.

[0046] An image (range image) SIMG generated by the image sensor 120 i The image sensor 120 outputs the image processing device 140 to process the image processing device 140.1 ~RNG N The multiple range images SIMG 1 ~SIMG N , generate the final output data CAMERAOUT. For example, the output data CAMERAOUT contains multiple range images SIMG 1 ~SIMG N Furthermore, in the present embodiment, the output data CAMERAOUT can include a normal image NIMG which is the same as an image captured by a normal camera.

[0047] The processing device 140 may be implemented on the same hardware as the camera controller 130 or may be composed of different hardware. Alternatively, part or all of the functions of the processing device 140 may be implemented as a processor or digital circuit built into the same module as the image sensor 120 .

[0048] The above is the basic structure of the ToF camera 100. Next, its operation will be described.

[0049] Figure 3 This is a diagram for explaining the basic operation of the ToF camera 100. Figure 3 The RNG for the i-th range is shown in i The lighting device 110 is synchronized with the light emitting timing signal S1 at time t 0 ~t 1 The light-emitting period τ 1 The top part shows a diagram of light with time on the horizontal axis and distance on the vertical axis. i The distance to the nearest boundary is d MINi , will be from the ToF camera 100 to the range RNG i The distance from the inner boundary to the MAXi .

[0050] At a certain moment, the light emitted from the lighting device 110 reaches a distance d MINi , the round trip time T for the reflected light to return to the image sensor 120 MINi for

[0051] T MINi =2×d MINi / c. c is the speed of light.

[0052] Similarly, at a certain moment, the light emitted from the lighting device 110 reaches the distance d MAXi, the round trip time T until the reflected light returns to the image sensor 120 MAXi for

[0053] T MAXi =2×d MAXi / c.

[0054] In the hope of shooting only range RNG i When the object OBJ is included in the camera, the camera controller 130 generates an exposure timing signal S2 so that at time t 2 =t 0 +T MINi Start exposure at time t 3 =t 1 +T MAXi End the exposure. This is one sensing operation.

[0055] i-th range RNG i The sensing may include a plurality of light emission and exposure sets. The camera controller 130 performs the sensing at a predetermined period τ. 2 Repeat the above sensing action multiple times.

[0056] As will be described in detail later, the image sensor 120 can perform multiple exposures, and multiple exposures are performed on multiple reflected lights obtained as a result of multiple pulsed light emission in the FD region (charge storage region) of each pixel px to generate one range image SIMG.

[0057] Figure 4A as well as Figure 4B 1 is a diagram for explaining a range image obtained by the ToF camera 100. Figure 4A In the example, in the range RNG 2 There are objects (pedestrians) OBJ 2 , in the range RNG 3 There is an object (vehicle) OBJ 3 .exist Figure 4B Shown in Figure 4A Multiple range images obtained in the situation SIMG 1 ~SIMG 3 . In the shooting range image SIMG 1 Since the image sensor only receives the RNG 1 The reflected light is exposed, so the range image SIMG 1 No objects are included in the image.

[0058] In the shooting range image SIMG 2 Since the image sensor only receives the RNG 2 The reflected light is exposed, so the range image SIMG2 In the image, only the object OBJ is captured 2 Similarly, in the shooting range image SIMG 3 Since the image sensor only receives the RNG 3 The reflected light is exposed, so the range image SIMG 3 In the image, only the object OBJ is captured 3 As described above, according to the ToF camera 100, objects can be separated and photographed for each range.

[0059] In the full range of RNG 1 ~RNG N After the sensing is completed, multiple range images SIMG 1 ~SIMG N By synthesizing, it is possible to generate an image (normal image) that is the same as an image taken by a normal camera. However, in this case, it takes a very long time to generate one normal image.

[0060] Back to Figure 2 The image sensor 120 is a multi-tap type, and each pixel has at least n FD regions fd 1 ~fd n The image sensor 120 can control exposure for each FD region fd. That is, n FD regions within a pixel can be exposed at different timings.

[0061] The camera controller 130 controls the lighting device 110 and the image sensor 120 to perform sensing as follows.

[0062] (i) The sensing is performed in units of zones. Specifically, the sensing is performed sequentially from the front to the inside across m zones (m≧2). Each zone j (1≦j≦m) contains n consecutive ranges RNG j1 ~RNG jn .

[0063] (ii) In the sensing of each zone ZONE j, the i-th (1≦i≦n) FD region fd of the image sensor 120 is detected. i By RNG from the i-th range of the zone ZONE j ji Exposure is done using reflected light.

[0064] Figure 5 This is a diagram for explaining the sensing of one partition.

[0065] Under the control of the camera controller 130, the illumination device 110 repeatedly irradiates the field of view with pulse illumination light L1, and the image sensor 120 performs multiple exposures on the FD region fd through the reflected light L2 from the field of view. In this time chart, the high of L1 indicates the emission of the pulse illumination light. 1 , fd 2 The high (high) indicates the exposure of each FD region. In addition, Tp1, Tp2... indicate the exposure of each FD region fd 1 , fd 2 ... during the exposure period.

[0066] In addition to the multiple FD regions, the pixel also includes a light receiving element such as a photodiode. Each FD region is exclusively connected to the light receiving element during its exposure period. 1 , Qp 2 ... indicates the FD area fd 1 , fd 2 ...the amount of charge of each. FD area fd 1 , fd 2 The exposure timing of ...is determined according to the position of the photographed range.

[0067] About Range RNG j1 ~RNG jn In order to obtain a sufficiently bright range image, it is necessary to 1 , fd 2 ... to store a sufficient amount of charge. To this end, the emission of the pulse illumination light L1 and the FD region fd 1 ~fd n The exposure needs to be repeated on the order of hundreds to hundreds of thousands of times.

[0068] In this example, the image sensor 120 is configured with a plurality of FD regions fd 1 ~fd n can only be read out at the same timing. That is, assume that if one readout is performed, the charges in multiple FD regions are reset. In this case, if at time t 1 After the sensing of one partition is completed, the FD area fd 1 ~fd n The charge Qp 1 ~Qp n is digitized and read out as pixel values.

[0069] The camera controller 130 performs sensing in order from the front zone ZONE 1 to the inner zone ZONE m using the same sequence.

[0070] Back to Figure 4A as well as Figure 4B After the exposure of a zone ZONE j is completed, the processing device 140 uses the information obtained in the j-1th zone ZONE j-1 that has completed the sensing to perform an exposure on the i-th (1≦i≦n) FD region fd of the j-th (2≦j≦m) zone ZONE j. i The pixel value is corrected to generate the i-th range RNG of the j-th zone ZONE j ji Range image SIMG ji .

[0071] In one embodiment, the processing device 140 obtains the FD region fd from the i-th (1≦i≦n) FD region fd of the j-th zone ZONE j. i The pixel value of the j-1th partition minus the nth FD area fd n The pixel value of the jth zone ZONE j generates the i-th range RNG ji Range image SIMG ji In addition, since there are no negative numbers in pixel values, when the subtraction result is negative, it can be set to 0.

[0072] Regarding the first partition ZONE 1, since there is no partition before it, the first FD area fd of the first partition ZONE 1 can also be used 1 The value of the FD area fd 1 ~fd n value to be corrected.

[0073] Figure 6 This is a diagram for explaining the generation of a range image by the arithmetic processing device 140 .

[0074] Here, assuming n=6, the generation of the range image SIMG is described for the first zone ZONE 1, the second zone ZONE 2, and the jth zone ZONE j where an object exists. Taps TAP1 to TAP6 represent the pixel values ​​of the FD region in each zone. In addition, the scale of the pixel values ​​is appropriately changed for easier understanding.

[0075] Regarding the first partition, the pixel values ​​of taps TAP1 to TAP6 can be corrected using the front pixel value TAP1. Specifically, TAP1 to TAP6 can be corrected by subtracting the value of TAP1. The value of tap TAP1 after correction becomes zero, and although the values ​​of other taps TAP2 to TAP6 become negative numbers, they also become zero in this case.

[0076] Using the corrected values ​​of the taps TAP1 to TAP6, a range image SIMG is generated. 11 ~SIMG 16 .

[0077] Regarding the second zone ZONE 2, the pixel values ​​of the taps TAP1 to TAP6 are corrected by subtracting the last tap TAP6 of the previous zone ZONE 1. When the corrected value is zero, the value becomes zero. Using the corrected values ​​of the taps TAP1 to TAP6, a range image SIMG is generated. 21 ~SIMG 26 .

[0078] For the j-th zone ZONE j, the pixel values ​​of taps TAP1 to TAP6 are corrected by subtracting the last tap TAP6 of the previous zone ZONE j-1. The values ​​of the third and fourth taps contain almost no influence of fog and represent reflected light from the object. Using the corrected values ​​of taps TAP1 to TAP6, a range image SIMG is generated. j1 ~SIMG j6 .

[0079] Figure 7 SIMG is a diagram for explaining the range image SIMG generated by the arithmetic processing device 140. Figure 7 2 shows a range image SIMG obtained by correction and a range image SIMG obtained without correction.

[0080] It can be seen that by performing correction, the range image SIMG ji Suitable for removing the effects of fog.

[0081] The effect of this correction becomes more significant when a plurality of range images SIMG are combined to generate one normal image.

[0082] In one embodiment, the processing device 140 synthesizes N=n×m range images to generate one normal image NIMG (synthesized image). The normal image NIMG includes information of all objects within the field of view, similar to an image captured by a general camera.

[0083] The processing device 140 compares the pixels corresponding to the N=n×m range images and sets the maximum value as the pixel value of the pixel corresponding to the normal pixel NIMG. 11 The value of is the largest, so the information of the object disappears. In contrast, if the range image SIMG obtained by the correction is synthesized, the pixel value of the range image containing the object becomes the largest, the influence of the fog can be removed, and a clear normal image NIMG can be generated.

[0084] Although the calibration by the calculation processing device 140 can be always performed, it will be useless calculation and waste power when there is no fog. Therefore, when there is no fog, the calibration by the calculation processing device 140 may not be performed, and the FD area fd obtained for each zone ZONE j may be set to i The value remains the same as the original state and is set to the range image SIMG ji The pixel value of .

[0085] (Variation Example)

[0086] Those skilled in the art will understand that the above-mentioned embodiments are merely examples, and that various modifications are possible in the combination of their constituent elements or processing steps. Such modifications will be described below.

[0087] In the embodiment, the value of the FD region of a certain j-th partition is corrected using the value of the n-th FD region of the previous j-1-th partition, but the present invention is not limited thereto. For example, the value of the FD region of the j-th partition may be corrected using the values ​​of a plurality of FD regions of the j-1-th partition. For example, the value of the FD region of the j-1-th partition may be corrected using the average value of the values ​​of the n-th and n-1-th FD regions of the j-1-th partition.

[0088] (use)

[0089] Figure 8 1 is a block diagram of a sensing system 10. The sensing system 10 includes a processing unit 40 in addition to the ToF camera 100. The sensing system 10 is mounted on a vehicle such as a car or a motorcycle, and is an object detection system that determines the type (category, or also called classification) of an object OBJ existing around the vehicle.

[0090] Through the ToF camera 100, multiple range RNGs are generated 1 ~RNG N Corresponding multiple range images SIMG 1 ~SIMG N The output data CAMERAOUT of the ToF camera 100 includes a plurality of range images SIMG 1 ~SIMG N , either or both of the usual images NIMG.

[0091] The calculation processing device 40 is configured to be able to recognize the type of the object based on the output data CAMERAOUT of the ToF camera 100 .

[0092] The functions of the operation processing device 40 can be implemented by software processing, hardware processing, or a combination of software processing and hardware processing. Software processing is specifically implemented by a combination of a processor (hardware) such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), a microcomputer, and a software program executed by the processor (hardware). In addition, the operation processing device 40 can also be a combination of multiple processors and software programs. Hardware processing is specifically implemented by hardware such as an ASIC (Application Specific Integrated Circuit) or a controller IC, an FPGA (Field Programmable Gate Array), etc. The functions of the operation processing device 40 and the functions of the operation processing device 140 can also be implemented on the same processor.

[0093] Fig. 9A as well as Fig. 9B 1 is a diagram showing a car 300 equipped with a ToF camera 100. Fig. 9A The automobile 300 includes headlights (lamps) 302L and 302R.

[0094] like Fig. 9A As shown, the lighting device 110 of the ToF camera 100 can also be built into at least one of the left and right headlights 302L, 302R. The image sensor 120 can be installed on a part of the vehicle, such as the back of the rearview mirror. Alternatively, the image sensor 120 can also be set on the front grille or the front bumper. The camera controller 130 can be set in the vehicle cabin, in the engine compartment, or built into the headlights 302L, 302R.

[0095] like Fig. 9B As shown, the image sensor 120 may be built into either of the left and right headlights 302L and 302R together with the lighting device 110 .

[0096] The lighting device 110 may also be disposed in a part of the vehicle, such as the back of a rearview mirror, or in a front grille or a front bumper.

[0097] Fig.101 is a block diagram showing a vehicle lamp 200 having a sensing system 10. The vehicle lamp 200 and the vehicle-side ECU 310 together constitute a lamp system 304. The vehicle lamp 200 includes a lamp-side ECU 210 and a lamp unit 220. The lamp unit 220 is a low beam or a high beam, and includes a light source 222, a lighting circuit 224, and an optical system 226. Furthermore, the vehicle lamp 200 is provided with the sensing system 10.

[0098] The information related to the object OBJ detected by the sensing system 10 can also be used for the light distribution control of the vehicle lamp 200. Specifically, the lamp-side ECU 210 generates an appropriate light distribution pattern based on the information related to the type and position of the object OBJ generated by the sensing system 10. The lighting circuit 224 and the optical system 226 operate to obtain the light distribution pattern generated by the lamp-side ECU 210. The processing device 40 of the sensing system 10 can also be set outside the vehicle lamp 200, that is, on the vehicle side.

[0099] Furthermore, the information related to the object OBJ detected by the sensing system 10 may be sent to the vehicle-side ECU 310. The vehicle-side ECU 310 may also use the information for autonomous driving or driving assistance.

[0100] It will be understood by those skilled in the art that the embodiments are merely illustrative and that there are various modifications in the combination of their constituent elements or processing steps, and that such modifications are also included in the scope of this disclosure or the present invention.

[0101] This application is based on Japanese Patent Application No. 2022-81841 filed on May 18, 2022, the contents of which are incorporated herein by reference.

Claims

1. A ToF camera divides the field of view into multiple ranges in the depth direction, generates multiple range images corresponding to the multiple ranges, and has: an illumination device for irradiating pulsed illumination light to the field of view; In a multi-tap image sensor, one pixel has n floating diffusion regions, i.e., FD regions, and exposure can be performed at different timings for each FD region. n≧2； A camera controller, controlling the light emission timing of the illumination device and the exposure timing of the image sensor; as well as The processing device generates a plurality of range images based on the output of the image sensor. The camera controller controls the lighting device and the image sensor to (i) perform sensing in sequence from the front to the inside of m partitions, taking one partition including n continuous ranges as a unit, and (ii) in the sensing of each partition, expose the i-th FD area of ​​the image sensor by the reflected light from the i-th range of the partition, wherein m≧2, 1≦i≦n, The processing device corrects the pixel value of the i-th FD region of the j-th partition using the information obtained in the j-1-th partition to generate a range image of the i-th range of the j-th partition, where 2≦j≦m, 1≦i≦n.

2. The ToF camera of claim 1, wherein: The arithmetic processing device generates a range image of the i-th range of the j-th partition by subtracting the pixel value of the n-th FD region of the j-1-th partition from the pixel value of the i-th FD region of the j-th partition, where 1≦i≦n.

3. The ToF camera according to claim 1 or 2, wherein: The arithmetic processing device synthesizes n×m range images to generate one synthesized image.

4. The ToF camera of claim 3, wherein: Each pixel of the synthesized image is generated based on the maximum value among the pixels corresponding to the n×m range images.

5. The ToF camera according to claim 1 or 2, wherein: In a condition where there is no fog, the arithmetic processing device does not correct the pixel value of the i-th FD region of the j-th partition, where 2≦j≦m and 1≦i≦n.

6. The ToF camera according to claim 1 or 2, wherein: The ToF camera is mounted on a vehicle.

7. A vehicle sensing system comprising the ToF camera according to claim 1 or 2. 8 . A vehicle lamp comprising the ToF camera according to claim 1 .

Images