Distance image capturing apparatus and distance image capturing method

Abstract

To improve distance measurement by suppressing an influence of a flare.SOLUTION: A distance image capturing apparatus includes: a light source part; a light-receiving part including a pixel including a photoelectric conversion element and three or more charge accumulation parts and a pixel driving circuit for distributing and accumulating a charge in each of the charge accumulation parts; and a distance image processing part for generating a distance image by controlling an irradiation timing and an accumulation timing, to measure a distance up to a subject on the basis of an accumulated charge amount. The distance image processing part generates the distance image by synthesizing a first distance image generated by controlling the light source part and the pixel driving circuit under a predetermined measurement condition, a second distance image generated by changing the predetermined measurement condition, so as to reduce the accumulated charge amount, and a third distance image generated by delaying the accumulation timing, so as not to accumulate the charge due to a flare in the charge accumulation part under the predetermined measurement condition.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a distance image capturing device and a distance image capturing method. [Background technology]

[0002] Taking advantage of the fact that the speed of light is known, a time-of-flight (hereinafter referred to as "TOF") distance imaging device has been realized, which measures the distance between a measuring device and an object based on the flight time of light in space (measurement space) (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4235729 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional distance imaging devices described above, flare can occur when capturing an image of a close-up subject at high power, which can reduce the accuracy of distance measurement. Here, flare is a phenomenon in which light reflected from a close-up object is re-reflected on the sensor surface, causing diffuse reflection between the lens and the sensor, resulting in the appearance of noise that reduces the accuracy of distance measurement, particularly for long-distance objects.

[0005] Furthermore, in conventional distance imaging devices, in situations where subjects with high reflectivity and subjects with low reflectivity are present at similar distances, distance images are generated using HDR (High Dynamic Range) to accurately measure the distance between the two. HDR is a technique for synthesizing a distance image captured using a drive with a high light source power or a high number of charge accumulations with another distance image captured using a drive with a low light source power or a low number of charge accumulations. However, with conventional distance imaging devices, even when generating a distance image using HDR in a situation where there are subjects with high and low reflectivity at similar distances, it was difficult to suppress the effects of flare that occurred in a distance image measured with high light source power or a high number of charge accumulations.

[0006] The present invention has been made to solve the above problems, and its object is to provide a distance image capturing device and a distance image capturing method that can suppress the effects of flare and improve the accuracy of distance measurement in a situation where subjects with high reflectivity and subjects with low reflectivity are present at similar distances. [Means for solving the problem]

[0007] In order to solve the above problem, one aspect of the present invention is a light receiving unit including a light source unit that irradiates a measurement space, which is a space to be measured, with a light pulse, pixels each having a photoelectric conversion element that generates a charge according to the incident light and three or more charge accumulation units that accumulate the charge, and a pixel drive circuit that distributes and accumulates charge in each of the charge accumulation units in the pixel at a timing synchronized with the irradiation of the light pulse according to a frame period, and a control circuit that controls the irradiation timing of the light pulse and the accumulation timing of the distribution and accumulation of charge in each of the charge accumulation units, and calculates a value based on the amount of charge accumulated in each of the charge accumulation units. and a distance image processing unit that measures the distance to a subject present in the measurement space and generates a distance image, wherein the distance image processing unit generates the distance image by combining a first distance image generated by controlling the light source unit and the pixel driving circuit under predetermined measurement conditions, which are the irradiation output of the light pulse and the number of charge accumulations in the charge accumulation unit; a second distance image generated by changing the predetermined measurement conditions so as to reduce the amount of accumulated charge; and a third distance image generated under the predetermined measurement conditions by delaying the accumulation timing so as to prevent charge due to flare from accumulating in the charge accumulation unit.

[0008] In one aspect of the present invention, in the above-described distance image capturing device, the distance image processing unit generates an HDR distance image by combining the first distance image and the second distance image using HDR (High Dynamic Range), and determines whether each pixel is a close-distance pixel at a distance equal to or less than a threshold distance; For each pixel, if it is a close-distance pixel, the pixel value of the HDR distance image may be selected, and if it is a long-distance pixel that is not a close-distance pixel, the pixel value of the third distance image may be selected and combined to generate the distance image.

[0009] In addition, one aspect of the present invention is that in the above-mentioned distance image capturing device, when a distance calculation error occurs for each pixel of the third distance image, the distance image processing unit may determine that the pixel is a close-distance pixel and select a pixel value of the HDR distance image for that pixel.

[0010] In addition, one aspect of the present invention is that in the above-mentioned distance image capturing device, the distance image processing unit determines whether each pixel of the first distance image is an object with low close-range reflectivity based on the amount of charge accumulated in each of the charge accumulation units, and further selects a pixel value of the HDR distance image for a pixel in the first distance image corresponding to the object with low close-range reflectivity.

[0011] In addition, according to one aspect of the present invention, in the above-described distance image capturing device, the distance image processing unit The second distance image may be generated by reducing the irradiation intensity of the light pulse or the number of times the electric charge is accumulated based on the predetermined measurement conditions.

[0012] Another aspect of the present invention is that, in the above-mentioned distance image capturing device, the frame period includes a plurality of sub-frame periods consisting of an accumulation period and a readout period for reading out the amount of charge accumulated in the charge accumulation section during the accumulation period, and the plurality of sub-frame periods may include a first sub-frame period for generating the first distance image, a second sub-frame period for generating the second distance image, and a third sub-frame period for generating the third distance image.

[0013] Furthermore, one aspect of the present invention is a distance image pickup device comprising: a light receiving unit having a light source unit that irradiates a measurement space, which is a space to be measured, with a light pulse; pixels each having a photoelectric conversion element that generates a charge in response to the incident light and three or more charge accumulation units that accumulate the charge; and a pixel drive circuit that distributes and accumulates charge in each of the charge accumulation units in the pixels at a timing synchronized with the irradiation of the light pulse according to a frame period; and a distance image processing unit that controls the irradiation timing of the light pulse and the accumulation timing of the distribution and accumulation of charge in each of the charge accumulation units, and measures the distance to a subject present in the measurement space based on the amount of charge accumulated in each of the charge accumulation units, and generates a distance image. a first generation step in which the distance image processing unit controls the light source unit and the pixel driving circuit to generate a first distance image under predetermined measurement conditions, which are the irradiation output of the light pulse and the number of charge accumulations in the charge accumulation unit; a second generation step in which the distance image processing unit generates a second distance image by changing the predetermined measurement conditions so as to reduce the amount of accumulated charge; a third generation step in which the distance image processing unit generates a third distance image by delaying the accumulation timing under the predetermined measurement conditions so as to prevent charge due to flare from accumulating in the charge accumulation unit; and a combination step in which the distance image processing unit combines the first distance image, the second distance image, and the third distance image to generate the distance image. [Effects of the Invention]

[0014] According to the present invention, it is possible to suppress the influence of flare and improve the accuracy of distance measurement in a situation where there are subjects with high reflectance and subjects with low reflectance at similar distances. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a block diagram showing a schematic configuration of a distance imaging device according to an embodiment of the present invention. [Figure 2]1 is a block diagram showing a schematic configuration of a range image sensor according to an embodiment of the present invention. [Figure 3] FIG. 2 is a circuit diagram showing an example of the configuration of a pixel of the distance imaging device according to the present embodiment. [Figure 4] FIG. 10 is a diagram showing an example of drive timing for distance measurement in a sub-frame period of the distance image pickup device according to the present embodiment. [Figure 5] 10A and 10B are diagrams illustrating a normal image showing an example of a subject in this embodiment. [Figure 6] 10A and 10B are diagrams illustrating an example of combining distance images by the distance image capturing apparatus according to the present embodiment. [Figure 7] 10A and 10B are diagrams illustrating an example of synthesis of an HDR range image by the range image capturing device according to the present embodiment. [Figure 8] 10A and 10B are diagrams showing an example of combining an HDR range image of the range image capturing device according to the present embodiment with a range image obtained by range shift driving. [Figure 9] 5 is a flowchart showing an example of the operation of the range imaging device according to the present embodiment. [Figure 10] 10 is a flowchart showing an example of a process for combining an HDR distance image and a distance image obtained by range shift driving in the distance image capturing device according to the present embodiment. [Figure 11] 10 is a flowchart showing another example of the process of combining an HDR range image and a range image obtained by range shift driving in the range image capturing device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] A distance image capturing device and a distance image capturing method according to an embodiment of the present invention will be described below with reference to the drawings.

[0017] FIG. 1 is a block diagram showing a schematic configuration of a distance image capturing device 1 according to this embodiment. As shown in Fig. 1, the distance image capturing device 1 includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Fig. 1 also shows a subject OB, which is an object to which the distance is to be measured by the distance image capturing device 1.

[0018] The light source unit 2 irradiates a light pulse PO into a space to be measured, in which a subject OB, the distance of which is to be measured by the distance image pickup device 1, is present, under the control of the distance image processor 4. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface-emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffuser plate 22.

[0019] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band of 850 nm to 940 nm) that becomes the light pulses PO to be irradiated onto the subject OB. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits pulsed laser light in response to control from the timing control unit 41.

[0020] The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the extent of the surface that is irradiated onto the subject OB. The pulsed laser light diffused by the diffusion plate 22 is emitted as a light pulse PO and is irradiated onto the subject OB.

[0021] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by an object OB, the distance of which is to be measured in the range image pickup device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a range image sensor 32.

[0022] The lens 31 is an optical lens that guides the incident reflected light RL to the range image sensor 32. The lens 31 outputs the incident reflected light RL to the range image sensor 32 side, and causes the light to be received (incident) by pixels provided in the light receiving region of the range image sensor 32.

[0023] The range image sensor 32 is an imaging element used in the range image capturing device 1. The range image sensor 32 has a plurality of pixels in a two-dimensional light receiving area. Each pixel of the range image sensor 32 is provided with one photoelectric conversion element, a plurality of charge accumulation units corresponding to this one photoelectric conversion element, and a component that distributes charge to each of the charge accumulation units. In other words, the pixel is an imaging element with a distribution configuration that distributes and stores charge among a plurality of (e.g., three or more) charge accumulation units.

[0024] The range image sensor 32 distributes the charges generated by the photoelectric conversion elements to the respective charge accumulation sections under the control of the timing control section 41. The range image sensor 32 also outputs pixel signals according to the amount of charge distributed to the charge accumulation sections. The range image sensor 32 has multiple pixels arranged in a two-dimensional matrix, and outputs pixel signals for one frame corresponding to each pixel.

[0025] The distance image processing unit 4 controls the distance image pickup device 1 and calculates the distance to the subject OB. The distance image processing unit 4 includes a timing control unit 41, a distance measurement unit , a measurement control unit 43, and a measurement storage unit .

[0026] The timing control unit 41 controls the timing of outputting various control signals required for measurement in accordance with the control of the measurement control unit 43. The various control signals here include, for example, a signal that controls the irradiation of the light pulse PO, a signal that distributes and accumulates charge in each charge accumulation unit CS in the pixel 321 at a timing synchronized with the irradiation of the light pulse PO according to the frame period (a signal that distributes reflected light RL to multiple charge accumulation units), and a signal that controls the number of distributions (number of charge accumulations) per frame. The number of distributions (number of charge accumulations) is the number of times the process of distributing charge to the charge accumulation units CS (see FIG. 3) is repeated. The exposure time is the product of this number of distributions and the time (accumulation time) for accumulating charge in each charge accumulation unit per charge distribution process.

[0027] The distance measurement unit 42 outputs distance information obtained by calculating the distance to the object OB based on the pixel signals output from the distance image sensor 32. The distance measurement unit 42 calculates the delay time from when the light pulse PO is emitted until when the reflected light RL is received based on the amount of charge accumulated in multiple (e.g., three or more) charge accumulation units. The distance measurement unit 42 calculates the distance to the object OB according to the calculated delay time. Details of the measurement of the distance to the object OB by the distance measurement unit 42 will be described later.

[0028] The measurement control unit 43 controls the timing control unit 41 and the distance measurement unit 42. For example, the measurement control unit 43 sets the number of allocations and accumulation time for one frame, and controls the timing control unit 41 so that imaging is performed according to the set contents.

[0029] The measurement control unit 43 causes the distance measurement unit 42 to measure the distance to the subject OB based on the imaging results executed using the timing control unit 41. The measurement control unit 43 also generates and outputs a distance image based on the measured distance to the subject OB. Here, the distance image is an image in which a pixel value corresponding to the measurement value (distance value) measured by the distance measurement unit 42 is inserted into each pixel.

[0030] The frame period (period of one frame) for generating a distance image includes a plurality of subframe periods each consisting of an accumulation period and a readout period for reading out the amount of charge accumulated in the charge accumulation unit CS during the accumulation period. The plurality of subframe periods includes a first subframe period, a second subframe period, and a third subframe period.

[0031] During the first sub-frame period, measurement control unit 43 controls light source unit 2 and pixel drive circuit (vertical scanning circuit 323) using timing control unit 41 under predetermined measurement conditions, measures distance using distance measurement unit 42, and generates a first distance image. Note that the predetermined measurement conditions (predetermined drive conditions) include the irradiation power (irradiation output) of light pulse PO and the number of charge accumulations in charge accumulation unit CS, for example, 100,000 times. Furthermore, the first sub-frame period is a sub-frame period for generating the first distance image.

[0032] Furthermore, measurement control unit 43 changes the predetermined measurement conditions to reduce the amount of accumulated charge during the second sub-frame period, and controls light source unit 2 and pixel drive circuit (vertical scanning circuit 323) using timing control unit 41, and distance measurement unit 42 measures the distance to generate a second distance image. Here, the predetermined measurement conditions that reduce the amount of accumulated charge are, for example, conditions in which the number of charge accumulations is reduced to 10,000. Furthermore, the second sub-frame period is a sub-frame period for generating the second distance image.

[0033] Furthermore, during the third sub-frame period, under predetermined measurement conditions, measurement control unit 43 delays the accumulation timing so that charge due to flare is not accumulated in charge accumulation unit CS, controls light source unit 2 and pixel drive circuit (vertical scanning circuit 323) using timing control unit 41, measures distance using distance measurement unit 42, and generates a third distance image. That is, measurement control unit 43 performs range shift drive during the third sub-frame period to generate the third distance image. Furthermore, the third sub-frame period is a sub-frame period for generating the third distance image.

[0034] The measurement control unit 43 generates a final distance image by combining the first distance image generated in the first sub-frame period, the second distance image generated in the second sub-frame period, and the third distance image generated in the third sub-frame period. Details of the processes by the measurement control unit 43 for generating the first distance image, the second distance image, and the third distance image and for combining the final distance image will be described later.

[0035] Measurement memory unit 44 stores various information used by distance image processor 4 when measuring the distance to subject OB and generating distance images. Measurement memory unit 44 stores, for example, the amount of charge accumulated in charge accumulation unit CS for each image, the first distance image, the second distance image, the third distance image, the final distance image, etc.

[0036] With the configuration shown in Figure 1, in the distance image capturing device 1, the light source unit 2 irradiates a light pulse PO in the near-infrared wavelength band onto the subject OB, and the light receiving unit 3 receives the reflected light RL that is reflected by the subject OB, and the distance image processing unit 4 outputs distance information (e.g., a distance image) that measures the distance to the subject OB.

[0037] Although FIG. 1 shows the distance image pickup device 1 having the distance image processing unit 4 built therein, the distance image processing unit 4 may be an element provided outside the distance image pickup device 1.

[0038] Next, the configuration of the distance image sensor 32 used as an imaging element in the distance image capturing device 1 will be described. FIG. 2 is a block diagram showing a schematic configuration of the distance image sensor 32 in this embodiment.

[0039] As shown in FIG. 2, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels 321 are arranged, a control circuit 322, a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324, and a pixel signal processing circuit 325.

[0040] The light receiving region 320 is a region in which a plurality of pixels 321 are arranged, and in FIG. 2, an example is shown in which the pixels are arranged in a two-dimensional matrix of 8 rows and 8 columns. The pixel 321 accumulates a charge corresponding to the amount of light received. The control circuit 322 performs overall control of the range image sensor 32. The control circuit 322 controls the operation of the components of the range image sensor 32, for example, in response to instructions from the timing control unit 41 of the range image processing unit 4. Note that the components of the range image sensor 32 may be directly controlled by the timing control unit 41, in which case the control circuit 322 may be omitted.

[0041] The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving region 320 for each row in accordance with control from the control circuit 322. The vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge accumulated in each charge accumulation unit CS of the pixels 321 to the pixel signal processing circuit 325. In this case, the vertical scanning circuit 323 distributes the charge converted by the photoelectric conversion element to each charge accumulation unit of the pixels 321. In other words, the vertical scanning circuit 323 is an example of a "pixel driving circuit."

[0042] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (e.g., noise suppression processing, A / D conversion processing, etc.) on the voltage signals output from the pixels 321 in each column to the corresponding vertical signal lines in accordance with control from the control circuit 322.

[0043] Horizontal scanning circuit 324 is a circuit that sequentially outputs signals output from pixel signal processing circuit 325 to horizontal signal lines in accordance with control from control circuit 322. As a result, pixel signals corresponding to the amount of charge accumulated for one subframe period are sequentially output to distance image processing unit 4 via the horizontal signal lines.

[0044] In the following description, it is assumed that the pixel signal processing circuit 325 performs A / D conversion processing and the pixel signals are digital signals. Here, the configuration of the pixels 321 arranged in the light receiving area 320 provided in the range image sensor 32 will be described.

[0045] FIG. 3 is a circuit diagram showing an example of the configuration of a pixel 321 of the range image pickup device 1 according to this embodiment. FIG. 3 shows an example of the configuration of one pixel 321 among a plurality of pixels 321 arranged in a light receiving region 320. The pixel 321 is an example of a configuration including three pixel signal readout units.

[0046] The pixel 321 includes one photoelectric conversion element PD, a drain gate transistor GD (charge discharging transistor), and three pixel signal readout units RU that output voltage signals from corresponding output terminals OUT. Each pixel signal readout unit RU includes a readout gate transistor G, a floating diffusion FD, a charge storage capacitance C, a reset gate transistor RT, a source follower gate transistor SF, and a select gate transistor SL. In each pixel signal readout unit RU, the floating diffusion FD and the charge storage capacitance C form a charge storage unit CS.

[0047] 3, the three pixel signal readout units RU are distinguished from one another by adding the numbers "1," "2," or "3" after the symbol "RU" of each pixel signal readout unit RU. Similarly, the components of the three pixel signal readout units RU are distinguished from one another by adding the number representing each pixel signal readout unit RU after the symbol.

[0048] In the pixel 321 shown in FIG. 3, the pixel signal readout unit RU1, which outputs a voltage signal from the output terminal OUT1, includes a readout gate transistor G1, a floating diffusion FD1, a charge storage capacitance C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a select gate transistor SL1. In the pixel signal readout unit RU1, the floating diffusion FD1 and the charge storage capacitance C1 form a charge storage unit CS1. The pixel signal readout units RU2 and RU3 also have a similar configuration. The charge storage unit CS1 is an example of a "first charge storage unit." The charge storage unit CS2 is an example of a "second charge storage unit." The charge storage unit CS3 is an example of a "third charge storage unit."

[0049] The photoelectric conversion element PD is a buried photodiode that photoelectrically converts incident light to generate electric charges and accumulates the generated electric charges. The photoelectric conversion element PD may have any structure. For example, the photoelectric conversion element PD may be a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined together, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. Furthermore, the photoelectric conversion element PD is not limited to a photodiode, and may be, for example, a photogate type photoelectric conversion element.

[0050] In pixel 321, the photoelectric conversion element PD photoelectrically converts incident light to generate electric charges, which are then distributed to each of three charge accumulation units CS, and voltage signals corresponding to the amount of distributed electric charges are output to the pixel signal processing circuit 325.

[0051] The configuration of the pixels arranged in the range image sensor 32 is not limited to the configuration including three pixel signal readout units RU as shown in Fig. 3, but may be any pixel configured to include multiple pixel signal readout units RU. In other words, the number of pixel signal readout units RU (charge accumulation units CS) included in the pixels arranged in the range image sensor 32 may be four or more.

[0052] 3 shows an example in which the charge storage section CS is configured by a floating diffusion FD and a charge storage capacitance C. However, the charge storage section CS only needs to be configured by at least a floating diffusion FD, and the pixel 321 may not have a charge storage capacitance C.

[0053] Furthermore, in the pixel 321 having the configuration shown in FIG. 3, an example of a configuration including a drain gate transistor GD is shown, but if there is no need to discard the charge accumulated (remaining) in the photoelectric conversion element PD, the pixel 321 may have a configuration not including a drain gate transistor GD.

[0054] Next, with reference to FIG. 4, the driving timing for distance measurement in a sub-frame period of the range image pickup device 1 according to this embodiment will be described. 4 is a diagram showing an example of the drive timing for distance measurement in a sub-frame period of the distance image pickup device 1 according to this embodiment. Fig. 4 shows an example in which there are four charge accumulation units CS, and further, an example in which the subject OB is at a short distance will be described.

[0055] In Figure 4, the timing of irradiating the light pulse PO is indicated by "PO," the timing of receiving the reflected light is indicated by "RL," the timing of drive signal TX1 is indicated by "G1," the timing of drive signal TX2 is indicated by "G2," the timing of drive signal TX3 is indicated by "G3," and the timing of drive signal TX4 is indicated by "G4." Note that drive signal TX1 is a signal that drives the read gate transistor G1. The same applies to drive signals TX2, TX3, and TX4.

[0056] The vertical scanning circuit 323 causes the charge storage units CS1, CS2, CS3, and CS4 to store electric charges in this order in synchronization with the irradiation of the light pulse PO. FIG. 4(a) shows an example of drive timing for distance measurement in the first sub-frame period and the second sub-frame period for generating the first distance image and the second distance image. 4(a), in the first and second subframe periods, the timing control unit 41 turns on the read gate transistor G1 that accumulates charges in the charge accumulation unit CS1 while irradiating the light pulse PO. Then, the timing control unit 41 sequentially turns on the read gate transistor G2 that accumulates charges in the charge accumulation unit CS2, the read gate transistor G3 that accumulates charges in the charge accumulation unit CS3, and the read gate transistor G4 that accumulates charges in the charge accumulation unit CS4.

[0057] In the example shown in FIG. 4(a), the subject OB is located at a close distance, so the reflected light RL of the light pulse PO is received at the accumulation timing of the read gate transistor G1 and the read gate transistor G2 described above.

[0058] FIG. 4(b) shows an example of the driving timing for distance measurement in the third sub-frame period for generating the third distance image, and also shows an example of the timing of range shift driving.

[0059] 4(b), in the third subframe period, the timing control unit 41 turns on the read gate transistor G1 that accumulates charge in the charge accumulation unit CS1 along with the irradiation of the light pulse PO, with a delay of the delay period DLY from the irradiation of the light pulse PO. Next, the timing control unit 41 sequentially turns on the read gate transistor G2 that accumulates charge in the charge accumulation unit CS2, the read gate transistor G3 that accumulates charge in the charge accumulation unit CS3, and the read gate transistor G4 that accumulates charge in the charge accumulation unit CS4.

[0060] In this case, the range shift drive delays the accumulation timing so that charges due to flare are not accumulated in the charge accumulation unit CS, and therefore reflected light RL from a close distance to the subject OB is received before the readout gate transistor G1 is turned on, and no charges are accumulated in the charge accumulation unit CS1.

[0061] The vertical scanning circuit 323 repeats the above-described driving a predetermined number of times in each subframe period. Then, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge allocated to each charge storage unit CS. Specifically, the vertical scanning circuit 323 turns on the selection gate transistor SL1 for a predetermined time, thereby outputting a voltage signal corresponding to the amount of charge accumulated in the charge storage unit CS1 from the output terminal OUT1 via the pixel signal readout unit RU1. Similarly, the vertical scanning circuit 323 sequentially turns on the selection gate transistors SL2 and SL3, thereby outputting voltage signals corresponding to the amounts of charge accumulated in the charge storage units CS2 and CS3 from the output terminals OUT2 and OUT3. Then, electrical signals corresponding to the amount of charge accumulated in each charge storage unit CS for each subframe period are output to the distance measurement unit 42 via the pixel signal processing circuit 325 and the horizontal scanning circuit 324.

[0062] Using this principle, the distance measurement unit 42 calculates the delay time Td in conventional short-distance light-receiving pixels using the following formula (1): Note that formula (1) is based on the premise that the amount of charge corresponding to the external light component among the amounts of charge accumulated in the charge accumulation units CS1 and CS2 is the same as the amount of charge accumulated in the charge accumulation unit CS3.

[0063] Td=To×(Q2-Q3) / (Q1+Q2-2×Q3) …(1) However, To is the period during which the light pulse PO is irradiated. Q1 is the amount of charge stored in the charge storage section CS1 Q2 is the amount of charge stored in the charge storage section CS2 Q3 is the amount of charge stored in the charge storage section CS3

[0064] In the short-distance light-receiving pixels, the distance measurement unit 42 calculates the round-trip distance to the subject OB by multiplying the delay time Td calculated by equation (1) by the speed of light (velocity).The distance measurement unit 42 then measures the distance to the subject OB by dividing the calculated round-trip distance by 1 / 2.

[0065] Next, the first distance image, second distance image, third distance image, and the final distance image obtained by combining these images will be described in detail with reference to FIGS. First, the subject OB used in the description of this embodiment will be described with reference to FIG.

[0066] FIG. 5 is a diagram illustrating a normal image showing an example of the subject OB in this embodiment. As shown in FIG. 5, the subjects OB in this embodiment include a close-distance object OB1 with high reflectance, a close-distance object OB2 with low reflectance, and a long-distance object OB3. Moreover, the image NG1 shown in FIG. 5 is a normal image NG1 obtained by capturing an image of a subject OB made up of objects OB1, OB2, and OB3.

[0067] Next, FIG. 6 is a diagram showing an example of synthesis of distance images by the distance image pickup device 1 according to this embodiment. FIG. 6(a) shows the first distance image DG1. The measurement control unit 43 of the distance image processing unit 4 generates the first distance image DG1 using the timing control of FIG. 4(a) described above, for example, when the charge accumulation count is 100,000. Note that flare FL1 occurs in the first distance image DG1, and the distance to a long-distance object OB3 cannot be accurately measured due to the influence of flare FL1. Also, the charge accumulation amount of a close-distance, high-reflectivity object OB1 is saturated, making it impossible to accurately measure the distance. Note that the first distance image DG1 allows accurate distance measurement to a close-distance, low-reflectivity object OB2.

[0068] 6(b) shows the second distance image DG2, which the measurement control unit 43 generates using the timing control of FIG. 4(a) described above, with the charge accumulation count set to 10,000. Note that while the occurrence of flare FL1 is suppressed in the second distance image DG2 by reducing the charge accumulation count to 10,000, the distances to the close-distance low-reflectivity object OB2 and the long-distance object OB3 cannot be accurately measured due to the reduced amount of accumulated charge. Note that the distance to the close-distance high-reflectivity object OB1 can be accurately measured in the second distance image DG2.

[0069] 6(c) shows a third distance image DG3, which is generated by the measurement control unit 43 using the timing control (range shift driving) shown in FIG. 4(b) above, with the charge accumulation count set to 100,000. Note that in the third distance image DG3, the range shift driving prevents accurate measurement of the distances to the close-range high-reflectivity object OB1 and the close-range low-reflectivity object OB2. Note that in the third distance image DG3, the distance to the long-distance object OB3 can be accurately measured.

[0070] 6(d) shows distance image DG4, which is a combination of first distance image DG1, second distance image DG2, and third distance image DG3. Measurement control unit 43 combines the portions of first distance image DG1, second distance image DG2, and third distance image DG3 for which distances were accurately measured, to generate final distance image DG4.

[0071] Next, the synthesis process of the distance image DG4 will be described in detail with reference to FIGS. FIG. 7 is a diagram showing an example of synthesis of an HDR range image by the range image capturing device 1 according to this embodiment.

[0072] Fig. 7(a) shows the first distance image DG1, Fig. 7(b) shows the second distance image DG2, and Fig. 7(c) shows the HDR distance image HDG. As shown in FIG. 7, the distance image processing unit 4 (measurement control unit 43) synthesizes the first distance image DG1 and the second distance image DG2 using HDR (High Dynamic Range) to generate an HDR distance image HDG. In the HDR distance image HDG, the distance to part of the distant object OB3 cannot be measured accurately due to the influence of flare FL1.

[0073] Next, the distance image processing unit 4 (measurement control unit 43) combines the generated HDR distance image HDG with the third distance image DG3 to generate a final distance image DG4, as shown in FIG. FIG. 8 is a diagram showing an example of combining an HDR distance image HDG of the distance image pickup device 1 according to this embodiment with a third distance image DG3 obtained by range shift driving.

[0074] Fig. 8(a) shows the HDR distance image HDG, Fig. 8(b) shows the third distance image DG3, Fig. 8(c) shows the distance image DG5 in which close-distance pixels are determined for the third distance image DG3, and Fig. 8(d) shows the final distance image DG4.

[0075] The distance image processing unit 4 (measurement control unit 43) first determines whether each pixel 321 in the third distance image DG3 is a close-distance pixel at a distance equal to or less than a threshold distance, and generates a distance image DG5 in which the close-distance pixels have been determined. For example, the measurement control unit 43 determines whether each pixel is a close-distance pixel depending on whether a distance calculation error has occurred. Here, a distance calculation error occurs, for example, when charge is accumulated only at timing G1 shown in FIG. 4(b) above (for example, when the amount of accumulation at timing G1 is the largest and is equal to or greater than the threshold).

[0076] The measurement control unit 43 determines whether or not the pixel is a close-distance pixel, and generates a distance image DG5 in which the close-distance pixels have been determined, as shown in FIG. 8(c). In the distance image DG5 shown in FIG. 8(c) in which the short-distance pixels have been determined, the areas NOB of the short-distance high reflectance object OB1 and the short-distance low reflectance object OB2 have been determined to be short-distance pixels.

[0077] Next, the measurement control unit 43 combines the HDR distance image HDG with the distance image DG5 determined to be a close-distance pixel. For each pixel 321, the measurement control unit 43 selects the pixel value of the HDR distance image HDG if it is a close-distance pixel, and selects the pixel value of the third distance image DG3 (distance image DG5) if it is a long-distance pixel that is not a close-distance pixel, and combines them to generate a distance image DG4.

[0078] 8, the measurement control unit 43 uses the pixel values of the HDR distance image HDG for pixels 321 in area NOB of distance image DG5, and uses the pixel values of the third distance image DG3 (distance image DG5) for pixels 321 in other areas (areas of the distant object OB3) to synthesize the images. As a result, the measurement control unit 43 generates distance image DG4 shown in FIG. 8(d).

[0079] Next, the operation of the range image capturing device 1 according to this embodiment will be described with reference to the drawings. FIG. 9 is a flowchart showing an example of the operation of the range image capturing device 1 according to this embodiment.

[0080] 9, distance image processing unit 4 of distance image pickup device 1 first sets measurement conditions (step S101). Measurement control unit 43 of distance image processing unit 4 causes timing control unit 41 to set predetermined measurement conditions for the first sub-frame period (e.g., 100,000 charge accumulations).

[0081] Next, the measurement control unit 43 measures the distance with the set number of charge accumulations and generates a first distance image DG1 (step S102). The measurement control unit 43 controls the timing control unit 41 to perform measurements, and the distance measurement unit 42 to perform distance measurements for each pixel 321, generating the first distance image DG1 shown in FIG. 6(a), for example. The measurement control unit 43 controls the measurement storage unit 44 to store the measurement data for generating the first distance image DG1 and the first distance image DG1.

[0082] Next, the measurement control unit 43 changes the measurement conditions to conditions in which the output of the light pulse PO is reduced or the number of charge accumulations is reduced (step S103). For example, the measurement control unit 43 causes the timing control unit 41 to set the measurement conditions for the second sub-frame period by reducing the number of charge accumulations to 10,000.

[0083] Next, the measurement control unit 43 measures the distance with the set number of charge accumulations and generates a second distance image DG2 (step S104). The measurement control unit 43 controls the timing control unit 41 to perform measurements, and controls the distance measurement unit 42 to perform distance measurements for each pixel 321, generating, for example, the second distance image DG2 shown in FIG. 6(b). The measurement control unit 43 controls the measurement storage unit 44 to store the measurement data for generating the second distance image DG2 and the second distance image DG2.

[0084] Next, the measurement control unit 43 changes the measurement conditions to those for the range shift measurement (step S105). For example, the measurement control unit 43 causes the timing control unit 41 to return the number of charge accumulations to 100,000, set the delay period DLY as shown in FIG. 4(b), and set the measurement conditions for the third sub-frame period.

[0085] Next, the measurement control unit 43 measures the distance with the set number of charge accumulations and generates a third distance image DG3 (step S106). The measurement control unit 43 controls the timing control unit 41 to perform measurements, and the distance measurement unit 42 to perform distance measurement of each pixel 321, generating, for example, the third distance image DG3 shown in FIG. 6(c). The measurement control unit 43 controls the measurement storage unit 44 to store the measurement data for generating the third distance image DG3 and the third distance image DG3.

[0086] Next, the measurement control unit 43 generates a final distance image DG4 by combining the first distance image DG1, the second distance image DG2, and the third distance image DG3 (step S106). The measurement control unit 43 generates the final distance image DG4, for example, as shown in FIG. 6(d), using the information stored in the measurement memory unit 44. After processing step S107, the measurement control unit 43 ends the processing.

[0087] Next, with reference to FIG. 10, the process of step S107 in FIG. 9 will be described in detail. FIG. 10 is a flowchart showing an example of a process for combining an HDR distance image HDG of the distance image capturing device 1 according to this embodiment with a distance image (third distance image DG3) obtained by range shift driving.

[0088] 10, the measurement control unit 43 first generates an HDR distance image HDG by combining the first distance image DG1 and the second distance image DG2 (step S201). The measurement control unit 43 uses information stored in the measurement storage unit 44 to generate an HDR distance image HDG, for example, as shown in FIG. 7. The measurement control unit 43 stores the generated HDR distance image HDG in the measurement storage unit 44.

[0089] Next, the measurement control unit 43 determines whether each pixel in the third distance image DG3 is a close-distance pixel (step S202). The measurement control unit 43 generates a distance image DG5 in which the close-distance pixels have been determined, as shown in FIG. 8(c), for example, and stores the distance image DG5 in the measurement storage unit 44.

[0090] Next, the measurement control unit 43 sets the first pixel 321 (step S203). The measurement control unit 43 sets the first pixel 321 for synthesis of the final distance image DG4.

[0091] Next, the measurement control unit 43 determines whether pixel 321 is a close-distance pixel (step S204). Using distance image DG5 in which a close-distance pixel has been determined, the measurement control unit 43 determines whether the set pixel 321 is a close-distance pixel. If pixel 321 is a close-distance pixel (step S204: YES), the measurement control unit 43 proceeds to step S205. If pixel 321 is not a close-distance pixel (step S204: NO), the measurement control unit 43 proceeds to step S206.

[0092] In step S205, the measurement control unit 43 selects pixel values of the HDR distance image HDG to synthesize a distance image DG4. The measurement control unit 43 substitutes the pixel value of the corresponding HDR distance image HDG for the target pixel 321 to synthesize a distance image DG4. After processing step S205, the measurement control unit 43 proceeds to step S207.

[0093] In step S206, the measurement control unit 43 selects pixel values in the third distance image DG3 and synthesizes a distance image DG4. The measurement control unit 43 substitutes the corresponding pixel value in the third distance image DG3 for the target pixel 321, and synthesizes a distance image DG4. After processing step S206, the measurement control unit 43 proceeds to step S207.

[0094] In step S207, the measurement control unit 43 determines whether or not the pixel is the final pixel. That is, the measurement control unit 43 determines whether or not the currently set pixel 321 is the final pixel of the distance image DG4. If the currently set pixel 321 is the final pixel of the distance image DG4 (step S207: YES), the measurement control unit 43 proceeds to step S209. If the currently set pixel 321 is not the final pixel of the distance image DG4 (step S207: NO), the measurement control unit 43 proceeds to step S208.

[0095] In step S208, the measurement control unit 43 sets the next pixel 321. After the process of step S208, the measurement control unit 43 returns the process to step S204.

[0096] Furthermore, in step S209, the measurement control unit 43 outputs the distance image DG4 generated by the synthesis as the final distance image. After the processing of step S209, the measurement control unit 43 ends the synthesis process of the HDR distance image HDG and the distance image generated by range shift driving (third distance image DG3).

[0097] Next, a modified example of the process of combining the HDR distance image HDG with a distance image (third distance image DG3) obtained by range shift driving will be described with reference to FIG. FIG. 11 is a flowchart showing another example of the process of combining the HDR distance image HDG of the distance image pickup device 1 according to this embodiment with a distance image (third distance image DG3) obtained by range shift driving.

[0098] In the example shown in Figure 11, the distance image processing unit 4 determines whether each pixel 321 in the first distance image DG1 is an object with low reflectivity at close range based on the amount of charge accumulated in each charge accumulation unit CS, and further selects the pixel value of the HDR distance image HDG for the pixel 321 in the first distance image DG1 that corresponds to an object with low reflectivity at close range.

[0099] The processes in steps S301 and S302 shown in FIG. 11 are the same as those in steps S201 and S202 shown in FIG. 10 described above, and therefore a description thereof will be omitted here.

[0100] In the next step S303, the measurement control unit 43 determines whether each pixel 321 in the first distance image DG1 is an object with low short-range reflectance. The measurement control unit 43 determines that the object is one with low short-range reflectance, for example, if the amount of charge Q1 accumulated in the charge accumulation unit CS1 at timing G1 is equal to or greater than a first threshold value and (amount of charge Q1 / amount of charge Q2) is equal to or less than a second threshold value. Here, the amount of charge Q2 is the amount of charge accumulated in the charge accumulation unit CS2. The measurement control unit 43 determines whether each pixel 321 in the first distance image DG1 is an object with low short-range reflectance. The measurement control unit 43 stores the determination result for each pixel 321 in the measurement memory unit 44.

[0101] The subsequent processes from step S304 to step S307 and step S308 to step S310 are similar to the processes from step S203 to step S206 and step S207 to step S209 shown in FIG. 10 described above, and therefore description thereof will be omitted here.

[0102] In step S308, the measurement control unit 43 determines whether pixel 321 is an object with low short-range reflectance. Using the determination result of step S303 stored in the measurement storage unit 44, the measurement control unit 43 determines whether the set pixel 321 is an object with low short-range reflectance. In the example shown in FIG. 6 described above, pixel 321 corresponding to short-range low-reflectance object OB2 corresponds to an object with low short-range reflectance. If pixel 321 is an object with low short-range reflectance (step S308: YES), the measurement control unit 43 proceeds to step S306, where it selects pixel values from the HDR distance image HDG to synthesize distance image DG4. If pixel 321 is not an object with low short-range reflectance (step S308: NO), the measurement control unit 43 proceeds to step S309.

[0103] As described above, the distance image capture device 1 according to this embodiment includes a light source 2, a light receiving unit 3, and a distance image processor 4. The light source 2 irradiates a measurement space, which is a space to be measured, with a light pulse PO. The light receiving unit 3 includes pixels 321 each having a photoelectric conversion element PD that generates charge in response to the incident light and three or more charge storage units CS that store the charge. The pixel drive circuit (vertical scanning circuit 323) distributes and accumulates the charge in each of the charge storage units CS in the pixels 321 at timing synchronized with the irradiation of the light pulse PO according to a frame period. The distance image processor 4 controls the irradiation timing of the light pulse PO and the accumulation timing of the distribution of charge in each of the charge storage units CS. Based on the amount of charge accumulated in each charge storage unit CS, the distance to the object OB in the measurement space is measured and a distance image is generated. The distance image processor 4 then generates a distance image DG4 by combining the first distance image DG1, the second distance image DG2, and the third distance image DG3. The first distance image DG1 is a distance image generated by controlling the light source unit 2 and pixel drive circuit (vertical scanning circuit 323) under predetermined measurement conditions, which are the irradiation output of the light pulse PO and the number of charge accumulations in the charge accumulation unit CS. The second distance image DG2 is a distance image generated by changing the predetermined measurement conditions so as to reduce the amount of accumulated charge. The third distance image DG3 is a distance image generated under predetermined measurement conditions by delaying the accumulation timing so as to prevent charge due to flare from accumulating in the charge accumulation unit CS.

[0104] As a result, the distance image pickup device 1 according to this embodiment can suppress the effects of flare and generate a distance image with correctly measured distances (for example, the distance image DG4 shown in FIG. 6 ) by selecting and combining pixel values from the first distance image DG1, the second distance image DG2, and the third distance image DG3, for example. Therefore, the distance image pickup device 1 according to this embodiment can suppress the effects of flare and improve the accuracy of distance measurement. For example, in a situation where there are subjects with high reflectivity and subjects with low reflectivity at similar distances, the distance image pickup device 1 according to this embodiment can suppress the effects of flare and improve the accuracy of distance measurement.

[0105] In this embodiment, the distance image processor 4 generates an HDR distance image HDG by combining the first distance image and the second distance image using HDR. The distance image processor 4 determines whether each pixel 321 is a close-distance pixel at a distance equal to or less than a threshold distance. For each pixel 321, the distance image processor 4 selects a pixel value from the HDR distance image HDG if the pixel is a close-distance pixel, and selects a pixel value from the third distance image DG3 if the pixel is a long-distance pixel that is not a close-distance pixel, and combines the pixel values to generate a distance image.

[0106] As a result, the distance image capturing device 1 according to this embodiment can generate a distance image (for example, the distance image DG4 shown in Figure 6) with correctly measured distance even for a long-distance object affected by flare FL1, such as the long-distance object OB3 in the HDR distance image HDG shown in Figure 7.

[0107] In addition, in this embodiment, when a distance calculation error occurs for each pixel 321 of the third distance image DG3, the distance image processing unit 4 determines that the pixel is a close-distance pixel and selects the pixel value of the HDR distance image HDG for that pixel 321.

[0108] As a result, the distance image pickup device 1 according to this embodiment can accurately determine whether a pixel is a close-distance pixel using a simple method, and can also suppress the influence of flare and improve the accuracy of distance measurement.

[0109] In this embodiment, the distance image processor 4 determines whether each pixel 321 in the first distance image DG1 is an object with low short-distance reflectance based on the amount of charge stored in each charge storage unit CS. Furthermore, the distance image processor 4 selects a pixel value in the HDR distance image HDG for each pixel 321 in the first distance image DG1 that corresponds to an object with low short-distance reflectance.

[0110] As a result, the distance image capturing device 1 according to this embodiment can accurately distinguish between a close-range object OB2 with low reflectivity and a long-range object OB3, and can further suppress the effects of flare and improve the accuracy of distance measurement.

[0111] In this embodiment, the distance image processing unit 4 generates the second distance image DG2 by reducing the irradiation intensity of the light pulse PO or the number of charge accumulations based on predetermined measurement conditions. As a result, the range image pickup device 1 according to this embodiment can easily generate the second range image DG2 in which the amount of accumulated charge is reduced and the influence of the flare FL1 is suppressed.

[0112] In this embodiment, a frame cycle includes multiple (e.g., three) subframe periods each consisting of an accumulation period and a readout period for reading out the amount of charge accumulated in the charge accumulation unit CS during the accumulation period. The multiple subframe periods include a first subframe period for generating a first distance image DG1, a second subframe period for generating a second distance image DG2, and a third subframe period for generating a third distance image DG3. This allows the distance image capturing device 1 according to this embodiment to easily and appropriately generate the first distance image DG1, the second distance image DG2, and the third distance image DG3.

[0113] The distance image capturing method according to this embodiment is a distance image capturing method for the distance image capturing device 1 described above, and includes a first generating step, a second generating step, a third generating step, and a combining step. The distance image capturing device 1 includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. The light source unit 2 irradiates a measurement space, which is a space to be measured, with a light pulse PO. The light receiving unit 3 includes pixels 321 each having a photoelectric conversion element PD that generates charge in response to the incident light and three or more charge storage units CS that accumulate the charge. The pixel drive circuit (vertical scanning circuit 323) distributes and accumulates the charge in each of the charge storage units CS in the pixels 321 at a timing synchronized with the irradiation of the light pulse PO according to a frame period. The distance image processing unit 4 controls the irradiation timing of the light pulse PO and the accumulation timing of the distribution and accumulation of the charge in each of the charge storage units CS. Based on the amount of charge accumulated in each charge storage unit CS, the distance to the object OB in the measurement space is measured, and a distance image is generated. Then, in a first generation step, distance image processor 4 controls light source unit 2 and pixel drive circuit (vertical scanning circuit 323) under predetermined measurement conditions, which are the irradiation output of light pulse PO and the number of charge accumulations in charge accumulation unit CS, to generate first distance image DG1. In a second generation step, distance image processor 4 changes the predetermined measurement conditions to reduce the amount of accumulated charge, to generate second distance image DG2. In a third generation step, distance image processor 4 delays the accumulation timing under the predetermined measurement conditions to prevent charge due to flare from accumulating in charge accumulation unit CS, to generate third distance image DG3. In a synthesis step, distance image processor 4 synthesizes the first, second, and third distance images to generate a distance image. As a result, the distance image capturing method according to this embodiment has the same effects as the distance image capturing device 1 described above, and can suppress the influence of flare and improve the accuracy of distance measurement.

[0114] The present invention is not limited to the above-described embodiments, and can be modified within the scope of the present invention. For example, in each of the above embodiments, the distance image processing unit 4 combines the first distance image DG1 and the second distance image DG2 to generate the HDR distance image HDG, and then combines the HDR distance image HDG with the third distance image DG3 to generate the final distance image DG4. However, this is not limiting. For example, the distance image processing unit 4 may select and combine pixel values from the first distance image DG1, the second distance image DG2, and the third distance image DG3 that allow for the most accurate distance measurement, without generating the HDR distance image HDG, or may use another method for combination.

[0115] Furthermore, in each of the above embodiments, an example has been described in which the measurement conditions are changed in the order of the first distance image DG1, the second distance image DG2, and the third distance image DG3 to perform distance measurement and generate distance images, but this is not limited to this, and distance measurement and distance images may be generated in a different order.

[0116] In addition, in each of the above embodiments, an example has been described in which the final distance image is synthesized from the first distance image DG1, the second distance image DG2, and the third distance image DG3 (distance images acquired in three subframes), but this is not limiting. The distance image capturing device 1 may also synthesize the final distance image from four or more distance images, for example, by changing the number of charge accumulations to further increase the number of distance images driven, or by using distance images captured with a dot light source.

[0117] Furthermore, in each of the above embodiments, an example of reducing the number of times charge is accumulated has been explained as an example of reducing the amount of accumulated charge, but this is not limited to this, and the irradiation intensity (irradiation power) of the light pulse PO may be reduced, or other methods may be applied.

[0118] In the above embodiments, the light source unit 2 has one light source, but the present invention is not limited to this and may use, for example, multiple light sources. The light source unit 2 may also be other light sources such as a dot light source, a wide-angle light source, or a high-power light source.

[0119] In addition, in the above embodiment, an example in which pixel 321 has three charge storage sections CS and an example in which pixel 321 has four charge storage sections CS are described, but this is not limited to this, and other forms are also possible as long as the pixel has three or more charge storage sections CS.

[0120] Each component of the above-described range image pickup device 1 has an internal computer system. A program for realizing the function of each component of the above-described range image pickup device 1 may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing in each component of the above-described range image pickup device 1. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices.

[0121] Furthermore, a "computer system" may include multiple computer devices connected via a network, including communication lines such as the Internet, WAN, LAN, and dedicated lines. Furthermore, a "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Thus, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.

[0122] The recording medium also includes an internal or external recording medium accessible from a distribution server for distributing the program. The program may be divided into multiple parts, downloaded at different times, and then combined by the components of the range image capture device 1, or each divided program may be distributed by a different distribution server. Furthermore, the term "computer-readable recording medium" also includes a medium that stores a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when a program is transmitted over a network. The program may also be a medium for implementing part of the above-described functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-described functions in combination with a program already stored in the computer system.

[0123] Furthermore, some or all of the above-described functions may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each of the above-described functions may be individually implemented as a processor, or some or all of the functions may be integrated into a processor. Furthermore, the integrated circuit implementation method is not limited to LSI, and may be implemented using a dedicated circuit or a general-purpose processor. Furthermore, if an integrated circuit implementation technology that can replace LSI emerges due to advances in semiconductor technology, an integrated circuit based on that technology may be used. [Explanation of symbols]

[0124] 1...Distance image capturing device 2...Light source section 3...Light receiving section 4...Distance image processing section 21...Light source device 22...Diffuser 31...Lens 32...Distance image sensor 41...Timing control section 42...Distance measurement unit 43...Measurement control section 44...Measurement storage section 320…Light receiving area 321...pixels 322...Control circuit 323...Vertical scanning circuit 324...Horizontal scanning circuit 325...Pixel signal processing circuit CS, CS1, CS2, CS3...Charge storage section FD, FD1, FD2, FD3...Floating diffusion G, G1, G2, G3, G4...Readout gate transistors GD: Drain gate transistor OB…Subject PD...photoelectric conversion element PO...light pulse RL…Reflected light RT1, RT2, RT3...Reset gate transistors SF, SF1, SF2, SF3...Source follower gate transistors SL, SL1, SL2, SL3...Select gate transistors

Claims

1. a light source unit that irradiates a measurement space, which is a space to be measured, with a light pulse; a light receiving unit including a pixel having a photoelectric conversion element that generates a charge according to incident light and three or more charge accumulation units that accumulate the charge, and a pixel drive circuit that distributes and accumulates the charge in each of the charge accumulation units in the pixel at a timing synchronized with the irradiation of the light pulse according to a frame period; a distance image processing unit that controls the irradiation timing of the light pulse and the accumulation timing of the charge distribution and accumulation in the charge accumulation units, measures the distance to a subject present in the measurement space based on the amount of charge accumulated in each of the charge accumulation units, and generates a distance image; Equipped with The distance image processing unit a first distance image generated by controlling the light source unit and the pixel drive circuit under predetermined measurement conditions, which are the irradiation output of the light pulse and the number of charge accumulations in the charge accumulation unit; and a second distance image generated by changing the predetermined measurement conditions so as to reduce the amount of accumulated charge; and a third distance image generated under the predetermined measurement conditions by delaying the accumulation timing so that charges due to flare are not accumulated in the charge accumulation unit; and and generate the distance image. Range imaging device.

2. The distance image processing unit synthesizing the first distance image and the second distance image using HDR (High Dynamic Range) to generate an HDR distance image; Determine whether each pixel is a close pixel with a distance equal to or less than a threshold distance; For each pixel, if it is a near-distance pixel, the pixel value of the HDR distance image is selected, and if it is a far-distance pixel that is not a near-distance pixel, the pixel value of the third distance image is selected and combined to generate the distance image.

2. The distance imaging device according to claim 1.

3. The distance image processing unit When a distance calculation error occurs for each pixel of the third distance image, the pixel is determined to be a near-distance pixel, and the pixel value of the HDR distance image is selected for the pixel.

3. The distance imaging device according to claim 2.

4. The distance image processing unit determining whether each pixel of the first distance image is an object with low short-distance reflectance based on the amount of charge accumulated in each of the charge accumulation units; Furthermore, a pixel value of the HDR distance image is selected for a pixel in the first distance image that corresponds to the object with low reflectance at a short distance.

4. The distance imaging device according to claim 3.

5. The distance image processing unit generating the second distance image by reducing the irradiation intensity of the light pulse or the number of times of charge accumulation under the predetermined measurement conditions; The distance imaging device according to claim 1 .

6. the frame period includes a plurality of subframe periods each including an accumulation period and a readout period for reading out the amount of charge accumulated in the charge accumulation section during the accumulation period; The plurality of subframe periods includes a first subframe period for generating the first distance image, a second subframe period for generating the second distance image, and a third subframe period for generating the third distance image.

6. The distance imaging device according to claim 5.

7. a light receiving unit having a light source unit that irradiates a measurement space, which is a space to be measured, with a light pulse; a pixel having a photoelectric conversion element that generates a charge according to the incident light and three or more charge accumulation units that accumulate the charge; and a pixel drive circuit that distributes and accumulates charge in each of the charge accumulation units in the pixel at a timing synchronized with the irradiation of the light pulse according to a frame period; and a distance image processing unit that controls the irradiation timing of the light pulse and the accumulation timing of the distribution and accumulation of charge in each of the charge accumulation units, and measures the distance to a subject present in the measurement space based on the amount of charge accumulated in each of the charge accumulation units, and generates a distance image, a first generation step in which the distance image processing unit controls the light source unit and the pixel drive circuit to generate a first distance image under predetermined measurement conditions, which are the irradiation output of the light pulse and the number of charge accumulations in the charge accumulation unit; a second generation step in which the distance image processing unit generates a second distance image by changing the predetermined measurement conditions so as to reduce the amount of accumulated charge; a third generation step in which the distance image processing unit generates a third distance image by delaying the accumulation timing under the predetermined measurement conditions so that charges due to flare are not accumulated in the charge accumulation unit; a synthesis step in which the distance image processing unit synthesizes the first distance image, the second distance image, and the third distance image to generate the distance image; A distance image capturing method comprising:

Images