Range image capturing device and range image capturing method

Abstract

A range image capturing device includes: a light source unit that irradiates a subject with a light pulse; a light receiving unit having a plurality of pixel circuits arranged in a two-dimensional matrix, and a pixel driving circuit; and a distance calculation unit that calculates the distance to the subject on the basis of the amount of charge accumulated by each of the charge accumulation units. The light pulse is structured light comprising a plurality of point lights, and the distance calculation unit calculates the distance between the light pulse and the pixel circuit using a value obtained by subtracting a second electric charge amount based on an electric charge amount accumulated by the electric charge accumulation unit in the second pixel circuit from a first electric charge amount accumulated by the electric charge accumulation unit in the first pixel circuit. The distance to the subject is calculated by calculating the distance between the first pixel circuit and the second pixel circuit, the first pixel circuit receives directly reflected light, and the second pixel circuit is located at the periphery of the first pixel circuit and does not receive the directly reflected light.

Description

Technical Field

[0001] The invention relates to a distance image capturing device and a distance image capturing method.

[0002] This application claims priority from Tokugan Application No. 2022-084758 filed in Japan on May 24, 2022, and the contents are incorporated herein by reference. Background Art

[0003] Time of Flight (hereinafter referred to as "TOF") distance image sensors have been realized, which use the fact that the speed of light is known to measure the distance between a measuring device and an object based on the flight time of light in a space (measurement space) (for example, see Patent Document 1). In such a distance image camera, reflected light is made incident on an imaging element, and charges corresponding to the amount of reflected light are distributed and accumulated in a plurality of charge accumulation parts, thereby obtaining a delay time from the time of irradiation of a light pulse to the time when the reflected light after being reflected by the subject returns, and the distance to the subject is calculated using the delay time and the speed of light.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent No. 4235729 Summary of the invention

[0007] Problems to be solved by the invention

[0008] However, the imaging element includes not only light that is reflected and directly incident from the light source to the subject (direct reflected light), but also light that is incident from the ground and the wall to the subject (multi-path reflected light). The multi-path reflected light reaches the imaging element after being reflected by the ground and the wall, so the optical path length is longer than the direct reflected light and it is incident on the imaging element with a delay compared to the direct reflected light. Therefore, since the multi-path reflected light is incident on the imaging element, it may not be possible to calculate the distance with high accuracy. In addition, light mixed with the direct reflected light and the multi-path reflected light is incident on the imaging element. Therefore, it is difficult to directly extract only the component of the reflected light from the amount of charge accumulated in the charge storage unit. In particular, in the case of measuring the distance to a subject (distant object) at a position far away from the distance image camera device, there is a tendency for the light pulse to diffuse before irradiating the subject, so there is a problem that the influence of the multi-path reflected light becomes greater than the case of measuring the distance to a subject (close object) at a close distance.

[0009] The present invention has been made based on the above-mentioned problems, and an object of the present invention is to provide a range image capturing device and a range image capturing method capable of suppressing the influence of multipath reflected light.

[0010] Means for solving problems

[0011] The distance image camera device of the present invention comprises: a light source unit for irradiating a light pulse to a subject; a light receiving unit having a pixel circuit and a pixel driving circuit, wherein a plurality of the pixel circuits are arranged in a two-dimensional matrix and have a photoelectric conversion element for generating charges corresponding to incident light, and N charge storage units for storing the charges, wherein N≥3, and the pixel driving circuit distributes the charges to the charge storage units respectively at storage timings synchronized with the irradiation of the light pulse and stores the charges; and a distance calculation unit for calculating the distance to the subject based on the charges respectively stored in the charge storage units, wherein the light pulse is structured light composed of a plurality of point lights, and the distance calculation unit calculates the distance to the subject using a value obtained by subtracting a second charge amount based on the charge amount stored in the charge storage unit in a second pixel circuit from a first charge amount stored in the charge storage unit in a first pixel circuit, wherein the first pixel circuit receives directly reflected light, and the second pixel circuit is located around the first pixel circuit and does not receive the directly reflected light.

[0012] In the distance image capturing device of the present invention, the distance calculation unit may calculate the second charge amount using the charge amount accumulated in the charge accumulation unit in each of the plurality of second pixel circuits.

[0013] In the distance image camera device of the present invention, the distance calculation unit may select, based on the position of the first pixel circuit having the charge storage unit having the largest amount of charge among the charge storage units located in a plurality of the first pixel circuits, a pixel circuit located more than one pixel away in a diagonal direction from the reference position as the second pixel circuit.

[0014] In the distance image camera device of the present invention, the distance calculation unit may select, based on the position of the first pixel circuit having the charge storage unit having the largest amount of charge among the charge storage units located in a plurality of the first pixel circuits, a pixel circuit located more than one pixel away in the vertical or horizontal direction from the reference position as the second pixel circuit.

[0015] In the distance image capturing device of the present invention, the distance calculation unit may calculate the second charge amount using a value obtained by weighting the charge amount accumulated in the charge accumulation unit in the second pixel circuit according to a position where the second pixel circuit is arranged.

[0016] In the distance image camera device of the present invention, when the distance calculation unit uses a plurality of the second pixel circuits, if the difference in the amount of charge accumulated in the charge storage units in adjacent pixel circuits is greater than a threshold value, the pixel circuit with a larger amount of charge accumulated in the charge storage unit is not selected as the second pixel circuit.

[0017] The distance image capturing method of the present invention is performed by a distance image capturing device, the distance image capturing device comprising: a light source unit for irradiating a light pulse to a subject; a light receiving unit having a pixel circuit and a pixel driving circuit, the pixel circuit being arranged in a two-dimensional matrix and having a plurality of photoelectric conversion elements for generating charges corresponding to incident light, and N charge storage units for storing the charges, wherein N is greater than or equal to 3, the pixel driving circuit respectively distributing the charges to the charge storage units at a storage timing synchronized with the irradiation of the light pulse and causing them to be stored; and a distance calculation unit for calculating a distance based on the charge storage. The distance calculation unit calculates the distance to the subject by using the amount of charge accumulated in each of the charge accumulation units. In the distance image capturing method, the light pulse is structured light composed of a plurality of point lights, and the distance calculation unit calculates the distance to the subject using a value obtained by subtracting a second charge amount based on the charge amount accumulated in the charge accumulation unit in the second pixel circuit from a first charge amount accumulated in the charge accumulation unit in the first pixel circuit, the first pixel circuit receives directly reflected light, and the second pixel circuit is located around the first pixel circuit and does not receive the directly reflected light.

[0018] Effects of the Invention

[0019] According to the present invention, the influence of multipath reflected light can be suppressed. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a block diagram showing a schematic configuration of a distance image capturing device according to an embodiment.

[0021] Figure 2 This is a block diagram showing a schematic configuration of a distance image sensor according to an embodiment.

[0022] Figure 3 This is a circuit diagram showing an example of the configuration of a pixel according to the embodiment.

[0023] Figure 4 It is a diagram for explaining the direct reflected light RLd and the multipath reflected light RLm according to the embodiment.

[0024] Figure 5 It is a diagram showing an example of light received by the light receiving area according to the embodiment.

[0025] Fig. 6AIt is a diagram showing an example of a pixel that receives direct reflected light RLd according to the embodiment.

[0026] Figure 6B This is a diagram showing an example of a pixel that does not receive the direct reflected light RLd according to the embodiment.

[0027] Figure 7 It is a diagram for explaining the weighting process according to the embodiment.

[0028] Fig. 8A It is a diagram showing an example of light received by the light receiving area according to the embodiment.

[0029] Figure 8B Yes means Fig. 8A A graph showing the changes in the amount of light received by each pixel in the image.

[0030] Fig. 9 This is a flowchart showing the flow of processing performed by the range image capturing device according to the embodiment. DETAILED DESCRIPTION

[0031] Hereinafter, a distance image capturing device according to an embodiment will be described with reference to the drawings.

[0032] Figure 1 1 is a block diagram showing a schematic configuration of a distance image capturing device according to an embodiment. The distance image capturing device 1 includes, for example, a light source unit 2 , a light receiving unit 3 , and a distance image processing unit 4 . Figure 1 4 also shows a subject OB, which is an object whose distance is measured by the distance image capturing device 1 .

[0033] The light source unit 2 irradiates the light pulse PO to the space of the measurement object where the object OB of the object to be measured in the distance image camera 1 exists, according to the control from the distance image processing unit 4. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffusion plate 22.

[0034] The light source device 21 is a light source that emits laser light in a near-infrared band (e.g., a band with a wavelength of 850 nm to 940 nm) as a light pulse PO irradiated to the subject OB. The light source device 21 is, for example, a semiconductor laser light emitting element. The light source device 21 emits pulsed laser light under control from the timing control unit 41.

[0035] The diffuser plate 22 is an optical member that diffuses the near-infrared laser light emitted from the light source device 21 to an area of ​​the surface irradiated to the object OB. The pulsed laser light diffused by the diffuser plate 22 is emitted as a light pulse PO and irradiates the object OB.

[0036] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by the object OB for measuring the distance in the distance image capturing device 1 , and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a distance image sensor 32 .

[0037] The lens 31 is an optical lens that guides the incident reflected light RL to the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32, and causes the pixels provided in the light receiving area of ​​the distance image sensor 32 to receive the light (enter).

[0038] The distance image sensor 32 is an imaging element used in the distance image imaging device 1. The distance image sensor 32 has a plurality of pixels in a two-dimensional light receiving area. Each pixel of the distance image sensor 32 is provided with one photoelectric conversion element, a plurality of charge storage units corresponding to the one photoelectric conversion element, and a component for distributing charges to each charge storage unit. That is, the pixel is an imaging element with a distribution structure for distributing charges to a plurality of charge storage units and accumulating them.

[0039] The distance image sensor 32 distributes the charges generated by the photoelectric conversion element to each charge storage unit according to the control from the timing control unit 41. In addition, the distance image sensor 32 outputs a pixel signal corresponding to the amount of charge distributed to the charge storage unit. In the distance image sensor 32, a plurality of pixels are arranged in a two-dimensional matrix, and the pixel signal corresponding to one frame of each pixel is output.

[0040] The distance image processing unit 4 controls the distance image capturing device 1 to calculate the distance to the object OB. The distance image processing unit 4 includes a timing control unit 41 , a distance calculation unit 42 , and a measurement control unit 43 .

[0041] The timing control unit 41 controls the timing of outputting various control signals required for measurement according to the control of the measurement control unit 43. The various control signals here include, for example, a signal for controlling the irradiation of the light pulse PO, a signal for distributing and accumulating the reflected light RL to a plurality of charge accumulation units, and a signal for controlling the number of accumulations per frame. The number of accumulations refers to the number of times the charge accumulation unit CS (see Figure 3 ) is repeated the number of times the charge distribution and accumulation process is repeated. The product of the number of accumulation times and the time period (accumulation time period) for accumulating charges in each charge accumulation section in each charge distribution and accumulation process becomes the exposure time.

[0042] The distance calculation unit 42 outputs distance information obtained by calculating the distance to the object OB based on the pixel signal output from the distance image sensor 32. The distance calculation unit 42 calculates the delay time from the irradiation light pulse PO to the reception of the reflected light RL based on the charge amount accumulated in the plurality of charge accumulation units. The distance calculation unit 42 calculates the distance to the object OB based on the calculated delay time Td.

[0043] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of accumulation times and the accumulation time period for one frame, and controls the timing control unit 41 to perform imaging according to the set contents.

[0044] According to this configuration, in the distance image camera 1, the light pulse PO in the near-infrared band irradiated by the light source unit 2 to the subject OB is reflected by the subject OB, and the reflected light RL is received by the light receiving unit 3, and the distance image processing unit 4 outputs the distance information obtained by measuring the distance to the subject OB.

[0045] in addition, Figure 1 2 shows the distance image capturing device 1 having a configuration in which the distance image processing unit 4 is provided inside the distance image capturing device 1 , but the distance image processing unit 4 may be a component provided outside the distance image capturing device 1 .

[0046] Here, use Figure 2 The configuration of the distance image sensor 32 used as an imaging element in the distance image imaging device 1 will be described. Figure 2 This is a block diagram showing a schematic configuration of an image pickup element (range image sensor 32 ) used in the range image pickup device 1 according to the embodiment.

[0047] like Figure 2 As shown, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels 321 are arranged, a control circuit 322 , a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324 , and a pixel signal processing circuit 325 .

[0048] The light receiving area 320 is an area where a plurality of pixels 321 are arranged. Figure 2 3 shows an example of being arranged in a two-dimensional matrix of 8 rows and 8 columns. The pixel 321 accumulates a charge corresponding to the amount of light received. The control circuit 322 generally controls the distance image sensor 32. The control circuit 322 controls the operation of the components of the distance image sensor 32, for example, according to the instruction from the timing control unit 41 of the distance image processing unit 4. In addition, it is also possible to have a configuration in which the control of the components of the distance image sensor 32 is directly performed by the timing control unit 41, in which case the control circuit 322 can also be omitted.

[0049] The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving area 320 for each row according to the control from the control circuit 322. The vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge respectively accumulated in the charge accumulation section CS of the pixel 321 to the pixel signal processing circuit 325. In this case, the vertical scanning circuit 323 distributes and accumulates the charge converted by the photoelectric conversion element to the charge accumulation section of the pixel 321. That is, the vertical scanning circuit 323 is an example of a "pixel driving circuit". Specifically, the vertical scanning circuit 323 distributes and accumulates the charge to the charge accumulation section CS at the accumulation timing synchronized with the irradiation of the light pulse PO.

[0050] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (for example, noise suppression processing, A / D conversion processing, etc.) on the voltage signal output from the pixels 321 of each column to the corresponding vertical signal line under the control of the control circuit 322 .

[0051] The horizontal scanning circuit 324 sequentially outputs the signal output from the pixel signal processing circuit 325 to the horizontal signal line according to the control from the control circuit 322. Thus, the pixel signal corresponding to the charge amount accumulated for one frame is sequentially output to the image processing unit 4 via the horizontal signal line.

[0052] In the following description, it is assumed that the pixel signal processing circuit 325 performs A / D conversion processing and the pixel signal is a digital signal.

[0053] Here, use Figure 3 The configuration of the pixels 321 arranged in the light receiving area 320 included in the distance image sensor 32 will be described. Figure 3 1 is a circuit diagram showing an example of the configuration of pixels 321 arranged in the light receiving region 320 of the distance image sensor 32 according to the embodiment. Figure 3 An example of the configuration of one pixel 321 among the plurality of pixels 321 arranged in the light receiving region 320 is shown. The pixel 321 is an example of a configuration including three pixel signal readout units.

[0054] The pixel 321 includes one photoelectric conversion element PD, a drain-gate transistor GD, and three pixel signal readout units RU that output voltage signals from corresponding output terminals O. The pixel signal readout units RU each include a readout gate transistor G, a floating diffusion region FD, a charge storage capacity C, a reset gate transistor RT, a source follower gate transistor SF, and a select gate transistor SL. In each pixel signal readout unit RU, the floating diffusion region FD and the charge storage capacity C constitute a charge storage unit CS.

[0055] In addition, Figure 3 In the figure, the numbers "1", "2" or "3" are added after the reference numerals "RU" of the three pixel signal readout units RU to distinguish each pixel signal readout unit RU. Similarly, for each component of the three pixel signal readout units RU, the numbers representing each pixel signal readout unit RU are shown after the reference numerals to distinguish the pixel signal readout units RU corresponding to each component.

[0056] exist Figure 3 In the pixel 321 shown, the pixel signal readout section RU1 that outputs a voltage signal from the output terminal O1 includes a readout gate transistor G1, a floating diffusion region FD1, a charge storage capacity C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a select gate transistor SL1. In the pixel signal readout section RU1, the floating diffusion region FD1 and the charge storage capacity C1 constitute a charge storage section CS1. The pixel signal readout sections RU2 and RU3 have the same configuration.

[0057] The photoelectric conversion element PD is an embedded photodiode that generates electric charge by photoelectric conversion of incident light and accumulates the generated electric charge. The structure of the photoelectric conversion element PD can be arbitrary. The photoelectric conversion element PD can be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined together, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. In addition, the photoelectric conversion element PD is not limited to a photodiode, and can be, for example, a grating-type photoelectric conversion element.

[0058] In the pixel 321 , charges generated by the photoelectric conversion element PD performing photoelectric conversion on incident light are distributed to the three charge storage units CS, respectively, and voltage signals corresponding to the amount of the distributed charges are output to the pixel signal processing circuit 325 .

[0059] The configuration of the pixels arranged in the distance image sensor 32 is not limited to Figure 3 The configuration with three pixel signal readout sections RU as shown above can be any pixel as long as it has a configuration with multiple pixel signal readout sections RU. That is, the number of pixel signal readout sections RU (charge storage sections CS) provided in the pixels arranged in the distance image sensor 32 can be two or four or more.

[0060] In addition, Figure 3In the pixel 321 of the structure shown, an example is shown in which the charge storage unit CS is composed of the floating diffusion region FD and the charge storage capacity C. However, the charge storage unit CS only needs to be composed of the floating diffusion region FD, and the pixel 321 may not have the charge storage capacity C.

[0061] In addition, Figure 3 The pixel 321 of the structure shown in the figure shows an example of a structure including a drain-gate transistor GD. However, when it is not necessary to discard the charge accumulated (remaining) in the photoelectric conversion element PD, a structure without the drain-gate transistor GD is also possible.

[0062] Figure 4 It is a diagram for explaining the direct reflected light RLd and the multipath reflected light RLm according to the embodiment. Figure 4 FIG. 2 schematically shows a situation in which the distance image capturing device 1 irradiates the light pulse PO to the object OB. Figure 4 As shown, the reflected light RL received by the distance image camera 1 includes direct reflected light RLd and multi-path reflected light RLm. That is, a part of the light pulse PO irradiated by the distance image camera 1 is reflected by the object OB and reaches the distance image camera 1, and is received by the light receiving unit 3 as direct reflected light RLd. On the other hand, another part of the light pulse PO irradiated by the distance image camera 1 is reflected by the floor etc. and then reflected by the object OB and reaches the distance image camera 1, and is received by the light receiving unit 3 as multi-path reflected light RLm. Such multi-path reflected light RLm has a longer optical path to be received by the light receiving unit 3 than the direct reflected light RLd, and is received with a delay compared to the direct reflected light RLd, so there is a problem that correct distance calculation cannot be performed.

[0063] As a countermeasure, in the present embodiment, point light is used as the light source for irradiation by the light source unit 2. The point light source is structured light composed of a plurality of point lights. By using the point light source, the object OB is irradiated with non-uniform local light pulses PO.

[0064] Figure 5 3 is a diagram showing an example of light received by the light receiving area 320 according to the embodiment. Figure 5 As shown, by using a point light source, there are areas illuminated by the point light and areas not illuminated by the point light in the object OB, and there are two types of pixels 321, namely, pixels 321A and pixels 321B, in the light receiving area 320 of the light receiving unit 3. For light coming from the area illuminated by the point light in the object OB, the pixel 321A receives light, while the pixel 321B does not receive light. The pixel 321A is an example of a "first pixel circuit". The pixel 321B is an example of a "second pixel circuit".

[0065] When a point light source is used, the pixel 321A receives the reflected light from the object OB. On the other hand, it can be considered that the pixel 321B does not receive the reflected light from the object OB. However, in reality, the light that reaches the object OB after being reflected by the floor, wall, etc. is received by the distance image camera 1 as multi-path reflected light RLm, and this multi-path reflected light RLm becomes one of the important factors of error in distance calculation.

[0066] In the presence of multipath reflected light RLm, by using a point light source, pixel 321A receives light in which reflected light from object OB and multipath reflected light RLm are mixed, and pixel 321B receives only multipath reflected light RLm. In the present embodiment, by utilizing this property, the influence of multipath reflected light RLm is suppressed by subtracting the amount of light received by pixel 321B from the amount of light received by pixel 321A.

[0067] More specifically, the distance calculation unit 42 subtracts the signal value corresponding to the amount of charge accumulated in the charge accumulation unit CS of the pixel 321B from the signal value corresponding to the amount of charge accumulated in the charge accumulation unit CS of the pixel 321A. The distance calculation unit 42 performs processing calculation using the subtracted value. Thus, the distance calculation can be performed using a signal corresponding to only the direct reflected light RLd excluding the multipath reflected light RLm, and the influence of the multipath reflected light RLm can be suppressed.

[0068] Fig. 6A as well as Figure 6B A timing diagram for pixel 321 is shown. Fig. 6A as well as Figure 6B , the timing at which the irradiated light (light pulse PO) is irradiated, the timing at which the direct reflected light RLd and the multipath reflected light RLm are received, and the timing at which the pixel 321 is driven are shown. As the timing at which the pixel 321 is driven, the item "G1" represents the driving signal TX1 for driving the readout gate transistor G1 of the pixel 321, the item "G2" represents the driving signal TX2 for driving the readout gate transistor G2, the item "G3" represents the driving signal TX3 for driving the readout gate transistor G3, and the item "GD" represents the driving signal RSTD for driving the drain gate transistor GD.

[0069] exist Fig. 6A as well as Figure 6B 2 is a timing chart showing the timing for driving the pixel 321A.

[0070] The vertical scanning circuit 323 turns off the drain gate transistor GD and turns on the read gate transistor G1 at the same timing as the timing of the irradiation light pulse PO. The vertical scanning circuit 323 turns off the read gate transistor G1 after a time corresponding to the accumulation time period has passed since the read gate transistor G1 was turned on. Thus, the charge obtained by the photoelectric conversion element PD during the period when the read gate transistor G1 is controlled to be in the on state is accumulated in the charge accumulation unit CS1 via the read gate transistor G1.

[0071] Next, the vertical scanning circuit 323 turns on the read gate transistor G2 for a time corresponding to the accumulation period at the timing when the read gate transistor G1 is turned off. Thus, the charge photoelectrically converted by the photoelectric conversion element PD during the period when the read gate transistor G2 is controlled to be turned on is accumulated in the charge accumulation unit CS2 via the read gate transistor G2.

[0072] Next, the vertical scanning circuit 323 turns on the read gate transistor G3 at the timing when the charge accumulation in the charge accumulation unit CS2 is completed, and turns off the read gate transistor G3 after a time corresponding to the accumulation time period has passed. Thus, the charge photoelectrically converted by the photoelectric conversion element PD during the period when the read gate transistor G3 is controlled to be in the on state is accumulated in the charge accumulation unit CS3 via the read gate transistor G3.

[0073] Then, the vertical scanning circuit 323 turns on the drain-gate transistor GD to discharge the charge at the timing when the charge accumulation in the charge accumulation unit CS3 is completed. Thus, the charge photoelectrically converted by the photoelectric conversion element PD is discarded via the drain-gate transistor GD.

[0074] like Fig. 6A As shown, the multipath reflected light RLm is received with a delay relative to the direct reflected light RLd. During the period when the gate transistor G2 of the pixel 321A is in the on state, the direct reflected light RLd1 (a part of the direct reflected light RLd) and the multipath reflected light RLm1 (a part of the multipath reflected light RLm) are received, and charges corresponding to the amount of light received are accumulated in the charge accumulation unit CS2. In addition, during the period when the gate transistor G3 of the pixel 321A is in the on state, the direct reflected light RLd2 (the remaining part of the direct reflected light RLd2) and the multipath reflected light RLm2 (the remaining part of the multipath reflected light RLm) are received, and charges corresponding to the amount of light received are accumulated in the charge accumulation unit CS3. That is, charges corresponding to the amount of light mixed with the direct reflected light RLd and the multipath reflected light RLm are distributed to the charge accumulation units CS2 and CS3 of the pixel 321A and accumulated.

[0075] like Figure 6B As shown, the pixel 321B does not receive the direct reflected light RLd, but only receives the multipath reflected light RLm. During the period when the gate transistor G2 of the pixel 321B is in the on state, the multipath reflected light RLm1 is received, and the charge corresponding to the amount of the received light is accumulated in the charge accumulation unit CS2. In addition, during the period when the gate transistor G3 of the pixel 321B is in the on state, the multipath reflected light RLm2 is received, and the charge corresponding to the amount of the received light is accumulated in the charge accumulation unit CS3. That is, the charge corresponding to the amount of the multipath reflected light RLm is distributed to and accumulated in the charge accumulation units CS2 and CS3 of the pixel 321B.

[0076] The vertical scanning circuit 323 repeats the above-mentioned accumulation cycle for a predetermined number of accumulation times. After the vertical scanning circuit 323 repeats the accumulation cycle for a predetermined number of accumulation times, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge allocated to each charge accumulation unit CS. Specifically, the vertical scanning circuit 323 turns on the selection gate transistor SL1 for a predetermined time, thereby outputting a voltage signal corresponding to the amount of charge accumulated in the charge accumulation unit CS1 from the output terminal O1 via the pixel signal readout unit RU1. Similarly, the vertical scanning circuit 323 turns on the selection gate transistors SL2 and SL3 in turn, thereby outputting a voltage signal corresponding to the amount of charge accumulated by the charge accumulation units CS2 and CS3 from the output terminals O2 and O3. Then, via the pixel signal processing circuit 325 and the horizontal scanning circuit 324, a voltage signal corresponding to the amount of charge accumulated by each charge accumulation unit CS is output as a signal value to the distance image processing unit 4.

[0077] The distance calculation unit 42 of the distance image processing unit 4 calculates the distance to the object OB based on the signal value output from the vertical scanning circuit 323. The distance calculation unit 42 determines the plurality of charge storage units CS to which the charges corresponding to the reflected light RL are distributed and accumulated, and calculates the distance to the object OB based on the ratio (distribution ratio) of the charge amounts distributed to the determined plurality of charge storage units CS.

[0078] The distance calculation unit 42 calculates the charge amount Q2 after removing the influence of the multipath reflected light RLm using the following formula (1). Also, the distance calculation unit 42 calculates the charge amount Q3 after removing the influence of the multipath reflected light RLm using the following formula (2).

[0079] Q2=Q2A-Q2B…(1)

[0080] Among them, Q2 is the amount of charge accumulated in the charge storage unit CS2 to which the direct reflected light RLd is distributed, Q2A is the amount of charge accumulated in the charge storage unit CS2 of the pixel 321A, and Q2B is the amount of charge accumulated in the charge storage unit CS2 of the pixel 321B.

[0081] Q3=Q3A-Q3B…(2)

[0082] Among them, Q3 is the amount of charge accumulated in the charge storage unit CS3 to which the direct reflected light RLd is distributed, Q3A is the amount of charge accumulated in the charge storage unit CS3 of the pixel 321A, and Q3B is the amount of charge accumulated in the charge storage unit CS3 of the pixel 321B.

[0083] The distance calculation unit 42 calculates the delay time Td according to the following formula (3).

[0084] Td=To×(Q2-Q1) / (Q2+Q3-2×Q1)…(3)

[0085] Q1 is the amount of charge accumulated in the charge accumulation section CS1 of the pixel 321A or the pixel 321B.

[0086] Q2 is the charge amount obtained from equation (1).

[0087] Q3 is the charge amount obtained from equation (2).

[0088] The distance calculation unit 42 calculates the round trip distance to the object OB by multiplying the delay time Td calculated by the equation (3) by the speed of light (velocity). Then, the distance calculation unit 42 calculates the distance to the object OB by halving the calculated round trip distance.

[0089] As described above, in this embodiment, according to equations (1) and (2), the charge amount obtained by subtracting the charge amount corresponding to the multipath reflected light RLm from the charge amount accumulated by the charge accumulation units CS2 and CS3, that is, the charge amount corresponding to the direct reflected light RLd can be calculated. Therefore, the influence of the multipath reflected light RLm can be suppressed and the distance can be calculated with high accuracy.

[0090] Here, which pixel 321 among the pixels 321 in the light receiving area 320 is set as the pixel 321A and which pixel is set as the pixel 321B is determined according to the position where the pixel 321 is arranged. Figure 2As shown, a plurality of pixels 321 are arranged in a two-dimensional matrix in the light receiving area 320. For example, the distance calculation unit 42 pre-classifies the pixels 321 arranged in the light receiving area 320 into pixels 321 that receive the direct reflected light RLd corresponding to the point light and pixels 321 that do not receive light, based on the dot pattern and the shape of the dots in the point light source. The distance calculation unit 42 sets the pixels 321 that receive the direct reflected light RLd corresponding to the point light as pixels 321A. The pixels 321 located around the pixels 321A are set as pixels 321 used as pixels 321B.

[0091] The distance calculation unit 42 may use a plurality of pixels 321 as the pixel 321B. In this case, the distance calculation unit 42 calculates the charge amount Q2B in the formula (1) and the charge amount Q3B in the formula (2) using a statistic obtained from the charge amount accumulated by each of the plurality of pixels 321 located around the pixel 321A, such as a simple addition mean or a median. By using the statistic obtained from the charge amount accumulated by each of the plurality of pixels 321, the influence of the noise accumulated by the pixel 321 can be reduced.

[0092] In addition, the distance calculation unit 42 uses the position of the pixel 321A having the charge accumulation unit CS with the largest amount of charge as a reference, and selects the pixel 321 located at a position more than one pixel away from the reference position in the diagonal direction as the pixel 321B. Although it also varies depending on the size and shape of the point light, the direct reflected light RLd corresponding to the point light is usually received across multiple pixels 321. Using this property, the position where the pixel 321 that accumulates the charge corresponding to the maximum amount of light is arranged among the multiple pixels 321 that receive the direct reflected light RLd corresponding to the point light is set as the reference. For example, the distance calculation unit 42 sets the pixel 321 arranged at a position that is away from the reference position by a predetermined number of pixels (for example, one pixel) in the diagonal direction as the pixel 321B. The shape of the point light is often circular. By selecting the pixel 321 arranged at a position away in the diagonal direction as the pixel 321B, the pixel 321 that is located closest to the reference position and does not receive the direct reflected light RLd can be selected as the pixel 321B. In addition, the selection of the pixel 321 located at a position more than several pixels away in the diagonal direction from the position where the reference pixel 321 is arranged can be determined according to the size of the spot light.

[0093] Alternatively, the distance calculation unit 42 may select, as the pixel 321B, a pixel 321 located at a position more than one pixel away in the horizontal direction or the vertical direction. That is, the distance calculation unit 42 may select, as the pixel 321B, a pixel 321 located at a position more than one pixel away in the horizontal direction or the vertical direction from the position serving as the reference, using the position of the pixel 321A having the charge storage unit CS with the largest amount of charge as the reference. In the case where the shape of the point light is a square shape rather than a circle, by selecting as the pixel 321B a pixel 321 arranged at a position away in the horizontal direction or the vertical direction, the pixel 321 located at a position closest to the reference position and not receiving the direct reflected light RLd may be selected as the pixel 321B.

[0094] In addition, the distance calculation unit 42 may also weight the charge amount (charge amount Q2B, Q3B) accumulated by the charge accumulation unit CS in the pixel 321B according to the position where the pixel 321 is arranged, and use the weighted charge amount for distance calculation. For example, it can be expected that the pixel 321 arranged on the lower side in the light receiving area 320 is greatly affected by the multi-path light reflected from the floor. In this way, when the degree of influence of the multi-path light in the light receiving area 320 varies depending on the position where the multi-path light is arranged, the influence of the multi-path light can be made uniform by performing weighting according to the position.

[0095] Figure 7 is a diagram illustrating weighting processing in an embodiment. Figure 7 As shown, in the light receiving area 320, a group of pixels 321 belonging to the group Gr on the lower side is arranged, and four pixels 321 existing in the diagonal direction relative to the position where the reference pixel 321-0 is arranged, that is, pixels 321-1 to 321-4 are used as pixels 321B.

[0096] In this case, it can be expected that among the pixels 321 belonging to the group Gr, the pixels 321-3 and 321-4 on the lower side are more affected by the multipath light than the pixels 321-1 and 321-2 on the upper side. In this case, the distance calculation unit 42 calculates the charge amount Q2B by performing weighting using, for example, the following formula (4). In addition, the charge amount Q3B is calculated by performing weighting using the following formula (5).

[0097] Q2B=Q2B1×α1

[0098] +Q2B2×α2

[0099] +Q2B3×α3

[0100] +Q2B4×α4…(4)

[0101] Here, Q2B is the amount of charge accumulated in the charge accumulation unit CS2 obtained by weighting.

[0102] Q2B1 is the amount of charge accumulated in the charge accumulation section CS2 of the pixel 321 - 1 .

[0103] Q2B2 is the amount of charge accumulated in the charge accumulation section CS2 of the pixel 321 - 2 .

[0104] Q2B3 is the amount of charge accumulated in the charge accumulation section CS2 of the pixel 321 - 3 .

[0105] Q2B4 is the amount of charge accumulated in the charge accumulation section CS2 of the pixel 321 - 4 .

[0106] α1 is a weighting coefficient by which the charge amount Q2B1 is multiplied.

[0107] α2 is a weighting coefficient by which the charge amount Q2B2 is multiplied.

[0108] α3 is a weighting coefficient by which the charge amount Q2B3 is multiplied.

[0109] α4 is a weighting coefficient by which the charge amount Q2B4 is multiplied.

[0110] Here, the total value of the coefficient α (coefficients α1 to α4) is set to a specific value (for example, 1). In addition, the coefficients (coefficients α1 and α2) multiplied by the pixel 321 on the upper side are set to values ​​smaller than the coefficients (coefficients α3 and α4) multiplied by the pixel 321 on the lower side. For example, the coefficient α1 is set to 1 / 8, the coefficient α2 is set to 1 / 8, the coefficient α3 is set to 3 / 8, and the coefficient α4 is set to 3 / 8.

[0111] Q3B=Q3B1×β1

[0112] +Q3B2×β2

[0113] +Q3B3×β3

[0114] +Q3B4×β4…(5)

[0115] Here, Q3B is the amount of charge accumulated in the charge accumulation unit CS3 obtained by weighting.

[0116] Q3B1 is the amount of charge accumulated in the charge accumulation section CS3 of the pixel 321 - 1 .

[0117] Q3B2 is the amount of charge accumulated in the charge accumulation section CS3 of the pixel 321 - 2 .

[0118] Q3B3 is the amount of charge accumulated in the charge accumulation section CS3 of the pixel 321 - 3 .

[0119] Q3B4 is the amount of charge accumulated in the charge accumulation section CS3 of the pixel 321 - 4 .

[0120] β1 is a weighting coefficient by which the charge amount Q2B1 is multiplied.

[0121] β2 is a weighting coefficient by which the charge amount Q2B2 is multiplied.

[0122] β3 is a weighting coefficient by which the charge amount Q2B3 is multiplied.

[0123] β4 is a weighting coefficient by which the charge amount Q2B4 is multiplied.

[0124] Here, the coefficient β (β1 to β4) may be the same value as the coefficient α or a different value. The total value of the coefficient β is set to a specific value (e.g., 1). In addition, the coefficients (coefficients β1 and β2) multiplied by the pixel 321 on the lower side are set to a value smaller than the coefficients (coefficients β3 and β4) multiplied by the pixel 321 on the lower side. For example, the coefficient β1 is set to 1 / 8, the coefficient β2 is set to 1 / 8, the coefficient β3 is set to 3 / 8, and the coefficient β4 is set to 3 / 8.

[0125] As described above, in the distance calculation unit 42 of the embodiment, when a plurality of pixels 321B (pixels 321-1 to 321-4) are used, the amount of charge accumulated in the charge accumulation unit CS in each pixel 321B (pixels 321-1 to 321-4) is weighted using equations (4) and (5) according to the positions at which the pixels 321B (pixels 321-1 to 321-4) are arranged. The distance calculation unit 42 calculates the amount of charge Q2B and Q3B using the weighted values. Thus, in the distance calculation unit 42 of the embodiment, even when the degree to which the pixel 321 is affected by the multipath light varies depending on the arranged position, the influence of the multipath light can be made uniform.

[0126] Here, a phenomenon in which a pixel 321 that should not receive the direct reflected light RLd corresponding to the point light but receives a larger amount of light than the multipath reflected light RLm is sometimes detected in the light receiving area 320. It is presumed that the reason for this is that the multipath reflected light RLm, which should originally be the multipath light derived from the diffuse reflected light RL, is mixed with the multipath reflected light RLm, which is mostly derived from the diffuse reflected light RL. This phenomenon is considered to be caused by, for example, the presence of a floor that performs specular reflection.

[0127] use Fig. 8A as well as Figure 8B The phenomenon of mixing multipath light originating from the direct reflected light RLd is described below. Fig. 8A An example of the light received by the light receiving area 320 is shown in FIG. Fig. 8A As shown, in pixel 321B# in light receiving area 320 , although it should not receive direct reflected light RLd corresponding to the point light, it sometimes receives multipath light originating from direct reflected light RLd corresponding to the point light.

[0128] Figure 8B It means that it is configured in Fig. 8A The dotted line L in FIG. 1 is a graph showing changes in the amount of light received by each pixel 321 in the column. Figure 8B The horizontal axis represents the pixel 321, and the vertical axis represents the light amount. Figure 8B As shown, there is a large difference in the amount of charge stored in each of the five pixel groups X arranged along the column to the right from the pixel 321B-Ls at the left end of the column, the 6th to 10th pixel groups Y, and the pixel group Z from the 11th to the pixel 321B-Le at the right end of the column.

[0129] Specifically, the amount of charge stored in the charge storage unit CS in the pixel group X and the pixel group Z is almost the same, with almost no difference. For example, the difference D1 between the amount of charge stored in the pixel 321B-L1 and the pixel 321B-L2 adjacent to the pixel 321B-L1 is less than ±1% of the pixel 321B-L1.

[0130] In contrast, the amount of charge accumulated in the charge storage unit CS in the pixel group Y is greater than that in the pixel group X and the pixel group Z. For example, the difference D2 between the amount of charge accumulated in the pixel 321B-L3 and the pixel 321B-L4 adjacent to the pixel 321B-L3 is greater than +3% of the pixel 321B-L3. This is believed to be caused by the fact that the pixel group Y receives multipath light from the direct reflected light RLd.

[0131] This multipath light derived from the directly reflected light RLd is received with a delay compared to the directly reflected light RLd, similarly to the originally expected multipath reflected light RLm. Figure 8B This indicates that the amount of multipath reflected light RLm is larger than originally expected, and therefore may become a significant factor causing a large error in distance calculation.

[0132] As a countermeasure, in the present embodiment, the pixel 321 that receives the multipath light from the direct reflected light RLd is not used as the pixel 321B. Specifically, the distance calculation unit 42 does not use, as the pixel 321B, a pixel 321 in which the charge storage unit CS stores a larger amount of charge than other pixels 321 among the adjacent pixels 321B.

[0133] Specifically, the distance calculation unit 42 obtains signal values ​​corresponding to the amount of charge accumulated by the charge storage unit CS for each adjacent pixel group, for example, in columns, for the pixel group configured at the position used as the pixel 321B. The distance calculation unit 42 determines whether to use the pixel 321 included in the pixel group as the pixel 321B based on the obtained signal value. For example, the distance calculation unit 42 calculates the average value (simple addition mean) of the amount of charge accumulated for each charge storage unit CS using the obtained signal value. The distance calculation unit 42 sets a threshold value based on the calculated average value, and the threshold value is, for example, a value obtained by adding 3% of the maximum margin allowed as a measurement error to the average value. The distance calculation unit 42 does not select the pixel 321 in the pixel group for which the signal value is obtained, and the amount of charge accumulated by the charge storage unit CS in the pixel 321 is greater than the threshold value, as the pixel 321B. That is, the distance calculation unit 42 selects, as the pixel 321B, a pixel 321 in which the amount of charge accumulated in the charge accumulation unit CS is smaller than the threshold value, among the pixel group for which the signal value is acquired.

[0134] Thus, in the distance image capturing device 1 of the embodiment, the pixel 321 that receives the multipath light originating from the direct reflection light RLd can be excluded from the pixel 321B. Therefore, even in a situation where there is a floor that performs specular reflection and multipath light originating from the direct reflection light RLd is generated, the originally expected multipath reflection light RLm can be excluded with high accuracy, and the distance can be calculated with high accuracy.

[0135] Here, use Fig. 9 A distance image capturing method performed by the distance image capturing device 1 will be described. Fig. 9 This is a flowchart showing the flow of processing performed by the distance image capturing device 1 according to the embodiment.

[0136] Step S10: The distance image capturing device 1 drives the pixel 321. For example, the distance image capturing device 1 sends a charge to the plurality of charge storage units CS ( Fig. 6A as well as Figure 6B The reflected light RL is distributed and accumulated by the charge accumulation units CS2 and CS3 in the image processing unit, and the accumulation times corresponding to one frame are performed, thereby driving the pixel 321.

[0137] Step S11: The distance image camera device 1 obtains a signal value VA corresponding to the amount of charge (charge amount QA) accumulated by the charge accumulation unit CS in the pixel 321A (pixel A). The distance image camera device 1 reads out a signal value corresponding to the amount of charge accumulated by each charge accumulation unit CS in the pixel 321A during the readout period set after driving the pixel 321 in step S1. Thus, the distance image camera device 1 obtains the signal value VA. More specifically, the distance image camera device 1 obtains a signal value VA1 corresponding to the amount of charge (charge amount QA1) accumulated by the charge accumulation unit CS1 in the pixel 321A. The distance image camera device 1 obtains a signal value VA2 corresponding to the amount of charge (charge amount QA2) accumulated by the charge accumulation unit CS2 in the pixel 321A. The distance image camera device 1 obtains a signal value VA3 corresponding to the amount of charge (charge amount QA3) accumulated by the charge accumulation unit CS3 in the pixel 321A.

[0138] Step S12: The distance image camera 1 obtains a signal value VB corresponding to the amount of charge (charge amount QB) accumulated by the charge accumulation section CS in the pixel 321B (pixel B). The distance image camera 1 reads out a signal value corresponding to the amount of charge accumulated by each charge accumulation section CS in the pixel 321B during the readout period set after driving the pixel 321 in step S1. Thus, the distance image camera 1 obtains a signal value VA. More specifically, the distance image camera 1 obtains a signal value VB1 corresponding to the amount of charge (charge amount QB1) accumulated by the charge accumulation section CS1 in the pixel 321B. The distance image camera 1 obtains a signal value VB2 corresponding to the amount of charge (charge amount QB2) accumulated by the charge accumulation section CS2 in the pixel 321B. The distance image camera 1 obtains a signal value VB3 corresponding to the amount of charge (charge amount QB3) accumulated by the charge accumulation section CS3 in the pixel 321B.

[0139] Step S13: The distance image camera 1 calculates a signal value V by subtracting the signal value VB from the signal value VA. More specifically, the distance image camera 1 calculates a signal value V2 by subtracting the signal value VB2 from the signal value VA2. The distance image camera 1 calculates a signal value V3 by subtracting the signal value VB3 from the signal value VA3.

[0140] Step S14: The distance image camera 1 calculates the distance using the signal value V (signal values ​​VA2 and V3) calculated in step S13. For example, the distance image camera 1 calculates the distance using formula (3). In this case, the signal value V is substituted for the charge amount Q in formula (3). Specifically, the signal value V1 is substituted for the charge amount Q1 in formula (3). The signal value V1 is a signal value corresponding to the charge amount accumulated by the charge storage unit CS1 in the pixel 321 (which can be either pixel 321A or 321B). In addition, the signal value V2 is substituted for the charge amount Q2 in formula (3). The signal value V3 is substituted for the charge amount Q3 in formula (3). The distance image camera 1 ends the measurement process.

[0141] As described above, in the distance image camera device 1 of the embodiment, a plurality of pixels 321 are arranged in a two-dimensional matrix. In addition, the light pulse PO is a structured light composed of a plurality of point lights. The distance calculation unit 42 calculates the distance to the object OB using a value obtained by subtracting the charge amount QB from the charge amount QA (first charge amount). The charge amount QA is the charge amount accumulated by the charge accumulation unit CS in the pixel 321A (first pixel circuit) that receives the direct reflected light RLd. The charge amount QB is the charge amount accumulated by the charge accumulation unit CS in the pixel 321B (second pixel circuit) that does not receive the direct reflected light RLd.

[0142] Thus, in the distance image capturing device 1 of the embodiment, the amount of light corresponding to the multipath reflected light RLm can be removed from the amount of light in which the direct reflected light RLd and the multipath reflected light RLm are mixed. Therefore, the influence of the multipath reflected light RLm, which is a major factor of error, can be suppressed, and the distance can be calculated with high accuracy.

[0143] In the range image capturing device 1 of the embodiment, the charge amount QB (second charge amount) may be calculated using a plurality of pixels 321B (second pixel circuits). By using a plurality of pixels 321B, noise included in the charge and variations in the amount of received light can be reduced.

[0144] In the distance image capturing device 1 of the embodiment, the distance calculation unit 42 may select a pixel 321 (pixel circuit) located at a position diagonally away from the reference position by one pixel or more as the pixel 321B (second pixel circuit).

[0145] In the distance image capturing device 1 of the embodiment, the distance calculation unit 42 may select a pixel 321 (pixel circuit) located one pixel or more away in the horizontal or vertical direction from the reference position as the pixel 321B (second pixel circuit).

[0146] The reference position here is the position of the pixel 321A (first pixel circuit) having the charge storage section CS having the largest amount of charge stored in the charge storage sections CS in the plurality of pixels 321A (first pixel circuit).

[0147] Thus, in the distance image capturing device 1 of the embodiment, the pixel 321 close to the position where the pixel 321A is arranged can be used as the pixel 321B. That is, the pixel 321 (pixel 321A and 321B) estimated to have the same light amount of the multipath reflected light RLm can be used. Therefore, the influence of the multipath reflected light RLm can be eliminated with high accuracy, and the distance can be calculated with high accuracy.

[0148] In addition, in the distance image camera device 1 of the embodiment, the distance calculation unit 42 calculates the charge amount QB (second charge amount) using a value obtained by weighting the charge amount accumulated by the charge accumulation unit CS in the pixel 321B (second pixel circuit) according to the position where the pixel 321B (second pixel circuit) is configured. Thus, in the distance image camera device 1 of the embodiment, even in the case where multi-path reflected light RLm with different light amounts is received depending on the position where the pixel 321 is configured in the light receiving area 320, the light amount of the multi-path reflected light RLm can be made uniform. Therefore, for example, even if the pixel 321 ( Figure 7 The pixels 321-3 and 321-4 in the middle are larger than the pixels 321 ( Figure 7 Even in a situation where the pixels 321-1 and 321-2 in the image are more affected by the multipath reflected light RLm, the deviation of its influence can be suppressed.

[0149] In addition, in the distance image camera device 1 of the embodiment, when the distance calculation unit 42 uses a plurality of the second pixel circuits, when the difference in the amount of charge accumulated in the charge accumulation unit CS in the adjacent pixels 321 is greater than the threshold value, the pixel 321 in which the charge accumulation unit CS has a larger amount of charge among the adjacent pixels 321 is not selected as the pixel 321B (second pixel circuit). Thus, in the distance image camera device 1 of the embodiment, even if the pixel 321, for example, which receives multipath light different from the originally expected multipath reflected light RLm, is included, Fig. 8A In the case of pixel 321B# shown in the figure, it is also possible to avoid being affected by multipath light different from the originally expected multipath reflected light RLm. Therefore, even when multipath light different from the originally expected multipath reflected light RLm is received, the distance can be calculated with high accuracy.

[0150] All or part of the distance image camera device 1 and the distance image processing unit 4 in the above-mentioned embodiment can also be realized by a computer. In this case, the program for realizing the function can be recorded in a computer-readable recording medium, and the program recorded in the recording medium can be read into the computer system and executed. In addition, the "computer system" mentioned here includes hardware such as OS and peripheral devices. In addition, "computer-readable recording medium" refers to removable media such as floppy disks, optical magnetic disks, ROMs, CD-ROMs, and storage devices such as hard disks built into the computer system. In addition, "computer-readable recording medium" also includes a medium that dynamically maintains the program for a short period of time, such as a communication line in the case of sending a program via a network such as the Internet, a communication line such as a telephone line, etc., and a medium that maintains the program for a certain period of time, such as a volatile memory inside the computer system of the server or client in this case. In addition, the above-mentioned program can be a program for realizing a part of the above-mentioned function, a program that realizes the above-mentioned function by combining with a program already recorded in the computer system, or a program realized using a programmable logic device such as an FPGA.

[0151] As mentioned above, although the embodiment of the present invention is described in detail with reference to the drawings, the specific configuration is not limited to the embodiment, and also includes designs and the like within the scope that does not depart from the gist of the present invention.

[0152] Industrial Applicability

[0153] According to the present invention, the influence of multipath reflected light can be suppressed.

[0154] Description of Reference Numerals

[0155] 1: Distance image camera

[0156] 2: Light source

[0157] 3: Light receiving part

[0158] 32: Distance image sensor

[0159] 321: Pixel (pixel circuit)

[0160] 321A: Pixel (first pixel circuit)

[0161] 321B: Pixel (second pixel circuit)

[0162] 42: Distance calculation unit

[0163] CS: Charge storage unit

[0164] PO: Light Pulse

[0165] RL: Reflected Light

[0166] RLd: Direct reflected light

[0167] RLm: Multipath reflected light

Claims

1. A distance image camera device, comprising: A light source unit irradiates light pulses to the subject; a light receiving unit having a pixel circuit and a pixel driving circuit, wherein a plurality of the pixel circuits are arranged in a two-dimensional matrix and have a photoelectric conversion element that generates charges corresponding to incident light, and N charge storage units that store the charges, wherein N ≥ 3, and wherein the pixel driving circuit distributes the charges to the charge storage units at storage timings synchronized with the irradiation of the light pulses and stores the charges; and a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated by the charge accumulation unit; The light pulse is a structured light composed of multiple point lights. The distance calculation unit calculates the distance to the subject using a value obtained by subtracting a second charge amount based on the charge amount accumulated in the charge accumulation unit in a second pixel circuit from a first charge amount accumulated in the charge accumulation unit in a first pixel circuit, wherein the first pixel circuit receives directly reflected light and the second pixel circuit is located around the first pixel circuit and does not receive the directly reflected light.

2. The distance image camera device according to claim 1, wherein: The distance calculation unit calculates the second charge amount using the charge amount accumulated in the charge accumulation unit in each of the plurality of second pixel circuits.

3. The distance image camera device according to claim 1 or 2, wherein: The distance calculation unit uses the position of the first pixel circuit having the charge storage unit having the largest charge amount among the charge storage units located in the plurality of first pixel circuits as a reference, and selects the pixel circuit located at a position more than one pixel away diagonally from the reference position as the second pixel circuit.

4. The distance image camera device according to claim 1 or 2, wherein: The distance calculation unit uses the position of the first pixel circuit having the charge storage unit having the largest amount of charge among the charge storage units located in the plurality of first pixel circuits as a reference, and selects the pixel circuit located at a location more than one pixel away in the vertical direction or horizontal direction from the reference position as the second pixel circuit.

5. The distance image camera device according to claim 1 or 2, wherein: The distance calculation unit calculates the second charge amount using a value obtained by weighting the charge amount accumulated in the charge accumulation unit in the second pixel circuit according to a position where the second pixel circuit is arranged.

6. The distance image camera device according to claim 1 or 2, wherein: When the distance calculation unit uses a plurality of the second pixel circuits, if the difference in the amount of charge accumulated in the charge storage units in adjacent pixel circuits is greater than a threshold, the pixel circuit having a larger amount of charge accumulated in the charge storage unit is not selected as the second pixel circuit.

7. A distance image capturing method, performed by a distance image capturing device, The distance image camera device comprises: A light source unit irradiates light pulses to the subject; a light receiving unit having a pixel circuit and a pixel driving circuit, wherein a plurality of the pixel circuits are arranged in a two-dimensional matrix and have a photoelectric conversion element that generates charges corresponding to incident light, and N charge storage units that store the charges, wherein N ≥ 3, and wherein the pixel driving circuit distributes the charges to the charge storage units at storage timings synchronized with the irradiation of the light pulses and stores the charges; and a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated by the charge accumulation unit; In the distance image capturing method, The light pulse is a structured light composed of multiple point lights. The distance calculation unit calculates the distance to the subject using a value obtained by subtracting a second charge amount based on the charge amount accumulated in the charge accumulation unit in a second pixel circuit from a first charge amount accumulated in the charge accumulation unit in a first pixel circuit, wherein the first pixel circuit receives directly reflected light and the second pixel circuit is located around the first pixel circuit and does not receive the directly reflected light.

Images