Image Generation Device

Abstract

To easily generate an image while suppressing influences caused by mirror reflection.SOLUTION: An image generating device comprises: a first light-emitting section 10 and a second light-emitting section 20; a light-receiving section 30; a drive control section 50 for controlling a light-emitting period and a light-receiving period; and a combination section 70 for combining a first pixel signal s1 outputted by light reception of the light-receiving section 30 corresponding to light emission of the first light-emitting section 10 and a second pixel signal s2 outputted by light reception of the light-receiving section 30 corresponding to light emission of the second light-emitting section 20. The first light-emitting section 10 and the second light-emitting section 20 are provided at such positions that they are not overlapped in a view from a subject 9. The drive control section 50 causes the first light-emitting section 10 and the second light-emitting section 20 to emit light in different timings in the same frame. In a case where at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal greater than that with a predetermined level, the combination section 70 generates a combination signal s10 without using the pixel signal greater than that with the predetermined level.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to an image generating device that generates an image of a subject. [Background technology]

[0002] In recent years, systems have been proposed that use cameras to capture the faces of people and judge their health, alertness, concentration, etc. in real time. The light sources used to capture images of subjects are classified into visible light and invisible light, but many invisible light sources are used to capture images with stable brightness without being affected by ambient light.

[0003] One of the issues when capturing an image by irradiating a subject with invisible light and receiving the reflected light is saturation of pixel signals due to specular reflection. For example, if specular reflection occurs on a part of the subject during imaging, saturation occurs due to an excessive amount of received light, and part of the image information of the subject is lost. If part of the image information of the subject is lost, it leads to a decrease in accuracy when determining a person's health condition, wakefulness state, concentration state, etc.

[0004] Non-Patent Document 1 proposes a technology for capturing images using a stereo camera system. In the stereo camera system, an object is captured using two cameras arranged at a distance. With this technology, even if one of the cameras cannot obtain necessary information due to the influence of specular reflection, an image of the object can be generated using information from the other camera. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] "Improvement of glasses reflection problem in pupil detection using stereo-calibrated cameras," by Hisahisa Ishida, Kiyotake Fukumoto, and Yoshinobu Ebisawa, ITE Annual Convention 2011 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in Non-Patent Document 1, a mechanism is required to simultaneously capture an image of a subject using two cameras installed at separate locations and to mutually complement the two captured images, which results in a problem of increased device size.

[0007] Therefore, the present disclosure provides an image generating device that suppresses the effects of specular reflection and generates an image easily. [Means for solving the problem]

[0009] The image generating device of the present disclosure includes a first light emitting unit that irradiates light toward a subject, a second light emitting unit that irradiates light toward the subject, a light receiving unit that receives reflected light generated when the light irradiated by the first light emitting unit and the second light emitting unit is reflected by the subject, using a plurality of pixels that are two-dimensionally arranged, a drive control unit that controls a light emitting period of the first light emitting unit, a light emitting period of the second light emitting unit, and a light receiving period of the light receiving unit, and a control unit that controls a light receiving period of the light receiving unit, and outputs a light signal from the pixels of the light receiving unit in response to the irradiation of light by the first light emitting unit and the second light emitting unit. a saturation level determination section that determines whether the first pixel signal and the second pixel signal that are detected by the saturation level determination section exceed a saturation level, and a synthesis section that generates a synthesis signal by synthesizing the first pixel signal and the second pixel signal based on a determination result of the saturation level determination section, wherein the first light-emitting section and the second light-emitting section are disposed apart from each other, and the drive control section generates, in the same frame, a first light-emitting pulse that causes the first light-emitting section to emit light and a second light-emitting pulse that causes the second light-emitting section to emit light, at different timings, and a first light receiving pulse for receiving the reflected light generated by the light irradiated by a first light emitting unit being reflected by the subject, and a second light receiving pulse for receiving the reflected light generated by the light irradiated by the second light emitting unit being reflected by the subject, the light receiving unit outputs the first pixel signal including information on a first amount of received light received in accordance with the first light receiving pulse, and outputs the second pixel signal including information on a second amount of received light received in accordance with the second light receiving pulse, and the saturation level determination unit determines the first amount of received light, The second amount of received light is compared with the saturation level, and the combining unit (1) combines the first pixel signal and the second pixel signal with a first combining coefficient when the first amount of received light and the second amount of received light are both smaller than the saturation level, (2) combines the first pixel signal and the second pixel signal with a second combining coefficient when only the first amount of received light is larger than the saturation level, and (3) combines the first pixel signal and the second pixel signal with a third combining coefficient when only the second amount of received light is larger than the saturation level. and further comprising a correction unit which corrects the information on the first amount of received light or the information on the second amount of received light, wherein the drive control unit controls a first light receiving period of the light receiving unit corresponding to the irradiation of light from the first light emitting unit and a second light receiving period of the light receiving unit corresponding to the irradiation of light from the second light emitting unit to be different from each other, the correction unit corrects the information on the first amount of received light or the information on the second amount of received light based on an amount of received light caused by a difference between the first light receiving period and the second light receiving period, and the combining unit combines the first pixel signal and the second pixel signal using the information on the first amount of received light or the information on the second amount of received light corrected by the correction unit. The image generating device of the present disclosure includes a first light-emitting unit that irradiates light toward a subject, a second light-emitting unit that irradiates light toward the subject, a light-receiving unit that receives, with a plurality of pixels arranged in a two-dimensional shape, each of the reflected lights generated when the light irradiated by the first light-emitting unit and the second light-emitting unit is reflected by the subject, and a drive control unit that controls an emission period of the first light-emitting unit, an emission period of the second light-emitting unit, and a light-receiving period of the light-receiving unit. A saturation level determination unit that determines whether a first pixel signal and a second pixel signal output from the pixel of the light-receiving unit in response to the irradiation of light by the first light-emitting unit and the second light-emitting unit exceed a saturation level, and a combining unit that combines the first pixel signal and the second pixel signal based on a determination result of the saturation level determination unit to generate a combined signal, wherein the first light-emitting unit and the second light-emitting unit are arranged apart from each other, and the drive control unit controls the first light-emitting unit to emit light and the second light-emitting unit to emit light in the same frame. and generates, at different timings, a first light-receiving pulse for receiving the reflected light generated by light irradiated by the first light-emitting unit being reflected by the subject and a second light-receiving pulse for receiving the reflected light generated by light irradiated by the second light-emitting unit being reflected by the subject, the light-receiving unit outputs the first pixel signal including information on a first amount of received light received in accordance with the first light-receiving pulse and outputs the second pixel signal including information on a second amount of received light received in accordance with the second light-receiving pulse, the saturation level determination unit compares the first amount of received light, the second amount of received light, and the saturation level, and the combining unit (1) combines the first pixel signal and the second pixel signal with a first combining coefficient when the first amount of received light and the second amount of received light are both smaller than the saturation level, and (2) combines the first pixel signal and the second pixel signal with a second combining coefficient when only the first amount of received light is greater than the saturation level, (3) when only the second amount of received light is greater than the saturation level, combining the first pixel signal and the second pixel signal by a third combining coefficient; a correction unit that corrects the information on the first amount of received light or the information on the second amount of received light, the drive control unit controls a first light-receiving start time from the start of irradiation of the first light-emitting unit to the start of light-receiving by the light-receiving unit to be different from a second light-receiving start time from the start of irradiation of the second light-emitting unit to the start of light-receiving by the light-receiving unit; the correction unit corrects the information on the first amount of received light or the information on the second amount of received light based on an amount of received light caused by a difference between the first light-receiving start time and the second light-receiving start time; The combining unit combines the first pixel signal and the second pixel signal using the information on the first amount of received light or the information on the second amount of received light corrected by the correction unit. Image generating device. Effect of the Invention

[0011] According to the image generating device of the present disclosure, it is possible to easily generate an image by suppressing the influence of specular reflection. [Brief description of the drawings]

[0012] [Figure 1] FIG. 2 is a diagram showing an example of a space in which an image generating device is provided. [Diagram 2] FIG. 13 is a diagram showing a state in which specular reflection occurs on a part of a subject. [Diagram 3] 1 is a diagram showing an image generating device and a subject according to a first embodiment as viewed from above. [Figure 4] FIG. 2 is a diagram showing the image generating device according to the first embodiment as viewed from the subject side. [Diagram 5] 1 is a block diagram of an image generating device according to a first embodiment. [Figure 6] 4 is a time chart showing the operation of the image generating device according to the first embodiment. [Figure 7] 2 is a diagram illustrating pixel information and the like acquired by a light receiving unit of the image generating device according to the first embodiment. FIG. [Figure 8] 4 is a diagram showing pixel signals output from a light receiving section of the image generating device according to the first embodiment. FIG. [Figure 9] 11 is a table showing the determination results of a saturation level determination section and the synthesis process in a synthesis section. [Figure 10] 6 is a diagram showing another example of the arrangement of the first light-emitting unit, the second light-emitting unit, and the light-receiving unit of the image generating device according to the first embodiment. FIG. [Figure 11] FIG. 11 is a block diagram of an image generating device according to a second embodiment. [Figure 12] 10 is a time chart showing the operation of the image generating device according to the second embodiment. [Figure 13] 13 is a time chart showing the operation of the image generating device according to the first modification of the second embodiment. [Figure 14] 13 is a time chart showing the operation of the image generating device according to the second modification of the second embodiment. [Figure 15] 13 is a time chart showing the operation of the image generating device according to the third modification of the second embodiment. [Figure 16] FIG. 11 is a block diagram showing the configuration of an image generating device according to a third embodiment. [Figure 17] 13 is a time chart showing the operation of the image generating device according to the third embodiment. [Figure 18] 11 is a table showing the determination results of a saturation level determination section and the synthesis process in a synthesis section. [Figure 19] 13 is a time chart showing the operation of an image generating device according to a modification of the third embodiment. [Figure 20] 11 is a table showing the determination results of a saturation level determination section and the synthesis process in a synthesis section. [Figure 21] FIG. 13 is a block diagram showing the configuration of an image generating device according to a fourth embodiment. [Figure 22] 13 is a time chart showing the operation of the image generating device according to the fourth embodiment. [Figure 23] 13 is a time chart showing the operation of an image generating device according to a modification of the fourth embodiment. [Figure 24] 13 is a time chart showing the operation of an image generating device according to a fifth embodiment. [Diagram 25] FIG. 13 is a diagram illustrating another example of the operation of the image generating device according to the fifth embodiment. [Figure 26] FIG. 13 is a diagram illustrating another example of the operation of the image generating device according to the fifth embodiment. [Figure 27] FIG. 13 is a diagram illustrating another example of the operation of the image generating device according to the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] (Foundational knowledge of this disclosure) The knowledge on which the present disclosure is based will be described with reference to Figures 1 and 2. The knowledge on which the present disclosure is based includes new knowledge not found in the prior art and conventional knowledge on which the new knowledge is based.

[0014] FIG. 1 is a diagram showing an example of a space 8 in which an image generating device is provided.

[0015] As shown in FIG. 1, the image generating device is installed in a predetermined space 8 such as a vehicle interior. The image generating device is a device that captures an image of a driver, which is a subject 9, and generates an image, and is used in a system that judges the health condition, etc., of the driver. The image generating device includes a light source that emits light and a camera that receives the light. In such an image generating device, the light emitted from the light source may be specularly reflected by a part of the subject.

[0016] 2 is a diagram showing a state where specular reflection occurs on a part of a subject 9. For example, when the subject 9 is wearing glasses, light emitted from a light source may be specularly reflected on the lenses or frames of the glasses. The figure shows an example where light from four light sources is specularly reflected on the lenses of the glasses.

[0017] As shown in FIG. 2, when specular reflection occurs on the eyeglasses of the subject 9, the amount of received light is excessive, causing saturation, and image information around the eyes of the subject 9 is lost. In this case, it is difficult to completely prevent saturation from occurring, even if the amount of light emitted from the light source is reduced. In addition, if the amount of light emitted from the light source is reduced too much, the brightness decreases, and an image of the subject 9 cannot be accurately captured. If an image of the subject 9 cannot be accurately captured in this way, it becomes impossible to correctly judge the health state, wakefulness state, concentration state, etc. of the subject 9.

[0018] In contrast, the image generating device of this embodiment has the following configuration in order to suppress the influence of specular reflection and generate an image simply.

[0019] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that each of the embodiments described below shows a specific example of the present disclosure. The numerical values, shapes, materials, components, the arrangement and connection forms of the components, steps, and the order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in the independent claims showing the realization of one embodiment of the present disclosure are described as optional components. The realization of the present disclosure is not limited to the current independent claims, and may also be expressed by other independent claims.

[0020] Note that each drawing is a schematic diagram and is not necessarily a precise illustration. In addition, in each drawing, the same reference numerals are used for substantially the same configurations, and duplicated explanations may be omitted or simplified.

[0021] (Embodiment 1) [Image generation device] The image generating device 1 according to the first embodiment will be described with reference to FIGS.

[0022] Fig. 3 is a diagram showing image generating device 1 and subject 9 according to embodiment 1 as viewed from above. Fig. 4 is a diagram showing image generating device 1 as viewed from the subject 9 side. Fig. 5 is a block configuration diagram of image generating device 1.

[0023] 3 and 4, the image generating device 1 includes a first light-emitting unit 10 and a second light-emitting unit 20 that emit light, a light-receiving unit 30 that receives the light, and a substrate 40. As shown in Fig. 5, the image generating device 1 further includes a drive control unit 50 that controls the first light-emitting unit 10, the second light-emitting unit 20, and the light-receiving unit 30, a saturation level determination unit 60, and a synthesis unit 70.

[0024] The substrate 40 is rectangular and has dimensions of, for example, 50 mm in height and 150 mm in width. The substrate 40 is provided with a first light-emitting unit 10, a second light-emitting unit 20, a light-receiving unit 30, a drive control unit 50, a saturation level determination unit 60, and a combining unit 70. The drive control unit 50, the saturation level determination unit 60, and the combining unit 70 may be provided on an external circuit board different from the substrate 40.

[0025] The first light-emitting unit 10 and the second light-emitting unit 20 are light sources that irradiate light to the subject 9. The irradiation light L1, L2 irradiated from the first light-emitting unit 10 and the second light-emitting unit 20 is, for example, infrared light or laser light. Each of the first light-emitting unit 10 and the second light-emitting unit 20 may be composed of a plurality of light sources (for example, light-emitting elements). FIG. 4 shows an example in which the first light-emitting unit 10 is composed of two light sources 10a and 10b, and the second light-emitting unit 20 is composed of two light sources 20a and 20b. Each of the first light-emitting unit 10 and the second light-emitting unit 20 may have a diffusion plate for diffusing and irradiating the emitted light.

[0026] The first light-emitting unit 10 and the second light-emitting unit 20 are provided at positions where they do not overlap when viewed from the subject 9, that is, at different positions when viewed from the subject 9. Specifically, the first light-emitting unit 10 and the second light-emitting unit 20 are disposed on both outer sides of the light-receiving unit 30, with the light-receiving unit 30 sandwiched between them, when viewed from a direction perpendicular to the substrate 40. In other words, the first light-emitting unit 10 and the second light-emitting unit 20 are disposed spaced apart from each other on the substrate 40.

[0027] The light receiving unit 30 is disposed between the first light emitting unit 10 and the second light emitting unit 20. The light receiving unit 30 receives reflected light R1 and R2, which are generated when the light emitted by the first light emitting unit 10 and the second light emitting unit 20 is irradiated on the subject 9 and reflected by the light receiving unit 30. Specifically, the light receiving unit 30 receives reflected light R1 generated when the irradiation light L1 irradiated by the first light emitting unit 10 is reflected on the subject 9, and also receives reflected light R2 generated when the irradiation light L2 irradiated by the second light emitting unit 20 is reflected on the subject 9. The light receiving unit 30 is composed of a plurality of pixels 31 arranged two-dimensionally. For example, a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) sensor is used as the light receiving unit 30.

[0028] The drive control unit 50 controls the first light emission period Tg1 of the first light-emitting unit 10, the second light emission period Tg2 of the second light-emitting unit 20, and the first light reception period Tr1 and second light reception period Tr2 of the light reception unit 30. The light emission periods and light reception periods will be described later.

[0029] The drive control unit 50 generates, at different timings, a first light emission pulse SG1 for causing the first light-emitting unit 10 to emit light and a second light emission pulse SG2 for causing the second light-emitting unit 20 to emit light in the same frame. The drive control unit 50 also generates, at different timings, a first light receiving pulse RG1 for causing reflected light R1 reflected by the subject 9 to be received and a second light receiving pulse RG2 for causing reflected light R2 reflected by the subject 9 to be received.

[0030] Fig. 6 is a time chart showing the operation of image generating device 1. Fig. 6 shows the operation of image generating device 1 in one frame.

[0031] 6, the drive control unit 50 generates a first light emission pulse SG1 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG1 output from the drive control unit 50.

[0032] In addition, the drive control unit 50 generates a second light emission pulse SG2 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG1. The second light emission unit 20 irradiates the subject 9 with irradiation light L2 in accordance with the second light emission pulse SG2 output from the drive control unit 50.

[0033] The illumination lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg1 of the first light emission pulse SG1 and the second light emission period Tg2 of the second light emission pulse SG2 to be equal to each other.

[0034] The drive control unit 50 generates a first light receiving pulse RG1 that controls the light receiving unit 30. Reflected light R1 that is irradiated from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 in accordance with the first light receiving pulse RG1 output from the drive control unit 50, and outputs pixel information related to a first amount of received light RL1.

[0035] Furthermore, the drive control unit 50 generates a second light receiving pulse RG2 that controls the light receiving unit 30 at a timing different from that of the first light receiving pulse RG1. Reflected light R2 that is irradiated from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG2 output from the drive control unit 50, and outputs pixel information related to a second amount of received light RL2.

[0036] The reflected lights R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls so that a first light-receiving start time ts1 from the start of light irradiation by the first light-emitting unit 10 to the start of light reception by the light-receiving unit 30 is equal to a second light-receiving start time ts2 from the start of light irradiation by the second light-emitting unit 20 to the start of light reception by the light-receiving unit 30. The drive control unit 50 also controls so that a first light-receiving period Tr1 of the light-receiving unit 30 corresponding to the light irradiation by the first light-emitting unit 10 is equal to a second light-receiving period Tr2 of the light-receiving unit 30 corresponding to the light irradiation by the second light-emitting unit 20.

[0037] FIG. 7 is a diagram showing a schematic of pixel information acquired by the light receiving unit 30 of the image generating device 1. FIG. 7(a) shows a state in which specular reflection occurs on a part of the subject 9 due to the light emitted by the first light emitting unit 10. FIG. 7(b) shows a state in which specular reflection occurs on another part of the subject 9 due to the light emitted by the second light emitting unit 20. Note that FIG. 7(a) and (b) show separate pixel information obtained from one frame, not two images generated by two frames. FIG. 7(c) shows one image generated by combining two pieces of pixel information.

[0038] As described above, the first light-emitting unit 10 and the second light-emitting unit 20 are provided at different positions and emit light at different timings. Therefore, the reflected light beams R1 and R2 reflected at the same position on the subject 9 are incident on different addresses on the light-receiving unit 30, i.e., different pixels 31.

[0039] Therefore, as shown in (a) of FIG. 7, even if specular reflection occurs on a part of the subject 9 due to the light emitted by the first light-emitting unit 10, it is possible to obtain pixel information that could not be obtained by the first light-emitting unit 10 using the light emitted by the second light-emitting unit 20. Also, as shown in (b) of FIG. 7, even if specular reflection occurs on another part of the subject 9 due to the light emitted by the second light-emitting unit 20, it is possible to obtain pixel information that could not be obtained by the second light-emitting unit 20 using the light emitted by the first light-emitting unit 10. In other words, even if specular reflection occurs on a part of the subject 9 due to the light emitted by one of the light-emitting units, it is possible to generate an image without specular reflection as shown in (c) of FIG. 7 by combining the pixel information shown in (a) and (b) of FIG. 7.

[0040] A specific configuration for generating an image based on the reflected light beams R1 and R2 received by the light receiving section 30 will be described below.

[0041] 8 is a diagram showing pixel signals output from the light receiving section 30 of the image generating device 1. The light receiving section 30 receives reflected light R1, R2 at each of a plurality of pixels 31 arranged two-dimensionally, and outputs a first pixel signal s1 including information on a first amount of received light RL1, and a second pixel signal s2 including information on a second amount of received light RL2. For ease of understanding, the following description focuses on one pixel 31.

[0042] The light receiving section 30 has a plurality of packets for reading out the signal charges accumulated in the pixels 31. Fig. 8 shows an example in which the light receiving section 30 has three packets.

[0043] The light receiving section 30 reads out the signal charge accumulated in the pixels 31 of the light receiving section 30 in response to the emission of the first light emitter 10 into a first packet p1, and outputs a first pixel signal s1 including information on a first amount of received light RL1 corresponding to the signal charge. The light receiving section 30 also reads out the signal charge accumulated in the pixels 31 of the light receiving section 30 in response to the emission of the second light emitter 20 into a second packet p2, and outputs a second pixel signal s2 including information on a second amount of received light RL2 corresponding to the signal charge.

[0044] The first pixel signal s1 and the second pixel signal s2 output from the light receiving section 30 are vertically transferred and horizontally transferred, and further converted from analog data to digital data, and then output to the saturation level determination section 60. Note that the conversion from analog data to digital data does not need to be performed by the light receiving section 30, and may be performed outside the light receiving section 30.

[0045] 5 judges whether the first amount of received light RL1 and the second amount of received light RL2 exceed a predetermined level. The predetermined level is set in advance based on a saturation level TH when the signal charge accumulated in the pixel 31 is saturated. For example, the predetermined level is set to 90% or more and 100% or less when the signal charge accumulated in the pixel 31 is saturated. Hereinafter, the predetermined level may be referred to as a saturation level TH.

[0046] For example, the saturation level determination unit 60 determines whether or not the first pixel signal s1 output from the pixel of the light receiving unit 30 in response to the irradiation of light from the first light emitting unit 10 exceeds the saturation level TH. The saturation level determination unit 60 also determines whether or not the second pixel signal s2 output from the pixel of the light receiving unit 30 in response to the irradiation of light from the second light emitting unit 20 exceeds the saturation level TH. The saturation level TH is set to one and the same value, but is not limited thereto. The saturation level TH may be set to different values ​​when comparing with the first pixel signal s1 and when comparing with the second pixel signal s2. The determination result determined by the saturation level determination unit 60 is output to the synthesis unit 70.

[0047] FIG. 9 is a table showing the determination results of the saturation level determination section 60 and the synthesis process in the synthesis section 70. As shown in FIG.

[0048] The combiner 70 combines the first pixel signal s1 and the second pixel signal s2 based on the determination result of the saturation level determiner 60 to generate a combined signal s10.

[0049] For example, when neither the first pixel signal s1 nor the second pixel signal s2 is saturated, the synthesis unit 70 generates the synthesis signal s10 as follows.

[0050] 9(1), when the first amount of received light RL1 and the second amount of received light RL2 are both smaller than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a first combination coefficient. The first combination coefficient is a coefficient that combines the first amount of received light RL1 as half and the second amount of received light RL2 as half. In other words, when the first amount of received light RL1 and the second amount of received light RL2 are both smaller than the saturation level TH, the combiner 70 generates a combined signal s10 using half each of the first pixel signal s1 and the second pixel signal s2.

[0051] Also, for example, when saturation occurs in at least one of the first pixel signal s1 and the second pixel signal s2, i.e., when at least one of them is a pixel signal greater than a predetermined level, the synthesis unit 70 generates the synthetic signal s10 without using the pixel signal greater than a predetermined level.

[0052] Specifically, when only the first amount of received light RL1 is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a second combination coefficient. The second combination coefficient is a coefficient for combining the first amount of received light RL1 as 0. In other words, when only the first amount of received light RL1 is greater than the saturation level TH, the combiner 70 generates a combined signal s10 using the second pixel signal s2 without using the first pixel signal s1 (see (2) in FIG. 9).

[0053] Furthermore, when only the second amount of received light RL2 is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a third combination coefficient. The third combination coefficient is a coefficient for combining the signals while regarding the second amount of received light RL2 as 0. In other words, when only the second amount of received light RL2 is greater than the saturation level TH, the combiner 70 generates a combined signal s10 using the first pixel signal s1 without using the second pixel signal s2 (see (3) in FIG. 9).

[0054] In this way, in the image generating device 1, when the illumination light L1 irradiated by the first light-emitting unit 10 is specularly reflected by the subject 9, the composite signal s10 is generated using information on the second amount of received light RL2 that is not affected by specular reflection, without using information on the first amount of received light RL1. Also, in the image generating device 1, when the illumination light L2 irradiated by the second light-emitting unit 20 is specularly reflected by the subject 9, the composite signal s10 is generated using the first amount of received light RL1 that is not affected by specular reflection, without using information on the second amount of received light RL2.

[0055] When an excessive amount of light is irradiated onto the subject 9, both the first amount of received light RL1 and the second amount of received light RL2 become larger than the saturation level TH, in which case the synthesis unit 70 may output an abnormal value. By setting an abnormal value, it is possible to notify that an abnormality has occurred in the image.

[0056] In this way, the synthesis unit 70 synthesizes the multiple pixel signals output from the pixels 31 based on the determination result of the saturation level determination unit 60, and outputs a synthesized signal s10. The image generating device 1 generates an image of the subject 9 based on the multiple synthesized signals s10 output from the multiple pixels 31.

[0057] According to this image generating device 1, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a predetermined level, the composite signal s10 is generated without using the pixel signal that is greater than the predetermined level. This makes it possible to easily generate an image by suppressing the influence of specular reflection.

[0058] In the above, an example has been shown in which the four light sources are arranged outside the light receiving unit 30, but the arrangement of the first light emitting unit 10 and the second light emitting unit 20 is not limited to this.

[0059] FIG. 10 is a diagram showing another example of the arrangement of the first light-emitting unit 10, the second light-emitting unit 20, and the light-receiving unit 30 of the image generating device 1. In FIG.

[0060] Fig. 10(a) shows an example in which the first light-emitting unit 10 and the second light-emitting unit 20 are arranged on both outer sides in the left-right direction with respect to the light-receiving unit 30. Fig. 10(b) shows an example in which the first light-emitting unit 10 and the second light-emitting unit 20 are arranged on both outer sides in the up-down direction with respect to the light-receiving unit 30. Fig. 10(c) shows an example in which the first light-emitting unit 10 and the second light-emitting unit 20 are arranged on both outer sides in a diagonal direction with respect to the light-receiving unit 30. The first light-emitting unit 10 and the second light-emitting unit 20 may be arranged in any manner as long as they are spaced apart.

[0061] (Embodiment 2) [Image generation device] An image generating device 1A according to the second embodiment will be described with reference to Fig. 11 and Fig. 12. In the second embodiment, an example will be described in which the image generating device 1A further includes a correction unit 80 and a distance calculation unit 90. Descriptions of the same configuration as in the first embodiment may be omitted.

[0062] The image generating device described below not only generates an image but also has a function of measuring distance, and may be used as a distance measuring device.

[0063] Fig. 11 is a block diagram of an image generating device 1A according to embodiment 2. Fig. 12 is a time chart showing the operation of the image generating device 1A. Fig. 12 shows the operation of the image generating device 1A in one frame.

[0064] 11, image generating device 1A of embodiment 2 also includes a first light-emitting unit 10, a second light-emitting unit 20, a light-receiving unit 30, a drive control unit 50, a saturation level determination unit 60, and a synthesis unit 70. Image generating device 1A also includes a correction unit 80 that corrects information on the amount of received light, and a distance calculation unit 90 that calculates the distance from image generating device 1A to subject 9.

[0065] 12, the drive control unit 50 generates a first light emission pulse SG3 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG3 output from the drive control unit 50.

[0066] The drive control unit 50 also generates a second light emission pulse SG4 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG3. The second light emission unit 20 irradiates the subject 9 with irradiation light L2 in accordance with the second light emission pulse SG4 output from the drive control unit 50.

[0067] The illumination lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg3 of the first light emission pulse SG3 and the second light emission period Tg4 of the second light emission pulse SG4 to be equal to each other.

[0068] The drive control unit 50 generates a first light receiving pulse RG3 that controls the light receiving unit 30. Reflected light R1 that is irradiated from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 in accordance with the first light receiving pulse RG3 output from the drive control unit 50, and outputs pixel information related to a first amount of received light RL3.

[0069] Further, the drive control unit 50 generates a second light receiving pulse RG4 that controls the light receiving unit 30 at a timing different from that of the first light receiving pulse RG3. Reflected light R2 that is irradiated from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG4 output from the drive control unit 50, and outputs pixel information related to a second amount of received light RL4.

[0070] The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light-receiving start time ts3 from the start of light irradiation by the first light-emitting unit 10 to the start of light reception by the light-receiving unit 30 to be equal to the second light-receiving start time ts4 from the start of light irradiation by the second light-emitting unit 20 to the start of light reception by the light-receiving unit 30. The drive control unit 50 also controls the first light-receiving period Tr3 of the light-receiving unit 30 corresponding to the irradiation of light by the first light-emitting unit 10 to be different from the second light-receiving period Tr4 of the light-receiving unit 30 corresponding to the irradiation of light by the second light-emitting unit 20. Specifically, the drive control unit 50 controls the first light-receiving period Tr3 to be 1 / 2 of the second light-receiving period Tr4.

[0071] A specific configuration for generating an image based on the reflected light beams R1 and R2 received by the light receiving section 30 and for determining the distance to the subject 9 will be described below.

[0072] 11 determines whether the first amount of received light RL3 and the second amount of received light RL4 exceed a predetermined level. The result of the determination made by the saturation level determination section 60 is output to the correction section 80.

[0073] For example, the correction unit 80 corrects the information on the first amount of received light RL3 when it is determined that the pixel signal indicating the information on the second amount of received light RL4 is greater than the saturation level TH. The reason for correcting the information on the first amount of received light RL3 is that the first light receiving period Tr3 is different from the second light receiving period Tr4. When performing the correction, the correction unit 80 corrects the information on the first amount of received light RL3 based on the amount of received light caused by the difference between the first light receiving period Tr3 and the second light receiving period Tr4. For example, the correction unit 80 corrects the information on the first amount of received light RL3 using the ratio of the amount of received light of the surrounding pixels that is smaller than the saturation level TH (second amount of received light of the surrounding pixels / first amount of received light of the surrounding pixels). The amount of received light of the surrounding pixels may be the amount of received light of the pixel closest to the pixel 31, or may be the average value of the amount of received light of a plurality of pixels located around the pixel 31.

[0074] Information on the first amount of received light RL3 after correction by the correction unit 80 is expressed by the following (Equation 1).

[0075]

number

[0076] The correcting section 80 outputs information on the corrected first amount of received light RL3 to the combining section 70 and the distance calculating section 90.

[0077] The combining unit 70 combines the first pixel signal s1 and the second pixel signal s2 using information on the first amount of received light RL3 after correction by the correction unit 80. Specifically, when only the second amount of received light RL4 is greater than the saturation level TH, the combining unit 70 generates a combined signal s10 using the first pixel signal s1 including information on the corrected first amount of received light RL3 without using the second pixel signal s2 including information on the second amount of received light RL4.

[0078] When the first amount of received light RL3 and the second amount of received light RL4 are both smaller than the saturation level TH, and when only the first amount of received light RL3 is greater than the saturation level TH, the combiner 70 generates a combined signal s10 in the same manner as in the first embodiment.

[0079] Next, the calculation process performed by the distance calculation unit 90 will be described.

[0080] The distance calculation unit 90 can calculate the distance to the subject 9 when the first amount of received light RL3 and the second amount of received light RL4 are both smaller than the saturation level TH. For example, the distance calculation unit 90 can calculate the distance to the subject 9 using the first amount of received light RL3 and the second amount of received light RL4. If the distance to the subject 9 is L, the light emission pulse width is Tn, and the speed of light is c (approximately 299,792,458 m / s), the distance L is expressed by the following (Equation 2).

[0081]

number

[0082] In this image generating device 1A as well, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a predetermined level (saturation level TH), a composite signal s10 is generated without using a pixel signal that is greater than the predetermined level. This makes it possible to easily generate an image by suppressing the influence of specular reflection. Furthermore, according to the image generating device 1A, when both the first amount of received light RL3 and the second amount of received light RL4 are smaller than the saturation level TH, the distance L to the subject 9 can be easily measured.

[0083] [Modification 1 of the second embodiment] An image generating device 1B according to a first modification of the second embodiment will be described with reference to Fig. 13. In the first modification of the second embodiment, an example in which the first light-receiving start time ts5 is delayed from the first light-receiving start time ts3 of the second embodiment will be described.

[0084] Fig. 13 is a time chart showing the operation of image generating device 1B. Fig. 13 shows the operation of image generating device 1B in one frame.

[0085] 13, the drive control unit 50 generates a first light emission pulse SG5 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG5 output from the drive control unit 50.

[0086] The drive control unit 50 also generates a second light emission pulse SG6 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG5. The second light emission unit 20 irradiates the subject 9 with irradiation light L2 in accordance with the second light emission pulse SG6 output from the drive control unit 50.

[0087] The illumination lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg5 of the first light emission pulse SG5 and the second light emission period Tg6 of the second light emission pulse SG6 to be equal to each other.

[0088] The drive control unit 50 generates a first light receiving pulse RG5 that controls the light receiving unit 30. Reflected light R1 that is irradiated from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 in accordance with the first light receiving pulse RG5 output from the drive control unit 50, and outputs pixel information related to a first amount of received light RL5.

[0089] Furthermore, the drive control unit 50 generates a second light receiving pulse RG6 that controls the light receiving unit 30 at a timing different from that of the first light receiving pulse RG5. Reflected light R2 that is irradiated from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG6 output from the drive control unit 50, and outputs pixel information related to a second amount of received light RL6.

[0090] The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light-receiving start time ts5 from the start of light irradiation by the first light-emitting unit 10 to the start of light reception by the light-receiving unit 30 so as to be different from the second light-receiving start time ts6 from the start of light irradiation by the second light-emitting unit 20 to the start of light reception by the light-receiving unit 30. Specifically, the drive control unit 50 controls the first light-receiving start time ts5 to be the sum of the second light-receiving start time ts6 and the first light-emitting period Tg5. The drive control unit 50 also controls the first light-receiving period Tr5 of the light-receiving unit 30 corresponding to the light irradiation of the first light-emitting unit 10 and the second light-receiving period Tr6 of the light-receiving unit 30 corresponding to the light irradiation of the second light-emitting unit 20 so as to be different from each other. Specifically, the drive control unit 50 controls the first light receiving period Tr5 to be half the second light receiving period Tr6.

[0091] The saturation level determination section 60 determines whether or not the first amount of received light RL5 and the second amount of received light RL6 exceed a predetermined level. The determination result obtained by the saturation level determination section 60 is output to the correction section 80.

[0092] For example, the correction unit 80 corrects the information on the first amount of received light RL5 when it is determined that the pixel signal indicating the information on the second amount of received light RL6 is greater than the saturation level TH. The reason for correcting the information on the first amount of received light RL5 is that the first light-receiving start time ts5 is different from the second light-receiving start time ts6. When performing the correction, the correction unit 80 corrects the information on the first amount of received light RL5 based on the amount of received light caused by the difference between the first light-receiving start time ts5 and the second light-receiving start time ts6. For example, the correction unit 80 corrects the information on the first amount of received light RL5 using the ratio of the amount of received light of the surrounding pixel that is smaller than the saturation level TH (the second amount of received light of the surrounding pixel / the first amount of received light of the surrounding pixel).

[0093] Information on the first amount of received light RL5 after correction by the correction unit 80 is expressed by the following (Equation 3).

[0094]

number

[0095] The correction section 80 outputs information on the corrected first amount of received light RL5 to the synthesis section 70 and the distance calculation section 90.

[0096] The combining unit 70 combines the first pixel signal s1 and the second pixel signal s2 using information on the first amount of received light RL5 after correction by the correction unit 80. Specifically, when only the second amount of received light RL6 is greater than the saturation level TH, the combining unit 70 generates a combined signal s10 using the first pixel signal s1 including information on the corrected first amount of received light RL5 without using the second pixel signal s2 including information on the second amount of received light RL6.

[0097] When the first amount of received light RL5 and the second amount of received light RL6 are both smaller than the saturation level TH, or when only the first amount of received light RL5 is larger than the saturation level TH, a composite signal s10 is generated as in the first embodiment.

[0098] The distance calculation unit 90 can calculate the distance to the subject 9 when the first amount of received light RL5 and the second amount of received light RL6 are both smaller than the saturation level TH. For example, the distance calculation unit 90 can calculate the distance to the subject 9 using the first amount of received light RL5 and the second amount of received light RL6. If the distance to the subject 9 is L, the light emission pulse width is Tn, and the speed of light is c, then the distance L is expressed by the following (Equation 4).

[0099]

number

[0100] In this image generating device 1B as well, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a predetermined level (saturation level TH), a composite signal s10 is generated without using a pixel signal that is greater than the predetermined level. This makes it possible to easily generate an image by suppressing the influence of specular reflection. Furthermore, according to the image generating device 1B, when both the first amount of received light RL5 and the second amount of received light RL6 are smaller than the saturation level TH, the distance L to the subject 9 can be easily measured.

[0101] [Modification 2 of the second embodiment] An image generating device 1C according to a second modification of the second embodiment will be described with reference to Fig. 14. In the second modification of the second embodiment, an example in which the second light receiving period Tr8 is set to 1 / 2 of the first light receiving period Tr7 will be described.

[0102] Fig. 14 is a time chart showing the operation of image generating device 1C in one frame.

[0103] 14, the drive control unit 50 generates a first light emission pulse SG7 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG7 output from the drive control unit 50.

[0104] In addition, the drive control unit 50 generates a second light emission pulse SG8 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG7. The second light emission unit 20 irradiates the subject 9 with irradiation light L2 in accordance with the second light emission pulse SG8 output from the drive control unit 50.

[0105] The illumination lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg7 of the first light emission pulse SG7 and the second light emission period Tg8 of the second light emission pulse SG8 to be equal to each other.

[0106] The drive control unit 50 generates a first light receiving pulse RG7 that controls the light receiving unit 30. Reflected light R1 that is irradiated from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 in accordance with the first light receiving pulse RG7 output from the drive control unit 50, and outputs pixel information related to a first amount of received light RL7.

[0107] Furthermore, the drive control unit 50 generates a second light receiving pulse RG8 that controls the light receiving unit 30 at a timing different from that of the first light receiving pulse RG7. Reflected light R2 that is irradiated from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG8 output from the drive control unit 50, and outputs pixel information related to a second amount of received light RL8.

[0108] The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light-receiving start time ts7 from the start of light irradiation by the first light-emitting unit 10 to the start of light reception by the light-receiving unit 30 to be equal to the second light-receiving start time ts8 from the start of light irradiation by the second light-emitting unit 20 to the start of light reception by the light-receiving unit 30. The drive control unit 50 also controls the first light-receiving period Tr7 of the light-receiving unit 30 corresponding to the irradiation of light by the first light-emitting unit 10 to be different from the second light-receiving period Tr8 of the light-receiving unit 30 corresponding to the irradiation of light by the second light-emitting unit 20. Specifically, the drive control unit 50 controls the second light-receiving period Tr8 to be 1 / 2 of the first light-receiving period Tr7.

[0109] The saturation level determination section 60 determines whether or not the first amount of received light RL7 and the second amount of received light RL8 exceed a predetermined level. The determination result obtained by the saturation level determination section 60 is output to the correction section 80.

[0110] For example, the correction unit 80 corrects the information on the second amount of received light RL8 when it is determined that the pixel signal indicating the information on the first amount of received light RL7 is greater than the saturation level TH. The reason for correcting the information on the second amount of received light RL8 is that the second light receiving period Tr8 is different from the first light receiving period Tr7. When performing the correction, the correction unit 80 corrects the information on the second amount of received light RL8 based on the amount of received light caused by the difference between the first light receiving period Tr7 and the second light receiving period Tr8. For example, the correction unit 80 corrects the information on the second amount of received light RL8 using a ratio of the amounts of received light of the surrounding pixels that is smaller than the saturation level TH (first amount of received light of the surrounding pixel / second amount of received light of the surrounding pixel).

[0111] Information on the second amount of received light RL8 after correction by the correction unit 80 is expressed by the following (Equation 5).

[0112]

number

[0113] The correction section 80 outputs information on the corrected second amount of received light RL8 to the synthesis section 70 and the distance calculation section 90.

[0114] The combining unit 70 combines the first pixel signal s1 and the second pixel signal s2 using information on the second amount of received light RL8 after correction by the correction unit 80. Specifically, when only the first amount of received light RL7 is greater than the saturation level TH, the combining unit 70 generates a combined signal s10 using the second pixel signal s2 including information on the corrected second amount of received light RL8 without using the first pixel signal s1 including information on the first amount of received light RL7.

[0115] When the first amount of received light RL7 and the second amount of received light RL8 are both smaller than the saturation level TH, and when only the second amount of received light RL8 is larger than the saturation level TH, a composite signal s10 is generated similarly to the first embodiment.

[0116] The distance calculation unit 90 can calculate the distance to the subject 9 when the first amount of received light RL7 and the second amount of received light RL8 are both smaller than the saturation level TH. For example, the distance calculation unit 90 can calculate the distance to the subject 9 using the first amount of received light RL7 and the second amount of received light RL8. If the distance to the subject 9 is L, the light emission pulse width is Tn, and the speed of light is c, the distance L is expressed by the following (Equation 6).

[0117]

number

[0118] In this image generating device 1C as well, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a predetermined level (saturation level TH), a composite signal s10 is generated without using a pixel signal that is greater than the predetermined level. This makes it possible to easily generate an image by suppressing the influence of specular reflection. Furthermore, according to the image generating device 1C, when both the first amount of received light RL7 and the second amount of received light RL8 are smaller than the saturation level TH, the distance L to the subject 9 can be easily measured.

[0119] [Third Modification of the Second Embodiment] An image generating device 1D according to a third modification of the second embodiment will be described with reference to Fig. 15. In the third modification of the second embodiment, an example in which the second light-receiving start time ts10 is delayed from the second light-receiving start time ts8 of the second modification will be described.

[0120] Fig. 15 is a time chart showing the operation of the image generating device 1D. Fig. 15 shows the operation of the image generating device 1D in one frame.

[0121] 15, the drive control unit 50 generates a first light emission pulse SG9 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG9 output from the drive control unit 50.

[0122] In addition, the drive control unit 50 generates a second light emission pulse SG10 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG9. The second light emission unit 20 irradiates the subject 9 with irradiation light L2 in accordance with the second light emission pulse SG10 output from the drive control unit 50.

[0123] The illumination lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg9 of the first light emission pulse SG9 and the second light emission period Tg10 of the second light emission pulse SG10 to be equal to each other.

[0124] The drive control unit 50 generates a first light receiving pulse RG9 that controls the light receiving unit 30. Reflected light R1 that is irradiated from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 in accordance with the first light receiving pulse RG9 output from the drive control unit 50, and outputs pixel information related to a first amount of received light RL9.

[0125] Further, the drive control unit 50 generates a second light receiving pulse RG10 that controls the light receiving unit 30 at a timing different from that of the first light receiving pulse RG9. Reflected light R2 that is irradiated from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG10 output from the drive control unit 50, and outputs pixel information related to a second amount of received light RL10.

[0126] The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the second light-receiving start time ts10, which is from the start of light irradiation by the second light-emitting unit 20 to the start of light reception by the light-receiving unit 30, so as to be different from the first light-receiving start time ts9. Specifically, the drive control unit 50 controls the second light-receiving start time ts10 to be the sum of the first light-receiving start time ts9 and the second light-emitting period Tg10. The drive control unit 50 also controls the first light-receiving period Tr9 of the light-receiving unit 30 corresponding to the irradiation of light by the first light-emitting unit 10 and the second light-receiving period Tr10 of the light-receiving unit 30 corresponding to the irradiation of light by the second light-emitting unit 20 so as to be different from each other. Specifically, the drive control unit 50 controls the second light-receiving period Tr10 so as to be 1 / 2 of the first light-receiving period Tr9.

[0127] The saturation level determination section 60 determines whether or not the first amount of received light RL9 and the second amount of received light RL10 exceed a predetermined level. The determination result obtained by the saturation level determination section 60 is output to the correction section 80.

[0128] For example, the correction unit 80 corrects the information on the second amount of received light RL10 when it is determined that the pixel signal indicating the information on the first amount of received light RL9 is greater than the saturation level TH. The reason for correcting the information on the second amount of received light RL10 is that the first light-receiving start time ts9 and the second light-receiving start time ts10 are different, and the first light-receiving period Tr9 and the second light-receiving period Tr10 are different. When performing the correction, the correction unit 80 corrects the information on the second amount of received light RL10 based on the amount of received light caused by the difference between the first light-receiving start time ts9 and the second light-receiving start time ts10, and the difference between the first light-receiving period Tr9 and the second light-receiving period Tr10. For example, the correction unit 80 corrects the information on the second amount of received light RL10 using a ratio of the amount of received light of the surrounding pixels that is smaller than the saturation level TH (first amount of received light of the surrounding pixels / second amount of received light of the surrounding pixels).

[0129] Information on the second amount of received light RL10 after correction by the correction unit 80 is expressed by the following (Equation 7).

[0130]

number

[0131] The correction section 80 outputs information on the corrected second amount of received light RL10 to the synthesis section 70 and the distance calculation section 90.

[0132] The combining unit 70 combines the first pixel signal s1 and the second pixel signal s2 using information on the second amount of received light RL10 after correction by the correction unit 80. Specifically, when only the first amount of received light RL9 is greater than the saturation level TH, the combining unit 70 generates a combined signal s10 using the second pixel signal s2 including information on the corrected second amount of received light RL10 without using the first pixel signal s1 including information on the first amount of received light RL9.

[0133] When the first amount of received light RL9 and the second amount of received light RL10 are both smaller than the saturation level TH, or when only the second amount of received light RL10 is larger than the saturation level TH, a composite signal s10 is generated similarly to the first embodiment.

[0134] The distance calculation unit 90 can calculate the distance to the subject 9 when the first amount of received light RL9 and the second amount of received light RL10 are both smaller than the saturation level TH. For example, the distance calculation unit 90 can calculate the distance to the subject 9 using the first amount of received light RL9 and the second amount of received light RL10. If the distance to the subject 9 is L, the light emission pulse width is Tn, and the speed of light is c, the distance L is expressed by the following (Equation 8).

[0135]

number

[0136] In this image generating device 1D, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a predetermined level (saturation level TH), a composite signal s10 is generated without using a pixel signal that is greater than the predetermined level. This makes it possible to easily generate an image by suppressing the influence of specular reflection. Furthermore, according to the image generating device 1D, when both the first amount of received light RL9 and the second amount of received light RL10 are smaller than the saturation level TH, the distance L to the subject 9 can be easily measured.

[0137] (Embodiment 3) [Image generation device] An image generating device 1E according to the third embodiment will be described with reference to Figs. 16 to 18. In the third embodiment, an example will be described in which two first light receiving pulses RG11 and RG11a and two first light receiving periods Tr11 and Tr11a are set. Note that the description of the same configuration as in the first and second embodiments may be omitted.

[0138] Fig. 16 is a block diagram of an image generating device 1E according to embodiment 3. Fig. 17 is a time chart showing the operation of the image generating device 1E. Fig. 17 shows the operation of the image generating device 1E in one frame.

[0139] 16, image generating device 1E of embodiment 3 also includes a first light-emitting unit 10, a second light-emitting unit 20, a light-receiving unit 30, a drive control unit 50, a saturation level determination unit 60, a synthesis unit 70, and a correction unit 80. Image generating device 1E also includes a distance calculation unit 90 that calculates the distance from image generating device 1E to subject 9. Image generating device 1E of embodiment 3 not only generates an image, but also has a function of measuring distance.

[0140] 17, the drive control unit 50 generates a first light emission pulse SG11 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG11 output from the drive control unit 50.

[0141] In addition, the drive control unit 50 generates a second light emission pulse SG12 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG11. The second light emission unit 20 irradiates the subject 9 with irradiation light L2 in accordance with the second light emission pulse SG12 output from the drive control unit 50.

[0142] The illumination lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg11 of the first light emission pulse SG11 and the second light emission period Tg12 of the second light emission pulse SG12 to be equal to each other.

[0143] The drive control unit 50 generates two first light receiving pulses RG11 and RG11a that control the light receiving unit 30. Reflected light R1 that is irradiated from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 in accordance with the first light receiving pulses RG11 and RG11a output from the drive control unit 50, and outputs pixel information related to the received first amounts of light RL11 and RL11a.

[0144] The light receiving unit 30 reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the first light receiving pulse RG11 into a first packet p1, and also reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the first light receiving pulse RG11a into a fourth packet (not shown), and outputs a first pixel signal s1 including information on the first light receiving amounts RL11 and RL11a corresponding to the above signal charges.

[0145] Furthermore, the drive control unit 50 generates a second light receiving pulse RG12 that controls the light receiving unit 30 at a timing different from the first light receiving pulses RG11, RG11a. Reflected light R2 that is irradiated from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG12 output from the drive control unit 50, and outputs pixel information related to the second amount of received light RL12.

[0146] The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light receiving period Tr11 according to the first light receiving pulse RG11 and the first light receiving period Tr11a according to the first light receiving pulse RG11a so that they are equal. The drive control unit 50 also controls the first light receiving start time ts11 from the start of light irradiation by the first light emitter 10 to the start of light reception by the light receiving unit 30 and the second light receiving start time ts12 from the start of light irradiation by the second light emitter 20 to the start of light reception by the light receiving unit 30 so that they are equal. The first light receiving start time ts11a according to the first light receiving pulse RG11a is controlled to be the sum of the first light receiving start time ts11 according to the first light receiving pulse RG11 and the first light receiving period Tr11. Furthermore, the drive control section 50 controls so that the sum of the first light receiving periods Tr11 and Tr11a is equal to the second light receiving period Tr12 of the light receiving section 30 corresponding to the emission of light by the second light emitting section 20.

[0147] The saturation level determination section 60 determines whether or not the first amount of received light RL11, RL11a and the second amount of received light RL12 exceed a predetermined level. The result of the determination made by the saturation level determination section 60 is output to the combining section .

[0148] FIG. 18 is a table showing the determination results of the saturation level determination section 60 and the synthesis process in the synthesis section 70. In FIG.

[0149] The combiner 70 combines the first pixel signal s1 and the second pixel signal s2 based on the determination result of the saturation level determiner 60 to generate a combined signal s10.

[0150] For example, when neither the first pixel signal s1 nor the second pixel signal s2 is saturated, the synthesis unit 70 generates the synthesis signal s10 as follows.

[0151] As shown in (1) of FIG. 18, when the sum of the first received light amounts RL11 and RL11a and the second received light amount RL12 are both smaller than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a first combination coefficient. The first combination coefficient is a coefficient that combines the first pixel signal s1 and the second pixel signal s2 by regarding the sum of the first received light amounts RL11 and RL11a as half and the second received light amount RL12 as half. In other words, when the sum of the first received light amounts RL11 and RL11a and the second received light amount RL12 are both smaller than the saturation level TH, the combiner 70 generates a combined signal s10 by using half each of the first pixel signal s1 and the second pixel signal s2.

[0152] Also, for example, when saturation occurs in at least one of the first pixel signal s1 and the second pixel signal s2, i.e., when at least one of them is a pixel signal greater than a predetermined level, the synthesis unit 70 generates the synthetic signal s10 without using the pixel signal greater than a predetermined level.

[0153] Specifically, when only the sum of the first amounts of received light RL11, RL11a is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with the second combination coefficient. The second combination coefficient is a coefficient for combining the first amounts of received light RL11, RL11a assuming that the sum of the first amounts of received light RL11, RL11a is 0. In other words, when only the sum of the first amounts of received light RL11, RL11a is greater than the saturation level TH, the combiner 70 generates the combined signal s10 using the second pixel signal s2 without using the first pixel signal s1 (see (2) in FIG. 18).

[0154] Furthermore, when only the second amount of received light RL12 is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a third combination coefficient. The third combination coefficient is a coefficient for combining the signals while regarding the second amount of received light RL12 as 0. In other words, when only the second amount of received light RL12 is greater than the saturation level TH, the combiner 70 generates a combined signal s10 using the first pixel signal s1 without using the second pixel signal s2 (see (3) in FIG. 18).

[0155] In this way, in the image generating device 1E, when the irradiation light L1 irradiated by the first light-emitting unit 10 is specularly reflected by the subject 9, the composite signal s10 is generated using information on the second amount of received light RL12 that is not affected by specular reflection, without using information on the first amount of received light RL11, RL11a. Also, in the image generating device 1E, when the irradiation light L2 irradiated by the second light-emitting unit 20 is specularly reflected by the subject 9, the composite signal s10 is generated using information on the first amount of received light RL11, RL11a that is not affected by specular reflection, without using information on the second amount of received light RL12.

[0156] When an excessive amount of light is irradiated onto the subject 9, the sum of the first received light amounts RL11 and RL11a and the second received light amount RL12 may both exceed the saturation level TH, in which case the synthesis unit 70 may output an abnormal value. By setting an abnormal value, it is possible to notify that an abnormality has occurred in the image.

[0157] In this way, the synthesis unit 70 synthesizes the multiple pixel signals output from the pixels 31 based on the determination result of the saturation level determination unit 60, and outputs a synthesized signal s10. The image generating device 1E generates an image of the subject 9 based on the multiple synthesized signals s10 output from the multiple pixels 31.

[0158] Moreover, in the image generating device 1E of the third embodiment, it is possible to obtain the distance L to the subject 9 except when both of the first received light amounts RL11 and RL11a exceed the saturation level TH.

[0159] For example, when only the second amount of received light RL12 is greater than the saturation level TH, the distance calculation unit 90 calculates the distance L to the subject 9 using the following (Equation 9). When only the first amount of received light RL11 is greater than the saturation level TH, the distance calculation unit 90 calculates the distance L to the subject 9 using the following (Equation 10). When only the first amount of received light RL11a is greater than the saturation level TH, the distance calculation unit 90 calculates the distance L to the subject 9 using the following (Equation 11).

[0160]

number

[0161]

number

[0162]

number

[0163] In this image generating device 1E, the influence of specular reflection can be suppressed and an image can be easily generated. Moreover, in this image generating device 1E, except for the case where both of the first received light amounts RL11 and RL11a exceed the saturation level TH, the distance L to the subject 9 can be easily measured.

[0164] [Modification of the third embodiment] An image generating device 1F according to a modification of the third embodiment will be described with reference to Fig. 19 and Fig. 20. In the modification of the third embodiment, an example in which two second light receiving pulses RG14 and RG14a and two second light receiving periods Tr14 and Tr14a are further set will be described.

[0165] Fig. 19 is a time chart showing the operation of the image generating device 1F. Fig. 19 shows the operation of the image generating device 1F in one frame.

[0166] 19, the drive control unit 50 generates a first light emission pulse SG13 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG13 output from the drive control unit 50.

[0167] In addition, the drive control unit 50 generates a second light emission pulse SG14 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG13. The second light emission unit 20 irradiates the subject 9 with irradiation light L2 in accordance with the second light emission pulse SG14 output from the drive control unit 50.

[0168] The illumination lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg13 of the first light emission pulse SG13 and the second light emission period Tg14 of the second light emission pulse SG14 to be equal to each other.

[0169] The drive control unit 50 generates two first light receiving pulses RG13 and RG13a that control the light receiving unit 30. Reflected light R1 that is irradiated from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 in accordance with the first light receiving pulses RG13 and RG13a output from the drive control unit 50, and outputs pixel information related to the received first amounts of light RL13 and RL13a.

[0170] The light receiving unit 30 reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the first light receiving pulse RG13 into a first packet p1, and also reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the first light receiving pulse RG13a into a fourth packet (not shown), and outputs a first pixel signal s1 including information on the first light receiving amounts RL13 and RL13a corresponding to the above signal charges.

[0171] Furthermore, the drive control unit 50 generates second light receiving pulses RG14 and RG14a that control the light receiving unit 30 at a timing different from that of the first light receiving pulses RG13 and RG13a. Reflected light R2 that is irradiated from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulses RG14 and RG14a output from the drive control unit 50, and outputs pixel information related to the received second amounts of light RL14 and RL14a.

[0172] The light receiving unit 30 reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the second light receiving pulse RG14 into a second packet p2, and also reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the second light receiving pulse RG14a into a fifth packet (not shown), and outputs a second pixel signal s2 including information on the second light receiving amounts RL14 and RL14a corresponding to the above signal charges.

[0173] The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light receiving period Tr13 according to the first light receiving pulse RG13, the first light receiving period Tr13a according to the first light receiving pulse RG13a, the second light receiving period Tr14 according to the second light receiving pulse RG14, and the second light receiving period Tr14a according to the second light receiving pulse RG14a so that they are equal to each other. The first light receiving start time ts13a according to the first light receiving pulse RG13a is controlled to be the sum of the first light receiving start time ts13 according to the first light receiving pulse RG13 and the first light receiving period Tr13. The drive control unit 50 also controls the first light receiving start time ts13 according to the first light receiving pulse RG13 and the second light receiving start time ts14 according to the second light receiving pulse RG14 so that they are equal to each other. The second light-receiving start time ts14a by the second light-receiving pulse RG14a is controlled to be equal to the sum of the second light-receiving start time ts14 and the second light-receiving period Tr14.

[0174] The saturation level determination section 60 determines whether or not the first amounts of received light RL13, RL13a and the second amounts of received light RL14, RL14a exceed a predetermined level. The result of the determination made by the saturation level determination section 60 is output to the combining section .

[0175] FIG. 20 is a table showing the determination results of the saturation level determination section 60 and the synthesis process in the synthesis section 70. In FIG.

[0176] The combiner 70 combines the first pixel signal s1 and the second pixel signal s2 based on the determination result of the saturation level determiner 60 to generate a combined signal s10.

[0177] For example, when neither the first pixel signal s1 nor the second pixel signal s2 is saturated, the synthesis unit 70 generates the synthesis signal s10 as follows.

[0178] As shown in (1) of FIG. 20, when the sum of the first received light amounts RL13 and RL13a and the sum of the second received light amounts RL14 and RL14a are both smaller than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a first combination coefficient. The first combination coefficient is a coefficient that combines the first pixel signal s1 and the second pixel signal s2 by regarding the sum of the first received light amounts RL13 and RL13a as half and the sum of the second received light amounts RL14 and RL14a as half. In other words, when the sum of the first received light amounts RL13 and RL13a and the sum of the second received light amounts RL14 and RL14a are both smaller than the saturation level TH, the combiner 70 generates a combined signal s10 by using half of both the first pixel signal s1 and the second pixel signal s2.

[0179] Also, for example, when saturation occurs in at least one of the first pixel signal s1 and the second pixel signal s2, i.e., when at least one of them is a pixel signal greater than a predetermined level, the synthesis unit 70 generates the synthetic signal s10 without using the pixel signal greater than a predetermined level.

[0180] Specifically, when only the sum of the first amounts of received light RL13 and RL13a is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with the second combination coefficient. The second combination coefficient is a coefficient for combining the first amounts of received light RL13 and RL13a, regarding the sum of the first amounts of received light RL13 and RL13a as 0. In other words, when only the sum of the first amounts of received light RL13 and RL13a is greater than the saturation level TH, the combiner 70 generates a combined signal s10 using the second pixel signal s2 without using the first pixel signal s1 (see (2) in FIG. 20).

[0181] Furthermore, when only the sum of the second amounts of received light RL14, RL14a is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a third combination coefficient. The third combination coefficient is a coefficient for combining the second amounts of received light RL14, RL14a as 0. In other words, when only the second amounts of received light RL14, RL14a is greater than the saturation level TH, the combiner 70 generates a combined signal s10 using the first pixel signal s1 without using the second pixel signal s2 (see (3) in FIG. 20).

[0182] In this way, in the image generating device 1F, when the irradiation light L1 irradiated by the first light-emitting unit 10 is specularly reflected by the subject 9, the composite signal s10 is generated using information on the second received light amounts RL14, RL14a that are not affected by specular reflection, without using information on the first received light amounts RL13, RL13a. Also, in the image generating device 1F, when the irradiation light L2 irradiated by the second light-emitting unit 20 is specularly reflected by the subject 9, the composite signal s10 is generated using information on the first received light amounts RL13, RL13a that are not affected by specular reflection, without using information on the second received light amounts RL14, RL14a.

[0183] When an excessive amount of light is irradiated onto the subject 9, the sum of the first received light amounts RL13, RL13a and the sum of the second received light amounts RL14, RL14a may both exceed the saturation level TH, in which case the synthesis unit 70 may output an abnormal value. By setting an abnormal value, it is possible to notify that an abnormality has occurred in the image.

[0184] In this way, the synthesis unit 70 synthesizes the multiple pixel signals output from the pixels 31 based on the determination result of the saturation level determination unit 60, and outputs a synthesized signal s10. The image generating device 1F generates an image of the subject 9 based on the multiple synthesized signals s10 output from the multiple pixels 31.

[0185] Moreover, in the image generating device 1F of the modified example of the third embodiment, it is possible to obtain the distance L to the subject 9.

[0186] For example, when only the first amount of received light RL13 is greater than the saturation level TH, the distance calculation unit 90 calculates the distance L to the subject 9 by the following (Equation 12). Note that when only the first amount of received light RL13a is greater than the saturation level TH, the distance L is calculated by the same equation.

[0187]

number

[0188] When only the second amount of received light RL14 is greater than the saturation level TH, the distance calculation unit 90 calculates the distance L to the subject 9 by the following (Equation 13). Note that when only the second amount of received light RL14a is greater than the saturation level TH, the distance L is calculated by the same equation.

[0189]

number

[0190] When the sum of the first amounts of received light RL13, RL13a and the sum of the second amounts of received light RL14, RL14a are both smaller than the saturation level TH, the distance calculation unit 90 calculates the distance to the subject 9 by the following (Equation 14).

[0191]

number

[0192] In this image generating device 1F as well, the influence of specular reflection can be suppressed to easily generate an image. Also, in this image generating device 1F as well, the influence of specular reflection can be suppressed to easily measure the distance L to the subject 9.

[0193] (Embodiment 4) [Image generation device] An image generating device 1G according to the fourth embodiment will be described with reference to Fig. 21 and Fig. 22. Note that the description of the same configuration as in the first and second embodiments may be omitted.

[0194] FIG. 21 is a block diagram of an image generating device 1G according to the fourth embodiment.

[0195] The image generating device 1G according to the fourth embodiment includes a first light emitting unit 10, a second light emitting unit 20, a light receiving unit 30, a drive control unit 50, a saturation level determining unit 60, a combining unit 70, a correcting unit 80, and a distance calculating unit 90.

[0196] The drive control unit 50 generates, in the same frame, a first light emission pulse SG15 to be emitted N times by the first light-emitting unit 10 and a second light emission pulse SG16 to be emitted M times by the second light-emitting unit 20, at different timings. In addition, in the same frame as the above, the drive control unit 50 generates, in different timings, a first light-receiving pulse RG15 to receive N times the reflected light R1 generated when the irradiation light L1 irradiated from the first light-emitting unit 10 is reflected by the subject 9, and a second light-receiving pulse RG16 to receive M times the reflected light R2 generated when the irradiation light L2 irradiated from the second light-emitting unit 20 is reflected by the subject 9. N and M are natural numbers equal to or greater than 1, and are controlled so that at least one of N and M is equal to or greater than 2.

[0197] The light receiving section 30 has a plurality of pixels 31 arranged two-dimensionally, receives reflected light R1 in accordance with N first light receiving pulses RG15, and generates signal charges accumulated in the pixels 31 by the light reception as N first amounts of received light RL15. Similarly, the light receiving section 30 receives reflected light R2 in accordance with M second light receiving pulses RG16, and generates signal charges accumulated in the pixels 31 by the light reception as M second amounts of received light RL16. The N first amounts of received light RL15 and the M second amounts of received light RL16 are converted into digital data and output.

[0198] The saturation level determination section 60 compares the N first amounts of received light RL15 with the saturation level, and compares the M second amounts of received light RL16 with the saturation level. The saturation level to be compared with the N first amounts of received light RL15 and the saturation level to be compared with the M second amounts of received light RL16 may be set individually.

[0199] The correction unit 80 corrects the N first amounts of received light RL15 and the M second amounts of received light RL16 that are determined to be greater than the saturation level. For example, the correction unit 80 may correct the information on the first amount of received light RL15 or the second amount of received light RL16 using the amounts of received light of surrounding pixels that are smaller than the saturation level. The amounts of received light of the surrounding pixels may be the amount of received light of the pixel closest to the pixel 31, or may be the average value of the amounts of received light of multiple pixels located around the pixel 31.

[0200] The combining unit 70 adds N first amounts of received light RL15 to obtain a total value of the first amounts of received light RL15. For example, when N is 1, the combining unit 70 obtains one first amount of received light RL15 to obtain a total value of the first amounts of received light RL15. Furthermore, the combining unit 70 adds M second amounts of received light RL16 to obtain a total value of the second amounts of received light RL16. For example, when M is 1, the combining unit 70 obtains one second amount of received light RL16 to obtain a total value of the second amounts of received light RL16.

[0201] The combining section 70 selects and combines the first combining coefficient when the N first amounts of received light RL15 and the M second amounts of received light RL16 are all smaller than the saturation level based on the comparison result of the saturation level determination section 60. The combining section 70 selects and combines the second combining coefficient when at least one of the N first amounts of received light RL15 is greater than the saturation level. The combining section 70 selects and combines the third combining coefficient when at least one of the M second amounts of received light RL16 is greater than the saturation level.

[0202] 22 is a time chart showing the operation of image generating device 1G. In FIG. 22, a case where N is 2 and M is 1 is shown.

[0203] The drive control unit 50 generates first light emission pulses SG15 and SG15a for controlling the first light-emitting unit 10. The first light-emitting unit 10 emits irradiation light L1 toward the subject 9 in accordance with the first light-emitting pulses SG15 and SG15a. The drive control unit 50 also generates a second light-emitting pulse SG16 for controlling the second light-emitting unit 20 at a timing different from that of the first light-emitting pulses SG15 and SG15a. The second light-emitting unit 20 emits irradiation light L2 toward the subject 9 in accordance with the second light-emitting pulse SG16.

[0204] The illumination lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg15 of the first light emission pulse SG15, the first light emission period Tg15a of the first light emission pulse SG15a, and the second light emission period Tg16 of the second light emission pulse SG16 so that they are equal to each other.

[0205] The light receiving section 30 receives the reflected light R1 in accordance with the first light receiving pulse RG15 generated by the drive control section 50, receives the reflected light R1 in accordance with the first light receiving pulse RG15a, and receives the reflected light R2 in accordance with the second light receiving pulse RG16.

[0206] The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls so that the first light-receiving start time ts15 from the start of irradiation to the first light-receiving pulse RG15 and the second light-receiving start time ts16 from the start of irradiation to the second light-receiving pulse RG16 are equal. The drive control unit 50 controls so that the first light-receiving period Tr15 and the first light-receiving period Tr15a are equal. The first light-receiving start time ts15a from the start of irradiation to the first light-receiving pulse RG15a is controlled to be the sum of the first light-receiving start time ts15 and the first light-receiving period Tr15. The second light-receiving period Tr16 is controlled to be the sum of the first light-receiving periods Tr15 and Tr15a.

[0207] The light receiving section 30 generates the signal charge accumulated in the pixel 31 by receiving light in response to the first light receiving pulse RG15 as a first amount of received light RL15. The light receiving section 30 also generates the signal charge accumulated in the pixel 31 by receiving light in response to the first light receiving pulse RG15a as a first amount of received light RL15a. The light receiving section 30 also generates the signal charge accumulated in the pixel 31 by receiving light in response to the second light receiving pulse RG16 as a second amount of received light RL16.

[0208] The saturation level determination section 60 determines whether or not the first amount of received light RL15, RL15a and the second amount of received light RL16 exceed a saturation level.

[0209] The combining unit 70 selects and combines the first combining coefficient when the first amounts of received light RL15, RL15a, and the second amount of received light RL16 are all smaller than the saturation level based on the comparison result of the saturation level determination unit 60. The combining unit 70 selects and combines the second combining coefficient when at least one of the first amounts of received light RL15, RL15a is greater than the saturation level. The combining unit 70 selects and combines the third combining coefficient when at least one of the second amounts of received light RL16 is greater than the saturation level.

[0210] In this way, the synthesis unit 70 synthesizes the multiple pixel signals output from the pixels 31 based on the determination result of the saturation level determination unit 60, and outputs a synthesized signal s10. The image generating device 1G generates an image of the subject 9 based on the multiple synthesized signals s10 output from the multiple pixels 31.

[0211] The distance calculation section 90 calculates the distance L to the subject 9 using the first amount of received light RL15, RL15a and the second amount of received light RL16.

[0212] For example, when the saturation level determination section 60 determines that only the first amount of received light RL15, RL15a or the second amount of received light RL16 is greater than the saturation level, the distance calculation section 90 calculates the distance L using (Equation 15) or (Equation 17). When the saturation level determination section 60 determines that only the second amount of received light RL16 is greater than the saturation level, the distance calculation section 90 calculates the distance L using (Equation 16). In other cases, the distance calculation section 90 calculates the distance L using (Equation 16) or (Equation 17).

[0213]

number

[0214]

number

[0215]

number

[0216] In this image generating device 1G as well, the influence of specular reflection can be suppressed to easily generate an image. Also, in this image generating device 1G as well, the influence of specular reflection can be suppressed to easily measure the distance L to the subject 9.

[0217] [Modification of the fourth embodiment] An image generating device 1H according to a modification of the fourth embodiment will be described with reference to FIG.

[0218] 23 is a time chart showing the operation of image generating device 1H. In FIG. 23, a case where N is 2 and M is 1 is shown.

[0219] The drive control unit 50 generates first light emission pulses SG17 and SG17a that control the first light emission unit 10. The first light emission unit 10 emits irradiation light L1 toward the subject 9 in accordance with the first light emission pulses SG17 and SG17a. The drive control unit 50 also generates second light emission pulses SG18 and SG18a that control the second light emission unit 20 at a timing different from that of the first light emission pulses SG17 and SG17a. The second light emission unit 20 emits irradiation light L2 toward the subject 9 in accordance with the second light emission pulses SG18 and SG18a.

[0220] The illumination lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg17 of the first light emission pulse SG17, the first light emission period Tg17a of the first light emission pulse SG17a, the second light emission period Tg18 of the second light emission pulse SG18, and the second light emission period Tg18a of the second light emission pulse SG18a so that they are equal to each other.

[0221] The light receiving unit 30 receives reflected light R1 in accordance with the first light receiving pulse RG17 generated by the drive control unit 50, receives reflected light R1 in accordance with the first light receiving pulse RG17a, receives reflected light R2 in accordance with the second light receiving pulse RG18, and receives reflected light R2 in accordance with the second light receiving pulse RG18a.

[0222] The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls so that the first light-receiving start time ts17 from the start of irradiation to the first light-receiving pulse RG17 and the second light-receiving start time ts18 from the start of irradiation to the second light-receiving pulse RG18 are equal. The drive control unit 50 controls so that the first light-receiving period Tr17, the first light-receiving period Tr17a, the second light-receiving period Tr18, and the second light-receiving period Tr18a are equal. The first light-receiving start time ts17a from the start of irradiation to the first light-receiving pulse RG17a is controlled to be the sum of the first light-receiving start time ts17 and the first light-receiving period Tr17. A second light-receiving start time ts18a from the start of irradiation to the second light-receiving pulse RG18a is controlled to be equal to the sum of the second light-receiving start time ts18 and the second light-receiving period Tr18.

[0223] The light receiving section 30 generates the signal charge accumulated in the pixel 31 by receiving light in response to the first light receiving pulse RG17 as a first amount of received light RL17. The light receiving section 30 also generates the signal charge accumulated in the pixel 31 by receiving light in response to the first light receiving pulse RG17a as a first amount of received light RL17a. The light receiving section 30 also generates the signal charge accumulated in the pixel 31 by receiving light in response to the second light receiving pulse RG18 as a second amount of received light RL18. The light receiving section 30 also generates the signal charge accumulated in the pixel 31 by receiving light in response to the second light receiving pulse RG18a as a second amount of received light RL18a.

[0224] The saturation level determination section 60 determines whether or not the first amounts of received light RL17, RL17a and the second amounts of received light RL18, RL18a exceed a saturation level.

[0225] The combining unit 70 selects and combines the first combining coefficient when the first amounts of received light RL17, RL17a and the second amounts of received light RL18, RL18a are all smaller than the saturation level based on the comparison result of the saturation level determination unit 60. The combining unit 70 selects and combines the second combining coefficient when at least one of the first amounts of received light RL17, RL17a is greater than the saturation level. The combining unit 70 selects and combines the third combining coefficient when at least one of the second amounts of received light RL18, RL18a is greater than the saturation level.

[0226] In this way, the synthesis unit 70 synthesizes the multiple pixel signals output from the pixels 31 based on the determination result of the saturation level determination unit 60, and outputs a synthesized signal s10. The image generating device 1H generates an image of the subject 9 based on the multiple synthesized signals s10 output from the multiple pixels 31.

[0227] A distance calculation section 90 calculates a distance L to the subject 9 using the first amounts of received light RL17, RL17a and the second amounts of received light RL18, RL18a.

[0228] When the saturation level determination section 60 determines that only the first amount of received light RL17 or RL17a is greater than the saturation level, the distance calculation section 90 calculates the distance L using (Equation 18). When the saturation level determination section 60 determines that only the second amount of received light RL18 or RL18a is greater than the saturation level, the distance calculation section 90 calculates the distance L using (Equation 19). In other cases, the distance calculation section 90 calculates the distance L using any of (Equation 18), (Equation 19), and (Equation 20).

[0229]

number

[0230]

number

[0231]

number

[0232] In this image generating device 1H as well, the influence of specular reflection can be suppressed to easily generate an image. Also, in this image generating device 1H as well, the influence of specular reflection can be suppressed to easily measure the distance L to the subject 9.

[0233] (Embodiment 5) An image generating device according to a fifth embodiment will be described.

[0234] Fig. 24 is a time chart showing the operation of the image generating device according to embodiment 5. Fig. 24 shows an example in which the light emission period and the light reception period of the drive timing shown in embodiment 1 are executed multiple times in the same frame.

[0235] 24, the image generating device may repeatedly accumulate electric charges by repeatedly receiving reflected light R1 corresponding to the emission of the first light-emitting unit 10 a plurality of times, and generate the accumulated electric charges as the first amount of received light. Similarly, the image generating device may repeatedly accumulate electric charges by repeatedly receiving reflected light R2 corresponding to the emission of the second light-emitting unit 20 a plurality of times, and generate the accumulated electric charges as the second amount of received light.

[0236] Fig. 25 is a diagram showing another example of the operation of the image generating device according to embodiment 5. Fig. 25 shows another example in which the light emission period and the light reception period of the drive timing shown in embodiment 1 are executed multiple times in the same frame.

[0237] As shown in FIG. 25, the image generating device may repeatedly accumulate electric charges by repeatedly receiving reflected light R1, R2 corresponding to the emission of light from the first light-emitting unit 10 and the emission of light from the second light-emitting unit 20, and generate the accumulated electric charges as the first amount of received light and the second amount of received light.

[0238] Fig. 26 is a diagram showing another example of the operation of the image generating device according to embodiment 5. Fig. 26 shows an example in which the light emission period and the light reception period are executed multiple times in the same frame when N is 2 and M is 2 in the form shown in embodiment 4.

[0239] As shown in FIG. 26, the image generating device may repeatedly accumulate charges by repeatedly receiving reflected light R1 in response to the emission of the first light-emitting unit 10 a plurality of times, and generate the accumulated charges as the first amount of received light of one of the devices. The image generating device may also repeatedly accumulate charges by repeatedly receiving reflected light R1 in response to the emission of the first light-emitting unit 10 a plurality of times, and generate the accumulated charges as the first amount of received light of the other device. Similarly, the image generating device may repeatedly accumulate charges by repeatedly receiving reflected light R2 in response to the emission of the second light-emitting unit 20 a plurality of times, and generate the accumulated charges as the second amount of received light of the other device. The image generating device may also repeatedly accumulate charges by repeatedly receiving reflected light R2 in response to the emission of the second light-emitting unit 20 a plurality of times, and generate the accumulated charges as the second amount of received light of the other device.

[0240] Fig. 27 is a diagram showing another example of the operation of the image generating device according to embodiment 5. Fig. 27 shows another example in which the light emission period and the light reception period are executed multiple times in the same frame when N is 2 and M is 2 in the form shown in embodiment 4.

[0241] As shown in FIG. 27, the image generating device may repeatedly accumulate electric charges by repeatedly receiving reflected light R1, R2 corresponding to the emission of light from the first light-emitting unit 10 and the emission of light from the second light-emitting unit 20, and generate the accumulated electric charges as the first amount of received light and the second amount of received light.

[0242] In this way, by repeating light emission and light reception (exposure) multiple times within the same frame, the amount of light emitted each time can be reduced. Also, by distributing light emission multiple times, the peak power of the first light-emitting unit 10 and the second light-emitting unit 20 can be reduced. Also, by repeating light emission multiple times, the device becomes less susceptible to the effects of noise, and the S / N ratio can be improved.

[0243] (summary) The image generating devices 1 to 1H of this embodiment include a first light-emitting unit 10 and a second light-emitting unit 20 that emit light, a light-receiving unit 30 that receives reflected light R1, R2 that is reflected when the light (illumination light L1, L2) emitted by the first light-emitting unit 10 and the second light-emitting unit 20 is irradiated onto a subject 9, using a plurality of pixels 31 arranged in a two-dimensional shape, a drive control unit 50 that controls the light-emitting period of the first light-emitting unit 10, the light-emitting period of the second light-emitting unit 20, and the light-receiving period of the light-receiving unit 30, and a combination unit 70 that combines a first pixel signal s1 output from the pixel 31 in response to light reception by the light-receiving unit 30 corresponding to the emission of the first light-emitting unit 10, and a second pixel signal s2 output from the pixel 31 in response to light reception by the light-receiving unit 30 corresponding to the emission of the second light-emitting unit 20 to generate a combined signal s10. The first light-emitting unit 10 and the second light-emitting unit 20 are provided at positions where they do not overlap when viewed from the subject 9. The drive control unit 50 causes the first light-emitting unit 10 and the second light-emitting unit 20 to emit light at different timings in the same frame. When at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal greater than a predetermined level, the synthesis unit 70 generates a synthesis signal s10 without using the pixel signal greater than the predetermined level.

[0244] In this way, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal greater than a predetermined level, the composite signal s10 is generated without using the pixel signal greater than the predetermined level, thereby suppressing the effects of specular reflection and easily generating an image.

[0245] The predetermined level may also be set in advance based on a saturation level TH at which the signal charge accumulated in the pixel 31 becomes saturated.

[0246] According to this, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a saturation level, the composite signal s10 can be generated without using the saturated pixel signal, thereby making it possible to easily generate an image by suppressing the influence of specular reflection.

[0247] In addition, the light receiving unit 30 may read out the signal charge accumulated in the pixel 31 corresponding to the emission of the first light emitting unit 10 into a first packet p1, output a first pixel signal s1 including information on the amount of light received corresponding to the signal charge, and read out the signal charge accumulated in the pixel 31 of the light receiving unit 30 corresponding to the emission of the second light emitting unit 20 into a second packet p2, and output a second pixel signal s2 including information on the amount of light received corresponding to the signal charge.

[0248] In this way, by reading out the signal charge accumulated in the pixel 31 into multiple packets and outputting the first pixel signal s1 and the second pixel signal s2, it is possible to generate one image using information within the same frame. That is, instead of generating one image using multiple frames, it is possible to generate one image using information within one frame. This makes it possible to easily generate an image by suppressing the effects of specular reflection.

[0249] In addition, the first light-emitting section 10 and the second light-emitting section 20 may be disposed on both outer sides of the light-receiving section 30.

[0250] In this way, by arranging the first light-emitting unit 10 and the second light-emitting unit 20 on both sides of the light-receiving unit 30, rather than on one side of the light-receiving unit 30, it is possible to prevent both the first pixel signal s1 and the second pixel signal s2 from exceeding a predetermined level. This makes it possible to easily generate an image by suppressing the influence of specular reflection.

[0251] The image generating devices 1 to 1H of the present embodiment include a first light-emitting unit 10 that irradiates light (irradiation light L1) toward a subject 9, a second light-emitting unit 20 that irradiates light (irradiation light L2) toward the subject 9, a light-receiving unit 30 that receives reflected light R1, R2 generated when the light irradiated by the first light-emitting unit 10 and the second light-emitting unit 20 is reflected by the subject 9, using a plurality of pixels 31 that are two-dimensionally arranged, and a light-emitting period of the first light-emitting unit 10, a light-emitting period of the second light-emitting unit 20, and a light-receiving unit 31 that receives the reflected light R1, R2 generated when the light irradiated by the first light-emitting unit 10 and the second light-emitting unit 20 is reflected by the subject 9. The image display device includes a drive control unit 50 that controls the light receiving period of the light receiving unit 30, a saturation level determination unit 60 that determines whether or not the first pixel signal s1 and the second pixel signal s2 output from the pixels 31 of the light receiving unit 30 in response to the irradiation of light by the first light emitting unit 10 and the second light emitting unit 20 exceed a saturation level TH, and a combining unit 70 that generates a combined signal s10 by combining the first pixel signal s1 and the second pixel signal s2 based on the determination result of the saturation level determination unit 60. The first light emitting unit 10 and the second light emitting unit 20 are arranged apart from each other. The drive control unit 50 generates, at different timings, a first light emission pulse for causing the first light emission unit 10 to emit light and a second light emission pulse for causing the second light emission unit 20 to emit light in the same frame, and generates, at different timings, a first light reception pulse for receiving reflected light R1 generated when light irradiated by the first light emission unit 10 is reflected by the subject 9, and a second light reception pulse for receiving reflected light R2 generated when light irradiated by the second light emission unit 20 is reflected by the subject 9. The light reception unit 30 outputs a first pixel signal s1 including information on a first amount of received light received in accordance with the first light reception pulse, and outputs a second pixel signal s2 including information on a second amount of received light received in accordance with the second light reception pulse. The saturation level determination unit 60 compares the first amount of received light, the second amount of received light, and the saturation level TH. The combination unit 70 (1) when the first amount of received light and the second amount of received light are both smaller than a saturation level TH, a first pixel signal s1 and a second pixel signal s2 are combined by a first combination coefficient; (2) when only the first amount of received light is greater than the saturation level TH, the first pixel signal s1 and the second pixel signal s2 are combined with a second combination coefficient; (3) When only the second amount of received light is greater than the saturation level TH, the first pixel signal s1 and the second pixel signal s2 are combined by a third combination coefficient.

[0252] In this way, by combining the first pixel signal s1 and the second pixel signal s2 based on the comparison result of the first amount of received light, the second amount of received light, and the saturation level TH, it is possible to easily generate an image while suppressing the effects of specular reflection.

[0253] The second synthesis coefficient may be a coefficient for synthesis considering the first amount of received light as zero, and the third synthesis coefficient may be a coefficient for synthesis considering the second amount of received light as zero.

[0254] According to this, when one of the first and second received light amounts is saturated, an image can be generated without using a pixel signal including information on the saturated received light amount, thereby suppressing the influence of specular reflection and generating an image easily.

[0255] The first synthesis coefficient may be a coefficient for synthesis that regards the first amount of received light as half and the second amount of received light as half.

[0256] According to this, when neither the first nor the second amount of received light is saturated, an image can be generated by using half each of the pixel signals including information on both the amounts of received light that are not saturated. This makes it possible to easily generate an image by suppressing the effects of specular reflection.

[0257] The image generating device 1A may further include a correction unit 80 that corrects the information on the first amount of received light or the information on the second amount of received light. The drive control unit 50 controls the first light receiving period of the light receiving unit 30 corresponding to the irradiation of light by the first light emitter 10 and the second light receiving period of the light receiving unit 30 corresponding to the irradiation of light by the second light emitter 20 so as to be different from each other. The correction unit 80 corrects the information on the first amount of received light or the information on the second amount of received light based on the amount of received light caused by the difference between the first light receiving period and the second light receiving period. The synthesis unit 70 may synthesize the first pixel signal s1 and the second pixel signal s2 using the information on the first amount of received light or the information on the second amount of received light corrected by the correction unit 80.

[0258] In this way, the correction unit 80 corrects the information on the first amount of received light or the information on the second amount of received light based on the amount of received light caused by the difference between the first light receiving period and the second light receiving period, so that the first pixel signal s1 or the second pixel signal s2 can be appropriately generated. This makes it possible to appropriately generate an image in which the influence of specular reflection is suppressed.

[0259] The image generating device 1B may further include a correction unit 80 that corrects the information on the first amount of received light or the information on the second amount of received light. The drive control unit 50 controls the first light receiving start time from the start of irradiation of the first light emitter 10 to the start of light receiving by the light receiving unit 30 to be different from the second light receiving start time from the start of irradiation of the second light emitter 20 to the start of light receiving by the light receiving unit 30. The correction unit 80 corrects the information on the first amount of received light or the information on the second amount of received light based on the amount of received light caused by the difference between the first light receiving start time and the second light receiving start time. The synthesis unit 70 may synthesize the first pixel signal s1 and the second pixel signal s2 using the information on the first amount of received light or the information on the second amount of received light corrected by the correction unit 80.

[0260] In this way, the correction unit 80 corrects the information on the first amount of received light or the information on the second amount of received light based on the amount of received light caused by the difference between the first light-receiving start time and the second light-receiving start time, thereby making it possible to appropriately generate the first pixel signal s1 or the second pixel signal s2. This makes it possible to appropriately generate an image in which the influence of specular reflection is suppressed.

[0261] Moreover, the image generating devices 1A to 1H may further include a distance calculation section 90 that calculates the distance L to the subject 9, and the distance calculation section 90 may calculate the distance L using the corrected first amount of received light or the corrected second amount of received light.

[0262] In this way, the distance calculation section 90 calculates the distance L using the corrected first amount of received light or the corrected second amount of received light, so that the distance L to the subject 9 can be measured appropriately.

[0263] Moreover, the image generating devices 1A to 1H of the present embodiment include a first light-emitting unit 10 that irradiates light (irradiation light L1) toward the subject 9, a second light-emitting unit 20 that irradiates light (irradiation light L2) toward the subject 9, a light-receiving unit 30 that receives reflected light R1, R2 generated when the light irradiated by the first light-emitting unit 10 and the second light-emitting unit 20 is reflected by the subject 9, using a plurality of pixels 31 arranged two-dimensionally, and a light-receiving unit 31 that receives a light-emitting period of the first light-emitting unit 10, a light-emitting period of the second light-emitting unit 20, and The image display device includes a drive control unit 50 that controls the light receiving period of the first light emitter 10 and the light receiving unit 30, a saturation level determination unit 60 that determines whether or not the first pixel signal s1 and the second pixel signal s2 output from the pixels 31 of the light receiving unit 30 in response to the irradiation of light by the first light emitter 10 and the second light emitter 20 exceed a saturation level TH, and a combination unit 70 that generates a combined signal s10 by combining the first pixel signal s1 and the second pixel signal s2 based on the determination result of the saturation level determination unit 60. The first light emitter 10 and the second light emitter 20 are arranged apart from each other. The drive control unit 50 generates, at different timings, one or more first light emission pulses for making the first light emission unit 10 emit light and one or more second light emission pulses for making the second light emission unit 20 emit light in the same frame, and generates, at different timings, one or more first light reception pulses for receiving reflected light R1 generated by light irradiated by the first light emission unit 10 being reflected by the subject 9, and one or more second light reception pulses for receiving reflected light R2 generated by light irradiated by the second light emission unit 20 being reflected by the subject 9. The light reception unit 30 outputs a first pixel signal s1 including information on one or more first received light amounts received in accordance with one or more first received light pulses, and outputs a second pixel signal s2 including information on one or more second received light amounts received in accordance with one or more second received light pulses. The saturation level determination unit 60 compares the information on one or more first received light amounts, the information on one or more second received light amounts, and the saturation level TH. The synthesis unit 70 (1) when the first amount of received light, which is equal to or greater than 1, and the second amount of received light, which is equal to or greater than 1, are all smaller than a saturation level TH, the first pixel signal s1 and the second pixel signal s2 are combined by a first combination coefficient; (2) when at least one of the first received light amounts, which is equal to or greater than one, is greater than a saturation level TH, the first pixel signal s1 and the second pixel signal s2 are combined with a second combination coefficient; (3) When at least one of the second amounts of received light that are equal to or greater than one is greater than a saturation level, the first pixel signal s1 and the second pixel signal s2 are combined by a third combination coefficient.

[0264] In this way, by combining the first pixel signal s1 and the second pixel signal s2 based on the comparison result of the first amount of received light which is equal to or greater than 1, the second amount of received light which is equal to or greater than 1, and the saturation level TH, it is possible to easily generate an image while suppressing the effects of specular reflection.

[0265] In addition, the image generating devices 1A to 1H may further include a distance calculation unit 90 that calculates the distance to the subject 9, and the distance calculation unit 90 may calculate the distance L using a sum of one or more first amounts of received light and a sum of one or more second amounts of received light.

[0266] This makes it possible to easily measure the distance L to the subject 9 while suppressing the effect of specular reflection.

[0267] Furthermore, the drive control unit 50 may execute each of the light emission period and the light reception period multiple times in the same frame.

[0268] In this way, by executing each of the light emission period and the light reception period multiple times within the same frame, the amount of light emitted per period can be reduced. Also, by distributing the light emission over multiple periods, the peak power of the first light-emitting unit 10 and the second light-emitting unit 20 can be reduced.

[0269] (Other embodiments) Although the embodiments have been described above, the present disclosure is not limited to the above-mentioned embodiments. The present disclosure also includes forms obtained by applying various modifications to the embodiments that a person skilled in the art can conceive, and forms realized by arbitrarily combining the components and functions of each embodiment within the scope of the present disclosure.

[0270] The image generating devices 1 to 1H are not limited to being installed in vehicles, and may be installed in a predetermined space 8 such as inside a building. The image generating devices 1 to 1H may be devices included in an imaging device. The image generating devices 1A to 1H may be devices included in a distance measuring device. [Industrial Applicability]

[0271] The present disclosure is useful for improving the performance of devices that detect a person's health state, alertness state, concentration state, and the like from images. [Explanation of symbols]

[0272] 1, 1A, 1B, 1C, 1D, 1E, 1F, 1G, 1H Image generating device 8 Space 9. Subject 10 First light emitting part 10a, 10b light source 20 Second light emitting part 20a, 20b light source 30 Light receiving section 31 pixels 40 Substrate 50 Drive control unit 60 Saturation level judgment section 70 Synthesis section 80 Correction section 90 Distance calculation section L1, L2 irradiation light p1 First packet p2 Second packet R1, R2 reflected light RG1, RG3, RG5, RG7, RG9, RG11, RG11a, RG13, RG13a, RG15, RG15a, RG17, RG17a First light receiving pulse RG2, RG4, RG6, RG8, RG10, RG12, RG14, RG14a, RG16, RG18, RG18a Second light receiving pulse RL1, RL3, RL5, RL7, RL9, RL11, RL11a, RL13, RL13a, RL15, RL15a, RL17, RL17a 1st light reception amount RL2, RL4, RL6, RL8, RL10, RL12, RL14, RL14a, RL16, RL18, RL18a 2nd light reception amount s1 First pixel signal s2 Second pixel signal s10 composite signal SG1, SG3, SG5, SG7, SG9, SG11, SG13, SG15, SG15a, SG17, SG17a First light emission pulse SG2, SG4, SG6, SG8, ​​SG10, SG12, SG14, SG16, SG18, SG18a Second light emission pulse Tg1, Tg3, Tg5, Tg7, Tg9, Tg11, Tg13, Tg15, Tg15a, Tg17, Tg17a First light emission period Tg2, Tg4, Tg6, Tg8, Tg10, Tg12, Tg14, Tg16, Tg18, Tg18a Second light emission period TH Saturation Level ts1, ts3, ts5, ts7, ts9, ts11, ts11a, ts13, ts13a, ts15, ts15a, ts17, ts17a First light reception start time ts2, ts4, ts6, ts8, ts10, ts12, ts14, ts14a, ts16, ts18, ts18a Second light reception start time Tr1, Tr3, Tr5, Tr7, Tr9, Tr11, Tr11a, Tr13, Tr13a, Tr15, Tr15a, Tr17, Tr17a First light receiving period Tr2, Tr4, Tr6, Tr8, Tr10, Tr12, Tr14, Tr14a, Tr16, Tr18, Tr18a Second light receiving period

Claims

1. a first light emitting unit that irradiates light toward a subject; A second light emitting unit that irradiates light toward the subject; a light receiving unit that receives reflected light generated by the light irradiated by the first light emitting unit and the light irradiated by the second light emitting unit being reflected by the subject, using a plurality of pixels that are two-dimensionally arranged; a drive control unit that controls a light emission period of the first light emitter, a light emission period of the second light emitter, and a light reception period of the light receiver; a saturation level determination unit that determines whether a first pixel signal and a second pixel signal output from the pixel of the light receiving unit in response to irradiation with light from the first light emitting unit and the second light emitting unit exceed a saturation level; a synthesis unit that synthesizes the first pixel signal and the second pixel signal based on a determination result of the saturation level determination unit to generate a synthesis signal; Equipped with The first light emitting unit and the second light emitting unit are disposed apart from each other, the drive control unit generates, at different timings in the same frame, a first light emission pulse for causing the first light emission unit to emit light and a second light emission pulse for causing the second light emission unit to emit light, and generates, at different timings, a first light reception pulse for receiving the reflected light generated by the light irradiated by the first light emission unit being reflected by the subject, and a second light reception pulse for receiving the reflected light generated by the light irradiated by the second light emission unit being reflected by the subject, the light receiving unit outputs the first pixel signal including information on a first amount of light received in accordance with the first light receiving pulse, and outputs the second pixel signal including information on a second amount of light received in accordance with the second light receiving pulse; the saturation level determination unit compares the first amount of received light, the second amount of received light, and the saturation level; The synthesis unit is (1) when the first amount of received light and the second amount of received light are both smaller than the saturation level, the first pixel signal and the second pixel signal are combined by a first combination coefficient; (2) when only the first amount of received light is greater than the saturation level, the first pixel signal and the second pixel signal are combined with a second combination coefficient; (3) when only the second amount of received light is greater than the saturation level, the first pixel signal and the second pixel signal are combined by a third combination coefficient; a correction unit that corrects the information on the first amount of received light or the information on the second amount of received light, the drive control unit controls a first light receiving period of the light receiving unit corresponding to irradiation of light from the first light emitting unit and a second light receiving period of the light receiving unit corresponding to irradiation of light from the second light emitting unit so as to be different from each other; the correction unit corrects information about the first amount of received light or information about the second amount of received light based on an amount of received light caused by a difference between the first light-receiving period and the second light-receiving period; The combining unit combines the first pixel signal and the second pixel signal using information on the first amount of received light or information on the second amount of received light corrected by the correction unit. Image generating device.

2. the second synthesis coefficient is a coefficient for synthesis with the first amount of received light regarded as 0, The third synthesis coefficient is a coefficient for synthesis by regarding the second amount of received light as 0.

2. The image generating device of claim 1.

3. The first synthesis coefficient is a coefficient for synthesizing the first received light amount by regarding it as half and the second received light amount by regarding it as half.

3. An image generating device according to claim 1 or 2.

4. a first light emitting unit that irradiates light toward a subject; A second light emitting unit that irradiates light toward the subject; a light receiving unit that receives reflected light generated by the light irradiated by the first light emitting unit and the light irradiated by the second light emitting unit being reflected by the subject, using a plurality of pixels that are two-dimensionally arranged; a drive control unit that controls a light emission period of the first light emitter, a light emission period of the second light emitter, and a light reception period of the light receiver; a saturation level determination unit that determines whether a first pixel signal and a second pixel signal output from the pixel of the light receiving unit in response to irradiation with light from the first light emitting unit and the second light emitting unit exceed a saturation level; a synthesis unit that synthesizes the first pixel signal and the second pixel signal based on a determination result of the saturation level determination unit to generate a synthesis signal; Equipped with The first light emitting unit and the second light emitting unit are disposed apart from each other, the drive control unit generates, at different timings in the same frame, a first light emission pulse for causing the first light emission unit to emit light and a second light emission pulse for causing the second light emission unit to emit light, and generates, at different timings, a first light reception pulse for receiving the reflected light generated by the light irradiated by the first light emission unit being reflected by the subject, and a second light reception pulse for receiving the reflected light generated by the light irradiated by the second light emission unit being reflected by the subject, the light receiving unit outputs the first pixel signal including information on a first amount of light received in accordance with the first light receiving pulse, and outputs the second pixel signal including information on a second amount of light received in accordance with the second light receiving pulse; the saturation level determination unit compares the first amount of received light, the second amount of received light, and the saturation level; The synthesis unit is (1) when the first amount of received light and the second amount of received light are both smaller than the saturation level, the first pixel signal and the second pixel signal are combined by a first combination coefficient; (2) when only the first amount of received light is greater than the saturation level, the first pixel signal and the second pixel signal are combined with a second combination coefficient; (3) when only the second amount of received light is greater than the saturation level, the first pixel signal and the second pixel signal are combined by a third combination coefficient; a correction unit that corrects the information on the first amount of received light or the information on the second amount of received light, the drive control unit controls a first light-receiving start time from a start of irradiation of the first light-emitting unit to a start of light-receiving by the light-receiving unit to be different from a second light-receiving start time from a start of irradiation of the second light-emitting unit to a start of light-receiving by the light-receiving unit; the correction unit corrects information about the first amount of received light or information about the second amount of received light based on an amount of received light caused by a difference between the first light-receiving start time and the second light-receiving start time; The combining unit combines the first pixel signal and the second pixel signal using information on the first amount of received light or information on the second amount of received light corrected by the correction unit. Image generating device.

5. Further, a distance calculation unit is provided for calculating a distance to the subject, The distance calculation unit calculates the distance using the first amount of received light or the second amount of received light after the correction.

5. An image generating device according to claim 1 or 4.

6. the drive control unit executes each of the light emission period and the light reception period a plurality of times in the same frame; The image generating device according to any one of claims 1 to 5.

Images