Photographing device and distance measuring method

Abstract

To provide a simple distance measuring technique which avoids the interference of light between a plurality of photographing devices.SOLUTION: In a photographing device 100, a distance measuring sensor 20 measures a distance to an object by measuring a time from when pulse light is irradiated to the object until when the pulse light is reflected to return. A light source unit 90 emits pulse light on the basis of a pulse signal obtained by encoding identification information of the photographing device. A determination unit 60 determines the identification information on the basis of the pulse signal acquired by receiving the pulse light. An adjustment unit 70 adjusts distance measuring start timing so as not to perform the distance measuring until the end of the distance measuring by another photographing device, when the determined identification information is identification information of the other photographing device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging device and a distance measuring method. [Background technology]

[0002] There is a Time of Flight (ToF) camera that calculates the distance to an object by shining infrared (IR) light on the object and measuring the time it takes for the reflected light to be received, and is used for object recognition and three-dimensional measurement of objects.

[0003] Patent Document 1 discloses a distance image capturing technique that controls the light receiving timings of a plurality of distance measuring image capturing devices so that they do not overlap in time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 188782 Summary of the Invention [Problem to be solved by the invention]

[0005] When measuring distances using multiple ToF cameras, there is a problem that the infrared light from each camera interferes with the other cameras, making it impossible to measure correctly. Therefore, it was necessary to take measures such as connecting the multiple ToF cameras with cables to synchronize them and shifting the emission periods of the infrared light to prevent interference between the cameras. However, if the cables cannot be connected due to layout restrictions, the multiple ToF cameras cannot be synchronized, making it impossible to operate a distance measurement system using multiple ToF cameras.

[0006] The present invention has been made in view of the above circumstances, and has an object to provide a simple distance measurement technique that avoids interference of light emitted by a plurality of imaging devices. [Means for solving the problem]

[0007] In order to solve the above problems, an imaging device of one embodiment of the present invention includes a ranging sensor that measures the distance to an object by measuring the time between when pulsed light is irradiated onto the object and when it is reflected and returned, a light source unit that irradiates pulsed light based on a pulse signal that encodes identification information of the imaging device, a determination unit that determines the identification information based on the pulse signal obtained by receiving the pulsed light, and an adjustment unit that, if the determined identification information is identification information of another imaging device, adjusts the timing of starting ranging so that ranging is not performed until ranging by the other imaging device is completed.

[0008] Another aspect of the present invention is a distance measuring method, which includes the steps of: measuring a distance to an object by measuring a time from when a pulsed light is irradiated onto the object until it is reflected and returned; irradiating the pulsed light based on a pulse signal encoding identification information of an image capturing device; determining the identification information based on the pulse signal obtained by receiving the pulsed light; and, if the determined identification information is identification information of another image capturing device, adjusting a timing to start distance measurement so as not to perform distance measurement until distance measurement by the other image capturing device is completed.

[0009] Any combination of the above components, and any transformation of the present invention into a method, device, system, recording medium, computer program, etc., are also effective as aspects of the present invention. Effect of the Invention

[0010] According to the present invention, it is possible to provide a simple distance measuring technique that avoids interference of light emitted by a plurality of imaging devices. [Brief description of the drawings]

[0011] [Figure 1] 1 is a configuration diagram of an imaging device according to an embodiment of the present invention. [Diagram 2] 2(a) and 2(b) are diagrams for explaining a situation in which the distance measurement periods of a plurality of image capturing devices overlap. [Diagram 3]3(a) to 3(c) are diagrams for explaining a method for preventing the occurrence of overlapping periods by providing identification periods before and after distance measurement periods. [Figure 4] 4(a) to 4(e) are diagrams illustrating a method of lighting a pulsed light in an identification period. [Diagram 5] 2A to 2C are diagrams illustrating the principle of event detection based on a luminance change by the EVS of FIG. 1. [Figure 6] 6(a) and 6(b) are diagrams for explaining a pulse recognition method by the transition detection unit of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] 1 is a configuration diagram of an imaging device 100 according to the present embodiment. A distance measuring system is configured using a plurality of imaging devices 100, and the distance to an object is measured by the plurality of imaging devices 100. Each of the plurality of imaging devices 100 is assigned an ID (identification) that is unique identification information.

[0013] The imaging device 100 includes lens units 10 and 12, a distance measurement sensor 20, a light receiving unit 30, a luminance change detection unit 40, a transition detection unit 50, a determination unit 60, an adjustment unit 70, a pulse generation unit 80, and a light source unit 90.

[0014] The light receiving unit 30 and the brightness change detection unit 40 are components of an event-based vision sensor (EVS) 200. Here, a configuration in which the EVS 200 is provided inside the image capturing device 100 is illustrated, but the EVS 200 may be provided externally in addition to the image capturing device 100 equipped with the distance measuring sensor 20. In this case, the event output of the EVS 200 is supplied to the image capturing device 100.

[0015] Also, here, a compound eye configuration is illustrated in which the lens unit 12 of the EVS 200 is provided separately from the lens unit 10 of the distance measurement sensor 20, but a single eye configuration in which the distance measurement sensor 20 and the EVS 200 share a single lens unit and the light source is divided into two using a beam splitter or the like may also be used.

[0016] The light source unit 90 irradiates an object with infrared pulsed light (hereinafter referred to as "IR pulse") based on a pulse signal generated during a distance measurement period by the pulse generating unit 80. The lens unit 10 focuses on the distance measuring sensor 20 the light reflected from the object irradiated with the IR pulse.

[0017] The distance measurement sensor 20 detects the distance to an object by measuring the time it takes for light irradiated onto the object to be reflected off the object and returned. As an example, a ToF sensor is used here, which irradiates the object with infrared light and captures the reflected light from the object to obtain a distance measurement image.

[0018] Furthermore, the pulse generating unit 80 generates a pulse signal that encodes an ID, which is identification information that uniquely identifies the image capturing device 100, during an identification period (hereinafter referred to as an "ID period") before and after the distance measuring period, as described below. The light source unit 90 irradiates an IR pulse to an object based on the pulse signal generated by the pulse generating unit 80 during the ID period.

[0019] The EVS200 captures an object, detects the brightness change of each pixel, and generates events containing the polarity (positive or negative), coordinates, and timestamp of the brightness change only for pixels that have a brightness change above a certain threshold. If the brightness change is positive, it is detected as a positive event, and if the brightness change is negative, it is detected as a negative event.

[0020] Unlike conventional frame-based image sensors, the EVS200 asynchronously outputs only information from pixels where an event has occurred, enabling it to detect luminance changes at high speed and with low power consumption.

[0021] The EVS 200 includes a light receiving unit 30 and a brightness change detection unit 40, and receives an IR pulse irradiated from the light source unit 90 to an object during the ID period. Note that, hereinafter, the EVS 200 will be described as receiving reflected light from an object, but it should be noted that in cases where multiple image capturing devices 100 face each other, the EVS 200 may directly receive pulsed light from another image capturing device 100.

[0022] The lens unit 12 focuses the reflected light from the object irradiated with the IR pulse onto the light receiving unit 30 of the EVS 200. In the lens unit 12, a visible light cut filter may be installed in front of the light receiving surface so that the light receiving unit 30 receives only infrared light. In addition, the visible light cut filter may be installed only in a part of the EVS 200 to limit the ID detection area, and the EVS 200 may be used for other purposes such as moving object detection in other parts.

[0023] The light receiving section 30 has a photodiode and a voltage conversion circuit, converts incident light into a voltage, and supplies the input voltage to the luminance change detection section 40 .

[0024] The luminance change detection unit 40 has a difference detection circuit and a comparator. The difference detection circuit generates a difference voltage between an input voltage and a reference voltage that is sequentially updated when an event occurs. When the difference voltage exceeds a preset positive or negative threshold, the comparator outputs the positive / negative polarity, the pixel coordinate value, and a timestamp as an event.

[0025] The event output by the luminance change detection unit 40 is supplied to the transition detection unit 50. The transition detection unit 50 correctly detects the timing of the rise and fall of the pulse by combining plus / minus events that may be detected consecutively during the pulse transition period into one plus / minus event, and supplies the detected pulse signal to the determination unit 60.

[0026] The determination unit 60 detects a pulse signal by identifying a pulse width from the difference between the occurrence time of a positive event and the occurrence time of a negative event in the ID period, and determines the ID by decoding the ID encoded in the pulse signal. The ID acquired by the determination unit 60 is supplied to the adjustment unit 70.

[0027] The adjustment unit 70 adjusts the distance measurement start timing based on the acquired ID. Specifically, when the acquired ID is the ID of another imaging device 100, the adjustment unit 70 adjusts its own distance measurement start timing by shifting its own distance measurement period until the distance measurement of the other imaging device 100 is completed in order to avoid interference with the pulsed light. The pulse generation unit 80 generates a pulse signal in the ID period and the distance measurement period based on the adjusted distance measurement start timing.

[0028] When measuring the distance to an object using a distance measuring system using multiple imaging devices 100 as shown in Figure 1, if the distance measuring periods of the multiple imaging devices 100 overlap, the pulsed light from each device will interfere with each other, making it impossible to measure the distance correctly.

[0029] 2(a) and 2(b) are diagrams for explaining a situation in which the distance measurement periods of a plurality of image capturing devices 100 overlap.

[0030] Fig. 2(a) shows IR pulses emitted during a ranging period of the first image capture device 100 (called "ToF1"), and Fig. 2(b) shows IR pulses emitted during a ranging period of the second image capture device 100 (called "ToF2"). An overlapping period exists between the ToF1 ranging period and the ToF2 ranging period, and the IR pulses interfere with each other, preventing correct ranging.

[0031] In order to prevent such overlapping periods from occurring, it is necessary to take measures such as connecting the multiple image capture devices 100 with cables to synchronize them and staggering the distance measurement periods so that the IR pulses do not interfere with each other. However, if it is not possible to connect the cables due to layout restrictions, it is not possible to operate a distance measurement system with multiple image capture devices 100.

[0032] Therefore, in the imaging device 100 of this embodiment, an ID period in which pulsed light is emitted based on a pulse signal that encodes identification information of the imaging device 100 is provided before and after the ranging period, and the imaging device 100 identifies the ID period to avoid overlapping ranging periods between multiple imaging devices 100.

[0033] Figures 3(a) to 3(c) are diagrams explaining a method for avoiding the occurrence of overlapping periods by providing ID periods before and after a ranging period. Figure 3(a) shows the IR pulse of ToF1, Figure 3(b) shows the IR pulse of ToF2 before improvement, and Figure 3(c) shows the IR pulse of ToF2 after improvement.

[0034] As shown in Fig. 3(a), a start ID period of ToF1 is provided before the ranging period of ToF1, and an end ID period of ToF1 is provided after the ranging period of ToF1. The end ID period of ToF1 ends at time t1. The pulse generating unit 80 generates a pulse signal that encodes an ID, which is identification information of ToF1, during the start ID period of ToF1 and the end ID period of ToF1, and the light source unit 90 emits a pulsed light based on the pulse signal.

[0035] As shown in Fig. 3(b) and Fig. 3(c), a start ID period of ToF2 is provided before the ranging period of ToF2, and an end ID period of ToF2 is provided after the ranging period of ToF2. The pulse generating unit 80 generates a pulse signal that encodes an ID, which is identification information of ToF2, during the ToF2 start ID period and the ToF2 end ID period, and the light source unit 90 emits a pulse light based on the pulse signal.

[0036] Figure 3(b) shows the ToF2 IR pulse before improvement. Because the start ID period of ToF2 starts at time t2 in the middle of the ToF1 ranging period, there is an overlapping period between the ToF1 ranging period and the ToF2 ranging period, and the IR pulses interfere with each other, preventing correct ranging.

[0037] Figure 3(c) shows the IR pulse of ToF2 after the improvement. ToF2 receives the ID signal of ToF1 during the start ID period and end ID period of ToF1, and starts its own ranging after waiting for the end ID period of ToF1 to end. That is, ToF2 starts its start ID period at time t3, which is after the end time t1 of the end ID period of ToF1. As a result, the start time of the start ID period of ToF2 shifts from time t2 before the improvement to time t3 after the improvement.

[0038] In this way, the start ID period of ToF2 starts after waiting for the end ID period of ToF1 to end, so there is no overlapping period between the ToF1 ranging period and the ToF2 ranging period, and interference between the IR pulses of each period can be avoided.

[0039] When the EVS 200 and transition detection unit 50 of an imaging device 100 detect an IR pulse during the ID period of another imaging device 100 and the judgment unit 60 identifies an ID other than its own from the pulse signal, the adjustment unit 70 adjusts the start timing of its own start ID period so as to wait for the end ID period of the other imaging device 100 before starting its own ranging period, thereby avoiding interference of IR pulses between multiple imaging devices 100.

[0040] Priority is assigned to each ID of the multiple image capture devices 100, and adjustment processing is performed so that distance measurement can be started in order starting from the image capture device 100 with the ID with the highest priority, thereby automating the adjustment of distance measurement timing among the multiple image capture devices 100. Specifically, when the determination unit 60 detects an ID with a higher priority than itself, the adjustment unit 70 performs adjustment processing to shift its own distance measurement period, and when it detects an ID with a lower priority than itself, adjustment processing to shift its own distance measurement period is not performed. In this way, distance measurement is automatically performed among the multiple image capture devices 100 in order of priority.

[0041] 4(a) to 4(e) are diagrams explaining a method of lighting a pulsed light in an ID period. Here, the start ID period and the end ID period are not distinguished and are referred to as "ID period." The IR pulsed light in the ID period includes a start portion, a sign portion, and an end portion.

[0042] The encoding section defines a one-bit period tb of the IR pulse, and if the IR pulse is on for tb period, it is set to "1", and if it is on for an integer multiple of tb, it is set to continuous "1". Similarly, if the IR pulse is off for tb period, it is set to "0", and if it is not on for an integer multiple of tb, it is set to continuous "0". The encoding section expresses the ID of the image capture device 100 as a bit string of IR pulses.

[0043] Here, in order to distinguish the pulse generated in the coding section from the pulse generated during the distance measurement period of the imaging device 100, the pulse width tb of one bit period is set sufficiently larger than the pulse width td of the distance measurement period.

[0044] For example, if the ID of imaging device 100 is a number ranging from 0 to 15 and the ID is expressed in 4 bits, then if the ID is 0, the ID bit string is "0000" as shown in Figure 4(a), if the ID is 1, the ID bit string is "0001" as shown in Figure 4(b), if the ID is 2, the ID bit string is "0010" as shown in Figure 4(c), if the ID is 3, the ID bit string is "0011" as shown in Figure 4(d), and if the ID is 15, the ID bit string is "1111" as shown in Figure 4(e).

[0045] In the sign section, the illumination period of the IR pulse is always an integer multiple of tb, but since this alone does not indicate the start position of the bit, a start section and an end section are added before and after the sign section.

[0046] At the beginning and end, a pulsed light is turned on for a period that is a non-integer multiple of tb, such as 2.5 times, so that it can be distinguished from the code portion. For example, at the beginning, a pulsed light is turned on for a period with a start pulse width of ts = 1.5 × tb, and at the end, a pulsed light is turned on for a period with a end pulse width of te = 2.5 × tb.

[0047] A transition period is provided at the end of the beginning part and at the beginning of the end part to separate them from the code part. For example, the transition period is set to 0.5 x tb.

[0048] In FIGS. 4(a) to 4(e), the configuration of the ID period has been described without distinguishing between the start ID period and the end ID period, but it is preferable to configure it so that the start ID period and the end ID period can be distinguished.

[0049] The start ID period and the end ID period may be configured to be distinguishable from each other by, for example, swapping the start pulse width and the end pulse width. For example, the start ID period may have a start pulse width ts = 1.5 × tb and an end pulse width te = 2.5 × tb, and the end ID period may have a start pulse width ts = 2.5 × tb and an end pulse width te = 1.5 × tb, so that the start ID period and the end ID period can be distinguished from each other. Alternatively, the start ID period and the end ID period may be configured to be distinguishable from each other by, for example, changing a specific position (e.g., the most significant bit) of the ID bit in the code section. For example, if the most significant bit of the code section is always 0 in the start ID period and the most significant bit of the code section is always 1 in the end ID period, the start ID period and the end ID period can be distinguished from each other by the most significant bit of the code section.

[0050] However, since it is known in advance that IR pulses are generated in the order of the start ID period, the ranging period, and the end ID period, it is possible to prevent the start ID period from being mistakenly detected as the end ID period even if the start ID period and the end ID period are not necessarily configured to be distinguishable.

[0051] FIG. 5 is a diagram for explaining the principle of event detection based on a luminance change by the EVS 200. In FIG.

[0052] 5, a reference threshold A, a positive threshold A+, and a negative threshold A- are set for the incident light luminance LA at time T1. The incident light luminance LA at time T1 changes to the incident light luminance LB at time T2, which exceeds the negative threshold A-, so a negative event occurs at time T2.

[0053] A reference threshold B, a positive threshold B+, and a negative threshold B- are newly set for the incident light luminance LB at time T2. The incident light luminance LB at time T2 changes to the incident light luminance LC at time T3, which exceeds the negative threshold B-, so a negative event occurs at time T3.

[0054] A reference threshold C, a positive threshold C+, and a negative threshold C- are newly set for the incident light luminance LC at time T3. The incident light luminance LC at time T3 changes to the incident light luminance LD at time T4, which exceeds the positive threshold C+, so a positive event occurs at time T4.

[0055] 6(a) and 6(b) are diagrams for explaining a pulse recognition method performed by the transition detection section 50. FIG.

[0056] When the EVS 200 of the imaging device 100 detects an IR pulse during the ID period, the EVS 200 detects all luminance changes exceeding a threshold as events, so there is a possibility that the EVS 200 may detect a transient luminance change that occurs during a pulse transition as an event and erroneously determine that an IR pulse has occurred. Therefore, it is necessary to accurately capture the occurrence of an IR pulse at the rising / falling edge of the pulse.

[0057] Fig. 6(a) shows an event that occurs during a pulse transition when the pulse rise time tr is 0. As can be seen from the example of Fig. 6(a), if the pulse rise / fall time tr is sufficiently smaller than the minimum event interval ti of the EVS 200, only one event is detected during the pulse transition.

[0058] Figure 6(b) shows events that occur during a pulse transition when the pulse rise time tr > 0. As can be seen from the example in Figure 6(b), if the pulse rise / fall time tr is greater than the minimum event interval ti of the EVS200, multiple events will be detected during the pulse transition.

[0059] The maximum number of times an event occurs is (tr / ti+1) times in succession, such as once when tr=0, twice when (tr / ti)<1, etc. Therefore, in order to accurately identify the occurrence of an IR pulse from the events detected by the EVS 200, the transition detection unit 50 determines that an IR pulse is rising when the brightness change detection unit 40 detects up to (tr / ti+1) consecutive positive events and outputs a High determination signal, and determines that an IR pulse is falling when the brightness change detection unit 40 detects up to (tr / ti+1) consecutive negative events. When an event is detected more than (tr / ti+1) times, it is a gradual brightness change, so it is not determined that a pulse is occurring.

[0060] The determination section 60 performs state transition based on the High / Low determination signal detected by the transition detection section 50, and performs decoding processing of the ID contained in the pulse signal.

[0061] First, the operation starts from an idle state, and when a High determination signal is input from the transition detection section 50, the determination section 60 transitions to a start determination state.

[0062] In the start portion determination state, when a Low determination signal is input after the start pulse width ts period has elapsed, the determination section 60 transitions to the code portion determination state after a transition period has elapsed.

[0063] In the code portion determination state, the determination section 60 repeats High / Low determination for each bit period tb for the number of code bits, and after identifying the bit string representing the ID, transitions to the end portion determination state.

[0064] In the end portion judgment state, when a High judgment signal is input after the transition period has elapsed and a Low judgment signal is input after the end pulse width te period has elapsed, the judgment section 60 completes the judgment of the ID period and returns to the idle state.

[0065] This enables the determination section 60 to read out the bit representation of the ID from the pulse signal in the ID period and acquire the ID.

[0066] When multiple IDs are detected by the determination unit 60, the adjustment unit 70 adjusts the distance measurement period starting from the imaging device 100 with the ID having the highest priority according to a predetermined priority order. For example, when imaging devices 100 with IDs 1, 2, 3, and 4 assigned in order of priority are used simultaneously, the imaging device 100 with ID=1 has the highest priority, so the imaging device 100 with ID=1 does not adjust the distance measurement period, and adjustments are made in order starting from the imaging device 100 with ID=2, and the start ID period of the imaging device 100 with ID=2 is adjusted so that the start ID period of the imaging device 100 with ID=2 starts after the end ID period of the imaging device 100 with ID=1 has elapsed. By repeating the adjustment of the distance measurement period according to the priority order of the IDs, the distance measurement periods of the multiple imaging devices 100 do not overlap.

[0067] As described above, according to the embodiment of the present invention, a distance measurement system is configured using multiple image capture devices 100, and distance measurement periods are automatically adjusted between the multiple image capture devices 100, thereby preventing interference between the pulsed lights and eliminating the need for cable connections between the image capture devices 100. Eliminating the need for cable connections between the image capture devices 100 simplifies the distance measurement system, and enables the operation of a distance measurement system using multiple image capture devices 100 even in environments where cables cannot be connected, which was previously impossible.

[0068] The present invention has been described above based on the embodiments. The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each treatment process, and that such modifications are also within the scope of the present invention.

[0069] In the present embodiment, the configuration for detecting the IR pulse during the ID period using the EVS 200 has been described, but other sensors capable of detecting the IR pulse may be used. For example, the IR pulse may be detected by detecting a change in luminance using the distance measuring sensor 20 or an image sensor. [Explanation of symbols]

[0070] 10, 12 lens unit, 20 distance measuring sensor, 30 light receiving unit, 40 brightness change detection unit, 50 transition detection unit, 60 determination unit, 70 adjustment unit, 80 pulse generation unit, 90 light source unit, 100 imaging device, 200 EVS.

Claims

1. a distance measuring sensor that measures a distance to an object by measuring a time from when a pulsed light is irradiated onto the object until when the pulsed light is reflected back; a light source unit that irradiates a pulsed light based on a pulse signal in which identification information of the imaging device is encoded; a determination unit that determines the identification information based on the pulse signal obtained by receiving the pulsed light; and an adjustment unit that, when the determined identification information is identification information of another imaging device, adjusts a timing to start distance measurement so that distance measurement is not performed until distance measurement by the other imaging device is completed.

2. An event-based vision sensor that captures an image of the object and detects a luminance change for each pixel, 2. The imaging device according to claim 1, wherein the event-based vision sensor receives the pulsed light and receives the pulse signal.

3. 3. The imaging device according to claim 2, wherein the determination unit specifies a pulse width of the pulse signal based on a difference between an occurrence time of a positive event and an occurrence time of a negative event detected by the event-based vision sensor.

4. 4. The imaging apparatus according to claim 1, further comprising an identification period during which pulsed light is emitted based on a pulse signal in which the identification information is encoded, before and after the distance measurement period.

5. measuring a distance to an object by measuring a time from when the pulsed light is irradiated onto the object until when the pulsed light is reflected back; irradiating a pulsed light based on a pulse signal in which identification information of the imaging device is encoded; determining the identification information based on the pulse signal obtained by receiving the pulsed light; If the determined identification information is identification information of another imaging device, a timing for starting distance measurement is adjusted so that distance measurement is not performed until distance measurement by the other imaging device is completed.

Images