Light reception device, information processing device, ranging device, and information processing method

Abstract

To propose a configuration for calculating an appropriate distance while suppressing an increase in a processing load.SOLUTION: A light reception device comprises: a light-receiving section for receiving reflected light obtained by reflection, by a subject, of light emitted from a light-emitting section by light emission commands issued on the basis of a predetermined processing cycle; and a calculation section for calculating information relating to the distance to the subject according to the difference between the light emission timing of the light-emitting section and the light reception timing of the light-receiving section. The light emission commands are timed so as to be delayed, every medium period, which includes a plurality of small periods, each of which includes one light emission, by a shift period with respect to reference timing in each of the predetermined processing cycles. A ratio of the shift period relative to the small period is modified every large period, which includes a plurality of the medium periods.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present technology relates to a light receiving device, an information processing device, a distance measuring device, and an information processing method for measuring distance information using a time-of-flight (ToF) method. [Background technology]

[0002] In distance measurement using the ToF (Time of Flight) method, pulsed light (pulsed light) is emitted toward the subject to be measured at a predetermined interval, and the reflected light from the subject is detected, thereby measuring the round-trip time of the light and calculating the distance to the subject.

[0003] If there is another light source that repeatedly emits light in synchronization with the irradiation cycle of the pulsed light, the distance to the subject may be measured erroneously by detecting the light emitted from the other light source.

[0004] To solve this problem, Patent Document 1 below discloses a configuration in which a random time offset is inserted for each irradiation at a predetermined cycle, thereby determining whether the detected light is normal reflected light or interfering light. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-056567 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technique of Patent Document 1 involves inserting a random offset each time a pulse of light is emitted, which causes problems such as a large processing load on the circuit and increased power consumption.

[0007] The present technology has been made in consideration of such problems, and aims to propose a configuration for calculating an appropriate distance while suppressing an increase in processing load. [Means for solving the problem]

[0008] The light receiving device according to the present technology includes a light receiving unit that receives light reflected from a subject when light is emitted from the light emitting unit in response to a light emission instruction issued based on a predetermined processing cycle, and a calculation unit that calculates information relating to the distance to the subject based on the difference between the light emission timing of the light emitting unit and the light reception timing of the light receiving unit, wherein the light emission instruction is issued at a timing delayed by a shift period from the reference timing in each cycle of the predetermined processing cycle for each medium period that includes multiple short periods that include one emission of light, and the ratio of the shift period to the short period is changed for each large period that includes multiple medium periods. By providing a shift period for each medium period in which short periods are successive, the periodicity of the light emission timing is disrupted.

[0009] The information processing device according to the present technology described above includes an other light source detection processing unit that detects light emitted from other light sources other than the light emitting unit based on information about the distance to the subject calculated according to the difference between the emission timing of light emitted from the light emitting unit in response to an emission instruction issued based on a predetermined processing cycle and the light reception timing at which the reflected light reflected by the subject is received, and the light emission instruction is set to a timing that is delayed by a shift period from the reference timing in each cycle of the predetermined processing cycle for each medium period that includes multiple short periods that include one emission of the light, and the ratio of the shift period to the short period is changed for each large period that includes multiple medium periods.

[0010] The distance measuring device according to the present technology described above includes a light receiving unit that receives light reflected from a subject when light is emitted from the light emitting unit in response to a light emitting instruction issued based on a predetermined processing cycle, and a calculation unit that calculates distance data to the subject based on the difference between the light emitting timing of the light emitting unit and the light receiving timing of the light receiving unit, wherein the light emitting instruction is issued at a timing delayed by a shift period from the reference timing in each cycle of the predetermined processing cycle for each medium period that includes multiple short periods that include one emission of light, and the ratio of the shift period to the short period is changed for each large period that includes multiple medium periods.

[0011] The information processing method executed by the information processing device relating to the above-mentioned present technology is such that, for light emitted from a light emitting unit in response to a light emitting instruction issued based on a predetermined processing cycle, information regarding the distance to the subject is calculated based on the difference between the light reception timing by a light receiving unit that receives reflected light of the light reflected by a subject and the light emission timing of the light emitting unit, the light emission instruction is set to a timing that is delayed by a shift period from the reference timing in each cycle of the predetermined processing cycle for each medium period that includes multiple short periods that include one emission of the light, and the ratio of the shift period to the short period is changed for each large period that includes multiple medium periods. Such an information processing device, distance measuring device, and information processing method can also provide the same effects as the light receiving device according to the present technology described above. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of the configuration of a distance measuring system according to a first embodiment of the present technology. [Figure 2] FIG. 2 is a block diagram illustrating an example of a time management unit. [Figure 3] 10A and 10B are diagrams illustrating an example of the relationship between a light emission timing signal, a received light intensity, and a counter value. [Figure 4] FIG. 10 is a diagram showing an example in which a sub-period is composed of only a light receiving period. [Figure 5]FIG. 10 is a diagram showing an example in which a sub-period is composed of a light-receiving period and a non-light-receiving period. [Figure 6] FIG. 10 is a diagram for explaining the relationship between a sub-period and a shift period. [Figure 7] FIG. 2 is a block diagram showing an example of a light-emitting unit. [Figure 8] FIG. 1 is a block diagram illustrating an example of a ToF sensor. [Figure 9] FIG. 2 is a block diagram illustrating an example of a distance calculation unit. [Figure 10] 10 is an example of a histogram generated based on unit distance data for each of a first intermediate period and a second intermediate period. [Figure 11] FIG. 10 is an explanatory diagram of detection of interfering light. [Figure 12] FIG. 10 is an explanatory diagram illustrating calculation of accurate distance data. [Figure 13] FIG. 2 is a diagram illustrating a configuration example of a long period in the first embodiment. [Figure 14] FIG. 10 is a diagram illustrating an example of a state in which information about distance is stored in a payload area of packet data conforming to MIPI. [Figure 15] FIG. 10 is a diagram illustrating another example of packet data that complies with MIPI. [Figure 16] FIG. 10 is a diagram showing another example of packet data conforming to MIPI, in which a virtual channel is used. [Figure 17] FIG. 10 is a diagram showing yet another example of packet data conforming to MIPI, in which both data for the first medium period and data for the second medium period are stored in the payload area. [Figure 18] FIG. 10 is a diagram showing another example of packet data conforming to MIPI, which includes other light source information. [Figure 19] 10 is a flowchart illustrating an example of processing executed by the distance measuring system. [Figure 20] 10 is a flowchart illustrating an example of medium-term measurement. [Figure 21] 10 is a flowchart illustrating an example of processing executed in a distance calculation unit. [Figure 22] FIG. 10 is a block diagram showing an example of a light-emitting unit according to a second embodiment. [Figure 23] FIG. 10 is a block diagram illustrating an example of a ToF sensor according to a second embodiment. [Figure 24] FIG. 10 is a block diagram showing an example of a distance measuring system according to a third embodiment. [Figure 25] FIG. 10 is a diagram showing a configuration example of a long period in Modification 2 regarding a shift period. [Figure 26] FIG. 10 is a diagram showing another example of the configuration of the long period in the second modification example regarding the shift period. [Figure 27] FIG. 10 is a diagram showing yet another example of the configuration of the long period in Modification 2 regarding the shift period. [Figure 28] FIG. 10 is a diagram showing another example of the configuration of the long period in the second modification example regarding the shift period. [Figure 29] FIG. 10 is a diagram showing a configuration example of a long period in Modification 2 regarding the shift period, and is a diagram showing an example in which an adjustment period is not provided. [Figure 30] FIG. 10 is a diagram showing a configuration example of a long period in Modification 3 regarding a shift period. [Figure 31] FIG. 10 is a diagram showing a configuration example of a long period in Modification 4 regarding a shift period. [Figure 32] FIG. 13 is a diagram showing a configuration example of a long period in a fifth modified example regarding a shift period. [Figure 33] FIG. 13 is a diagram showing a configuration example of a long period in a sixth modification regarding a shift period. [Figure 34] FIG. 13 is a diagram showing an example of the configuration of a long period in a seventh modified example regarding a shift period. [Figure 35] FIG. 13 is a diagram showing another example of the configuration of the long period in the seventh modification regarding the shift period. [Figure 36] FIG. 13 is a diagram showing an example of the configuration of a long period in Modification 8 regarding the shift period. [Figure 37] FIG. 13 is a diagram showing a configuration example of a long period in a ninth modification example regarding a shift period. [Figure 38]FIG. 13 is a diagram showing another example of the configuration of the long period in the ninth modification example regarding the shift period. [Figure 39] FIG. 10 is a diagram illustrating an example of a selection mode of the length of the shift period. [Figure 40] FIG. 10 is a diagram illustrating an example of the configuration of a long period in which no shift period is provided. [Figure 41] FIG. 10 is a diagram illustrating another example of the configuration of a long period in which no shift period is provided. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, with reference to the accompanying drawings, embodiments according to the present technology will be described in the following order. <1. First embodiment> <1-1. Distance measurement system configuration> <1-2. Data transmission> <1-3. Processing flow> <2. Second embodiment> <3. Third Embodiment> <4. Variations of the staggered period> <4-1. Variation 1> <4-2. Variation 2> <4-3. Variation 3> <4-4. Variation 4> <4-5. Variation 5> <4-6. Variation 6> <4-7. Variation 7> <4-8. Variation 8> <4-9. Variation 9> <4-10. Variation 10> <5. Other Modifications> <6. Summary> <7. This Technology>

[0014] <1. First embodiment> <1-1. Distance measurement system configuration> An overview of the configuration of a ranging system 1 of the present technology will be described with reference to Fig. 1. Note that the configuration shown below is an application of the present technology to dToF (direct ToF), but the present technology is not limited to this and can also be applied to iToF (indirect ToF).

[0015] The distance measuring system 1 measures the distance between the subject OB, which is the object to be measured, and the subject OB by calculating the difference in time between when light is irradiated onto the subject OB and when the reflected light is received.

[0016] The distance measuring system 1 includes a time management unit 2, a light emitting unit 3, a light receiving unit 4, and a distance calculation unit 5.

[0017] The time management unit 2 determines the light emission timing of the light emitting elements provided in the light emitting unit 3 and issues a light emission instruction to the light emitting unit 3.

[0018] The light emitting unit 3 emits light in accordance with a light emitting timing signal supplied at a predetermined timing based on a light emitting instruction issued by the time management unit 2.

[0019] The light receiving section 4 receives light that is emitted from the light emitting section 3 and reflected by the object OB, and outputs a light receiving pulse signal to the time management section 2.

[0020] The time management unit 2 receives the pulsed light reception signal output from the light receiving unit 4 and outputs time information relating to the light emission timing and time information relating to the light reception timing to the distance calculation unit 5 .

[0021] The distance calculation unit 5 calculates distance data between the distance measurement system 1 and the object OB using time information relating to the light emission timing and time information relating to the light reception timing output from the time management unit 2.

[0022] In calculating distance data in distance calculation unit 5, intermediate data is used.

[0023] Here, the intermediate data will be described. In this embodiment, one unit distance data obtained as a result of a unit distance measurement that combines one light emission and one light reception is treated as intermediate data. Then, the distance calculation unit 5 calculates one distance data based on multiple unit distance data obtained as a result of multiple unit distance measurements.

[0024] Specifically, a histogram is generated using multiple intermediate data (unit distance data) obtained as a result of multiple unit distance measurements, and the distance calculation unit 5 outputs the distance with the highest frequency in the histogram as the distance data obtained in the series of distance measurements.

[0025] This prevents the output of erroneous distance data based on accidentally received light from another light source.

[0026] The histogram data may be treated as intermediate data.

[0027] An example of a specific configuration of the time management unit 2 is shown in FIG.

[0028] The time management unit 2 includes a counter unit 6, a delay amount instruction unit 7, and a light emission instruction unit 8.

[0029] The counter unit 6 is configured to have a counter that counts up from 0 to a predetermined number at predetermined time intervals, and outputs the counter value at the timing when the light receiving signal is received. The counter of the counter unit 6 can be reset to its initial value (0) by a reset command.

[0030] FIG. 3 shows an example of the relationship between the light emission timing signal, the intensity of light received by the light receiving unit 4, and the counter value of the counter managed by the counter unit 6. 3, after a time corresponding to the distance to the subject OB has elapsed since the pulsed light emission timing signal, reception of light based on the pulsed reflected light is detected by the light receiving unit 4. The difference time between these two can be determined by the increment of the counter value.

[0031] In this embodiment, a period that includes one of the above-mentioned unit distance measurements is defined as a "short period TS." During the short period TS, the light-emitting unit 3 can emit light once and the light-receiving unit 4 can receive light once. However, since there is no guarantee that the light emitted from the light-emitting unit 3 and reflected by the subject OB can be reliably detected by the light-receiving unit 4, there may be cases during the short period TS where only the light-emitting unit 3 emits light and the light-receiving unit 4 does not detect light reception.

[0032] The period during which unit distance measurement is performed is the period during which reflected light from the subject OB is detected, and can be considered as the "light-receiving period Ta." The short period TS may be composed of only the light-receiving period Ta (see FIG. 4), or may be composed of the light-receiving period Ta and the non-light-receiving period Tb (see FIG. 5).

[0033] It should be noted that there is a possibility that reflected light from the subject OB may be detected (received) even during the non-light-receiving period Tb depending on the configuration of the distance measurement system 1. In this case, for example, the light-receiving period Ta may be considered to be a period during which unit distance measurement is substantially valid, and the non-light-receiving period Tb may be considered to be a period during which unit distance measurement is substantially invalid.

[0034] The light-receiving period Ta is a period during which the reflected light is received so that the distance to the object OB located within the range of distance measurement can be measured. The length of the light-receiving period Ta is uniquely determined by the range of distance measurement.

[0035] The non-light-receiving period Tb is a time other than the light-receiving period Ta in the short period TS, and may be provided, for example, before light emission or after the light-receiving period Ta. An example in which the short period TS includes both the light-receiving period Ta and the non-light-receiving period Tb will be explained later.

[0036] 2 supplies a start signal indicating the start timing of the short period TS to the delay amount instruction unit 7 and the light emission instruction unit 8. The start timing of the short period TS may be the same as the light emission timing, or may be earlier than the light emission timing. In other words, the light emission timing may arrive a predetermined time after the start timing of the short period TS.

[0037] The counter section 6 resets the counter value to 0 in synchronization with the start timing of the short period TS.

[0038] The delay amount instruction unit 7 determines the amount of delay (first delay amount) from the start timing based on the start signal supplied from the counter unit 6, and instructs the light emission instruction unit 8 on the amount of delay. As a result, light is emitted at a timing delayed by the first delay amount from the start timing of the short period TS, as shown in Fig. 6. Of course, light may be emitted approximately simultaneously with the start timing of the short period TS by setting the first delay amount to "0".

[0039] Furthermore, the delay amount instruction unit 7 determines a delay amount (second delay amount) for delaying the start timing of the short period TS and supplies it to the counter unit 6. The second delay amount is inserted between some of the short periods TS. The period between the short periods defined by this delay amount is referred to as a "shift period Tsft."

[0040] The counter unit 6 resets the counter value to 0 in accordance with the length of the shift period Tsft (second delay amount) supplied from the delay amount specifying unit 7. The delay amount specifying unit 7 may issue a reset command to the counter unit 6 after waiting for the length of the shift period Tsft.

[0041] The counter value of the counter during standby for the shift period Tsft may continue to count up from the previous short period TS and may be reset at the end of standby for the shift period Tsft, i.e., at the start of the next short period TS, or may be reset once at the end of standby for the previous short period TS and then reset again at the end of standby for the shift period Tsft.

[0042] Furthermore, the shift period Tsft is not inserted between all short periods TS, but is inserted between multiple short periods TS. In other words, the shift period Tsft is a period provided to prevent short periods TS from continuing too closely together at regular intervals, and is a period provided to periodically shift the start timing of the short periods TS, as shown in FIG.

[0043] Furthermore, by providing the shift period Tsft periodically (or irregularly), it is possible to periodically (or irregularly) shift the timing of light emission that is performed for each short period TS. By periodically or irregularly shifting the light emission timing, it is possible to detect interference light, which will be described later.

[0044] A continuous section of short periods TS is referred to as a medium period TM. The medium period TM is a period that constitutes a long period TL, which will be described later. Specifically, the long period TL includes multiple medium periods TM. The long period TL is a frame period during which a final piece of distance data is calculated based on multiple unit distance data for the object OB.

[0045] However, the long period TL may be a subframe period (subframe period) or a line period (line period) that constitutes a frame period (frame period), or the long period TL may be a period that changes at least one of the physical irradiation position and irradiation range of the laser light from the light emitting element 11. In the following explanation, an example in which the long period TL is a frame period will be described.

[0046] The start period of the short period TS is delayed by the shift period Tsft for each medium period TM. Accordingly, the light emission instruction (light emission timing) is also delayed by the shift period Tsft for each medium period TM.

[0047] The light emission instruction unit 8 supplies the light emission unit 3 with a light emission instruction according to the start signal supplied from the counter unit 6 and the first delay amount supplied from the delay amount instruction unit 7 .

[0048] In this way, the time management unit 2 manages various delay amounts for the light emission timing, supplies light emission instructions to the light emission unit 3 according to the delay amount, and outputs a counter value according to the light reception signal to the distance calculation unit 5.

[0049] Each of the parts shown in Figure 2 that the time management unit 2 has may be provided as a part separate from the light-emitting unit 3 and the light-receiving unit 4 as shown in Figure 1 (for example, a standalone information processing device), but each of the parts that the time management unit 2 has may be configured to be included in either the light-emitting unit 3, the light-receiving unit 4, or the distance calculation unit 5.

[0050] In this embodiment, an example in which the various units constituting the time management unit 2 are included in the light receiving unit 4 will be described.

[0051] First, the specific configuration of the light emitting unit 3 is shown in FIG. The light emitting unit 3 is configured to include a light emitting timing signal generating unit 9, a driver 10, and a light emitting element 11.

[0052] The light emission timing signal generating unit 9 generates a pulsed light emission timing signal in response to a light emission instruction supplied from the light emission instruction unit 8, and supplies the generated signal to the driver 10. This light emission timing signal takes into account the first delay amount and the second delay amount (the length of the shift period Tsft).

[0053] The driver 10 drives the light emitting element 11 in accordance with the light emitting timing signal supplied by the light emitting timing signal generating unit 9. This causes the light emitting element 11 to output a pulsed laser beam.

[0054] The light emitting element 11 is configured to include a VCSEL (Vertical Cavity Surface Emitting Laser) or the like as a light source.

[0055] 8 shows an example of the configuration of the light receiving unit 4. The light receiving unit 4 is provided as part of a ToF (Time of Flight) sensor 12 that includes each part of the time management unit 2.

[0056] The ToF sensor 12 includes a light receiving unit 4, a counter unit 6, a delay amount instruction unit 7, a light emission instruction unit 8, an intermediate data calculation unit 13, and a communication unit .

[0057] The light receiving unit 4 is compatible with distance measurement using the ToF method, and is configured to have a SPAD (Single Photon Avalanche Diode) element as the light receiving element for each pixel.

[0058] The counter unit 6, the delay amount instruction unit 7, and the light emission instruction unit 8 will not be described again.

[0059] The intermediate data calculation unit 13 supplies one unit distance data obtained as a result of the unit distance measurement as intermediate data to the communication unit 14. Specifically, the intermediate data calculation unit 13 calculates the difference time between the light emission timing and the light reception timing using the counter value supplied from the counter unit 6 and the first delay amount supplied from the delay amount instruction unit 7, and calculates the distance to the subject OB by multiplying it by the speed of light. The intermediate data calculation unit 13 may be configured to output the difference between the counter values as intermediate data, and the distance calculation unit 5 at the subsequent stage may calculate the distance using the speed of light.

[0060] The communication unit 14 transmits the intermediate data to the distance calculation unit 5. When the intermediate data transmitted by the communication unit 14 is received by another device, the transmitted data may be protected using a protection function of the communication unit 14. Specifically, data for protecting the integrity of the transmitted data may be transmitted together.

[0061] 9 shows an example of the configuration of the distance calculation unit 5. The distance calculation unit 5 includes a communication unit 15, a histogram generation unit 16, an other light source detection unit 17, and a distance data calculation unit .

[0062] The communication unit 15 receives the intermediate data calculated by the intermediate data calculation unit 13 via the communication unit 14 .

[0063] The histogram generator 16 generates a histogram using the results of multiple unit distance measurements received over a predetermined period (e.g., the period of one frame). The histogram is generated as frequency information for each predetermined distance, such as 10 cm. However, the histogram may also be generated as frequency information for each differential time.

[0064] The histogram generated by the histogram generating unit 16 is supplied to the other light source detecting unit 17 .

[0065] The other light source detection unit 17 detects noise components due to light emission from other light sources based on the histogram supplied by the histogram generation unit 16 .

[0066] A specific description will be given with reference to FIGS. 10, 11 and 12. FIG.

[0067] An intermediate period TM consisting of four short periods TS is defined as a first intermediate period TM1, and an intermediate period TM consisting of four short periods TS following the first intermediate period TM1 with a shift period Tsft therebetween is defined as a second intermediate period TM2.

[0068] If we assume that the interfering light is emitted at the same cycle as the short period TS, two peaks are detected in the first medium period TM1. One of the two peaks is the detection of the reflected light from the object OB, and the other is the detection of the interfering light.

[0069] Similarly, two peaks are detected in the second intermediate period TM2. However, while the position of the peak detected by the reflected light from the subject OB is the same as (i.e., the same distance as) the position of the peak detected in the first intermediate period TM1, the position of the peak detected by the interference light is different from the position of the peak detected in the first intermediate period TM1.

[0070] This is because the interference light is emitted repeatedly at regular intervals without taking into account the shift period Tsft.

[0071] In addition, in order to make the peak position of the interfering light detected in the first intermediate period TM1 different from the peak position of the interfering light detected in the second intermediate period TM2, it is necessary that the length of the shift period Tsft is different from an integer multiple of the length of the short period TS.

[0072] Because the shift period Tsft satisfies this condition, when the difference between the histogram for the first intermediate period TM1 (the histogram on the left in Figure 10) and the histogram for the second intermediate period TM2 (the histogram on the right in Figure 10) is calculated, only the peak detected by the interfering light remains, as shown in Figure 11.

[0073] This makes it possible to identify false distance information about the object OB detected due to the interfering light.

[0074] The other light source detection unit 17 supplies false distance information as information about the other light source to the distance data calculation unit 18 at the subsequent stage together with histogram data.

[0075] The distance data calculation unit 18 calculates correct distance data for the object OB by using only appropriate data based on the false distance information supplied by the other light source detection unit 17 .

[0076] Specifically, when the histogram for the first intermediate period TM1 and the histogram for the second intermediate period TM2 are added together, three pieces of distance information are obtained as shown in Figure 12, as both the detection results of the interfering light and the detection results of the reflected light appear as peaks.

[0077] Of these three types of distance information, the distance data calculation unit 18 identifies the distance information that does not correspond to false distance information as normal distance data and outputs it.

[0078] <1-2. Data transmission> The intermediate data and distance data may be transmitted between different devices, and a predetermined standard such as MIPI (Mobile Industry Processor Interface) may be used for the data transmission.

[0079] For example, if the ToF sensor 12 shown in Fig. 8 and the distance calculation unit 5 shown in Fig. 9 are different devices, the ToF sensor 12 transmits intermediate data processed in accordance with a predetermined standard to the distance calculation unit 5. Furthermore, the distance calculation unit 5 transmits distance data processed in accordance with a predetermined standard to the other device.

[0080] Here, we will explain using MIPI-compliant communication as an example.

[0081] The intermediate data and distance data are transmitted for each long period TL, which includes multiple medium periods TM. Here, the long period TL will be described with reference to FIG.

[0082] First, the short period TS is a unit distance measurement in which light is emitted and received once, as described above. The medium period TM is a period in which the short periods TS are successively repeated.

[0083] The long period TL corresponds to one frame period for generating a distance image and is made up of multiple medium periods TM. The long period TL shown in Figure 13 includes a first medium period TM1 and a second medium period TM2.

[0084] The long period TL is composed of a start period ("Start" in the figure) in which start processing of the frame is performed, a first medium period TM1 and a second medium period TM2, a shift period Tsft set between the medium periods TM, an adjustment period Tadj, a processing period ("Processing" in the figure) in which storage processing to memory etc. is performed, and an end period ("End" in the figure) in which end processing of the frame is performed.

[0085] The time lengths of the long periods TL are unified, and the total time of the shift periods Tsft and the adjustment periods Tadj is the same for all long periods TL. That is, the total time of the shift periods Tsft1 and the adjustment periods Tadj1 and the total time of the shift periods Tsft2 and the adjustment periods Tadj2 are the same.

[0086] The time length of the adjustment period Tadj8 corresponding to the shift period Tsft8 is set to 0. That is, the time length of the shift period Tsft8 is set to the same length as the total time of the shift period Tsft1 and the adjustment period Tadj1.

[0087] In other words, the adjustment period Tadj is a period provided according to the length of the shift period Tsft so that each long period TL has the same time length. Depending on the length of the shift period Tsft, there may be cases where the adjustment period Tadj is not provided.

[0088] It should be noted that the length of each of the shifted periods Tsft1 to Tsft8 is different from an integral multiple of the length of the short period TS.

[0089] FIG. 14 shows an example of a packet structure in which intermediate data output for each long period TL is stored. Intermediate data for one frame period is stored and transmitted at a predetermined position in a series of MIPI data formats that start with a frame start ("FS" in the figure) and end with a frame end ("FE" in the figure).

[0090] Here, the MIPI-compliant packet data (hereinafter referred to as "MIPI data") may be transmitted during the processing period ("processing" in the figure) shown in Fig. 13, or may be transmitted while ranging for the next frame is being performed. Also, it may be transmitted over multiple frames.

[0091] MIPI data is transmitted as a series of multiple packets between the frame start and frame end. Each packet consists of a packet header ("PH" in the diagram), a payload area, and a packet footer ("PF" in the diagram), with various transmission data stored in the payload area.

[0092] Figure 14 shows an example in which embedded data ("Embedded Data" in the figure), first medium-term data, second medium-term data, and MAC (Message Authentication Code) values and CRC (Cyclic Redundancy Code) values as additional data for protecting the transmitted data by ensuring the confidentiality and integrity of the transmitted data are stored in the payload area.

[0093] Here, the first intermediate period data is one or more unit distance data measured in the first intermediate period TM1 in FIG. 13, and is used as intermediate data for calculating distance data.

[0094] The second intermediate period data is one or more unit distance data measured in the second intermediate period TM2 in FIG. 13, and is used as intermediate data for calculating distance data.

[0095] The first medium-term data and the second medium-term data may be stored successively as multiple unit distance data measured at the same pixel, or may be stored successively as unit distance data for multiple pixels measured in one short period TS.

[0096] Another example of MIPI data is shown in Figure 15. The example shown in FIG. 15 may utilize the packet structure of an extended packet. For example, the payload area of the packet data stores an extended packet header ("ePH" in the figure), transmission data (intermediate data), a MAC value, and a CRC value. As in the following examples, the transmission data may contain either a MAC value or a CRC value, or both.

[0097] The extended packet footer may be stored in the payload area of the packet data.

[0098] Here, some examples of modified examples of how intermediate data is stored in the payload area will be given.

[0099] The first modification is shown in Figure 16. In this modification, the first virtual channel CN1 and the second virtual channel CN2 in MIPI are used, and the first virtual channel CN1 transmits a packet having first medium-term data stored in its payload area, and the second virtual channel CN2 transmits a packet having second medium-term data stored in its payload area. Note that "FN" in the figure indicates the frame number.

[0100] That is, the first medium-term data and the second medium-term data are separated and transmitted separately.

[0101] The second modification is shown in FIG. In the second modification, the first medium-term data and the second medium-term data are mixed in the payload area of one packet and transmitted. Here, the manner in which the first medium period data and the second medium period data are mixed may be such that the first medium period data is stored consecutively along the pixel arrangement, and then the second medium period data corresponding to the first medium period data is stored (Figure 17), or the first medium period data and the second medium period data in a pixel are paired and the pair of data is stored consecutively, i.e., the first medium period data and the second medium period data alternate in one payload area.

[0102] When the long term TL includes three or more n medium term data TM, the payload area of one packet may include n medium term data.

[0103] A third modification is shown in FIG. In a third modification, packet data having other light source information stored in a payload area is transmitted. The other light source information may be stored in any manner, and for example, as shown in Fig. 18, a packet having first medium-term data stored in a payload area and a packet having second medium-term data stored in a payload area may be transmitted followed by a packet having other light source information stored in a payload area.

[0104] In addition, the first medium-term data, the second medium-term data, and the other light source information may be transmitted using multiple virtual channels CN as shown in Figure 16, or the first medium-term data, the second medium-term data, and the other light source information may be stored as a set in the payload area of one packet as shown in Figure 17.

[0105] The other light source information may be stored in a packet header, an extended packet header, or in an area other than the payload area, such as embedded data.

[0106] The first medium-term data and second medium-term data shown in each figure may be unit distance data calculated as intermediate data in each medium-term TM, or may be frequency data of a histogram as intermediate data generated based on the unit distance data calculated in the medium term TM.

[0107] The embedded data shown in each figure may include setting information for each device, register values according to the standard, vendor-specific register values, frame format descriptions, statistical values, and the like. The embedded data may also include information on distance data, image data, or user-defined data.

[0108] <1-3. Processing flow> An example of the processing executed by the ToF sensor 12 and the distance calculation unit 5 will be described.

[0109] 19 and 20 show an example of processing executed by the ToF sensor 12. FIG. In step S101, a processing unit such as a CPU (Central Processing Unit) in the ToF sensor 12 (hereinafter simply referred to as the "processing unit") determines whether or not to end the loop process. This determination process is a process for determining whether or not to end a series of loop processes related to distance measurement.

[0110] If it is determined that the loop processing is to be ended, the processing unit of the ToF sensor 12 ends the series of processing shown in FIG.

[0111] On the other hand, if it is determined that the loop processing should be continued, the processing unit of the ToF sensor 12 determines whether or not to start the long period TL in step S102. If it is determined that the timing to start the long period TL has not yet come, the processing unit of the ToF sensor 12 returns to the processing of step S101.

[0112] On the other hand, if it is determined that the start timing of the long period TL has arrived, the delay amount specifying unit 7 of the ToF sensor 12 determines the first delay amount and the second delay amount in step S103. The process of step S103 may be a process of selecting one of the first delay amount and the second delay amount for each long period TL that are determined in advance using random numbers (including pseudo-random numbers) or the like, or a process of determining each delay amount using random numbers or the like at the execution timing of step S103.

[0113] Alternatively, in the process of step S103, the delay amount may be determined for each long period TL according to a pattern table prepared in any of the light emitting unit 3, the ToF sensor 12, the distance calculation unit 5, or the like.

[0114] Only one such pattern table may be prepared, or multiple pattern tables may be prepared. If multiple pattern tables are prepared, a process of selecting one pattern table may be performed at the start of distance measurement.

[0115] After determining each delay amount, the processing unit of the ToF sensor 12 starts measurement for a first intermediate period TM1 in step S104.

[0116] An example of the processing executed by the processing unit of the ToF sensor 12 in the measurement during the medium period TM is shown in Fig. 20. Note that the processing shown in Fig. 20 is also executed in the measurement during the subsequent second medium period TM2, and is a generalized illustration of the processing executed during each medium period TM.

[0117] In the first intermediate period TM1, in step S201, the light emission instruction unit 8 of the ToF sensor 12 supplies a light emission instruction to the light emission unit 3 based on the start signal supplied by the counter unit 6 and the first delay amount supplied by the delay amount instruction unit 7, as described above.

[0118] Next, in step S202, the processing unit of the ToF sensor 12 resets the counter managed by the counter unit 6. Note that if a light emission instruction is to be given after a predetermined time has elapsed since the start of the short period TS, the counter is reset in step S202, and the light emission instruction in step S201 is issued after waiting for the predetermined time.

[0119] In step S203, the processing unit of the ToF sensor 12 determines whether or not a light reception signal has been detected by the light receiving unit 4. If it is determined that a light reception signal has not been detected, the processing unit of the ToF sensor 12 further determines whether or not the short period TS has elapsed in step S204. If it is determined that the short period TS has not elapsed, the processing unit of the ToF sensor 12 returns to the processing of step S203 again.

[0120] That is, the processing unit of the ToF sensor 12 repeats the determination processes of steps S203 and S204 until a light reception signal is detected or the short period TS has elapsed.

[0121] In addition, if the short period TS includes a light-receiving period Ta and a non-light-receiving period Tb, and at least a part of the non-light-receiving period Tb is provided after the light-receiving period Ta, it is determined in step S204 whether the light-receiving period Ta has elapsed.

[0122] If it is determined in step S203 that a received light signal has been detected, in step S205, the intermediate data calculation unit 13 of the ToF sensor 12 calculates unit distance data as intermediate data based on the first delay amount determined in step S103.

[0123] The processing unit of the ToF sensor 12 determines whether the medium period TM has ended in step S206. If it is determined that the medium period TM has ended, the processing unit of the ToF sensor 12 ends the series of processes shown in FIG.

[0124] If it is determined in step S204 that the short period TS has elapsed, i.e., if the reflected light is not received during the period in the short period TS when it should be received, the calculation of the unit distance data in step S205 is not performed and the process proceeds to the determination process in step S206.

[0125] Returning to the description of FIG. After completing the measurement in the first intermediate period TM1, in step S105, the processing unit of the ToF sensor 12 waits for a shift period based on the second delay amount determined in step S103.

[0126] After waiting for the shift period, the processing unit of the ToF sensor 12 starts measurement in the second intermediate period TM2 in step S106, thereby executing the series of processes shown in FIG.

[0127] After completing the measurement in the second intermediate period TM2, the processing unit of the ToF sensor 12 waits for the adjustment period Tadj in step S107.

[0128] In step S108, the processing unit of the ToF sensor 12 causes the communication unit 14 to execute processing for outputting the first medium-term data and the second medium-term data to the distance calculation unit 5.

[0129] Next, an example of processing executed by a processing unit such as a CPU in distance calculation unit 5 (hereinafter simply referred to as "processing unit") is shown in FIG. In step S301, the processing unit of the distance calculation unit 5 determines whether the communication unit 15 has received the first medium-term data and the second medium-term data from the ToF sensor 12.

[0130] If it is determined that the message has not been received, the process of step S301 is repeated.

[0131] If it is determined that the signal has been received, the histogram generating unit 16 of the distance calculation unit 5 generates a histogram in step S302.

[0132] Next, the other light source detection unit 17 of the distance calculation unit 5 performs other light source detection processing in step S303.

[0133] In step S304, the processing unit of the distance calculation unit 5 determines whether or not another light source has been detected. If it is determined that another light source has been detected, the processing unit of the distance calculation unit 5 excludes data of the other light source in step S305.

[0134] In step S306, the distance data calculation unit 18 of the distance calculation unit 5 calculates distance data from the first intermediate period data and the second intermediate period data. If it is determined in step S304 that no other light source is detected, the processing unit of distance calculation unit 5 does not execute the processing of step S305, but causes distance data calculation unit 18 to calculate distance data in step S306.

[0135] <2. Second embodiment> In the second embodiment, the ToF sensor 12 in the first embodiment is replaced with a ToF sensor 12A having the function of the distance calculation unit 5.

[0136] The distance measuring system 1A in this embodiment includes a light emitting unit 3A and a ToF sensor 12A.

[0137] The light-emitting unit 3A includes a part of the time management unit 2 shown in Fig. 2. Specifically, the light-emitting unit 3A includes a delay amount instruction unit 7, a light-emitting instruction unit 8, a light-emitting timing signal generation unit 9, a driver 10, and a light-emitting element 11, as shown in Fig. 22.

[0138] The light emitting unit 3A receives a start signal from the ToF sensor 12 that indicates the start timing of the short period TS, and the delay amount specifying unit 7 determines the first delay amount based on the start signal.

[0139] The light emission instruction unit 8 supplies a light emission instruction to the light emission timing signal generation unit 9 based on the start signal and the first delay amount determined by the delay amount instruction unit 7 . In response to this light emission instruction, the light emission timing signal generating unit 9, the driver 10 and the light emitting element 11 perform predetermined operations, thereby outputting pulsed laser light at a timing delayed by a predetermined first delay amount from the start timing of the short period TS.

[0140] The delay amount specifying unit 7 determines the length of the shift period as a second delay amount, and supplies the second delay amount together with the first delay amount to the ToF sensor 12A.

[0141] The ToF sensor 12A includes a counter unit 6 as part of the function of the time management unit 2. Specifically, as shown in FIG. 23, the ToF sensor 12A includes a light receiving unit 4, a counter unit 6, an intermediate data calculation unit 13, a histogram generation unit 16, an other light source detection unit 17, a distance data calculation unit 18, a communication unit 14, and a protection unit 19.

[0142] The counter unit 6 has a counter that is reset at the start timing of the short period TS, and supplies a start signal for notifying the start timing of the short period TS to the light-emitting unit 3 A. The counter unit 6 also resets the counter based on the second delay amount as the length of the shift period.

[0143] When the light receiving unit 4 detects the reception of light by the SPAD element, it outputs a light reception signal to the counter unit 6 .

[0144] The counter unit 6 outputs a counter value to the intermediate data calculation unit 13 in accordance with the timing at which the light reception signal is received.

[0145] The intermediate data calculation unit 13 calculates unit distance data as intermediate data based on the counter value supplied from the counter unit 6 and the first delay amount supplied from the light emitting unit 3A, and outputs the calculated data to the histogram generation unit 16.

[0146] The histogram generating unit 16 generates a histogram using the unit distance data received over one frame period.

[0147] The other light source detection unit 17 detects the presence of other light sources based on the histogram generated by the histogram generation unit 16 .

[0148] The distance data calculation unit 18 uses the false distance information supplied by the other light source detection unit 17 to calculate correct distance data for the object OB.

[0149] The protection unit 19 performs a process to protect the calculated distance data. For example, the protection unit 19 may perform various processes using a CRC value, a MAC value, or the like to ensure the integrity, or may perform a process to encrypt the distance data as transmission data using some encryption technology to ensure confidentiality.

[0150] The communication unit 14 transmits the distance data protected by the protection unit 19 to another device.

[0151] By performing each process up to the calculation of distance data within the ToF sensor 12A, privacy protection for the measurement data can be strongly ensured.

[0152] In addition, the distance calculation unit 5 in the first embodiment shown in Figure 9 may be provided with a protection unit 19 that applies the above-mentioned protection function when transmitting the distance data calculated by the distance data calculation unit 18 to an external device.

[0153] <3. Third Embodiment> The distance measuring system 1B in the third embodiment includes a light emitting unit 3B, a ToF sensor 12B, and a distance calculation unit 5. The ToF sensor 12B determines the light emission timing by receiving light that is emitted from the light emitting unit 3B and reflected inside the distance measuring system 1B. For this purpose, the distance measuring system 1B further includes a reflecting unit 20.

[0154] Specifically, the light emitting section 3B includes a counter section 6, a delay amount indicating section 7, a light emitting timing signal generating section 9, a driver 10, and a light emitting element 11, as shown in FIG.

[0155] We will avoid redundant explanations of the configuration of each part. Unlike the second embodiment, the light emitting unit 3B does not include the light emission instruction unit 8, but instead includes a ToF sensor 12B.

[0156] Specifically, the ToF sensor 12B includes a first light receiving unit 4a, a second light receiving unit 4b, a light emission instruction unit 8, an intermediate data calculation unit 13, and a communication unit .

[0157] In the ToF sensor 12B, the same configuration as in other embodiments will not be described again.

[0158] The first light receiving section 4a performs the same function as the light receiving sections 4 and 4A in the first and second embodiments, and receives the first reflected light that is light emitted from the light emitting section 3B and reflected by the subject OB.

[0159] The second light receiving unit 4b is a configuration unique to the third embodiment, and determines the light emission timing of the light emitting unit 3B by receiving the second reflected light that is formed when light emitted from the light emitting unit 3B is reflected by the reflecting unit 20 provided inside the distance measuring system 1B.

[0160] The light reception signal from the second light receiving unit 4b is supplied to the light emission instruction unit 8. This light reception signal corresponds to the start signal in the other embodiments described above. That is, the light emission instruction unit 8 generates a light emission instruction for the next light emission based on the reception signal supplied from the second light receiving unit 4b and the first delay amount determined by the delay amount instruction unit 7, and supplies the light emission instruction to the light emission timing signal generation unit 9.

[0161] The configuration of the distance calculation unit 5 is the same as that of the other embodiments described above, so a duplicated description will be avoided.

[0162] The first light receiving section 4a and the second light receiving section 4b may be provided as a light receiving section having a single pixel array section. For example, the first reflected light may be received in the effective pixel region of the pixel array section, and the second reflected light may be received in the peripheral region of the effective pixel region of the pixel array section. This reduces the number of components and contributes to cost reduction.

[0163] The reflector 20 may be provided in the light-emitting unit 3B, in the ToF sensor 12B, or in a part of the distance measurement system 1B other than the light-emitting unit 3B and the ToF sensor 12B.

[0164] <4. Variations of the staggered period> The shift periods Tsft1 to Tsft8 for each long period TL shown in Fig. 13 are shown as an embodiment in which the shift period Tsft becomes longer for each long period TL. Several modified examples of other embodiments will be described below. Note that, in the following modified examples, an example will be described in which eight types of shift periods Tsft1 to Tsft8 are prepared, as in Fig. 13, but any number of types of shift periods may be prepared as long as there is more than one type.

[0165] <4-1. Variation 1> The shift periods Tsft1 to Tsft8 may be rearranged at random and applied to the first to eighth long periods TL in order.

[0166] Alternatively, one of the eight staggered periods Tsft1 to Tsft8 may be selected randomly for each major period TL. That is, the staggered period Tsft1 may be selected multiple times or consecutively within the eight major periods TL.

[0167] In order to unify the lengths of the long periods TL, the adjustment period Tadj in the long periods TL in which the shifted period Tsft1 is selected is set to the adjustment period Tadj1, and the adjustment period Tadj in the long periods TL in which the shifted period Tsft2 is selected is set to the adjustment period Tadj2.

[0168] <4-2. Variation 2> A plurality of shift periods Tsft may be provided in one long period TL. For example, in the example of FIG. 25, four medium periods TM are included in the long period TL, and three staggered periods Tsft are provided between the medium periods TM.

[0169] The lengths of the multiple shift periods Tsft included in one long period TL are uniform.

[0170] Specifically, first, shift periods Tsft1 to Tsft8 are prepared. The shift period Tsft1 is the shortest period, and the shift period Tsft8 is the longest period.

[0171] A shift period Tsft1 is provided between the first medium period TM1, the second medium period TM2, the third medium period TM3, and the fourth medium period TM4 in the first long period TL1. A shift period Tsft2 is provided between each of the intermediate periods TM in the second major period TL2. A staggered period Tsft3 is provided between each intermediate period TM in the third long period TL3. A staggered period Tsft8 is provided between each of the intermediate periods TM in the eighth major period TL8.

[0172] After the fourth intermediate period TM4 of each of the first long period TL1, the second long period TL2, and the third long period TL3, an adjustment period Tadj having a different length is provided to unify the time lengths of the long periods TL.

[0173] Note that the multiple shift periods Tsft included in the long period TL may be combined so as to shorten the length of the long period TL.

[0174] For example, as shown in FIG. 26, the shortest shift period Tsft1 and the longest shift period Tsft8 may be combined and included in the first major period TL1, and the second shortest shift period Tsft2 and the second longest shift period Tsft7 may be combined and included in the second major period TL2.

[0175] In this way, by skillfully combining the long shift period Tsft and the short shift period Tsft, the length of the long period TL can be shortened. Moreover, instead of shortening the length of the long period TL, the processing period ("processing" in the figure) can be lengthened, which increases the amount of calculation that can be performed and makes it possible to use a calculation unit with low processing power.

[0176] Alternatively, four shift periods Tsft may be prepared to be included in each long period TL, and three selected shift periods Tsft may be placed between each medium period TM, while an adjustment period Tadj having the same length as the unselected shift periods Tsft may be placed after the fourth medium period TM4.

[0177] This makes it possible to unify the length of the long period TL.

[0178] The selection of the three shift periods Tsft may be different for each long period TL (see FIG. 27), or may be the same for each long period TL (see FIG. 28).

[0179] 29, three types of shift periods Tsft (shift periods Tsft1, Tsft2, and Tsft3) may be prepared and arranged so as not to overlap with the medium periods TM. This eliminates the need to provide an adjustment period Tadj, and allows the length of the long periods TL to be shortened.

[0180] <4-3. Variation 3> The short period TS may be configured to include a light-receiving period Ta and a non-light-receiving period Tb, and the non-light-receiving period Tb may further include a pre-delay period TwA provided before the light-receiving period Ta and a post-delay period TwB provided after the light-receiving period Ta.

[0181] A specific description will be given with reference to Fig. 30. In Fig. 30 and the subsequent figures, the front delay period TwA and the rear delay period TwB are shown as hatched areas in different directions.

[0182] As shown in the figure, the first long period TL1 is divided into multiple short periods TS, and each short period TS is composed of a light receiving period Ta, a pre-delay period TwA1 that is provided before the light receiving period Ta, and a post-delay period TwB1 that is provided after the light receiving period Ta.

[0183] In the first long period TL1, a staggered period Tsft1 is provided between the first medium period TM1 and the second medium period TM2, and an adjustment period Tadj1 corresponding to the length of the staggered period Tsft1 is provided after the second medium period TM2.

[0184] The short period TS of the second long period TL2 is composed of a pre-delay period TwA2, a light-receiving period Ta, and a post-delay period TwB2. The pre-delay period TwA2 is longer than the pre-delay period TwA1, and the post-delay period TwB2 is shorter than the post-delay period TwB1.

[0185] The total length of the front delay period TwA2 and the rear delay period TwB2 is set equal to the total length of the front delay period TwA1 and the rear delay period TwB1. Therefore, the length of the short period TS is set equal in the first long period TL1 and the second long period TL2.

[0186] In the second long period TL2, a staggered period Tsft2 is provided between the first medium period TM1 and the second medium period TM2, and an adjustment period Tadj2 corresponding to the length of the staggered period Tsft2 is provided after the second medium period TM2. The total length of the shift period Tsft2 and the adjustment period Tadj2 is set to be the same as the total length of the shift period Tsft1 and the adjustment period Tadj1.

[0187] Similarly, for the third major period TL3 to the eighth major period TL8, the length of the pre-delay period TwA gradually increases, the length of the post-delay period TwB gradually decreases, the length of the shift period Tsft gradually increases, and the length of the adjustment period Tadj gradually decreases.

[0188] In addition, the adjustment period Tadj is not provided in the eighth major period TL8. In other words, the adjustment period Tadj is provided after the second intermediate period TM2 in the first major period TL1 to the seventh major period TL7 according to the length of the eighth major period TL8 in which the adjustment period Tadj is not provided.

[0189] In FIG. 30, the length of the pre-delay period TwA included in the large period TL gradually increases from the first large period TL1 to the eighth large period TL8, and the length of the post-delay period TwB gradually decreases from the first large period TL1 to the eighth large period TL8. However, the length of the pre-delay period TwA and the post-delay period TwB for each long period TL may be set to be random.

[0190] <4-4. Variation 4> The short period TS may be configured to include a light-receiving period Ta and a non-light-receiving period Tb, and further the non-light-receiving period Tb may include only the front delay period TwA without including the rear delay period TwB.

[0191] 31, the short period TS is composed of only a pre-delay period TwA and a light-receiving period Ta as a non-light-receiving period Tb, and the ratio of the lengths of the pre-delay period TwA and the light-receiving period Ta is constant within the same long period TL. Furthermore, when comparing different long periods TL, the length of the pre-delay period TwA relative to the light-receiving period Ta differs.

[0192] The adjustment period Tadj that follows the second intermediate period TM2 is provided according to the lengths of the pre-delay period TwA and the shift period Tsft in order to unify the lengths of the long periods TL.

[0193] The length of the short period TS can be shortened by the amount that the short period TS does not include the post-delay period TwB, so that one long period TL can include many short periods TS and medium periods TM.

[0194] In the example shown in Figure 31, the length of the pre-delay period TwA gradually increases with each long period TL, but the length of the pre-delay period TwA may also be set to gradually decrease, or may be set to a random length.

[0195] 31, the length of the pre-delay period TwA2 in the second major period TL2 may be made longer than the pre-delay period TwA1 in the first major period TL1, and the length of the shift period Tsft2 in the second major period TL2 may be made shorter than the shift period Tsft1 in the first major period TL1. That is, the increase in the length of the pre-delay period TwA2 may be absorbed by shortening the shift period Tsft2. This makes it possible to shorten or eliminate the adjustment period Tadj, thereby shortening the length of the major period TL and shortening the frame period.

[0196] <4-5. Variation 5> The short period TS may be configured to include a light-receiving period Ta and a non-light-receiving period Tb, and further the non-light-receiving period Tb may be configured to include the post-delay period TwB but not the pre-delay period TwA.

[0197] Specifically, as shown in FIG. 32, the short period TS consists of only a light-receiving period Ta and a post-delay period TwB, and within the same long period TL, the ratio of the lengths of the post-delay period TwB and the light-receiving period Ta is constant. Furthermore, when comparing different long periods TL, the length of the post-delay period TwB relative to the light-receiving period Ta is different.

[0198] The adjustment period Tadj that follows the second intermediate period TM2 is provided as needed based on the lengths of the post-delay period TwB and the shift period Tsft in order to unify the lengths of the long periods TL.

[0199] The length of the short period TS can be shortened by the amount that the short period TS does not include the pre-delay period TwA, and therefore many medium periods TM can be included in one long period TL.

[0200] In the example shown in FIG. 32, the length of the post-delay period TwB gradually becomes shorter for each long period TL, but the length of the post-delay period TwB may be set to gradually become longer, or may be set to a random length.

[0201] 32, by gradually shortening the post-delay period TwB and gradually lengthening the shift period Tsft, it is possible to unify or approximate the total lengths of the post-delay period TwB and the shift period Tsft. This makes it possible to shorten or eliminate the adjustment period Tadj, shorten the length of the long period TL, and shorten the frame period.

[0202] <4-6. Variation 6> In a configuration in which the short period TS includes a light-receiving period Ta and a non-light-receiving period Tb, and the non-light-receiving period Tb does not include a post-delay period TwB but includes only a pre-delay period TwA, the length of the pre-delay period TwA may be different for each medium period TM included in the long period TL.

[0203] That is, even within the same long period TL, the length of the pre-delay period TwA may be different.

[0204] Specifically, as shown in FIG. 33, in the first long period TL1, the short period TS in the first intermediate period TM1 includes the pre-delay period TwA1, and in the second intermediate period TM2, the short period TS includes the pre-delay period TwA4.

[0205] Similarly, in the first intermediate period TM1 of the second long period TL2, the short period TS includes the pre-delay period TwA3, and in the second intermediate period TM2, the short period TS includes the pre-delay period TwA2.

[0206] The adjustment period Tadj that follows the second intermediate period TM2 is provided according to the lengths of the pre-delay period TwA and the shift period Tsft in order to unify the lengths of the long periods TL.

[0207] The combination of the pre-delay period TwA provided in the first intermediate period TM1 included in the long period TL and the pre-delay period TwA provided in the second intermediate period TM2 may be selected in consideration of the length of the shift period Tsft.

[0208] For example, when the length of the shift period Tsft is long, the length of the pre-delay period TwA provided in each intermediate period TM may be shortened.

[0209] Furthermore, by unifying the total length of the pre-delay period TwA provided in the first intermediate period TM1, the pre-delay period TwA provided in the second intermediate period TM2, and the shift period Tsft for each long period TL, it becomes possible to shorten or eliminate the adjustment period Tadj after the second intermediate period TM2, thereby shortening the length of the long period TL.

[0210] In this modified example, an example has been described in which only the pre-delay period TwA is provided as the non-light-receiving period Tb in the short period TS, but this may also be applied to a case in which only the post-delay period TwB is provided as the non-light-receiving period Tb in the short period TS. Specifically, the length of the post-delay period TwB provided in the first intermediate period TM1 may be different from the length of the post-delay period TwB provided in the second intermediate period TM2.

[0211] <4-7. Variation 7> The length of the shift period Tsft provided for each long period TL does not have to be different for each long period TL, but may be the same.

[0212] 34, the long period TL includes two medium periods TM, and a shift period Tsft is provided between the two medium periods TM. In addition, an adjustment period Tadj is provided after the second medium period TM2.

[0213] The shift period Tsft provided between the medium periods TM has the same length in all the long periods. The pre-delay period TwA included in the short period TS in the first medium period TM1 and the pre-delay period TwA included in the short period TS in the second medium period TM2 are of different lengths, and the lengths of the pre-delay periods TwA also differ when comparing the long periods TL.

[0214] When comparing the long periods TL, the difference in the length of the pre-delay period TwA is absorbed by the adjustment period Tadj, so that the length of each long period TL is unified.

[0215] By unifying the shift period Tsft, it becomes unnecessary to select a shift period Tsft from among a plurality of shift periods Tsft, thereby reducing the processing load.

[0216] In FIG. 34, only the pre-delay period TwA is provided as the non-light-receiving period Tb of the short period TS, but it is also possible to unify the shift period Tsft even when only the post-delay period TwB is provided as the non-light-receiving period Tb of the short period TS.

[0217] 34, the total length of the first medium period TM1 and the second medium period TM2 may be the same for each long period TL. That is, if the short period TS included in the first medium period TM1 is long, the short period TS included in the second medium period TM2 may be shortened accordingly. This eliminates the need to provide an adjustment period Tadj for unifying the lengths of the long periods TL.

[0218] Furthermore, when the long period TL includes four medium periods TM, four different lengths of pre-delay periods TwA may be prepared, and the timing of the light-receiving period Ta may be shifted for each long period TL by changing the combination of the four medium periods TM and the four types of pre-delay periods.

[0219] Specifically, this is shown in Figure 35. In the first medium period TM1 of the first long period TL1, a pre-delay period TwA1 is used, in the second medium period TM2, a pre-delay period TwA2 is used, in the third medium period TM3, a pre-delay period TwA3 is used, and in the fourth medium period TM4, a pre-delay period TwA4 is used.

[0220] That is, the first long period TL1 includes the same number (one each) of pre-delay periods TwA1, TwA2, TwA3, and TwA4.

[0221] Also, in the second major period TL2, the third major period TL3, and the fourth major period TL4, the pre-delay periods TwA1, TwA2, TwA3, and TwA4 are selected so that the same number (one) of pre-delay periods are included, although in a different order.

[0222] As a result, the total length of the first medium period TM1, the second medium period TM2, the third medium period TM3, and the fourth medium period TM4 is the same for all of the long periods TL. In addition, since the lengths of the shift periods Tsft provided between the medium periods TM are also the same, it is possible to unify the lengths of the long periods TL without providing an adjustment period Tadj after the fourth medium period TM4.

[0223] This makes it possible to shorten the length of the long period TL and shorten the frame period.

[0224] Although FIG. 35 shows an example in which the combination of the medium period TM and the previous delay period TwA is changed randomly for each long period TL, the combination may be fixed. That is, measurements may be performed in the second long period TL2 and the third long period TL3 in the same manner as the first long period TL1 shown in FIG.

[0225] In addition, in Figure 35, pre-delay periods TwA1, TwA2, TwA3, and TwA4 are prepared and rearranged in random order before going through a cycle, but the long period TL may include more medium periods TM (for example, 8 or 12), and the four types of pre-delay periods TwA may be selected repeatedly multiple times within the long period TL.

[0226] When the pre-delay period TwA is selected randomly for the first intermediate period TM1 and the second intermediate period TM2, pre-delay periods TwA of the same length may be selected for the first intermediate period TM1 and the second intermediate period TM2 in one long period TL. In such a case, by providing a shift period Tsft between the intermediate periods TM, it is possible to avoid erroneous ranging due to the detection of another light source.

[0227] <4-8. Variation 8> In each of the above-mentioned examples, although the lengths may differ, the configuration of the short periods TS is the same regardless of the medium periods TM. For example, in the example shown in Fig. 30, each of the short periods TS includes both a pre-delay period TwA and a post-delay period TwB in addition to the light-receiving period Ta, and in the example shown in Fig. 31, each of the short periods TS includes only a pre-delay period TwA in addition to the light-receiving period Ta.

[0228] In this modification, some of the short periods TS are composed of only the light-receiving periods Ta. For example, Fig. 36 is a modified example of the example shown in Fig. 35. In the example shown in Fig. 35, one is selected for each medium period TM from four types of pre-delay periods TwA1, TwA2, TwA3, and TwA4, but in the example shown in Fig. 36, one is selected for each medium period TM from four options: three types of pre-delay periods TwA1, TwA2, and TwA3, and no pre-delay period TwA (this can also be considered as providing a pre-delay period TwA of length 0).

[0229] When the pre-delay period TwA and the post-delay period TwB are changed for each intermediate period TM, one of the options may be to provide an option of not providing a delay period.

[0230] This allows the length of the long period TL to be shortened.

[0231] <4-9. Variation 9> Unlike the above-described examples, the shift period Tsft may include a processing period ("processing" in each drawing) and a first non-processing period TnpA and a second non-processing period TnpB provided before and after the processing period. In other words, storage processing in memory may be performed as appropriate during the processing period of the shift period Tsft provided for each intermediate period TM.

[0232] This is specifically shown in Figure 37.

[0233] The first long period TL1 is provided with four intermediate periods TM, and a stagger period Tsft is provided between each of the intermediate periods TM.

[0234] The shift period Tsft includes a processing period ("processing" in the drawing), a first non-processing period TnpA, and a second non-processing period TnpB.

[0235] Furthermore, a second non-processing period TnpB is provided before the first intermediate period TM1, so that the start timing of the first short period TS for each long period TL varies.

[0236] Considering a medium period TM consisting of four short periods TS as a base, a second non-processing period TnpB is provided before the medium period TM, and a first non-processing period TnpA is provided after the medium period TM.

[0237] A total of eight types of first non-processing periods TnpA are available, including an option with a length of 0 (i.e., an option where the first non-processing period TnpA is not provided), and periods ranging from a short first non-processing period TnpA1 to a long first non-processing period TnpA7. Correspondingly, a total of eight types of second non-processing periods TnpB are available, ranging from a long second non-processing period TnpB7 to a short second non-processing period TnpB1, and an option with a length of 0 (i.e., an option where the second non-processing period TnpB is not provided).

[0238] The second non-processing period TnpB and the first non-processing period TnpA before and after the medium period TM are selected so that their total lengths are the same, thereby unifying the length of the long period TL without providing an adjustment period Tadj after the fourth medium period TM4.

[0239] Furthermore, since the total length of the medium period TM and the second non-processing period TnpB and first non-processing period TnpA before and after it is the same, the position of the processing period relative to the start period ("Start" in the figure) is kept constant for each long period TL. Therefore, the example shown in FIG. 37 can be said to be an example in which the processing executed during the processing period within the long period TL is executed periodically.

[0240] Depending on the selection manner of the first non-processing period TnpA and the second non-processing period TnpB, the shifted period Tsft may be configured to include only a processing period, such as the shifted period Tsft provided between the second medium period TM2 and the third medium period TM3 of the fourth long period TL4 shown in Figure 37.

[0241] In this modification, the selection process for the second non-processing period TnpB or the first non-processing period TnpA can be executed in the processing period provided after the medium period TM. Therefore, since it is not necessary to execute the process for selecting the length of each period in advance before the start of the long period TL, the long period TL can be started promptly.

[0242] In addition, four types of first non-processing periods TnpA and four types of second non-processing periods TnpB (which may include one with a length of 0) may be prepared, and the order may be selected in advance so that each appears once in one long period TL in no particular order.

[0243] Furthermore, the configuration of the first long period TL1 obtained by randomly arranging the first non-processing period TnpA and the second non-processing period TnpB before and after the intermediate period TM may be repeated in the second long period TL2 and the third long period TL3. For example, in the second long period TL2, the third long period TL3, and the fourth long period TL4 shown in FIG. 37, measurements may be repeated at the same timing as in the first long period TL1.

[0244] In this case, the process of randomly arranging the first non-processing periods TnpA and the second non-processing periods TnpB only needs to be performed once, thereby reducing the processing load.

[0245] In addition, Figure 37 shows an example in which all shift periods Tsft can include both the first non-processing period TnpA and the second non-processing period TnpB, but the shift period Tsft may be configured so as not to include either the first non-processing period TnpA or the second non-processing period TnpB.

[0246] For example, FIG. 38 shows an example in which each shift period Tsft is configured without including the second non-processing period TnpB. In this case, the position of the processing period relative to the start period is not constant. Therefore, the example shown in Fig. 38 can be said to be an example in which the processing executed in the processing period within the long period TL is executed non-periodically.

[0247] <4-10. Variation 10> There are various possible ways to select the length of the shift period Tsft, an example of which is shown in FIG.

[0248] First, a plurality of shift periods Tsft with different lengths are prepared (16 types in FIG. 39), and shift period IDs (Identifications) are assigned to the periods in order from shortest to longest.

[0249] Then, an initial value for the shift period ID is selected ("7" in FIG. 39), and shift period IDs are selected in order by repeatedly incrementing and decrementing from the initial value. At this time, the fluctuation range is increased each time increment and decrement are repeated.

[0250] Specifically, after selecting "7" as the staggered period ID, add 1 to select "8" as the staggered period ID, then subtract 2 to select "6" as the staggered period ID, then add 3 to select "9" as the staggered period ID.

[0251] If the calculated shift period ID is out of range (other than 0 to 15), the shift period ID is reset to the initial value. At this time, as shown in FIG. 39, the initial value of the shift period ID may be changed (for example, from "7" to "8").

[0252] By selecting the shift period ID in such a manner that it is difficult to guess, it is possible to make it difficult to guess the light emission timing in the short period TS, and it is possible to make it difficult for interference light to interfere with distance measurement.

[0253] Although the manner of selecting the length of the shift period Tsft has been described here, a similar manner may be applied to the lengths of various other periods, such as the length of the front delay period TwA and the length of the rear delay period TwB.

[0254] <5. Other Modifications> Before transmitting the first medium-term data and the second medium-term data, other light source detection may be performed in the ToF sensor 12. Then, the ToF sensor 12 may store information about the result of the other light source detection in the payload area of the MIPI data. The information about the result of other light source detection may be, for example, one-bit data indicating whether or not another light source has been detected, or one-bit data indicating whether or not the light reception data for the other light source has been excluded. Alternatively, the information indicating the influence of receiving light from other light sources on the calculation of distance measurement data may be expressed, for example, in two bits. The two-bit data may, for example, be "00" indicating that the influence of other light sources is small, "01" indicating that the influence of other light sources is medium, "10" indicating that the influence of other light sources is large, or "11" indicating that the presence or absence of other light sources cannot be determined. Note that these are merely examples, and information indicating the influence of other light sources in other ways may also be stored in the payload area of the MIPI data.

[0255] In the above examples, a shift period Tsft is provided between medium periods TM, but a front delay period TwA or a rear delay period TwB included in the short period TS may be used instead of providing a shift period Tsft.

[0256] A specific description will be given with reference to FIGS.

[0257] 40, the long period TL is made up of two medium periods TM, one medium period TM is made up of four short periods TS, and one short period TS is made up of a pre-delay period TwA and a light-receiving period Ta.

[0258] Eight different lengths of pre-delay periods TwA (one of which has a length of 0) are prepared, and after being rearranged in a predetermined order using random numbers or pseudo-random numbers, etc., they are combined with the eight short periods TS included in the long period TL.

[0259] As a result, a pre-delay period TwA is provided between the last light-receiving period Ta in the first intermediate period TM1 and the first light-receiving period Ta in the second intermediate period TM2, making it possible to shift the light-emitting timing in the first intermediate period TM1 and the light-emitting timing in the second intermediate period TM2.

[0260] This makes it possible to make it difficult for interference light to interfere with distance measurement even in the configuration shown in FIG. As shown in FIG. 40, by changing the order of the eight types of pre-delay periods TwA for each long period TL, it is possible to further increase the difficulty of jamming.

[0261] The example shown in FIG. 41 is the same as the example shown in FIG. 40 except that a post-delay period TwB is used instead of the pre-delay period TwA. This configuration can also provide the same effect.

[0262] 40, multiple types of pre-delay periods TwA are provided, but only one type of pre-delay period TwA of a predetermined length may be provided. Then, it may be selected whether or not to include the pre-delay period TwA in each short period TS. As a result, the medium period TM is formed by irregularly mixing short periods TS consisting of only the light-receiving period Ta and short periods TS consisting of the pre-delay period TwA and the light-receiving period Ta. This configuration also makes it difficult for interference light to interfere with distance measurement. Such a configuration can also be applied to the post-delay period TwB in FIG.

[0263] The above-described examples are not limited to the ToF sensor 12 (12A, 12B) as a single chip in which a pixel array section and a processing section such as a CPU are stacked, but can also be applied to a light receiving device in which the pixel array section and the CPU are provided as separate chips.

[0264] <6. Summary> As explained in each of the above examples, the ToF sensor 12 (12A, 12B) as a light receiving device includes a light receiving unit 4 that receives light reflected from the subject OB when light is emitted from the light emitting unit 3 (3A, 3B) in response to a light emission instruction issued based on a predetermined processing cycle (the cycle of the start timing of the short period TS), and a calculation unit (intermediate data calculation unit 13) that calculates information regarding the distance to the subject OB (unit distance data as intermediate data, data regarding a histogram, or distance data) based on the difference between the light emission timing of the light emitting unit 3 and the light reception timing of the light receiving unit 4, and the light emission instruction is issued at a timing delayed by a shift period Tsft from the reference timing in each cycle of the predetermined processing cycle for each medium period TM that includes multiple short periods TS that include one emission of light, and the ratio of the shift period Tsft to the short period TS is changed for each long period TL that includes multiple medium periods TM. By providing a shift period Tsft for each medium period TM in which short periods TS are consecutive, the periodicity of the light emission timing can be broken, which prevents erroneous calculation of distance data (or distance-related information) due to interference or crosstalk from other light sources, etc. It also makes it difficult for interference light to interfere with distance measurement.

[0265] As explained with reference to Figure 14 etc., the ToF sensor 12 (12A, 12B) as a light receiving device may be provided with a transmitting unit (communication unit 14) that transmits information regarding distance (unit distance data as intermediate data, data regarding histograms, or distance data) and information for ensuring the completeness of at least a portion of the information regarding distance. The information for protecting the integrity is, for example, a CRC value or a MAC value. By transmitting such information together with information about distance, it is possible to appropriately calculate distance data and perform processing using the distance data at a later stage based on correct information.

[0266] As explained with reference to Figure 14 etc., in the ToF sensor 12 (12A, 12B) as a light receiving device, the long period TL includes a first medium period TM1 and a second medium period TM2 as medium periods TM, information regarding distance is intermediate data for calculating distance data, and a transmitting unit (communication unit 14) may be provided that transmits first data (e.g., packet data shown in Figure 14 etc.) in which the intermediate data of the first medium period TM1 is stored in a payload area, and second data (e.g., packet data shown in Figure 14 etc.) in which the intermediate data of the second medium period TM2 is stored in a payload area. The first data and the second data are transmitted by the transmitting unit to a device such as distance calculation unit 5, so that the device can calculate appropriate distance data taking other light sources into consideration.

[0267] As explained with reference to Figure 17 etc., in the ToF sensor 12 (12A, 12B) as a light receiving device, the long period TL includes a first medium period TM1 and a second medium period TM2 as medium periods TM, information regarding distance is intermediate data for calculating distance data, and a transmitting unit (communication unit 14) may be provided that transmits data stored in the payload area as both the intermediate data of the first medium period TM1 and the intermediate data of the second medium period TM2. To obtain information about other light sources, both intermediate data for the first medium period TM1 and intermediate data for the second medium period TM2 are required. According to this configuration, the first medium period data and the second medium period data are stored as a set in a single data and transmitted, which facilitates the process of detecting noise due to other light sources in the subsequent stage and reduces the processing load.

[0268] As explained with reference to Figure 25 etc., in the ToF sensor 12 (12A, 12B) as a light receiving device, the long period TL includes three or more medium periods TM and two or more shift periods Tsft respectively provided between the medium periods TM, and the multiple shift periods Tsft included in one long period TL may be of the same time length. This eliminates the need to perform the process of selecting the shift period Tsft for each medium period TM within the long period TL multiple times, thereby reducing the processing load.

[0269] As explained with reference to Figures 26, 27, 28, 29, 37, 38, etc., in the ToF sensor 12 (12A, 12B) as a light receiving device, the long period TL includes three or more medium periods TM and two or more shift periods Tsft respectively provided between the medium periods TM, and at least some of the multiple shift periods Tsft included in one long period TL may be of different time lengths. This allows the light emission timing to be appropriately shifted for each medium period TM, making it possible to more reliably detect other light sources.

[0270] As described with reference to FIG. 29 etc., in the ToF sensor 12 (12A, 12B) as the light receiving device, the total sum of the shift periods Tsft included in the long periods TL may be set to the same time length for each long period TL. This makes it possible to unify the lengths of the long periods TL without providing the adjustment period Tadj. By not providing the adjustment period Tadj, it becomes possible to shorten the length of the long periods TL or to lengthen the processing period.

[0271] As explained with reference to each of Figures 31 to 36, in the ToF sensor 12 (12A, 12B) as a light receiving device, the short period TS consists of a light receiving period Ta and a non-light receiving period Tb provided at least either before or after the light receiving period Ta, and the time length of the short period TS may be changed for each long period TL. This allows the cycle of the short periods TS in the medium periods TM to be made different for each long period TL, making it difficult for interference light to interfere with distance measurement.

[0272] As explained with reference to each of Figures 33 to 36, in the ToF sensor 12 (12A, 12B) as a light receiving device, the time lengths of multiple short periods TS included in the same long period TL may be at least partially different. For example, the short period TS may be changed for each medium period TM within the long period TL. This makes it difficult for interference light to interfere with distance measurement because the cycle of the short period TS changes for each medium period.

[0273] As explained with reference to Figures 35 and 36, in the ToF sensor 12 (12A, 12B) as a light receiving device, the sum of the time lengths of multiple short periods TS included in a long period TL may be the same time length for each long period TL. As shown in Figure 13, it is possible to unify the total length of the short periods TS within the long period TL, not only when the short period TS is composed only of the light-receiving period Ta, but also when the short period TS includes a pre-delay period TwA (see Figure 35) or a post-delay period TwB (see Figure 36) as a non-light-receiving period Tb. This makes it difficult to interfere with distance measurement by changing the cycle of the short period TS for each medium period, and also makes it possible to shorten the length of the long period TL, etc. Therefore, it is possible to make the frame rate high frequency.

[0274] As explained with reference to each of Figures 31 to 36, in the ToF sensor 12 (12A, 12B) as a light receiving device, the short period TS consists of a light receiving period Ta and a non-light receiving period Tb provided at least either before or after the light receiving period Ta, and the time length of the non-light receiving period Tb may be changed for each long period TL. This allows the light emission timing of the light emitting unit 3 to be shifted for each long period TL, making it difficult to interfere with distance measurement.

[0275] As explained with reference to each of Figures 33 to 36, in the ToF sensor 12 (12A, 12B) as a light receiving device, the time lengths of multiple non-light receiving periods Tb included in the same long period TL may be at least partially different. This makes it possible to make the light emission timing of the light-emitting unit 3 non-periodic even within the long period TL, thereby increasing the resistance to interference caused by interference light.

[0276] As explained with reference to Figures 30, 35, and 36, in the ToF sensor 12 (12A, 12B) as a light receiving device, the sum of the time lengths of the multiple non-light receiving periods Tb included in the long period TL may be the same time length for each long period TL.

[0277] As shown in Figure 37, in the ToF sensor 12 (12A, 12B) as a light receiving device, the long period TL includes a processing period (period marked "processing" in each figure) in which processing different from the processing performed in the short period TS is performed, the processing period is set periodically, and the shift period Tsft may include at least a portion of the processing period. By periodically executing the processes, it is possible to prevent the processes from concentrating.

[0278] As explained with reference to Figure 2 etc., the ToF sensor 12 (12A, 12B) as a light receiving device may be provided with a delay amount instruction unit 7 that reflects the time length of the shift period Tsft as a delay amount (second delay amount) in the light emission instruction. Specifically, the delay amount specifying unit 7 determines the length of the shift period Tsft to determine the amount by which the short period TS is delayed (second delay amount), and supplies the delay amount to the counter unit 6 to reset the counter value to "0." When reset, the counter unit 6 supplies the start timing of the short period TS to the light emission instruction unit 8. This allows the delay amount instruction unit 7 to indirectly delay the light emission instruction in accordance with the length of the shift period Tsft. Therefore, the light emission timing can be made difficult to discern from the outside, and resistance to interference light can be improved.

[0279] As explained with reference to each figure, the ToF sensor 12A as an information processing device, or the distance calculation unit 5, is equipped with an other light source detection unit 17 that detects noise based on the light emission of other light sources other than the light emitting unit 3 based on information about the distance to the subject OB (unit distance data as intermediate data, data on a histogram, or distance data) calculated according to the difference between the emission timing of light emitted from the light emitting unit 3 (3A, 3B) in response to an emission instruction issued based on a predetermined processing cycle and the reception timing at which the reflected light reflected by the subject OB is received, and the light emission instruction is issued at a timing delayed by a shift period Tsft from the reference timing in each cycle of the predetermined processing cycle for each medium period TM that includes multiple short periods TS that include one emission of light, and the ratio of the shift period Tsft to the short period TS may be changed for each long period TL that includes multiple medium periods TM. By providing a shift period Tsft for each medium period TM in which short periods TS are consecutive, the periodicity of the light emission timing can be broken, which prevents erroneous calculation of distance data (or distance-related information) due to interference or crosstalk from other light sources, etc. It also makes it difficult for interference light to interfere with distance measurement.

[0280] As explained with reference to each of Figures 10 to 12, in the ToF sensor 12A as an information processing device or the distance calculation unit 5, the long period TL includes the first medium period TM1 and the second medium period TM2 as the medium periods TM, information regarding distance is intermediate data for calculating distance data, and the other light source detection unit 17 may detect noise based on the comparison result between the intermediate data of the first medium period TM1 and the intermediate data of the second medium period TM2. Specifically, this can be realized by generating respective histograms based on the unit distance data acquired in the first intermediate period TM1 and the unit distance data acquired in the second intermediate period TM2. This allows other light sources to be suitably detected.

[0281] As described with reference to FIG. 23 and the like, the ToF sensor 12A as an information processing device may include a transmitter (communication unit 14) that transmits information related to other light sources. For example, by generating information about other light sources in the ToF sensor 12A,

[0282] As explained in each of the above examples, the distance measuring system 1 (1A, 1B) as a distance measuring device comprises a light receiving unit 4 that receives light reflected from the subject OB when light is emitted from the light emitting unit 3 (3A, 3B) in response to a light emitting instruction issued based on a predetermined processing cycle, and a calculation unit (intermediate data calculation unit 13) that calculates distance data to the subject OB based on the difference between the light emitting timing of the light emitting unit 3 and the light receiving timing of the light receiving unit 4, and the light emitting instruction is issued at a timing delayed by a shift period Tsft from the reference timing in each cycle of the predetermined processing cycle for each medium period TM that includes multiple short periods TS that include one light emission, and the ratio of the shift period Tsft to the short period TS is changed for each long period TL that includes multiple medium periods TM. By providing a shift period Tsft for each medium period TM in which short periods TS are consecutive, the periodicity of the light emission timing can be broken, which prevents erroneous calculation of distance data (or distance-related information) due to interference or crosstalk from other light sources, etc. It also makes it difficult for interference light to interfere with distance measurement.

[0283] The information processing method in the embodiment is an information processing method in which, for light emitted from a light-emitting unit 3 (3A, 3B) in response to a light-emitting instruction issued based on a predetermined processing cycle, distance data to the subject OB is calculated based on the difference between the light-receiving timing by a light-receiving unit 4 that receives reflected light of the light reflected by the subject OB and the light-emitting timing of the light-emitting unit 3, the light-emitting instruction is set to a timing that is delayed by a shift period Tsft from the reference timing in each cycle of the predetermined processing cycle for each medium period TM that includes multiple short periods TS that include one emission of light, and the information processing method executes a process in which the information processing device changes the ratio of the shift period Tsft to the short period TS for each long period TL that includes multiple medium periods TM. This information processing method can also provide the same effects and advantages as the ToF sensors 12 (12A, 12B) of the above-described embodiments and modifications.

[0284] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0285] Furthermore, the above-described examples may be combined in any manner, and even when various combinations are used, the various effects described above can be obtained.

[0286] The timing and position of each element shown in the block diagrams and flowcharts in the drawings are merely examples, and at least some of the elements may be configured differently. Various modifications exist for the embodiments described in each example. That is, some of the components of each example described may be the same, some may be integrated, some may be integrated, or some may be separate. Furthermore, some of the components of each example described may be omitted, some or all may be changed, or some or all may be modified. Furthermore, some of the components of each example described may be replaced with other components, or other components may be added to some or all of the components. Furthermore, some or all of the components of each example described may be divided into multiple components, or some or all of the components may be separated into multiple components, or at least some of the divided or separated components may have different functions and / or features.

[0287] Furthermore, different embodiments may be implemented by moving at least some of the components. Furthermore, different embodiments may be implemented by adding at least one of a coupling element and a relay element to a combination of at least some of the components. Furthermore, different embodiments may be implemented by adding at least one of a switching function and a selection function to a combination of at least some of the components.

[0288] In this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order according to the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing that is executed in parallel or individually (for example, parallel processing or processing by objects).

[0289] The program may be processed by a single computer (processor), or may be distributed among multiple computers. Furthermore, the program may be transferred to and executed on a remote computer.

[0290] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0291] Also, for example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0292] Furthermore, for example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by multiple devices via a network.

[0293] Furthermore, for example, the above-described program can be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0294] Also, for example, each step described in the above flowchart can be executed by one device or can be shared and executed by multiple devices. Furthermore, if one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0295] In addition, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0296] It should be noted that the present technologies described in this specification can be implemented independently and singly, unless a contradiction arises. Of course, any two or more of the present technologies can also be implemented in combination. For example, part or all of the present technologies described in any embodiment can be implemented in combination with part or all of the present technologies described in other embodiments. Furthermore, part or all of any of the present technologies described above can also be implemented in combination with other technologies not described above.

[0297] <7. This Technology> The present technology can also be configured as follows. (1) a light receiving unit that receives light that is emitted from the light emitting unit in response to a light emission instruction issued based on a predetermined processing cycle and reflected by a subject; a calculation unit that calculates information about the distance to the subject based on a difference between a light emission timing of the light emitter and a light reception timing of the light receiver, the light emission instruction is issued at a timing delayed by a shift period with respect to a reference timing in each cycle of the predetermined processing cycle, for each medium period including a plurality of short periods each including one light emission; The ratio of the shift period to the small period is changed for each large period including a plurality of the medium periods. Light receiving device. (2) a transmitter for transmitting information relating to the distance and information for ensuring the integrity of at least a part of the information relating to the distance; The light receiving device according to (1) above. (3) The long period includes a first medium period and a second medium period as the medium period, The information about the distance is used as intermediate data for calculating distance data, a transmitter for transmitting first data in which the intermediate data of the first medium period is stored in a payload area and second data in which the intermediate data of the second medium period is stored in a payload area; The light receiving device according to any one of (1) and (2) above. (4) The long period includes a first medium period and a second medium period as the medium period, The information about the distance is used as intermediate data for calculating distance data, a transmitter for transmitting data in which both the intermediate data of the first intermediate period and the intermediate data of the second intermediate period are stored in a payload area; The light receiving device according to any one of (1) and (2) above. (5) the long period includes three or more of the medium periods and two or more of the staggered periods provided between the medium periods, The plurality of shift periods included in one long period are set to have the same time length. A light receiving device according to any one of (1) to (4) above. (6) the long period includes three or more of the medium periods and two or more of the staggered periods provided between the medium periods, At least some of the plurality of shift periods included in one of the large periods have different time lengths. A light receiving device according to any one of (1) to (4) above. (7) The total of the shift periods included in the large period is set to the same time length for each large period. The light receiving device according to (6) above. (8) the short period is composed of a light-receiving period and a non-light-receiving period provided at least either before or after the light-receiving period, The time length of the small period is changed for each large period. A light receiving device according to any one of (1) to (7) above. (9) The time lengths of the plurality of sub-periods included in the same major period are at least partially different. The light receiving device according to (8) above. (10) The sum of the lengths of the plurality of small periods included in the large period is set to the same length for each large period. The light receiving device according to (9) above. (11) the short period is composed of a light-receiving period and a non-light-receiving period provided at least either before or after the light-receiving period, The time length of the non-light-receiving period is changed for each of the long periods. A light receiving device according to any one of (1) to (10) above. (12) The time lengths of the plurality of non-light-receiving periods included in the same long period are at least partially different. The light receiving device according to (11) above. (13) The total time length of the plurality of non-light-receiving periods included in the long period is set to the same time length for each long period. The light receiving device according to (12) above. (14) the long period includes a processing period in which a process different from the process executed in the short period is executed, The processing period is periodically set, The shift period includes at least a portion of the processing period. A light receiving device according to any one of (1) to (13) above. (15) a delay amount instruction unit that reflects the time length of the shift period as a delay amount in the light emission instruction; A light receiving device according to any one of (1) to (14) above. (16) an other light source detection unit that detects noise due to light emitted from a light source other than the light emitting unit based on information about a distance to the subject calculated according to a difference between a light emission timing of light emitted from the light emitting unit in response to a light emission instruction issued based on a predetermined processing cycle and a light reception timing of light reflected from the light emitting unit by the subject, the light emission instruction is issued at a timing delayed by a shift period with respect to a reference timing in each cycle of the predetermined processing cycle, for each medium period including a plurality of short periods each including one light emission; The ratio of the shift period to the small period is changed for each large period including a plurality of the medium periods. Information processing device. (17) The long period includes a first medium period and a second medium period as the medium period, The information about the distance is used as intermediate data for calculating distance data, The other light source detection unit detects the noise based on a comparison result between the intermediate data of the first medium period and the intermediate data of the second medium period. The information processing device according to (16) above. (18) a transmitter for transmitting information about the other light source; The information processing device according to any one of (16) to (17) above. (19) a light receiving unit that receives light that is emitted from a light emitting unit in response to a light emission instruction issued based on a predetermined processing cycle and reflected from a subject, and a calculation unit that calculates distance data to the subject in accordance with a difference between a light emission timing of the light emitting unit and a light reception timing of the light receiving unit, the light emission instruction is issued at a timing delayed by a shift period with respect to a reference timing in each cycle of the predetermined processing cycle, for each medium period including a plurality of short periods each including one light emission; The ratio of the shift period to the small period is changed for each large period including a plurality of the medium periods. Ranging device. (20) Regarding light emitted from a light emitting unit in response to a light emission instruction issued based on a predetermined processing cycle, information regarding the distance to the subject is calculated based on the difference between the timing at which the light is received by a light receiving unit that receives reflected light of the light reflected by a subject and the timing at which the light is emitted by the light emitting unit, the light emission instruction is issued at a timing delayed by a shift period with respect to a reference timing in each cycle of the predetermined processing cycle, for each medium period including a plurality of short periods each including one light emission; The ratio of the shift period to the short period is changed for each long period including a plurality of the medium periods. Information processing methods. [Explanation of symbols]

[0298] 1, 1A, 1B Ranging System (Ranging Device) 3, 3A, 3B light emitting part 4 Light receiving section 7 Delay amount indicator 13 Intermediate data calculation unit (calculation unit) 14 Communication unit (transmitter) 17 Other light source detection unit OB subject TS short period TM Medium Term TM1 1st Medium Term TM2 Second Medium-Term TM3 3rd interim period TM4 Fourth Medium-Term Plan TL Long Period TL1 First period TL2 Second period TL3 3rd major period TL8 8th period Tsft staggered period Ta Light receiving period Tb Non-photosensitive period

Claims

1. a light receiving unit that receives light that is emitted from the light emitting unit in response to a light emission instruction issued based on a predetermined processing cycle and reflected by a subject; a calculation unit that calculates information about the distance to the subject based on a difference between a light emission timing of the light emitter and a light reception timing of the light receiver, the light emission instruction is issued at a timing delayed by a shift period with respect to a reference timing in each cycle of the predetermined processing cycle, for each medium period including a plurality of short periods each including one light emission; The ratio of the shift period to the small period is changed for each large period including a plurality of the medium periods. Light receiving device.

2. a transmitter for transmitting information relating to the distance and information for ensuring the integrity of at least a part of the information relating to the distance; The light receiving device according to claim 1 .

3. The long period includes a first medium period and a second medium period as the medium period, The information about the distance is used as intermediate data for calculating distance data, a transmitter for transmitting first data in which the intermediate data of the first medium period is stored in a payload area and second data in which the intermediate data of the second medium period is stored in a payload area; The light receiving device according to claim 1 .

4. the long period includes a first medium period and a second medium period as the medium period, The information about the distance is used as intermediate data for calculating distance data, a transmitter for transmitting data in which both the intermediate data of the first intermediate period and the intermediate data of the second intermediate period are stored in a payload area; The light receiving device according to claim 1 .

5. the long period includes three or more of the medium periods and two or more of the staggered periods provided between the medium periods, The plurality of shift periods included in one long period are set to have the same time length. The light receiving device according to claim 1 .

6. the long period includes three or more of the medium periods and two or more of the staggered periods provided between the medium periods, At least some of the plurality of shift periods included in one of the large periods have different time lengths. The light receiving device according to claim 1 .

7. The total of the shift periods included in the large period is set to the same time length for each large period. The light receiving device according to claim 6 .

8. the short period is composed of a light-receiving period and a non-light-receiving period provided at least either before or after the light-receiving period, The time length of the small period is changed for each large period. The light receiving device according to claim 1 .

9. The time lengths of the plurality of sub-periods included in the same major period are at least partially different. The light receiving device according to claim 8 .

10. The sum of the lengths of the plurality of small periods included in the large period is set to the same length for each large period. The light receiving device according to claim 9 .

11. the short period is composed of a light-receiving period and a non-light-receiving period provided at least either before or after the light-receiving period, The time length of the non-light-receiving period is changed for each of the long periods. The light receiving device according to claim 1 .

12. The time lengths of the plurality of non-light-receiving periods included in the same long period are at least partially different. The light receiving device according to claim 11 .

13. The total time length of the plurality of non-light-receiving periods included in the long period is set to the same time length for each long period. The light receiving device according to claim 12.

14. the long period includes a processing period in which a process different from the process executed in the short period is executed, The processing period is periodically set, The shift period includes at least a portion of the processing period. The light receiving device according to claim 1 .

15. a delay amount instruction unit that reflects the time length of the shift period as a delay amount in the light emission instruction; The light receiving device according to claim 1 .

16. an other light source detection unit that detects noise due to light emitted from a light source other than the light emitting unit based on information about a distance to the subject calculated according to a difference between a light emission timing of light emitted from the light emitting unit in response to a light emission instruction issued based on a predetermined processing cycle and a light reception timing of light reflected from the light emitting unit by the subject, the light emission instruction is issued at a timing delayed by a shift period with respect to a reference timing in each cycle of the predetermined processing cycle, for each medium period including a plurality of short periods each including one light emission; The ratio of the shift period to the small period is changed for each large period including a plurality of the medium periods. Information processing device.

17. The long period includes a first medium period and a second medium period as the medium period, The information about the distance is used as intermediate data for calculating distance data, The other light source detection unit detects the noise based on a comparison result between the intermediate data of the first medium period and the intermediate data of the second medium period. The information processing device according to claim 16.

18. a transmitter for transmitting information about the other light source; The information processing device according to claim 16.

19. a light receiving unit that receives light that is emitted from a light emitting unit in response to a light emission instruction issued based on a predetermined processing cycle and reflected from a subject, and a calculation unit that calculates distance data to the subject in accordance with a difference between a light emission timing of the light emitting unit and a light reception timing of the light receiving unit, the light emission instruction is issued at a timing delayed by a shift period with respect to a reference timing in each cycle of the predetermined processing cycle, for each medium period including a plurality of short periods each including one light emission; The ratio of the shift period to the small period is changed for each large period including a plurality of the medium periods. Ranging device.

20. Regarding light emitted from a light emitting unit in response to a light emission instruction issued based on a predetermined processing cycle, information regarding the distance to the subject is calculated based on the difference between the timing at which the light is received by a light receiving unit that receives reflected light of the light reflected by a subject and the timing at which the light is emitted by the light emitting unit, the light emission instruction is issued at a timing delayed by a shift period with respect to a reference timing in each cycle of the predetermined processing cycle, for each medium period including a plurality of short periods each including one light emission; The ratio of the shift period to the short period is changed for each long period including a plurality of the medium periods. An information processing method executed by an information processing device.

Images