ToF SYSTEM, CONTROL METHOD THEREOF, PROGRAM, AND ToF APPARATUS

Abstract

To make a reflected light proper by controlling an amount of light applied to an object according to reflected light received from the object.SOLUTION: A ToF system is configured to apply light to an object (14) and find distance information by receiving reflected light from the object. The ToF system includes a light source portion (light emission portion 4) configured to apply the light (Li) to the object; a light receiver (6) configured to receive the reflected light (Lr) from the object; an information processing unit (8) configured to acquire signal amount information from a received light output of the light receiver and calculate light amount information for making the signal amount information arise at a predetermined value or within a predetermined range of a lower limit threshold (LimL) to an upper limit threshold (LimU); and a light amount control part (10) configured to control a light amount of the light source portion using the light amount information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a ToF (Time of Flight) system used, for example, for measuring the distance to an object, and relates to light amount control according to received light information of reflected light from the object. [Background technology]

[0002] The ToF system irradiates light onto an object, receives the reflected light from the object with a light-receiving unit, and calculates distance information to the object from the signal amount information obtained at the light-receiving unit. For this reason, it is known that the accuracy of distance information is affected by factors such as the amount of light emitted by the light-emitting means that irradiates the object with light.

[0003] It is known that the amount of light emitted by the light-emitting means is adjusted by selecting an optimal pattern from a combination of distance information based on the gesture of the target object and the amount of light (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-179997 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the light receiving output of the light receiving unit that receives the reflected light from the target object may become saturated or have a low S / N ratio depending on the amount of reflected light received, and appropriate signal amount information may not be obtained from the light receiving unit. A saturated state occurs when the amount of reflected light received is greater than necessary, and a low S / N state occurs when the noise (N) level is high and the ratio of the signal (S) representing the amount of received light to the noise (N) is small. In either case, an error occurs in the distance information, and the error rate of the distance information increases.

[0006] To avoid such a phenomenon, it is necessary to adjust the intensity of the reflected light received by the light receiving unit. The intensity of the reflected light needs to take into account parameters such as the distance to the target object and the reflectance of the target object. However, these parameters are governed by the installation environment of the ToF system, and are difficult to control on the system.

[0007] In view of such problems, the inventors of the present disclosure have found that since the reflected light from an object depends on the amount of light received by the object, optimizing the amount of light is effective in improving the accuracy of distance information. Patent Document 1 does not disclose or suggest such problems, and the disclosed configuration cannot solve such problems.

[0008] In view of the above problems and findings, an object of the present disclosure is to optimize reflected light by controlling the amount of light irradiated to an object in accordance with reflected light received from the object. [Means for solving the problem]

[0009] In order to achieve the above-mentioned object, according to the ToF system disclosed herein, the ToF system irradiates light onto an object and receives reflected light from the object to obtain distance information, and includes a light source unit that irradiates light onto the object, a light receiving unit that receives the reflected light from the object, an information processing unit that obtains signal amount information from the received light output of the light receiving unit and calculates light amount information to generate this signal amount information at a predetermined value, or within a predetermined range between a lower threshold value and an upper threshold value, and a light amount control unit that uses the light amount information to control the light amount of the light source unit.

[0010] In this ToF system, the information processing unit may be configured to generate the light amount information by comparing an average value of all or a part of the signal amount information with the lower limit threshold or the upper limit threshold.

[0011] In this ToF system, the information processing unit may be configured to select a specific target area or target frame from pixels representing the light receiving output of the light receiving unit, and calculate the average value of the signal amount information.

[0012] In this ToF system, the light source unit may include at least one light-emitting element or a plurality of light-emitting elements, and the light intensity control unit may be configured to control the amount of current flowing to the light-emitting elements or control the number of light-emitting elements that are caused to emit light.

[0013] In order to achieve the above object, according to the control method for a ToF system disclosed herein, there is provided a control method for a ToF system that irradiates an object with light and receives reflected light from the object to obtain distance information, the control method including a step of a light source unit emitting light and irradiating the object with light, a step of a light receiving unit receiving the reflected light from the object, a step of an information processing unit acquiring signal amount information from the received light output of the light receiving unit and calculating light amount information for generating this signal amount information to be a predetermined value or within a predetermined range between a lower threshold value and an upper threshold value, and a step of a light amount control unit controlling the light amount of the light source unit using the light amount information.

[0014] In order to achieve the above-mentioned object, according to the program disclosed herein, the program is executed by a computer, and causes the computer to execute a function of irradiating light from a light source unit onto an object, a function of acquiring signal amount information from the light receiving output of a light receiving unit that receives the reflected light from the object, and a function of calculating light amount information to generate this signal amount information at a predetermined value or within a predetermined range between a lower threshold value and an upper threshold value, and a function of controlling the light amount of the light source unit using the light amount information.

[0015] In order to achieve the above object, according to the ToF device disclosed herein, the ToF device irradiates light onto an object and receives reflected light from the object to obtain distance information, and includes a light source unit that irradiates light onto the object, a light receiving unit that receives the reflected light from the object, an information processing unit that obtains signal amount information from the received light output of the light receiving unit and calculates light amount information to generate this signal amount information at a predetermined value, or within a predetermined range between a lower threshold value and an upper threshold value, and a light amount control unit that uses the light amount information to control the light amount of the light source unit. Effect of the Invention

[0016] According to the present disclosure, any of the following effects can be obtained. (1) Light is irradiated onto an object from an emitting unit, the light reflected from the object is received by a receiving unit, and signal amount information of the reflected light is obtained from the received light output obtained by the receiving unit. The amount of light irradiated onto the object can be controlled so that this signal amount information is obtained above a lower threshold and below an upper threshold.

[0017] (2) The light amount of the light emitting unit is controlled so that the signal amount information of the reflected light obtained by the light receiving unit is above a lower threshold and below an upper threshold. This makes it possible to optimize the signal amount information of the reflected light, and prevents the amount of light received by the light receiving unit from becoming saturated or falling into a low S / N state due to insufficient received light. This improves the accuracy of the distance information calculated from the signal amount and also improves the reliability of the distance information.

[0018] (3) When calculating distance information of an object, the information to be processed is optimized, thereby reducing the load of information processing. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram illustrating a ToF system according to a first embodiment. [Diagram 2] FIG. 2A is a diagram showing a light receiving output of a light receiving section according to the second embodiment, and FIG. 2B is a diagram showing a pixel array representing the light receiving output. [Diagram 3] FIG. 3A is a diagram showing a pixel array representing received light output, and FIG. 3B is a diagram showing calculation of an average signal amount. [Figure 4] FIG. 4A is a diagram showing a pixel array and a target area for calculating an average value, and FIG. 4B is a diagram showing a target area for calculating an average value. [Diagram 5] FIG. 5 is a flowchart showing a processing procedure of the ToF system. [Figure 6] FIG. 6 is a flowchart illustrating a processing procedure of the ToF system according to the third embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a light receiving unit according to the embodiment. [Figure 8] FIG. 8 is a flowchart showing the procedure of the light receiving process. [Figure 9] FIG. 9 is a diagram showing the operation timing of the light receiving process. [Figure 10] FIG. 10A shows the case where the target object is in close range, FIG. 10B shows the light emission period of the light emitting unit, FIG. 10C shows the period of close-range light reception, and FIG. 10D shows the period of long-range light reception by the ToF system from the target object. [Figure 11] FIG. 11A shows the case where the target object is at a long distance, FIG. 11B shows the light emission period of the light emitting unit, FIG. 11C shows the period of close-range light reception, and FIG. 11D shows the period of long-range light reception by the ToF system from the target object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] First Embodiment Fig. 1 shows a ToF system 2 according to a first embodiment of the present disclosure. The configuration shown in Fig. 1 is an example, and the present disclosure is not limited to such a configuration.

[0021] The ToF system 2 shown in FIG. 1 includes a light emitting unit 4, a light receiving unit 6, an information processing unit 8 (hereinafter referred to as the "processing unit 8"), a light amount control unit 10, and a calculation unit 12. Therefore, according to this ToF system 2, signal amount information is acquired from the light receiving output of the light receiving unit 6, and the light amount of the light emitting unit 4 is controlled so that the signal amount information is in a range of a predetermined lower limit threshold or more and an upper limit threshold or less, thereby optimizing the distance information obtained by the calculation unit 12. The light amount of the light emitting unit 4 to be controlled may be any of the light emission amount, the diffusion amount, and the light amount (light projection amount) projected from the light emitting unit 4 to the object 14. In addition, this ToF system 2 is applied to a ToF camera module, a ToF camera, a distance measuring system, etc. that express contrast or coloring information representing distance information in captured image information.

[0022] The light emitting unit 4 is an example of a light source unit that emits light Li to be irradiated to the object 14. The light emitting unit 4 is configured to include at least one light emitting element or a plurality of light emitting elements, emits light under the control of the light quantity control unit 10, and irradiates (or projects) the light Li to the object 14. For example, an LED (Light Emitting Diode) is used as the light emitting element. The object 14 is an object for acquiring reflected light Lr to acquire distance information, and may be any object that generates reflected light Lr, such as a human being or an object.

[0023] The light receiving unit 6 receives the reflected light Lr from the object 14 and generates a light receiving output. This light receiving output is an electrical signal that indicates the amount of reflected light Lr received. For example, the light receiving unit 6 may be a light receiving means such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a light receiving element that converts the received light into an electrical signal.

[0024] The processing unit 8 is, for example, configured as a computer and equipped with a processor, a storage unit, etc., and receives the received light output from the light receiving unit 6 to obtain signal amount information of the reflected light Lr, and performs information processing as a control function for optimizing the light emission amount of the light emitting unit 4. This information processing includes a process of calculating light amount information for generating signal amount information at a predetermined value, or within a predetermined range between a lower threshold value and an upper threshold value.

[0025] The light amount information calculated by the processing unit 8 through this information processing is fed back to the light amount control unit 10 and is used for the light amount control of the light emitting unit 4 by the light amount control unit 10 .

[0026] The light amount control unit 10 controls the light amount of the light-emitting unit 4 based on the light amount information fed back from the processing unit 8. The light amount of the light-emitting unit 4 may be the amount of light projected onto the target object 14. The light amount control unit 10 may use a computer constituting the processing unit 8.

[0027] Then, the calculation unit 12 acquires the signal amount information from the light receiving unit 6. As described above, this signal amount information is based on the light amount optimized by the light amount control of the light emitting unit 4 and on the reflected light Lr received by the light receiving unit 6. Therefore, the calculation unit 12 uses the optimized signal amount information to calculate distance information indicating the distance to the target object 14 and outputs this distance information. The calculation unit 12 may use the above-mentioned computer constituting the processing unit 8. In this case, the information processing unit 8, the light amount control unit 10 and the calculation unit 12 may be constituted by using a plurality of computers.

[0028] <Advantages of the First Embodiment> According to the first embodiment, any one of the following effects can be obtained. (1) When light Li is irradiated from the light-emitting unit 4 to the object 14 and the reflected light Lr from the object 14 is received by the light-receiving unit 6, a light-receiving output having a bit depth of [n] bits is obtained from the light-receiving unit 6. Therefore, in this ToF system, distance information of the object 14 can be calculated using the light-receiving output of the light-receiving unit 6.

[0029] (2) By setting a predetermined value or a predetermined range of the upper threshold LimU and lower threshold LimL for the signal amount information, the signal amount information can be controlled within a range of LimL or more and LimU or less. Therefore, according to this ToF system 2, the reflected light Lr received by the light receiving unit 6 occurs such that the signal amount information obtained from the light receiving output of the light receiving unit 6 is in a range of a predetermined lower threshold or more and an upper threshold or less, and the light receiving output of the light receiving unit 6 is prevented from being in a saturated state or a low S / N state, so that highly accurate distance information can be obtained.

[0030] (3) According to this ToF system 2, the amount of light from the light-emitting unit 4 can be controlled using the light receiving output according to the light receiving sensitivity of the light receiving unit 6, and the influence of the light receiving sensitivity of the light receiving unit 6 on the distance information, which is the calculation result of the calculation unit 12, can be reduced.

[0031] (4) When calculating distance information of an object, the information to be processed is optimized, thereby reducing the load of information processing.

[0032] Second Embodiment In this second embodiment, an average value of all or part of the signal amount information acquired from the light receiving output of the light receiving unit 6 is calculated, and this average value is used to control the light amount of the light emitting unit 4. In other words, the calculation of the average value of the signal amount information may use pixels in the entire area of ​​the signal amount information, or pixels in an arbitrary area may be extracted and used. Therefore, this second embodiment includes a method of calculating an average value using all pixels of the signal amount information, a method of calculating an average value by extracting pixels in an arbitrary area that is a part of the signal amount information, and a method of controlling on a frame-by-frame basis.

[0033] <Any area> If the arbitrary area is expressed as an image, which is a collection of pixel data, it is signal amount information including all pixel data, and the number of pixels constituting the arbitrary area may be 1 (minimum) to n (maximum). For example, if the (X, Y) coordinates are one or more pieces of signal amount information including pixel data in a certain range, the selection parameter may be "coordinates", and the range and the number of ranges may be n (=any number). Also, if the signal amount information is one or more pieces of signal amount information including pixel data in a certain range, the selection parameter may be "signal amount information threshold", and the threshold or threshold range may be n (=any number).

[0034] <Range of average value Ave(α) of signal amount information> If the amount of current flowing through the light-emitting element that constitutes the light-emitting unit 4 is set to α, the average value of the signal amount information is set to Ave(α), and the aforementioned LimL and LimU are set for this Ave(α), the range of Ave(α) can be expressed by Equation 1.

[0035] LimL≦Ave(α)≦LimU ··· Formula 1

[0036] If the current amount α is controlled so as to satisfy the formula 1, the light amount of the light-emitting unit 4, that is, the signal amount information, can be optimized. Hereinafter, the average value of the signal amount information is represented as Ave(α).

[0037] <Acquisition range of average value Ave(α)> For example, if the light receiving unit 6 is equipped with a detector of QVGA (Quarter Video Graphics Array: screen resolution) = 320 pixels x 240 pixels, the output information of the light receiving unit 6 will be a pixel array from the 1st pixel (0,0) to the 76800th pixel (320,240). This pixel array is input in series to the processing unit 8.

[0038] The processing unit 8 may either calculate the average value of all pixels (320 pix × 240 pix = 76,800 pix) and compare it with LimL and LimU, or extract a portion of all pixels, calculate the average value, and compare it with LimL and LimU. In the latter method, it is possible to compare the average value calculated from pixels in an arbitrary area (for example, 30 pix × 30 pix = 900 pix) of all pixels with LimL and LimU.

[0039] <Light receiving output and pixels of the light receiving section 6> FIG. 2A shows a pixel array which is the light receiving output of the light receiving section 6 according to the second embodiment.

[0040] The light receiving output of the light receiving unit 6, that is, the signal amount information obtained from the light receiving unit 6, is a collection of all pixels in a unit of frame F, as shown in Fig. 2A, and as described above, these all pixels are composed of the first pixel (0,0) to the nth pixel (ny,nx). The arrows in the figure indicate the output order of the pixel array. Each pixel has a signal amount that represents individual shade or brightness according to the reflected light Lr.

[0041] B of Fig. 2 shows a pixel array representing all the pixels. As shown in B of Fig. 2, the light receiving output of the light receiving unit 6 is output in the form of a pixel array in which pixel information from the first pixel (0,0), the second pixel (0,1), ..., to the nth pixel (ny,nx) is arranged in series. The light receiving output consisting of this pixel array is input to the processing unit 8 in series and in sequence.

[0042] <When targeting all pixels> FIGS. 3A and 3B show the process of calculating the average value for all pixels.

[0043] When all pixels are the targets of average value calculation, as shown in FIG. 3A, all pixels in one frame (F) output from the light receiving section 6 are the targets.

[0044] As shown in B of Figure 3, the processing unit 8 calculates an average value using all pixels consisting of the first pixel (0,0), the second pixel (0,1), ..., the nth pixel (ny,nx), and compares this average value with LimU and LimL.

[0045] <When targeting any pixel among all pixels> A in Fig. 4 shows a case where an average value is calculated for any pixel among all pixels output in time series from the light receiving unit 6. For example, in this case, as shown in A in Fig. 4, areas A1, A2, A3, ... are set in the processing unit 8 for extracting pixels at regular intervals from the pixels output in time series from the light receiving unit 6. The average value of the signal amounts of, for example, three pixels included in the areas A1, A2, A3, ... is calculated.

[0046] When any pixel is to be targeted from among all pixels, the processing unit 8 acquires all pixels output from the light receiving unit 6 as shown in B of Figure 4, and periodically acquires, for example, three pixels included in areas A1, A2, A3, ... from all the pixels according to a certain rule, calculates the average value, and compares this average value with LimU and LimL.

[0047] According to such processing, the amount of information to be handled in the information processing executed by the processing unit 8 can be reduced, making it possible to speed up the processing and reduce the load on the computer.

[0048] <Comparison of Ave(α) with LimL and LimU for each frame> The target for obtaining the average value may control the frame for obtaining the average value from the output information of the light receiving unit 6. For example, when the light receiving unit 6 is equipped with a detector capable of 30 drawings per second, the processing unit 8 may obtain Ave(α) from the pixels of each frame and compare this Ave(α) with LimU and LimL. Using this comparison result, the light quantity control unit 10 can obtain light quantity information 30 times per second from the processing unit 8. Such processing is excellent in followability.

[0049] Alternatively, Ave(α) may be obtained from the pixels every 30 frames and compared with LimU and LimL. Using this comparison result, the light quantity control unit 10 can obtain light quantity information once per second from the processing unit 8. Such processing is excellent in noise resistance. Therefore, an appropriate setting can be made for the light quantity irradiated on the object 14.

[0050] <Relationship between the attributes of the object 14 and light quantity control> When the reflectivity of the central part of the object 14 to be distance-measured is extremely high and the reflectivity of its surroundings is extremely low, if the average of all pixels is handled, when the pixels with a large light quantity are defined as strong pixels and the pixels with a small light quantity are defined as weak pixels, the relationship between the central part and the peripheral part of the image is that the number of strong pixels << the number of weak pixels. Therefore, in this case, the average value of the signal quantity information becomes a small value, so the process is to increase the current amount of the light emitting unit 4. As a result, the entire image of the object 14 becomes brighter, but the part with high reflectivity is likely to be in a saturated state.

[0051] On the contrary, if the part with high reflectivity of the object 14 is set to an arbitrary area and the average value of the signal quantity information is taken, the relationship between the central part and the peripheral part is reversed, and the number of strong pixels >> the number of weak pixels. In this case, since the average value of the signal quantity information becomes a large value, the process is to decrease the current amount of the light emitting unit 4. As a result, an appropriate signal quantity tends to be obtained from the part with high reflectivity of the object 14.

[0052] <Processing procedure of the ToF system 2> 5 shows a process procedure for controlling the amount of light by using the average value of the luminance values ​​of the reflected light Lr. This process procedure is an example of a control method or program for the ToF system or ToF device of the present disclosure.

[0053] This processing procedure is information processing that is mainly executed by the processing unit 8. This processing procedure includes a process for acquiring signal amount information of the reflected light Lr (process F11) and a process for calculating light amount information (process F12). Process F12 constitutes a part of process F11, and in this process F12, the average light receiving signal amount is calculated from the luminance value included in the light receiving output of the light receiving unit 6, and the light amount information used for controlling the light amount of the light emitting unit 4 is increased or decreased.

[0054] Process F11 includes setting an initial value of the light amount (S101), controlling the light amount (S102), emitting light from the light-emitting unit 4 (S103), receiving reflected light Lr (S104), outputting a frame from the light-receiving unit 6 (S105), inputting the frame to the processing unit 8 (S106), extracting an arbitrary area (S107), process F12 (S108), and outputting signal amount information (S109).

[0055] Setting an initial value of the amount of light (S101): Before the ToF system 2 starts operating, the processing unit 8 sets an initial value of the amount of light in the light-emitting unit 4.

[0056] Control of light amount (S102): The processing unit 8 controls the light amount of the light-emitting unit 4 by using the light amount control unit 10. For this light amount control, light amount information obtained from process F12 is used.

[0057] Light emission by light-emitting unit 4 (S103): The light-emitting unit 4 emits light under the control of light amount control unit 10, and the amount of light is controlled based on the light amount information obtained in step F12.

[0058] Receiving reflected light Lr (S104): When the light emitter 4 emits light, the object 14 is irradiated with light Li. This results in reflected light Lr from the object 14, which is received by the light receiver 6, and a received light output is obtained from the light receiver 6.

[0059] Frame output of the light receiving unit 6 (S105): As described above, a CMOS image sensor is used for the light receiving unit 6. Therefore, the light receiving unit 6 provides a frame output (=received light output) which is a collection of pixels.

[0060] Frame input to processing unit 8 (S106): The frame output obtained by the light receiving unit 6 is provided to the processing unit 8. This is the frame input from the light receiving unit 6 to the processing unit 8.

[0061] Extraction of an arbitrary area (S107): In the processing unit 8, an arbitrary area including a target pixel for the average value is extracted from all pixels included in the provided frame.

[0062] Process F12 (S108): This process F12 (S108) performs comparison processing for all groups based on a comparison for each group divided from all pixels, and outputs signal amount information. Details of this process F12 (S108) will be described later.

[0063] Output of signal amount information (S109): After process F12 (S108), the processing unit 8 outputs and acquires the signal amount information.

[0064] Therefore, the light amount control unit 10 acquires light amount information from the processing unit 8, and the light amount of the light-emitting unit 4 is optimized based on this light amount information. In this embodiment, the light amount of the light-emitting unit 4 is controlled so that the luminance value of the reflected light Lr falls within the range from LimL to LimU.

[0065] <Processing of Process F12 (S108)> Process F12 includes calculation of an average value of luminance values ​​(for example, Ave(α) described above) (S201), comparison of the average value with an upper threshold value (for example, LimU described above) (S202), comparison of the average value with a lower threshold value (for example, LimL described above) (S203), decrement of light quantity value (S204), increment of light quantity value (S205), output of light quantity information (S206), etc. This process F12 uses a light quantity register of processing unit 8, which includes an upper limit register that reduces the light quantity when a pixel whose average value is equal to or greater than the upper limit threshold is detected, and a lower limit register that increases the light quantity when a pixel whose average value is equal to or greater than the lower limit threshold is detected.

[0066] Calculation of average brightness values ​​(S201): The processing unit 8 acquires brightness values ​​from pixels in a given area, and calculates the average brightness values.

[0067] Comparison of average value with upper threshold (S202): The processor 8 compares the average value of the luminance values ​​with the upper threshold and judges whether the average value is equal to or smaller than the upper threshold. If the average value is equal to or smaller than the upper threshold (YES in S202), the process proceeds to S203, and if the average value is greater than the upper threshold (NO in S202), the process proceeds to S204.

[0068] Comparison of average value with lower threshold (S203): If the average value is equal to or less than the upper threshold (YES in S202), the processing unit 8 judges whether the average value is equal to or greater than the lower threshold. If the average value is equal to or greater than the lower threshold (YES in S203), the process F12 ends and the process proceeds to S109, and if the average value is less than the lower threshold (NO in S203), the process proceeds to S205.

[0069] Decrementing the light quantity value (S204): If the average value is greater than the upper threshold value (NO in S202), the processing unit 8 decrements the light quantity value (reduces the amount of light) since the average value is large (outside the appropriate range).

[0070] Incrementing the light quantity value (S205): If the average value is smaller than the lower threshold value (NO in S203), the processing unit 8 increments the light quantity value (increases the amount of light) since the average value is small (outside the appropriate range).

[0071] Output of light quantity information (S206): After the process of S204 or S205, the processing unit 8 outputs the adjusted light quantity information and provides it to the light quantity control unit 10.

[0072] <Groups divided from all pixels> A group divided from all pixels represents a group of pixels classified according to a specific case when expressed in an image, which is a collection of pixel data. For example, groups are formed by classification of all pixel data, classification of charge Q11 data and charge Q12 data representing pixels described below, classification of data included in any area described above, etc.

[0073] <Advantages of the second embodiment> According to the second embodiment, any one of the following effects can be obtained. (1) The light quantity information, which is the received light output obtained by the light receiving unit 6 from the reflected light Lr from the object 14, is averaged over all or a part of the signal quantity information and compared with a lower threshold or an upper threshold as a predetermined value to generate light quantity information and control the light quantity of the light emitting unit 4. This makes it possible to optimize the amount of light irradiated to the object 14 and calculate distance information with high accuracy.

[0074] (2) A specific target area or target frame can be selected from the pixels representing the light receiving output of the light receiving unit 6 to calculate the average value of the signal amount information, and this average value can be compared with a lower threshold or an upper threshold as a predetermined value to generate light amount information and control the light amount of the light emitting unit 4. This makes it possible to optimize the amount of light irradiated to the target object 14 and calculate distance information with high accuracy.

[0075] Third embodiment 6 shows a processing procedure when a counter is used to optimize the luminance value obtained from the reflected light Lr. This processing procedure is an example of a control method or program for a ToF system or ToF device according to the present disclosure. In this processing procedure, an upper limit counter that counts pixels whose luminance value exceeds an upper limit threshold and a lower limit counter that counts pixels whose luminance value is lower than a lower limit threshold are set in the counter.

[0076] As in the second embodiment, this processing procedure is information processing that is mainly executed by the processing unit 8. This processing procedure includes a process for acquiring signal amount information of the reflected light Lr (process F21), a process for acquiring an arbitrary frame (process F22), a comparison process for each pixel (process F23), and a comparison process for each group (process F24).

[0077] Process F21 includes setting an initial value of the light amount (S301), controlling the light amount (S302), initializing the upper limit counter and the lower limit counter (S303), process F22 (S304), determining the count value of the upper limit counter (S305), determining the count value of the lower limit counter (S306), decrementing the light amount value (S307), incrementing the light amount value (S308), and outputting light amount information (S309).

[0078] Setting initial value of light intensity (S301): Before the ToF system 2 starts operating, the processing unit 8 sets an initial value of the light intensity for the light emitting unit 4. Therefore, in this ToF system 2, the light intensity is controlled to be optimal for the target object 14 compared with the initial value.

[0079] Control of light amount (S302): The processing unit 8 controls the light amount of the light-emitting unit 4 by the light amount control unit 10 based on the light amount information.

[0080] Initialization of upper limit counter and lower limit counter (S303): The upper limit counter and the lower limit counter are initialized, and the process proceeds to S304, where step F22 is executed.

[0081] Process F22 (S304): In process F22, signal amount information is acquired from an arbitrary frame, and this signal amount information acquisition process is executed for all frames. The process after the end of process F22 will be described first, and the contents of process F22 will be described later.

[0082] Determination of count value of upper limit counter (S305): After process F22, the processing unit 8 determines the count value of the upper limit counter. If the count value>the specified value (NO in S305), the process proceeds to S307, and if the count value≦the specified value (YES in S305), the process proceeds to S306.

[0083] Determination of count value of lower limit counter (S306): If the count value is less than or equal to the specified value (YES in S305), the processing unit 8 determines the count value of the lower limit counter (S306). If the count value is greater than or equal to the specified value (NO in S306), the process proceeds to S308, and if the count value is less than or equal to the specified value (YES in S306), the process proceeds to S303.

[0084] Decrementing the light amount value (S307): If the count value is greater than the specified value (NO in S305), the processing unit 8 decrements the light amount value because the light amount is excessive (outside the appropriate range), thereby decreasing the light amount.

[0085] Incrementing the light amount value (S308): If the count value is greater than the specified value (NO in S306), the processing unit 8 increments the light amount value because the light amount is too low (outside the appropriate range) and increases the light amount.

[0086] Output of light amount information (S309): The processing unit 8 outputs the light amount information indicating the increased or decreased light amount, and inputs it to the light amount control unit 10. Therefore, the light amount control unit 10 controls the light amount of the light-emitting unit 4 based on the light amount information.

[0087] <Processing of Process F22 (S304)> Process F22 (S304) includes light emission from the light-emitting unit 4 (S401), reception of reflected light Lr (S402), frame output from the light-receiving unit 6 (S403), frame input to the processing unit 8 (S404), extraction of an arbitrary area (S405), process F23 (S406), process F24 (S407), and output of signal amount information (S408).

[0088] Emission of light from light-emitting unit 4 (S401): The processing unit 8 causes the light-emitting unit 4 to emit light under the control of the light amount control unit 10. The amount of light from the light-emitting unit 4 is controlled with the light amount information obtained by process F21 as a target value. As a result, the object 14 is irradiated with light Li.

[0089] Receiving reflected light Lr (S402): When light Li is irradiated, reflected light Lr is obtained from the object 14, and this reflected light Lr is received by the light receiving unit 6. Therefore, the light receiving unit 6 provides a received light output of the reflected light Lr.

[0090] Frame output of the light receiving unit 6 (S403): The light receiving unit 6 provides a frame output that indicates the received light output.

[0091] Frame input to processing unit 8 (S404): The frame output from the light receiving unit 6 is input to the processing unit 8.

[0092] Extraction of arbitrary area (S405): The processing unit 8 extracts an arbitrary area from the frame input of all pixels input from the light receiving unit 6.

[0093] Process F23 (S406): After S405, the process proceeds to process F23. In this process F23, a comparison is made for each pixel in the arbitrary area extracted by the processing unit 8, and this comparison is executed for all pixel determinations.

[0094] Process F24 (S407): Process F23 includes process F24, and when process F23 is entered, process F24 is executed. In this process F24, a comparison is performed for each group including a plurality of pixels, and this comparison process is executed for all groups. When process F24 is completed, the process transitions to process F23.

[0095] Output of signal amount information (S408): Upon completion of process F23, the processing unit 8 outputs the signal amount information and provides it to the calculation unit 12.

[0096] <Processing of Process F24 (S407)> Process F24 includes comparing the brightness value with an upper threshold (S501), incrementing an upper limit counter (S502), comparing the brightness value with a lower threshold (S503), and incrementing the lower limit counter (S504).

[0097] Comparison of brightness value with upper threshold (S501): The processing unit 8 compares the brightness value with the upper threshold. If the brightness value is greater than the upper threshold (NO in S501), the processing proceeds to S502. If the brightness value is less than or equal to the upper threshold (YES in S501), the processing proceeds to S503.

[0098] Incrementing the upper limit counter (S502): If the luminance value is greater than the upper limit threshold (NO in S501), the processing unit 8 increments the upper limit counter to increase the count value.

[0099] Comparison of brightness value with lower threshold (S503): The processing unit 8 compares the brightness value with the lower threshold. If the brightness value is less than the lower threshold (NO in S503), the processing proceeds to S504. If the brightness value is greater than or equal to the lower threshold (YES in S503), the processing unit 8 completes process F24.

[0100] Incrementing the lower limit counter (S504): If the luminance value is smaller than the lower limit threshold (NO in S503), the processing unit 8 increments the lower limit counter to increase the count value.

[0101] The process proceeds from the group-by-group comparison in process F24 (S407) to process F23 (S406), and through the pixel-by-pixel comparison in process F23, the process proceeds to process F22.

[0102] Then, when this process F22 is completed, the process proceeds to the above-mentioned steps S305 to S309 of the process F21, and this process procedure is terminated.

[0103] Therefore, according to this processing procedure, the light amount information is increased or decreased according to the luminance value obtained from the reflected light Lr in the same manner as in the second embodiment, thereby optimizing the light amount of the light-emitting unit 4.

[0104] <Advantages of the Third Embodiment> According to this third embodiment, the light amount of the light-emitting unit 4 can be controlled to a predetermined value or a predetermined appropriate value range by using the counter's count value to set the brightness value representing the signal amount information obtained from the reflected light Lr above a lower threshold and below an upper threshold, thereby improving the accuracy of the distance information.

[0105] [Fourth embodiment] The ToF device according to the fourth embodiment can be realized by, for example, incorporating the above-described ToF system 2 in a housing. The device configuration of this ToF device is common to the ToF system 2, and the above-described methods and programs are similar, so the description thereof will be omitted.

[0106] <Advantages of the Fourth Embodiment> According to the fourth embodiment, any one of the following effects can be obtained. (1) The same effects as those of the first to third embodiments can be obtained.

[0107] (2) A ToF device can be configured using the ToF system 2, for example, a distance measuring device or a camera. EXAMPLES

[0108] This example identifies how the distance between the object 14 and the ToF system 2 affects the distance information obtained from the ToF system 2.

[0109] In this embodiment, a short distance gate, a long distance gate, and a gate for eliminating disturbances due to environmental light are provided in the means for processing the received light output of the reflected light Lr, and the received light output of the reflected light Lr is processed.

[0110] The signal amount information is information obtained by measuring and quantifying the characteristics of the reflected light Lr received by the light receiving unit 6. For example, when the light intensity of the reflected light Lr is measured in the light receiving unit 6, the luminance value of the reflected light Lr is output to the light receiving unit 6. Therefore, in this embodiment, the luminance value included in the light receiving output of the light receiving unit 6 is treated as the signal amount information.

[0111] Fig. 7 shows an example of the light receiving section 6 according to this embodiment. The light receiving section 6 shown in Fig. 7 is just an example, and the present disclosure is not limited to this configuration.

[0112] The light receiving section 6 includes a light receiving element 20 for receiving reflected light Lr from the object 14. The light receiving element 20 receives light from the environment surrounding the object 14 (ambient light) in addition to the reflected light Lr.

[0113] The capacitor 21 (hereinafter referred to as "Cap21") stores the charge obtained from the light receiving element 20 when the light receiving element 20 receives ambient light.

[0114] The capacitor 22 (hereinafter referred to as "Cap22") stores the electric charge obtained in the light receiving element 20 when the light receiving element 20 receives reflected light Lr from an object 14 located in a short distance (the light takes a short time to reach the object).

[0115] The capacitor 23 (hereinafter referred to as "Cap23") stores the electric charge obtained in the light receiving element 20 when the light receiving element 20 receives reflected light Lr from an object 14 at a long distance (the light takes a long time to reach the object).

[0116] A gate circuit 31 (hereinafter referred to as “Gate 31”) sets a charge storage time for Cap 21.

[0117] A gate circuit 32 (hereinafter referred to as “Gate 32”) sets a charge storage time for Cap 22.

[0118] A gate circuit 33 (hereinafter referred to as “Gate 33”) sets a charge storage time for Cap 23.

[0119] A gate circuit 34 (hereinafter referred to as “GateDrain 34 ”) sets the operation timing for removing electric charges unnecessary for distance measurement from the light receiving output of the light receiving element 20 .

[0120] The cathode of the light receiving element 20 is connected to Cap 21 via Gate 31, to Cap 22 via Gate 32, to Cap 23 via Gate 33, and to a reference potential point 36 (hereinafter referred to as "Drain 36") via Gate Drain 34.

[0121] <Light receiving processing> 8 shows a processing procedure when distance information is acquired by the light receiving unit 6. This processing procedure is an example of a control method or program for the ToF system or ToF device of the present disclosure.

[0122] When obtaining distance information, the GateDrain 34 is transitioned from the ON state (conducting state) to the OFF state (non-conducting state) (S601), and the light receiving element 20 is set to a light receiving preparation state.

[0123] Before the light-emitting unit 4 emits light, the Gate 31 is made conductive and cut off at any timing. During this conductive period, the charge obtained in the light-receiving element 20 is stored in the Cap 21 (S602). As a result, the charge obtained in the light-receiving element 20 by receiving ambient light is stored in the Cap 21.

[0124] The light emitting unit 4 is caused to emit light (S603), and the object 14 is irradiated with light Li.

[0125] Gate 32 is turned on and then turned off at an arbitrary timing. The charge obtained in light receiving element 20 during this conduction period is stored in Cap 22 (S604).

[0126] Gate 33 is turned on and then cut off at an arbitrary timing. The charge obtained in light receiving element 20 during this conduction period is stored in Cap 23 (S605).

[0127] The GateDrain 34 is shifted from the OFF state to the ON state (S606), and the charge generated by the light receiving element 20 receiving light is discharged to the Drain 36.

[0128] This series of processes (S601 to S606) is repeated n times. The number of repetitions is checked (S607), and when this example of the process has been repeated n times (YES in S607), an output signal Sout (charge signal) is taken out (S608). This output signal Sout corresponds to the signal amount information described above.

[0129] <Light reception processing operation timing> 9A shows the ON / OFF timing of the GateDrain 34. The GateDrain 34 transitions from the ON state (conducting state) to the OFF state (non-conducting state) at time t1, and transitions from the OFF state to the ON state at time t6. In other words, the GateDrain 34 is in an OFF section from time t1 to time t6.

[0130] 9B shows the ON / OFF timing of Gate 31. Gate 31 transitions from the OFF state to the ON state at time t2 immediately after time t1, and transitions from the ON state to the OFF state at time t3. That is, Gate 31 is conductive in the period from time t2 to t3. The period from time t2 to t3 is the period during which ambient light is received.

[0131] 9C shows the light emission timing of the light-emitting unit 4. The light-emitting unit 4 starts emitting light at time t3 and turns off at time t4. The section from time t3 to time t4 is the light-emitting period.

[0132] 9D shows the ON / OFF timing of Gate 32. Gate 32 transitions from the OFF state to the ON state at time t3, and transitions from the ON state to the OFF state at time t4. That is, Gate 32 is conductive in the section from time t3 to t4. The section from time t3 to t4 is the period during which reflected light Lr from a close-range object 14 is received.

[0133] 9E shows the ON / OFF timing of Gate 33. Gate 33 transitions from the OFF state to the ON state at time t4, and transitions from the ON state to the OFF state at time t5. That is, Gate 33 is conductive in the section from time t4 to t5. The section from time t4 to t5 is the period during which reflected light Lr from a long-distance object 14 is received.

[0134] <When Object 14 is in close range> 10A shows a distance d1 (= a short distance) between the object 14 and the ToF system 2. In FIG.

[0135] B in FIG. 10 indicates the light emission period of the light emitting unit 4 from time t3 to time t4. During this light emission period, the object 14 is irradiated with the light Li. It is assumed that the light receiving element 20 receives the reflected light Lr from the object 14, and a charge Q1 is obtained as a light receiving output from the light receiving element 20.

[0136] C in FIG. 10 indicates the conduction period of Gate 32 from time t3 to time t4. A delay time Δt1 occurs in receiving the reflected light Lr according to the distance d1 between the object 14 and the light receiving unit 6. For this reason, the charge from the light receiving element 20 is stored in Cap22 during the period from time (t3 + Δt1), which is the time after a lapse of the delay time Δt1 from time t3, to time t4. This charge is denoted as Q11.

[0137] D in FIG. 10 indicates the conduction period of Gate 33 from time t4 to time t5. The charge from the light receiving element 20 is stored in Cap23 during the period from time t4 to time (t4 + Δt1), which is the time after a lapse of the delay time Δt1 from time t4. This charge is denoted as Q12.

[0138] In this way, most of the charge Q1 generated in the light receiving element 20, i.e., the charge Q11, is stored in Cap22, and the charge Q12 stored in Cap23 becomes less. That is, the distance to the object 14 is proportional to the amount of charge (signal amount) stored in the charge Q11 stored in Cap22 and the charge Q12 stored in Cap23. Therefore, it can be determined that the distance to the object 14 is short based on the magnitudes of the charges Q11 and Q12.

[0139] <When the object 14 is at a long distance> A in FIG. 11 shows the distance d2 (= long distance) between the object 14 and the ToF system 2.

[0140] B in FIG. 11 indicates the light emission period of the light emitting unit 4 from time t3 to time t4. During this light emission period, the object 14 is irradiated with the light Li. It is assumed that the light receiving element 20 receives the reflected light Lr from the object 14, and a charge Q2 (< Q1) is obtained as a light receiving output from the light receiving element 20.

[0141] C in Fig. 11 shows the conduction period of Gate 32 from time t3 to time t4. A delay time Δt2 (>Δt1) occurs in the reception of reflected light Lr depending on the distance d2 between the object 14 and the light receiving unit 6. Therefore, charge from the light receiving element 20 is stored in Cap22 during the period from time (t3+Δt2), which is the delay time Δt2 after time t3, to time t4. This charge is designated as Q21.

[0142] 11D shows the conduction period of Gate 33 from time t4 to time t5. During the period from time t4 to time (t4+Δt2) after the delay time Δt2 has elapsed, the charge from the light receiving element 20 is stored in Cap 23. This charge is designated as Q22.

[0143] In this way, most of the charge Q22 generated in the light receiving element 20 is stored in Cap23, and the charge Q21 stored in Cap22 is reduced. In other words, the distance to the object 14 is proportional to the charge amount (signal amount) of the charge Q21 stored in Cap22 and the charge Q22 stored in Cap23. Therefore, the distance to the object 14 can be determined to be long depending on the magnitude of the charges Q21 and Q22.

[0144] <Relationship between distance to object 14 and amount of light received> The amount of light received by the light receiving unit 6 that receives the reflected light Lr depends on the light density of the reflected light Lr. This light density can be regarded as the density of light emitted from the target 14 as the light source. If the total amount of light is P, this light density attenuates in proportion to the square of the distance d. If the light density at a point at a certain distance is Pd, this Pd can be expressed by Equation 2.

[0145] Pd = P / (4πd 2 )...Equation 2

[0146] It is possible to calculate distance from the charge ratio in the above manner, but the light density varies depending on the distance d, which may affect the accuracy of the distance information. That is, in the case of a short distance, Pd becomes large, so the light receiving output of the light receiving unit 6 may become saturated. Also, in the case of a long distance, Pd becomes small, so the S / N ratio may deteriorate. Therefore, in order to obtain an appropriate light density Pd at any distance d, it is preferable to control the light amount of the light emitting unit 4.

[0147] Therefore, in the embodiment, the charge amount is the signal amount, the output signal amount of Cap21 for measuring ambient light is S1, the output signal amount of Cap22 is S2, and the output signal amount of Cap23 is S3, and the total signal amount (Intensity) of these can be expressed by Equation 3.

[0148] Intensity=(S2-S1)+(S3-S1)...Equation 3

[0149] In formula 3, (S2-S1) represents the net signal amount obtained by subtracting the output signal amount S1 (=environmental noise component) of Cap21 from the output signal amount S2 of Cap22. (S3-S1) represents the net signal amount obtained by subtracting the output signal amount S1 (=environmental noise component) of Cap21 from the output signal amount S3 of Cap23. Therefore, Intensity is signal amount information excluding the environmental noise component, and represents the amount of information offset excluding the environmental noise component.

[0150] The target value of this Intensity was set to 2500. This target value is a value that prevents the light receiving section 6 from becoming saturated or causing a low S / N ratio, and is an estimated value of approximately 60% of the signal amount of 12 bits (2 to the power of 12 = 4096).

[0151] In this embodiment, a PC (personal computer) was set outside the ToF system 2, and a control command was sent from this PC to the light amount control unit 10 to obtain the intensity and distance information when the light amount obtained by the light emitting unit 4 was set to three levels: "high", "medium", and "low". The results are shown in Table 1.

[0152] [Table 1]

[0153] In this embodiment, when the target distance to the object 14 is set to 500 mm and the light intensity of the light-emitting unit 4 is set to "large," an intensity of 4025 is obtained. The distance information calculated from this intensity is 576 mm, which is significantly different from the target value. Furthermore, when the light intensity of the light-emitting unit 4 is set to "small," an intensity of 652 is obtained. The distance information calculated from this intensity is 148 mm, which is significantly different from the target value.

[0154] In contrast, when the light amount of the light-emitting unit 4 was set to "medium," the obtained Intensity was 2531. The distance information calculated from this Intensity was 496 mm, which is almost the target value.

[0155] As is clear from this embodiment, when the light amount is set to large or small without controlling the signal amount information, the signal amount of the intensity significantly exceeds or falls short of the target value, resulting in the distance information deviating significantly from the actual distance to the object 14.

[0156] Therefore, it was confirmed that the signal amount can be optimized by controlling the signal amount information obtained by the light receiving unit 6 within the range of LimL and LimU, and the distance information can be optimized. It was also confirmed that the signal amount can be further optimized and high accuracy of the distance information can be realized by setting LimL and LimU according to the measurement distance to the object 14.

[0157] Other Embodiments The present disclosure includes the following modifications:

[0158] (1) In the above embodiment and example, the amount of current flowing through the light-emitting element of the light-emitting unit 4 was given as an example of a parameter for controlling the amount of light obtained from the light-emitting unit 4, but parameters such as electrical quantities other than the amount of current may also be used.

[0159] (2) The energy density (=luminous flux) of the light Li irradiated from the light emitting unit 4 to the object 14 may be controlled.

[0160] (3) The wavelength of the light Li may be controlled.

[0161] (4) The amount of diffusion of the light Li may be controlled. It is possible to control the amount of diffusion of the light Li by increasing or decreasing the operating current of the light-emitting element installed in the light-emitting unit 4. Also, by providing an aperture in the light-emitting unit 4, it is possible to control the amount of diffusion of the light projected from the light-emitting unit 4 to the object 14 by the aperture. Also, by providing a plurality of light-emitting elements in the light-emitting unit 4, the number of light-emitting elements that are turned on may be controlled. Therefore, the light-emitting unit 4 can be configured to include at least one light-emitting element or a plurality of light-emitting elements, and the light amount control unit 10 can control the amount of current flowing to the light-emitting element or the number of light-emitting elements that are caused to emit light, thereby controlling the signal amount information obtained in the light-receiving output of the light-receiving unit 6.

[0162] As described above, the most preferred embodiments and examples of the present disclosure have been described. The present disclosure is not limited to the above description, and various modifications and changes can be made by those skilled in the art based on the technical contents described in the claims or disclosed in the specification. It goes without saying that such modifications and changes are included in the scope of the present disclosure. [Industrial Applicability]

[0163] The present disclosure provides a method for controlling the amount of light irradiated to an object from an light emitting unit, receiving the light reflected from the object with a light receiving unit, and acquiring signal amount information of the reflected light from the received light output obtained in the light receiving unit, so that the signal amount information is obtained above a lower threshold and below an upper threshold, thereby optimizing the amount of reflected light received by the light receiving unit, and improving the accuracy of distance information obtained from a ToF system or ToF device. [Explanation of symbols]

[0164] 2 ToF System 4 Light emitting part 6 Light receiving part 8. Information Processing Section 10 Light quantity control section 12 Arithmetic section 14 Object 20 Photodetector 21 Capacitor 22 Capacitor 23 Capacitor 31 Gate Circuit 32 Gate Circuit 33 Gate Circuit 34 Gate Circuit 36 Reference potential point A1 Area A2 Area A3 Area Li Light Lr reflected light LimL Lower Threshold LimU Upper threshold

Claims

1. A ToF system that irradiates a target object with light and receives reflected light from the target object to obtain distance information, A light source unit that irradiates light onto the object; A light receiving unit that receives reflected light from the object; an information processing unit that acquires signal amount information from the received light output of the light receiving unit and calculates light amount information for generating the signal amount information within a predetermined value or a predetermined range between a lower threshold value and an upper threshold value; a light amount control unit that controls the light amount of the light source unit using the light amount information; A ToF system comprising:

2. The ToF system according to claim 1 , wherein the information processing unit generates the light amount information by comparing an average value of all or a part of the signal amount information with the lower limit threshold or the upper limit threshold.

3. The ToF system according to claim 2 , wherein the information processing unit selects a specific target area or target frame from pixels representing the light receiving output of the light receiving unit and calculates the average value of the signal amount information.

4. The ToF system according to claim 1 or 2, wherein the light source unit includes at least one light-emitting element or a plurality of light-emitting elements, and the light amount control unit controls the amount of current flowing through the light-emitting element or controls the number of light-emitting elements that are caused to emit light.

5. A method for controlling a ToF system that irradiates an object with light and receives reflected light from the object to obtain distance information, comprising: A step of a light source unit emitting light and irradiating the object with light; A step of a light receiving unit receiving reflected light from the object; An information processing unit acquires signal amount information from the light receiving output of the light receiving unit, and calculates light amount information for generating the signal amount information within a predetermined value or a predetermined range between a lower limit threshold and an upper limit threshold; a step of a light amount control unit controlling the light amount of the light source unit using the light amount information; A method for controlling a ToF system, comprising:

6. A program for causing a computer to execute the program, A function of irradiating light from a light source unit onto an object; A function of acquiring signal amount information from the light receiving output of the light receiving unit that receives the reflected light from the object, and calculating light amount information for generating the signal amount information within a predetermined value or a predetermined range between a lower threshold value and an upper threshold value; a function of controlling the amount of light of the light source unit using the light amount information; A program for causing the computer to execute the above.

7. A ToF device that irradiates a target with light and receives reflected light from the target to obtain distance information, A light source unit that irradiates light onto the object; A light receiving unit that receives reflected light from the object; an information processing unit that acquires signal amount information from the received light output of the light receiving unit and calculates light amount information for generating the signal amount information within a predetermined value or a predetermined range between a lower threshold value and an upper threshold value; a light amount control unit that controls the light amount of the light source unit using the light amount information; A ToF device comprising:

Images