Distance information acquisition device

Abstract

The purpose of the present invention is to provide a distance information acquisition device capable of measuring the distance to an object with high accuracy. This distance information acquisition device (10) is provided with: an irradiation unit (11) that irradiates an object (OB) with light; a distance information acquisition device (10) for acquiring distance information to an object (OB), the distance information acquisition device (10) including an irradiation unit (151) for irradiating light emitted by the irradiation unit and reflected by the object (OB) via an optical system lens (12), and a Time of Flight (ToF) sensor (13) for receiving light irradiated by the irradiation unit and reflected by the object (OB) via the optical system lens (12), the distance information acquisition device (10) acquiring distance information to the object (OB), the distance information acquisition device (10) including: a distance value acquisition unit (151) for acquiring a distance value from the object (OB); acquiring a distance value obtained on the basis of the timing of irradiation by the irradiation unit (11), the timing of light reception by the ToF sensor (13), and the light velocity; a storage unit (152) that stores a predetermined numerical expression that includes at least information relating to an angle of view (AFOV) of the optical lens (12) as a parameter; and a correction calculation unit (153) that calculates distance information to the object by applying the acquired distance value to a predetermined mathematical expression stored in the storage unit (152).

Description

Technical Field

[0001] The present invention relates to a distance information acquisition device.

[0002] This application claims the priority of Japanese Patent Application No. 2022-205108 filed in Japan on December 22, 2022, and incorporates its content herein. Background Art

[0003] Conventionally, there has been a device in which a light-emitting element such as a VCSEL (Vertical Cavity Surface Emitting Laser) and a light-receiving element such as a ToF sensor are provided around the lens of a photographing device, and the distance to an object is measured by measuring the time it takes for light emitted from the light-emitting element to be reflected by the object and received by the ToF sensor (for example, see Patent Document 1). Such a distance measurement method is widely known as the TOF (Time of Flight) method.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2021-26236. Summary of the Invention

[0007] However, in the case of using the above-described technology, since the optical axis of the lens and the optical axis of the irradiation unit are arranged offset from each other, there is a problem that an error occurs in the measured distance in space. Since the error increases as the distance between the optical axis of the lens and the optical axis of the irradiation unit increases, according to such prior art, it is not preferable to use a large lens (specifically, a lens having a large lens barrel diameter and a long lens barrel length). That is, when a zoom lens that is likely to be large is applied to the prior art, there is a problem that an error occurs in the measured distance.

[0008] In addition, even if the optical axis of the lens and the optical axis of the irradiation unit can be arranged to coincide, the object to be the distance measurement target may not exist at the center of the optical axis. When the object does not exist at the center of the optical axis, the distance from the irradiation unit to the object until it is reflected (hereinafter, referred to as the first optical path length) is different from the distance from the object to the light-receiving unit after being reflected (hereinafter, referred to as the second optical path length). According to the prior art, since the first optical path length and the second optical path length are regarded as the same distance, and the time of flight of light is divided by 2 to be used as the time of flight of light to the object for distance measurement calculation, there is a problem of generating an error.

[0009] The present invention has been made in view of such circumstances, and an object thereof is to provide a distance information acquisition device capable of measuring the distance to an object with high accuracy.

[0010] [1] In one aspect of the present embodiment, a distance information acquisition device includes: an irradiation unit that irradiates light onto an object; and a time-of-flight sensor (ToF, Time of Flight) that receives, via an optical system lens, the light irradiated by the irradiation unit and reflected by the object. The distance information acquisition device is configured to acquire distance information to the object, and the distance information acquisition device includes: a distance value acquisition unit that acquires a distance value obtained based on the timing of irradiation by the irradiation unit, the timing of the time-of-flight sensor receiving light, and the speed of light; a storage unit that stores a prescribed mathematical formula that includes at least information related to the field of view angle of the optical system lens as a parameter; and a correction calculation unit that calculates the distance information to the object by applying the acquired distance value to the prescribed mathematical formula stored in the storage unit.

[0011] [2] Additionally, in one mode of the present embodiment, in the distance information acquisition device described in the above [1], a correction value storage unit is further included, which stores information related to the difference between the result of actually measuring the distance to the object and the distance information to the object calculated by the correction calculation unit as a correction value. The correction calculation unit calculates the distance information to the object based on the correction value stored in the correction value storage unit.

[0012] [3] Further, in one mode of the present embodiment, in the distance information acquisition device described in the above [1] or [2], the optical system lens is a variable focal length zoom lens, and the distance information acquisition device further includes: a field of view angle information storage unit that stores the focal length of the optical system lens and the field of view angle corresponding to the focal length in association with each other. The correction calculation unit calculates the distance information to the object based on the field of view angle corresponding to the focal length of the optical system lens.

[0013] [4] Additionally, in one mode of the present embodiment, in the distance information acquisition device described in any one of the above [1] to [3], the optical system lens is a replaceable lens on which any one of a plurality of lenses is mounted, and the distance information acquisition device further includes a lens identification information acquisition unit that acquires lens identification information for identifying the mounted optical system lens. The field of view angle information storage unit stores the focal length and the field of view angle corresponding to the focal length in association with each other for each of the plurality of lenses. The correction calculation unit acquires the field of view angle corresponding to the focal length of the lens identified based on the lens identification information acquired by the lens identification information acquisition unit from the field of view angle information storage unit, and calculates the distance information to the object based on the acquired field of view angle.

[0014] [5] Further, in one aspect of the present embodiment, in the distance information acquisition device described in any one of [1] to [4] above, the irradiation unit changes the distance between the irradiation unit and the time-of-flight sensor by moving along the optical axis direction of the optical system lens, and the calibration calculation unit calculates the distance information to the object based on the distance between the irradiation unit and the time-of-flight sensor.

[0015] [6] Further, in one aspect of the present embodiment, the distance information acquisition device described in any one of [1] to [4] above further includes: a prism disposed between the optical system lens and the time-of-flight sensor for allowing light irradiated by the irradiation unit and reflected by the object to enter; a visible light reflection dichroic film that reflects visible light in the light incident on the prism; and an RGB sensor that receives the visible light reflected by the visible light reflection dichroic film.

[0016] [7] Further, in one aspect of the present embodiment, in the distance information acquisition device described in [6] above, the calibration calculation unit further calculates the distance information to the object based on the thickness and reflectivity of the glass of the prism.

[0017] According to the present embodiment, the distance to the object can be measured with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic diagram for explaining the distance information acquisition system according to Embodiment 1.

[0019] Figure 2 is a functional block diagram showing the functional structure of the arithmetic unit according to Embodiment 1.

[0020] Figure 3 is a first diagram for explaining the calculation performed by the arithmetic unit according to Embodiment 1.

[0021] Figure 4 is a second diagram for explaining the calculation performed by the arithmetic unit according to Embodiment 1.

[0022] Figure 5 is a diagram showing an example of the three-dimensional information after calibration calculation by the distance information acquisition device according to Embodiment 1.

[0023] Figure 6 is a diagram for explaining a modification of the distance information acquisition system according to Embodiment 1.

[0024] Figure 7 is a diagram for explaining the outline of calibration according to Embodiment 2.

[0025] Figure 8This is a diagram for explaining the functional structure of the distance information acquisition system according to Embodiment 3.

[0026] Figure 9 This is a diagram for explaining the functional structure of the distance information acquisition system according to Embodiment 4.

[0027] Figure 10 This is a diagram for explaining the problem to be solved by the distance information acquisition device according to Embodiment 5.

[0028] Figure 11 This is a diagram for explaining the structure of the distance information acquisition device according to Embodiment 5.

[0029] Figure 12 This is a diagram for explaining the functional structure of the distance information acquisition device according to Embodiment 5.

[0030] Figure 13 This is a diagram showing an example of the correspondence table between the lens and the LD according to Embodiment 5.

[0031] Figure 14 This is a diagram for explaining the structure of a modified example of the distance information acquisition device according to Embodiment 5.

[0032] Figure 15 This is a diagram for explaining the functional structure of the distance information acquisition device according to Embodiment 6.

[0033] Figure 16 This is a diagram showing an example of the reflection spectral characteristics of the visible light reflection dichroic film according to Embodiment 6;

[0034] Figure 17 This is a diagram showing an example of the arrangement of the lens and the irradiation unit included in the distance information acquisition device 10 according to Embodiment 7.

[0035] Figure 18 This is a diagram for explaining the ToF camera system according to the prior art.

[0036] Figure 19 This is a diagram showing an example of the arrangement of the lens and the ToF sensor included in the ToF camera system according to the prior art.

[0037] Figure 20 This is a diagram showing an example of the situation of generating point cloud data based on the distance values obtained by the ToF camera system according to the prior art. Detailed implementation manners

[0038] Hereinafter, preferred embodiments will be disclosed, and the distance information acquisition device according to the embodiments will be described in detail with reference to the accompanying drawings. In addition, the embodiments described below are merely examples, and the present embodiments are not limited to the following embodiments. In addition, "based on XX" as used in this application means "at least based on XX", and also includes cases where other elements are also based on in addition to XX. In addition, "based on XX" is not limited to directly using XX, and also includes cases based on elements obtained by calculating and processing XX. "XX" is an arbitrary element (for example, arbitrary information). In addition, in the following drawings, in order to facilitate understanding of each structure, the scale, quantity, etc. in each structure may sometimes be different from those in the actual structure.

[0039] [Prior Art]

[0040] First, refer to Figures 18 to 20 to describe the prior art.

[0041] Figure 18 FIG. is a diagram for explaining a ToF camera system of the prior art. Referring to this figure, the ToF camera system 9 of the prior art will be described. Hereinafter, the positional relationship and the like of the components included in the ToF camera system 9 may sometimes be represented by a three-dimensional orthogonal coordinate system of the x-axis, y-axis, and z-axis.

[0042] The ToF camera system 9 includes a ToF camera device 90 and an information processing device 95. The ToF camera device 90 includes an irradiation unit 91, a lens 92, a ToF sensor 93, and a distance measurement unit 94. The ToF camera device 90 measures the distance to the object OB. In an example shown in the figure, in order to explain the problems of the prior art, a white chart WC (white chart) is placed on the surface of the object OB. In addition, the white chart WC is arranged in parallel with the ToF sensor 93.

[0043] The irradiation unit 91 includes a light emitting element capable of irradiating laser light, and irradiates the object OB with laser light. The laser light irradiated by the irradiation unit 91 may be visible light or infrared light. The irradiation unit 91 may be, for example, a laser diode array such as a VCSEL (Vertical Cavity Surface Emitting Laser). The irradiation unit 91 irradiates laser light in a direction perpendicular to the ToF sensor 93 (the z-axis direction in the figure) based on an instruction from the distance measurement unit 94. The light irradiated by the irradiation unit 91 is reflected by the object OB and enters the ToF sensor 93 via the lens 92. The distance measurement unit 94 measures the distance to the object OB based on the time required from when the light is irradiated from the irradiation unit 91 until the ToF sensor 93 receives the light.

[0044] The ToF sensor 93 has a plurality of pixels on two-dimensional coordinates arranged in the vertical direction (y-axis direction in the figure) and the horizontal direction (x-axis direction in the figure). Each pixel included in the ToF sensor 93 detects the timing of receiving light. The ToF sensor 93 outputs the detection information of each pixel in the two-dimensional coordinates to the distance measuring unit 94 while scanning. The distance measuring unit 94 calculates the distance value for each pixel and generates a distance image as two-dimensional array data. As the distance measuring method of the ToF sensor 93, a dToF (direct Time of Flight) method or an iToF (indirect Time of Flight) method or the like may also be used. In the illustrated example, the iToF method is used. The distance image generated by the distance measuring unit 94 is converted into three-dimensional point group data by an information processing device 95. The information processing device 95 may also be an information processing device such as a personal computer or a tablet terminal.

[0045] Here, it is preferable that the distance L91 from the irradiation unit 91 of the ToF camera device 90 to the object OB is the same as the distance L92 from the object OB to the ToF sensor 93 of the ToF camera device 90. However, in reality, the distance L91 and the distance L92 may be different depending on the irradiation angle of the irradiation unit 91, the optical path in the lens 92, etc. According to the prior art, the distance L91 and the distance L92 are regarded as the same and the distance to the object OB is calculated. That is, according to the prior art, the difference between the distance L91 and the distance L92 is ignored and the distance to the object OB is calculated, so there is a problem of error in the measurement result.

[0046] Figure 19 1 is a diagram showing an example of the arrangement of a lens and a ToF sensor included in a conventional ToF camera system. Referring to this diagram, an example of the arrangement of a lens 92 and a ToF sensor 93 will be described. Figure 19 (A) is in Figure 18 0 is a diagram of a ToF camera device 90 observed from an object OB in the z-axis direction. As shown in the figure, a lens 92 and an irradiation unit 91 are arranged at positions perpendicular to the y-axis. The distance from the optical axis OA of the lens 92 to the center position of the irradiation unit 91 is recorded as a distance dv. It is known that the irradiation unit 91 suppresses the error of the distance by giving an inclination to the light axis of the light. However, arranging the irradiation unit 91 such as a VCSEL at an inclination on the substrate will lead to the complication of the design man-hour and the shape of the substrate. In addition, it is expected that by arranging the irradiation unit 91 at an inclination, the error of the distance may be suppressed for a certain range of distances, but for distances exceeding this range, the error becomes larger.

[0047] Figure 19(B) is an example of a method designed to avoid such problems. In the illustrated example, a plurality of irradiation units 91 are arranged near the lens 92. Further, in one example shown, the irradiation units 91-1 to 91-4 as the irradiation units 91 are arranged at symmetric positions in the up, down, left, and right directions with the lens 92 as the center. Specifically, in the up and down directions of the lens 92, the irradiation units 91-1 and 91-3 are arranged at positions separated from the optical axis OA of the lens 92A by a distance dv. Further, in the left and right directions of the lens 92, the irradiation units 91-2 and 91-4 are arranged at positions separated from the optical axis OA of the lens 92A by a distance dh. In this case, it is necessary to use a plurality of irradiation units 91, which causes problems such as an increase in component costs, an increase in the size of the apparatus, and an increase in man-hours. Moreover, when using a plurality of irradiation units 91, simultaneously lighting a plurality of irradiation units 91 will simultaneously cause problems such as complication of wiring and heat treatment generated by the plurality of irradiation units 91.

[0048] Figure 20 FIG. is an example showing a case of generating point cloud data based on distance values obtained by a ToF camera system of the prior art. Referring to this figure, an example of three-dimensional information obtained in the case of the configuration shown will be described. The example shown is a view observed from the same angle as Figure 19 and, as the camera view point direction, is from the right direction to the left direction. Figure 18 The (A) of Figure 20 corresponds to the configuration of the (A) of Figure 19 and the (B) of Figure 20 corresponds to the configuration of the (B) of Figure 19 The white board WC is arranged in parallel with the ToF sensor 93, so the white board WC should be parallel to the axis AX. However, in the example shown in the (A) of Figure 20 , an inclination occurs in the up and down directions. Specifically, according to the example shown in the (A) of Figure 20 , in the upper direction (i.e., the position close to the irradiation unit 91), the measured distance of the ToF camera device 90 is short. Further, in the lower direction (i.e., the position far from the irradiation unit 91), the measured distance of the ToF camera device 90 is long. This is an error caused by the fact that the distance and time for the light emitted from the irradiation unit 91 to reach the upper side of the white board WC are different from those to reach the lower side. In particular, in a system that generates point cloud data from various angles using a plurality of ToF camera devices 90, due to such a deviation, the positions of the distance data between the cameras do not match, so it is difficult to perform synthesis. Further, in a system that generates point cloud data from various angles using a plurality of ToF camera devices 90, a problem of three-dimensional shape deformation obtained by synthesizing the distance data of a plurality of cameras sometimes occurs. In addition, in Figure 20In an example shown in (B) thereof, although such an error is not generated, a problem caused by using a plurality of irradiation units 91 as described above is generated.

[0049] [Embodiment 1]

[0050] This embodiment is used to solve the above problems. Hereinafter, with reference to Figures 1 to 6 Embodiment 1 will be described.

[0051] Figure 1 FIG. is a diagram for schematically explaining the distance information acquisition system of Embodiment 1. Referring to this figure, the outline of the distance information acquisition system 1 will be described. Hereinafter, the positional relationship and the like of the components included in the distance information acquisition system 1 may sometimes be represented by a three-dimensional orthogonal coordinate system of the x-axis, the y-axis, and the z-axis.

[0052] The distance information acquisition system 1 includes a distance information acquisition device 10 and an information processing device 20. The distance information acquisition device 10 includes an irradiation unit 11, a lens 12, a ToF sensor 13, a distance measurement unit 14, and an arithmetic unit 15. The distance information acquisition device 10 acquires the distance to the object OB. In an example shown in the figure, in order to illustrate the effect of the calibration operation of this embodiment, a white board WC is placed on the surface of the object OB. In addition, the white board WC is arranged in parallel with the light receiving surface of the ToF sensor 13.

[0053] The irradiation unit 11 includes a light emitting element capable of irradiating light such as laser, and irradiates the object OB with light. The laser irradiated by the irradiation unit 11 may be visible light or infrared light. The number of light emitting elements included in the irradiation unit 11 may also be plural. Specifically, the irradiation unit 11 may be a laser diode array such as a VCSEL. The irradiation unit 11 irradiates laser light in a direction perpendicular to the ToF sensor 13 (the z-axis direction in the figure) based on an instruction from the distance measurement unit 14. The light irradiated by the irradiation unit 11 is reflected by the object OB and enters the ToF sensor 13 via the lens 12. The lens 12 may specifically be an optical system lens. The distance measurement unit 14 measures the distance to the object OB based on the time required from when the light is irradiated from the irradiation unit 11 until the ToF sensor 13 receives the light.

[0054] The ToF sensor 13 has a plurality of pixels on a two-dimensional coordinate arranged in the vertical direction (the y-axis direction in the figure) and the lateral direction (the x-axis direction in the figure). Each pixel included in the ToF sensor 13 detects the timing of receiving light. The ToF sensor 13 outputs the detection information of each pixel in the two-dimensional coordinate to the distance measurement unit 14 while scanning. The distance measurement unit 14 calculates a distance value for each pixel and generates a distance image as two-dimensional array data. As a distance measurement method of the ToF sensor 13, a dToF method, an iToF method, etc. can be used. In the illustrated example, iToF is used. The distance image generated by the distance measurement unit 14 is subjected to a correction operation by the operation unit 15. The correction operation performed by the operation unit 15 is for correcting the calculation of the distance value generated due to the offset between the optical axis of the lens 12 and the position of the irradiation unit 11. The distance image generated by the distance measurement unit 14 is converted into three-dimensional point cloud data by the information processing device 20 after the correction operation by the operation unit 15. The information processing device 20 can also be an information processing device such as a personal computer or a tablet terminal.

[0055] In addition, in the present embodiment, the case where the function of the operation unit 15 is included in the distance information acquisition device 10 has been described, but the function equivalent to the operation unit 15 can also be included in the information processing device 20. Further, the function equivalent to the operation unit 15 can also be included in a device different from the distance information acquisition device 10 and the information processing device 20. That is, the structure included in the distance information acquisition device 10 and the structure having the function equivalent to the operation unit 15 included in the information processing device 20 can be used as the distance information acquisition device 10.

[0056] Figure 2 It is a functional block diagram showing the functional structure of the operation unit in Embodiment 1. Referring to this figure, an example of the functional structure of the operation unit 15 will be described. The operation unit 15 includes a distance value acquisition unit 151, a storage unit 152, and a correction operation unit 153. These respective functional units are implemented using an electronic circuit, for example. In addition, each functional unit may include storage units such as a semiconductor memory and a magnetic hard disk device inside as needed. Further, each functional unit can also be implemented by a computer and software.

[0057] The distance value acquisition unit 151 acquires a distance image DI from the distance measurement unit 14. The distance image DI includes the distance values (Depth values) of each point on the two-dimensional coordinate. This distance value can also be said to be the distance value before correction. This distance value is obtained based on the timing of light irradiation by the irradiation unit 11, the timing of the ToF sensor 13 receiving light, and the speed of light. The distance value acquisition unit 151 outputs the acquired distance image DI to the correction operation unit 153.

[0058] The storage unit 152 stores the static parameter PM and the arithmetic expression F. The static parameter PM is a static parameter related to the distance information acquisition device 10. The static parameter PM can also be a parameter based on the structure and configuration of the lens 12. The static parameter PM at least includes information related to the field of view angle of the lens 12. The information related to the field of view angle of the lens 12 can also be information related to the AFOV (Angular Field of View). The arithmetic expression F is a mathematical expression for correction calculation. Details of the correction calculation will be described later with reference to Figure 3 and Figure 4 describe the details of the correction calculation. In addition, the static parameter PM and the arithmetic expression F do not need to be stored in separate independent forms. For example, the static parameter PM can also be included in the arithmetic expression F. That is, the arithmetic expression F can also be a mathematical expression that includes the static parameter PM such as the AFOV as a parameter. In addition, the storage unit 152 does not necessarily need to be included in the arithmetic unit 15, and the arithmetic unit 15 can also be configured to obtain information including the static parameter PM and the arithmetic expression F from an external device.

[0059] The correction arithmetic unit 153 acquires the distance image DI from the distance value acquisition unit 151, and acquires the static parameter PM and the arithmetic expression F from the storage unit 152. The correction arithmetic unit 153 calculates the distance information to the object by applying the pre-correction distance value included in the acquired distance image DI to the arithmetic expression F stored in the storage unit 152. Specifically, the calculation performed by the correction arithmetic unit 153 is a calculation for correcting the pre-correction distance value acquired from the distance value acquisition unit 151 based on the structure, configuration, etc. of the lens 12. Therefore, the calculation performed by the correction arithmetic unit 153 is sometimes referred to as a correction calculation. The correction arithmetic unit 153 outputs the result of the correction calculation as the corrected distance image CDI to the information processing device 20.

[0060] Next, details of the correction calculation performed by the correction arithmetic unit 153 will be described with reference to Figure 3 and Figure 4 In addition, the correction arithmetic unit 153 does not need to perform all the correction calculations described below, and the correction calculations described below can also be pre-calculated and stored in the storage unit 152 as the arithmetic expression F. That is, the description with reference to Figure 3 and Figure 4 can also be said to be an explanation of the principle of correction performed by applying the arithmetic expression F. In addition, in Figure 3 and Figure 4 for simplicity of explanation, the scale, quantity, etc. are sometimes made different from those in the actual structure.

[0061] Figure 3This is the first diagram for explaining the operations performed by the arithmetic unit in Embodiment 1. In this diagram, the irradiation unit 11, the first substrate SB1, the lens 12, the ToF sensor 13, the second substrate SB2, the distance measurement unit 14, the flexible cable FL, and the object OB are shown. In the description of this diagram, sometimes the description is omitted by assigning the same reference numerals to the structures that have been described. Figure 1 or Figure 2 The description of the already described structures is omitted by assigning the same reference numerals.

[0062] In the illustrated example, the irradiation unit 11 is fixed to the first substrate SB1, and the ToF sensor 13 and the distance measurement unit 14 are fixed to the second substrate SB2. That is, the irradiation unit 11, the ToF sensor 13, and the distance measurement unit 14 are fixed to different substrates. The irradiation unit 11 and the distance measurement unit 14 are electrically connected by the flexible cable FL. The flexible cable FL transmits the electrical signal from the distance measurement unit 14 to the irradiation unit 11. Specifically, the distance measurement unit 14 outputs a light emission pulse LP to the irradiation unit 11. The light emission pulse LP contains information about the timing of irradiation by the irradiation unit 11. The light emission pulse LP may also be a control signal for directly irradiating the irradiation unit 11. The irradiation unit 11 irradiates light when it receives the light emission pulse LP. In addition, the distance measurement unit 14 outputs a shutter pulse SUB to the ToF sensor 13. The shutter pulse SUB is a pulse output at the timing when the ToF sensor 13 receives light. When the ToF sensor 13 receives the shutter pulse SUB, it detects the light irradiated by the irradiation unit 11 and reflected by the object OB. In addition, the distance measurement unit 14 outputs drive pulses and the like for scanning the detection results from a plurality of elements two-dimensionally arranged in the ToF sensor 13. The distance measurement unit 14 manages the timing from the light emission of the irradiation unit 11 to the reception of light by the ToF sensor 13.

[0063] Since both the distance measurement unit 14 and the ToF sensor 13 are arranged on the second substrate SB2, the propagation time of the shutter pulse SUB and the drive pulse from the distance measurement unit 14 to the ToF sensor 13 is about several hundred [psec (picoseconds)] to several [nsec (nanoseconds)]. Considering the distance measurement accuracy of the ToF sensor 13, this propagation time can be regarded as negligible. On the other hand, since the irradiation unit 11 is fixed to the first substrate SB1 different from the second substrate SB2 to which the distance measurement unit 14 is fixed, signals are transmitted via the flexible cable FL and the like. Therefore, in the signal transmission from the distance measurement unit 14 to the irradiation unit 11, a delay that cannot be ignored even considering the distance measurement accuracy of the ToF sensor 13 occurs. In addition, as a reason for having to fix the irradiation unit 11 to the first substrate SB1 different from the second substrate SB2, there are reasons related to the structure of the camera. Instead of the flexible cable FL, a thin coaxial cable or the like can be used, but in any case, a non-negligible delay will occur in the signal transmission from the distance measurement unit 14 to the irradiation unit 11.

[0064] In the example shown in the figure, the center of the effective image range of the ToF sensor 13 coincides with the optical axis of the lens 12. The distance between the optical axis of the lens 12 and the center position of the light emission surface of the irradiation unit 11 is recorded as a distance dV. Figure 1 As shown, in the case of a structure in which the irradiation unit 11 is located on the upper side of the ToF sensor 13, Figure 3 The left side is the upper side of the camera. Figure 3 The right side is the lower side of the camera. The ToF sensor 13 is usually located inside the camera body for the flange focal length of the lens 12. The distance between the light emitting surface of the irradiation unit 11 and the light receiving surface of the ToF sensor 13 is recorded as distance LD. The entrance pupil position is defined in the lens 12. In the illustrated model, the distance from the light receiving surface of the ToF sensor 13 to the entrance pupil position of the lens 12 is recorded as distance LI. In addition, the maximum field of view of the lens is defined according to the entrance pupil position. The maximum field of view of the lens is recorded as AFOV.

[0065] The distance from the entrance pupil position of the lens 12 to the optical axis direction of the object OB is recorded as distance AD. The distance from the optical axis to the lower side of the camera (right side in the figure) of the object OB is recorded as distance YD. In this case, the light irradiated from the irradiation unit 11 reaches the object OB at a distance Da. In addition, the light reflected by the object OB reaches the entrance pupil position of the lens 12 at a distance Db. Here, the distance from the entrance pupil position of the lens 12 to the position of the rear principal plane (Rear PrincipalPlane) of the lens is recorded as distance Dc, and the distance from the rear principal plane to the light receiving surface of the ToF sensor 13 is recorded as distance Dd.

[0066] As shown in the figure, the distance Dd is a distance with an angle relative to the ToF sensor 13 according to the angle of the incident main light. In the case of a small sensor such as 1 / 4 inch in size, the effective image range of the ToF sensor 13 is as small as H3.6[mm]×V2.7[mm]. In addition, in the case of a C-mount lens, the flange focal length is as short as 17.526[mm]. In the example shown in the figure, in order to simplify the explanation, the angle between the distance Dc and the distance Dd is ignored and processed as Dc+Dd≒LI.

[0067] In addition, the distance Dc and the distance Dd can be accurately calculated by finding the angle between the distance Dc and the distance Dd and calculating the distance Dd based on the distance from the rear principal plane to the light receiving surface of the ToF sensor 13 and the pixel position of the ToF sensor 13 corresponding to the position of the object OB. Detailed description of the calculation of the distance Dc and the distance Dd is omitted.

[0068] In the ToF camera device 90 in the prior art, generally, the time from when it is irradiated by the irradiation unit 91 until it is received by the ToF sensor 93 is multiplied by the speed of light C = 3×10^8 [m / s] and divided by 2, thereby calculating the distance. Here, dividing by 2 is to make the round-trip time from when it is irradiated by the irradiation unit 91 until it returns to the ToF sensor 93 be a one-way trip. That is, according to the prior art, the distance Da, the distance Db, the distance Dc, and the distance Dd as described above are regarded as the same distance for calculation. The reference of time in this model is based on the timing when the ranging unit 14 outputs the light emission pulse LP to the irradiation unit 11. The value obtained by multiplying the actual time from the generation timing of the light emission pulse LP until it is received by the ToF sensor 13 by the speed of light C becomes the ranging data. In this model, the distance is not simply obtained by dividing by 2 as in the prior art, but is obtained by the detailed calculation described later. Specifically, the distance is obtained by adding the propagation time of the flexible cable FL, the propagation time from the irradiation unit 11 to the object OB, the propagation time from the position of the object OB to the entrance pupil position of the lens 12, and the propagation time from the entrance pupil position of the lens 12 to the light receiving surface of the ToF sensor 13. The specific calculation is described in detail below.

[0069] First, when the transmission time of the flexible cable FL is set to Flt, Flt can be approximated by the following formula (1) according to the dielectric constant of the flexible cable FL. In addition, Ve is the propagation speed of the flexible cable FL, and Fl is the length of the flexible cable FL.

[0070] Flt = Fl / Ve…(1)

[0071] Here, if the speed of light is set to C and the dielectric constant of the flexible cable FL is set to ε, the propagation speed Ve of the flexible cable FL can be recorded as C / SQRT(ε). In addition, if the propagation time from the entrance pupil position of the lens 12 to the light receiving surface of the ToF sensor 13 is set to Lt, Lt can be approximated by the following formula (2).

[0072] Lt = (LI - Lk) / C + Lk / C×Lr…(2)

[0073] Here, Lk is the total thickness of the glass inside the lens, and Lr is the reflectivity of the glass.

[0074] If the propagation time from when it is irradiated by the irradiation unit 11 to the object OB is set to Dat, the propagation time from the object OB to the entrance pupil position of the lens 12 is set to Dbt, and the ranging time actually obtained by the ToF sensor 13 is set to TALL, then TALL can be obtained by the following formula (3).

[0075] TALL = Flt + Lt + Dat + Dbt…(3)

[0076] Moreover, if the measured distance calculated by the distance measurement unit 14 (i.e., the distance measured by the prior art, i.e., the distance value including errors) is set as Depth, then Depth can be obtained by the following formula (4).

[0077] Depth = TALL × C / 2…(4)

[0078] The Depth value shown in formula (4) includes the distance difference between Fl, Da, and Db. However, the actual physical distance of the object OB is LI + Db in the figure. Therefore, the distance shown in formula (4) is different from the actual physical distance and includes errors. Here, since the Fl value is known according to the length of the flexible cable FL, if the distances of Da and Db are accurately obtained, then LI + Db can be accurately obtained. Therefore, if the distance obtained by adding Da and Db is set as Dab, then the distance Dab can be obtained by the following equation.

[0079] Dab = Da + Db…(5)

[0080] Dab = (TALL - Flt - Lt) × C…(6)

[0081] Figure 4 It is the second figure for explaining the calculation performed by the arithmetic unit in Embodiment 1. This figure is for converting the actual distance into the number of LINEs of the ToF sensor 13 (number of lines, hereinafter, also only recorded as LINE conversion (line conversion)). In this figure, the relationship between AVL and AL representing the position of the object OB using the unit of the LINE number (line number) of the ToF sensor 13 is used to record the distances YD, AD, and θ1 representing the position of the actual object OB. The triangle composed of YD, AD, and θ1 can be conceptually understood as a triangle similar to the triangle composed of AVL, AL, and θ1.

[0082] Refer to Figure 4 , and the method for obtaining θ1 will be described. Let the LINE number of V of the HV pixel captured by the ToF sensor 13 with respect to the position of the object OB be AV. When the center line of the effective pixels is set as the center VC and the number of lines of AV from this center position is set as AVL, AVL can be represented by the following formula (7).

[0083] AVL = AV - VC…(7)

[0084] Here, for the maximum number of LINEs of the ranging points that can be the maximum field of view angle from the AFOV, the distance from the center VC is the largest when counted from the center VC. The position of the object OB for making AVL equal to VC is the field of view angle of the AFOV, which is the maximum field of view angle with respect to the lens 12 of the camera. This AFOV is determined by the lens. Generally, when the aspect ratio of the effective pixels of the ToF sensor 13 is different in the vertical and horizontal directions, such as 4:3, the field of view angles in the vertical and horizontal directions change. Therefore, the AFOV in the vertical direction and the AFOV in the horizontal direction are defined. For the AFOV processed here, when the irradiation unit 11 is offset in the vertical direction with respect to the lens optical axis, the AFOV in the vertical direction is applied to the AFOV, and when it is offset in the horizontal direction, the AFOV in the horizontal direction is applied to the AFOV. In the illustrated example, since it is offset in the vertical direction, the AFOV in the vertical direction is input.

[0085] If the position of the object OB at the maximum field of view angle is considered by line conversion and the distance AD is set as AL obtained by the ratio of line conversion, then AL can be described by Equation (8) according to the relationship between VC and the right triangle formed by the angle AFOV / 2.

[0086] AL = VC / TAN(AFOV / 2)…(8)

[0087] Therefore, the angle θ1 can be described by the following Equation (9).

[0088] θ1 = ATAN(AVL / AL)…(9)

[0089] Using θ1 described in Equation (9), the actual distance AD is obtained. AD, which is the ratio of AL, can be described by the following Equation (10).

[0090] AD = Db×COS(θ1)…(10)

[0091] Similarly, YD, which is the ratio of AVL, can be described by the following Equation (11).

[0092] YD = Db×SIN(θ1)…(11)

[0093] Return Figure 3 , if the distance in the z direction between the irradiation surface of the irradiation unit 11 and the light-receiving surface of the ToF sensor 13 is set as LD, then Da can be described by the following Equation (12). In addition, Lz = LD - LI.

[0094] Da = SQRT((AD - Lz)^2+(YD + dv)^2)…(12)

[0095] By solving Da from the simultaneous equations of equation (12) and equation (5), the following equation (13) can be derived.

[0096] Da=(Dab^2+Lz^2+dv^2-2×Lz×Dab×COS(θ1)+2×dv×Dab×SIN(θ1)) / (2×Dab-2×Lz×COS(θ1)+2×dv×SIN(θ1))…(13)

[0097] Therefore, Db can be derived from equation (14) based on equation (5).

[0098] Db=Dab-Da…(14)

[0099] In addition, Dab can derive equation (15) from equation (4) and equation (6).

[0100] Dab=Depth×2-(Flt-Lt)×C…(15)

[0101] Based on these calculations, if the distance after correction is obtained as Dcr, Dcr can be derived as the following formula (16).

[0102] Dcr=Db+LI

[0103] =Dab-(Dab^2+Lz^2+dv^2-2×Lz×Dab×COS(θ1)+2×dv×Dab×SIN(θ1))

[0104] / (2×Dab-2×Lz×COS(θ1)+2×dv×SIN(θ1))+LI…(16)

[0105] Here, Dab, θ1, and Lz are

[0106] Dab=Depth×2-(Flt-Lt)×C

[0107] θ1=ATAN(AVL / AL)=ATAN(AVL / (VC / TAN(AFOV / 2)))

[0108] Lz=LD-LI

[0109] In Equation (16), Flt, Lt, LD, LI, dv, VC, and AFOV are static parameters determined by the structures of the camera and the lens. That is, these parameters are stored as static parameter PM in the storage unit 152. In addition, Equation (16) is an example of the arithmetic expression F. The static parameter PM can be preset to a fixed value. The storage unit 152 may store, instead of the static parameter PM and the arithmetic expression F, a mathematical expression in which the static parameter PM is set to a fixed value. By substituting the dynamic parameters of the Depth value obtained by converting the measurement time obtained from the ToF sensor 13 by the distance measurement unit 14 for the position of the object OB and the line number AVL of the pixels of the ToF sensor 13 into this mathematical expression (Equation (16) with fixed values substituted), the correct distance with the position offset between the lens optical axis and the irradiation unit 11 corrected for each pixel of the automatic scan can be obtained. The correction arithmetic unit 153 obtains the Depth value and the line number AVL from the distance image DI, substitutes them into this mathematical expression, and thereby calculates the corrected distance, which is output as the corrected distance image CDI to the information processing device 20.

[0110] In addition, looking at Equation (16), the value of the correction value changes according to the field of view angle of the AFOV of the lens and the angle of the ToF sensor 13 with respect to the scanned object pixel with respect to the ranging distance. Also, looking at Equation (16), it can be seen that the correction is not a correction at a fixed value, and the correction amount varies spatially. Flt in Equation (16) is the propagation time based on the delay of the flexible cable FL, but the light emission pulse LP output from the distance measurement unit 14 can be advanced by Flt in advance, so that the delay time without Flt can be set. In this case, Flt in Equation (16) can be set to 0.

[0111] Figure 5 is a diagram showing an example of three-dimensional information after correction calculation by the distance information acquisition device of Embodiment 1. Referring to this diagram, an example of the case of generating point cloud data based on the result of correction calculation by the distance information acquisition device 10 will be described. In an example shown in this diagram, instead of the white board WC, a plane diagram with grid lines is used. This diagram is a view from directly in front of the diagram from the left direction, and the point cloud in the diagram is displayed at an angle where the line of sight of the camera observes from left to right on the diagram. An example shown in the diagram is the result when the distance between the irradiation unit 11 and the center of the lens optical axis is 0.07 [m (meter)] and the distance to the center of the object OB is 0.6 [m]. Figure 5 (A) is an example before correction, Figure 5 (B) is an example when it is set to point cloud data after correction calculation for each pixel by Equation (16). As Figure 5 (A) shown, it can be seen that it is inclined with the upper part of the screen being closer and the lower part being farther before correction. However, as Figure 5 (B) shown, it can be seen that it becomes straight and the inclination has been eliminated after correction.

[0112] Figure 6 This is a diagram for explaining a modified example of the distance information acquisition system of Embodiment 1. Referring to this diagram, a modified example of the distance information acquisition system 1 will be described. In the above example, the case where the position of the irradiation unit 11 is in the direction perpendicular to the optical axis of the lens 12 was described. In a modified example of the distance information acquisition system 1, an example is the case where the position of the irradiation unit 11 is further in the horizontal direction with respect to the optical axis of the lens 12. The vertical offset is denoted as dv, and the horizontal offset is denoted as dh.

[0113] The horizontal offset can be obtained by replacing AVL in Equation (16) with the pixel number AHP scanned in the horizontal direction and setting dv as dh. In the illustrated example, the arithmetic unit 15 includes a vertical correction arithmetic unit 15A and a horizontal correction arithmetic unit 15B. The vertical correction arithmetic unit 15A performs a calculation using Equation (16) for the vertical position offset and outputs the corrected distance as Dcrv. The horizontal correction arithmetic unit 15B performs a calculation by further replacing AVL in Equation (16) with the pixel number AHP and replacing dv with dh for the horizontal position offset with respect to Dcrv output from the vertical correction arithmetic unit 15A. Finally, the arithmetic unit 15 can obtain Dcr corrected in both the vertical and horizontal directions. At this time, the AFOV in Equation (16) uses the vertical AFOV for the vertical correction arithmetic unit 15A and the horizontal AFOV for the horizontal correction arithmetic unit 15B.

[0114] [Summary of Embodiment 1]

[0115] According to the embodiment described above, the distance information acquisition device 10 irradiates the object OB with light by including the irradiation unit 11, and receives the light irradiated by the irradiation unit 11 and reflected by the object OB via the optical system lens by including the ToF sensor 13. The distance information acquisition device 10 acquires the distance information of the object OB by including the irradiation unit 11 and the ToF sensor 13. The distance information acquisition device 10 further includes a distance value acquisition unit 151, and thereby acquires a distance value (i.e., the Depth value before correction) based on the timing of irradiation by the irradiation unit 11, the timing of light reception by the ToF sensor 13, and the speed of light. In addition, the distance information acquisition device 10 stores a prescribed mathematical formula (i.e., the above formula (16) substituting static parameters) including at least information related to the field of view angle of the optical system lens (i.e., AFOV) as a parameter by including the storage unit 152. In addition, the distance information acquisition device 10 calculates the distance information to the object OB by applying the acquired distance value to the prescribed mathematical formula stored in the storage unit 152 by including the correction calculation unit 153. The prescribed mathematical formula includes information related to the correction calculation based on the distance from the irradiation unit 11 to the object OB and the distance from the object OB to the ToF sensor 13. According to the present embodiment, by performing a correction calculation on the result obtained from the distance measurement unit 14, the distance to the object can be measured with high accuracy. In addition, according to the present embodiment, since a correction calculation is performed on the results obtained from the conventional ToF sensor 13 and the distance measurement unit 14, the structure for additionally performing a correction calculation on the conventional technology enables easy correction of the measured distance.

[0116] [Embodiment 2]

[0117] Next, with reference to Figure 7A description is given of Embodiment 2. In Embodiment 2, in addition to the structure of Embodiment 1 described above, correction calculation is performed in consideration of distance calibration, which is different from Embodiment 1 in this regard. By performing the correction calculation as described above, it is logically possible to correct errors with high precision, but the actually obtained values sometimes still contain errors. As a main cause of further errors, it is possible to cite the blunting of the waveform of the pulsed light irradiated from the irradiation unit 11 such as a VCSEL. As an ideal pulsed light, a rectangular shape is preferred, but in reality, depending on the performance of the driver of the irradiation unit 11, the waveform of the pulsed light sometimes blunts rather than being rectangular, or ringing occurs. Due to these waveform blunts, errors sometimes occur in the measurement results. In Embodiment 2, the difference between the actual distance and the distance obtained from the ToF sensor 13 is measured in the calibration mode in advance, and the error is stored in a storage area such as an EEPROM (Electrically Erasable Programmable Read-Only Memory), and the result of the correction calculation performed by the arithmetic unit 15 is further corrected.

[0118] That is, according to the distance information acquisition device 10 according to Embodiment 2, a correction value storage unit (for example, an EEPROM) is further included, and thus the correction value obtained as a result of calibration is stored. Specifically, the correction value may also be information related to the difference between the result of actually measuring the distance to the object OB and the distance information to the object calculated by the arithmetic unit 15. The arithmetic unit 15 calculates the distance information to the object OB based on the correction value stored in the correction value storage unit.

[0119] Figure 7 FIG. is a diagram for explaining the outline of calibration in Embodiment 2. Referring to this figure, the specific procedure of calibration is described. In calibration, first, while varying the distance between the distance information acquisition device 10 and the object OB, the correspondence between the actual distance (theoretical value) and the distance (measured value) measured by the distance information acquisition device 10 including correction calculation is obtained. Specifically, when the ranging range based on the distance information acquisition device 10 is 1 to 2 [m], the ranging range of 1 to 2 [m] is divided into 10 equal parts, and while moving the distance between the distance information acquisition device 10 and the object OB at 10 [cm] intervals, this range is photographed to obtain the correspondence between the theoretical value and the measured value. This correspondence is stored as a correction value in a correction value storage unit such as an EEPROM. Hereinafter, the act of obtaining a correction value based on the theoretical value and the measured value is sometimes referred to as calibration.

[0120] The specific calibration sequence will be described. As shown in the figure, an object OB is arranged on the optical axis of the lens 12 (i.e., YD = 0). At 10 locations with an interval of 10 [cm] between values of AD + LI in the range of 1 to 2 [m], Depth values (i.e., the Depth values before calibration) are obtained from the ToF sensor 13. The obtained Depth values are substituted into Equation (16) to calculate the calibrated distance Dcr. The difference between the calibrated distance Dcr and the actual physical distance of AD + LI is set as Δe. The correspondence between the distance and Δe is stored in the calibration value storage unit. Δe is obtained by Δe = physical distance AD + LI - Dcr.

[0121] Next, the calculation when exiting the calibration mode and actually performing distance measurement in the distance measurement mode will be described. First, the distance information acquisition device 10 reads calibration values at 10 [cm] intervals close to the measured distance from the calibration value storage unit and performs addition calculation to correct the distance. When the position of the distance information acquisition device 10 is at a position between 10 [cm] intervals, it is also possible to select calibration values of 2 points close to this distance from the calibration value storage unit, and interpolate the calibration values by linear interpolation or the like to obtain the calibration value of the distance between 10 [cm] intervals. In such a calibration case, it is also possible to perform it at the center point of the effective screen.

[0122] The specific process in the distance measurement mode will be described. First, based on the Depth value before calibration obtained by distance measurement, Δe corresponding to the Depth value before calibration obtained by distance measurement is read from the Δe at 10 [cm] intervals stored in the calibration value storage unit to obtain the calibration value ΔE to be applied. For the distance between 10 [cm] intervals, 2 points close to Δe are read, and linear interpolation is performed in proportion to the interval to obtain the calibration value ΔE to be applied. In the second embodiment, it is possible to add the calibration value ΔE to the above Equation (16) and derive the following Equation (17).

[0123] Dcr′ = Db + LI + ΔE

[0124] = Dab - (Dab^2 + Lz^2 + dv^2 - 2 × Lz × Dab × COS(θ1) + 2 × dv × Dab × SIN

[0125] (θ1)) / (2 × Dab - 2 × Lz × COS(θ1) + 2 × dv × SIN(θ1)) + LI + ΔE… (17)

[0127] Here, Dab, θ1, and Lz are respectively

[0128] Dab = Depth × 2 - (Flt - Lt) × C

[0129] θ1 = ATAN(AVL / AL) = ATAN(AVL / (VC / TAN(AFOV / 2)))

[0130] Lz = LD - LI

[0131] In Embodiment 2, the correction operation unit 153 performs a correction operation by using Equation (17) instead of Equation (16).

[0132] [Summary of Embodiment 2]

[0133] According to the embodiment described above, the distance information acquisition device 10 further includes a correction value storage unit, and thereby stores correction values obtained as a result of calibration. Specifically, the correction value may also be information related to the difference between the result of actually measuring the distance to the object OB and the distance information to the object calculated by the arithmetic unit 15. The arithmetic unit 15 calculates the distance information to the object OB based on the correction value stored in the correction value storage unit. Therefore, according to the present embodiment, it is possible to measure the distance to the object with high accuracy in consideration of the error caused by the blunting of the waveform of the pulsed light irradiated from the irradiation unit 11 such as VCSEL.

[0134] In addition, in Embodiment 2, since a correction value is added to Equation (16), it is possible to obtain an accurate distance even for an object OB that is not of a predetermined shape, and also even when the object OB exists outside the center of the screen. In this regard, it is particularly different from the prior art.

[0135] In addition, in the above description, the case where the distance information acquisition device 10 performs calibration to correct the error based on the blunting of the waveform has been described. However, the error corrected by calibration may not be an error based on the blunting of the waveform. For example, as an error other than the error based on the blunting of the waveform, it is possible to exemplify errors caused by assembly deviations of each individual such as the installation position and angle of the irradiation unit 11 and the ToF sensor 13, and the environment such as the temperature at which the distance information acquisition device 10 is used. In Embodiment 2, by calibrating each distance information acquisition device 10, it is also possible to correct these various individual-specific errors.

[0136] [Embodiment 3]

[0137] Next, refer to Figure 8 Embodiment 3 will be described. Embodiment 3 is an embodiment that takes into account a zoom lens. In the above-described embodiment, the distance information acquisition device 10 performs a correction operation based on the arithmetic expression F of the static parameters PM including the field of view angle AFOV, the incident pupil position LI, etc. The lens 12 is premised on a single-focus lens with a fixed field of view angle AFOV, incident pupil position LI, etc., but in Embodiment 3, it is different from the above-described embodiment in that it performs calculations considering a zoom lens.

[0138] Figure 8 This is a diagram for explaining the functional structure of the distance information acquisition system in Embodiment 3. Referring to this diagram, the distance information acquisition system 1C will be described. The difference between the distance information acquisition system 1C and the distance information acquisition system 1 is that it includes a distance information acquisition device 10C instead of the distance information acquisition device 10. In the description of the distance information acquisition system 1C, sometimes the description will be omitted by labeling the same structural components as those in the distance information acquisition system 1 with the same reference numerals.

[0139] In the distance information acquisition device 10C, a lens 12C which is a zoom lens with variable focal length is used instead of the lens 12 which is a single - focus lens. In addition, the distance information acquisition device 10C further includes a lens control unit 31 and a lens information storage unit 33. The lens control unit 31 controls the magnification of the lens 12C according to the indication of the ZOOM position (zoom position) from the information processing device 20. The lens control unit 31 controls the focal length (magnification) of the lens 12C through automatic control or manual control. The lens control unit 31 outputs the information related to the focal length of the lens 12C to the lens information storage unit 33. In the lens information storage unit 33, the focal length of the lens 12C and the corresponding field - of - view angle are stored in association with each other. Specifically, the lens information storage unit 33 stores the arrangement data corresponding to the field - of - view angle AFOV and the entrance pupil position LI respectively. This arrangement data corresponds to the specifications of the lens 12C, the ZOOM position signal, the size of the effective area of the ToF sensor 13, etc. The field - of - view angle AFOV stored in the lens information storage unit 33 becomes the vertical AFOV when the irradiation unit 11 is vertically offset with respect to the lens optical axis, and becomes the horizontal AFOV when horizontally offset. This arrangement data has the values of the field - of - view angle AFOV and the entrance pupil position LI at regular intervals. The regular interval can also be an interval obtained by dividing the interval between the wide - angle end and the telephoto end of the zoom ratio that the lens 12C can set at an arbitrary interval. For example, in a 2 - fold zoom lens with a focal length of 8 - 16 [mm], when the value of the ZOOM signal indicating the ZOOM position is 8 [BIT], with 8 [mm] set to 0 and 16 [mm] set to 255 at a resolution of 256, the lens information storage unit 33 stores 256 field - of - view angles AFOV and entrance pupil positions LI. Whenever the magnification is made variable by the lens control unit 31, the lens information storage unit 33 outputs the field - of - view angle AFOV and the entrance pupil position LI corresponding to this ZOOM position to the arithmetic unit 15.

[0140] In the following description, the lens information storage unit 33 is also referred to as the field of view angle information storage unit. The lens information storage unit 33 outputs information related to the lens at that focal length (specifically, the angular field of view AFOV and the entrance pupil position LI) to the arithmetic unit 15 based on information related to the focal length of the lens 12C from the lens control unit 31. The correction arithmetic unit 153 calculates the distance information to the object OB based on the angular field of view corresponding to the focal length of the lens 12C (specifically, AFOV and LI). More specifically, the correction arithmetic unit 153 substitutes the angular field of view AFOV and the entrance pupil position LI corresponding to the ZOOM position output from the lens information storage unit 33 as dynamic variables into Equation (16) or Equation (17) to obtain the corrected distance as Dcr.

[0141] In addition, when the lens 12C is a zoom lens capable of electrically changing the zoom position (i.e., in the case of automatic control), the lens control unit 31 outputs a ZOOM signal indicating the zoom position to the lens 12C. When the ZOOM signal is output from the lens control unit 31 to the lens 12C, the lens 12C is set to the focal length corresponding to that ZOOM position. Further, when the lens 12C is a zoom lens whose zoom position is manually changed (i.e., in the case of manual control), the zoom position is manually input through the information processing device 20, and the lens control unit 31 acquires the ZOOM position from the information processing device 20.

[0142] [Summary of Embodiment 3]

[0143] According to the embodiment described above, the lens 12C may also be a zoom lens with a variable focal length. In addition, the distance information acquisition device 10C further includes a lens information storage unit 33, thereby storing the focal length of the lens 12C and the angular field of view corresponding to that focal length in association with each other. Further, the arithmetic unit 15 of the distance information acquisition device 10C calculates the distance information to the object OB based on the angular field of view corresponding to the focal length of the lens 12C. Therefore, according to the present embodiment, even when the lens 12C is a zoom lens and the zoom position can be changed, the distance to the object can be measured with high accuracy.

[0144] [Embodiment 4]

[0145] Next, Figure 9 Embodiment 4 will be described. Embodiment 4 is an embodiment that takes into account interchangeable lenses. In the above embodiments, embodiments in which the single-focus lens or the zoom lens is fixed were described. However, in actual use, interchangeable lenses are sometimes used. By using interchangeable lenses, lenses with different adjustable focal length ranges can be installed, further improving the versatility. Therefore, in Embodiment 4, it is different from other embodiments in that interchangeable lenses (including both single-focus lenses and zoom lenses) can be used.

[0146] Figure 9 This is a diagram for explaining the functional structure of the distance information acquisition system in Embodiment 4. Referring to this diagram, the distance information acquisition system 1D will be described. The difference between the distance information acquisition system 1D and the distance information acquisition system 1C is that it includes a distance information acquisition device 10D instead of the distance information acquisition device 10C. In the description of the distance information acquisition system 1D, sometimes the description will be omitted by labeling the same structural components as those in the distance information acquisition system 1C with the same reference numerals.

[0147] In the distance information acquisition device 10D, a replaceable lens on which any one of a plurality of lenses is mounted is used. In an example shown in the figure, the replaceable lens is denoted as lens 12D. The lens 12D is attached to the distance information acquisition device 10D via a lens mounting portion 121. The lens mounting portion 121 has electrical contact points on the lens and the bayonet, for example, and can acquire lens identification information from the lens. The lens control unit 31 also has a lens identification information acquisition unit (not shown), and thereby acquires the lens identification information from the lens 12D via the lens mounting portion 121. The lens identification information identifies the type of the optical system lens mounted. The lens identification information identifies the lens based on, for example, the model number of the lens. The lens control unit 31 outputs the acquired lens identification information to the information processing device 20.

[0148] The distance information acquisition device 10D includes a lens information storage unit 33D instead of the lens information storage unit 33. The lens information storage unit 33D stores in association the focal length (zoom signal), the corresponding angle of view AFOV, and the entrance pupil position LI for each of the plurality of lenses according to their model numbers, etc. In addition, the lens information storage unit 33D may store information such as the thickness of different lenses and the reflectivity of the lens glass for each lens. The arithmetic unit 15 acquires information related to the angle of view AFOV and the entrance pupil position LI corresponding to the focal length of the lens 12D from the lens information storage unit 33D, and calculates the distance information to the object OB based on the acquired angle of view, where the lens 12D is identified based on the lens identification information acquired by the lens identification information acquisition unit.

[0149] In addition, depending on the mounting shape supported by the lens mounting portion 121, there may be no electrical contact points. For example, most C-mount lenses may not have electronic contacts. In the absence of electronic contacts, the model number of the lens cannot be acquired from the lens 12D. Therefore, for lenses for which the model number cannot be acquired, the information processing device 20 can input the model number of the mounted lens in association with an INDEX number (index number), and send the model number to the lens identification information acquisition unit included in the lens control unit 31.

[0150] The lens information storage unit 33D can also obtain information on the angle of view AFOV and the incident pupil position LI that vary for each lens from the information processing device 20. The lens information storage unit 33D can also store information on a plurality of lenses (e.g., N types of lenses) that can be replaced. In an example shown in the figure, information related to three different lenses is stored as the first lens information, the second lens information, and the third lens information. For example, INDEX can be assigned from 1 to 3 in these 3 storage areas, and when writing lens data from the information processing device 20, the INDEX number can be specified for writing. If the writing is completed, the INDEX number corresponding to the lens mounted on the lens mounting portion 121 can also be specified from the information processing device 20. When the lens 12D is replaced with a lens different from the currently mounted lens, the angle of view AFOV and the incident pupil position LI corresponding to the replaced lens (if the lens is a zoom lens, the zoom position of the lens) are changed, and the arithmetic unit 15 performs a calculation based on Equation (16) or Equation (17) to obtain the corrected distance Dcr.

[0151] In addition, in the case of more strict correction, Dab in Equation (16) or Equation (17) can also calculate the propagation time from the incident pupil position LI of the lens to the light receiving surface of the ToF sensor 13 according to Lt as shown in Equation (6). Lt is calculated according to the thickness of the lens 12D, Lk, and the reflectance Lr as shown in Equation (2). Therefore, Lk and Lr can also be used as dynamic parameters for replacing the lens.

[0152] [Summary of Embodiment 4]

[0153] According to the embodiment described above, the lens 12D can also be a replaceable lens on which any one of a plurality of lenses is mounted. The lens control unit 31 also has a lens identification information acquisition unit (not shown) for identifying the mounted lens 12D. The lens identification information acquisition unit can obtain the identification information of the lens 12D from the lens 12D via an electrical contact point, or can obtain the identification information of the lens through the operation of the user. The lens information storage unit (angle of view information storage unit) 33D stores the focal length, the angle of view AFOV corresponding to the focal length, and the incident pupil position LI in association with each other for a plurality of lenses 12D as replaceable lenses. The arithmetic unit 15 obtains the angle of view AFOV and the incident pupil position LI corresponding to the focal length of the lens from the lens information storage unit 33D, and calculates the distance information to the object OB based on the obtained angle of view. The lens is identified based on the lens identification information obtained by the lens identification information acquisition unit. Therefore, according to this embodiment, the lens 12D is a replaceable lens, and even when it is replaced with a different lens, the distance to the object can be measured with high accuracy.

[0154] [Embodiment 5]

[0155] Next, a description will be given with reference to Figures 10 to 14 Embodiment 5. First, the problems to be solved in Embodiment 5 will be described. As described in the fourth embodiment, according to this embodiment, a plurality of zoom lenses of different types can be used. Here, the lengths of the bodies of the zoom lenses are different from each other depending on the lenses. Generally, since a zoom lens has a large number of constituent lenses, its shape tends to be long. When the irradiation unit 11 is fixed to the lens mounting unit 121, there is a problem that the light irradiated by the irradiation unit 11 collides with the lens barrel of the lens.

[0156] Figure 10 FIG. is a diagram for explaining the problems to be solved by the distance information acquisition device of Embodiment 5. The problems to be solved by the distance information acquisition device 10E according to Embodiment 5 will be described in detail with reference to the accompanying drawings. As shown in the figure, when light is irradiated at an irradiation angle α from the irradiation unit, depending on the type of the zoom lens, the lens barrel of the lens may enter the irradiation angle and block the light. In an example shown in the figure, at position P, the lens barrel of the lens enters the irradiation angle and blocks the light. In such a case, there is a problem that although the object OB exists within the irradiation angle, the light cannot reach it, and accurate distance measurement cannot be performed. Therefore, in Embodiment 5, the purpose is to solve such a problem.

[0157] Figure 11 FIG. is a diagram for explaining the structure of the distance information acquisition device of Embodiment 5. Referring to this figure, an example of the structure of the distance information acquisition device 10E will be described. In the description of the distance information acquisition device 10E, the description may sometimes be omitted by assigning the same reference numerals to the same structures as those of the distance information acquisition device 10. The distance information acquisition device 10E includes an irradiation unit 11E instead of the irradiation unit 11 and includes a lens 12E instead of the lens 12. The lens 12E is an example of a lens whose lens barrel is long in the optical axis direction. The irradiation unit 11E is configured to be slidable in the optical axis direction (the direction indicated by the double-headed arrow AR). By moving the irradiation unit 11E in the optical axis direction of the lens 12E, the distance between the irradiation unit 11E and the ToF sensor 13 changes. As a mechanism for sliding the irradiation unit 11E in the optical axis direction, for example, a rod 111, a set screw 112, and a rod 113 can be exemplified. As shown in the figure, the irradiation unit 11E is supported by two rods 111. The irradiation unit 11E moves in the optical axis direction along the rod 111. The position of the irradiation unit 11E is fixed by the set screw 112 and the rod 113.

[0158] It is also possible to attach a scale like a ruler to the two rods 111. By referring to this scale, the user can grasp how much the irradiation unit 11E has moved. This scale can also be, for example, the distance from the light receiving surface of the ToF sensor 13 to the irradiation surface of the irradiation unit 11E.

[0159] Here, when moving the irradiation unit 11E in the optical axis direction, the movable range of the irradiation unit 11E is considered to be restricted by the flexible cable FL. Therefore, when moving the irradiation unit 11E to the most forward position (i.e., the position farthest from the light receiving surface of the ToF sensor 13) by the rod 111, the length of the flexible cable FL is preferably a fixed length with a certain degree of surplus so as not to be stretched.

[0160] Figure 12 FIG. is a diagram for explaining the functional structure of the distance information acquisition device according to Embodiment 5. The functional configuration of the distance information acquisition device 10E will be described in more detail with reference to the accompanying drawings. The user moves the irradiation unit 11E in the direction of arrow AR to a position where the light of the irradiation unit 11E is not blocked by the lens 12E according to the length of the lens 12E in the optical axis direction. After the user moves the irradiation unit 11E to an appropriate position, the position of the irradiation unit 11E is fixed by operating the set screw 112 and the rod 113. After the user fixes the position of the irradiation unit 11E, the scale marked on the rod 111 is read, and the value of the read scale and the model of the lens 12E at this time are input to the information processing device 20.

[0161] The information processing device 20 stores the LD value of each lens 12E, and associates and stores the model of the lens 12E with the LD value. The information processing device 20 grasps the length of LD based on the input scale value and the model of the lens 12E at this time. The model of the lens 12E and the LD value corresponding to the lens 12E are displayed, for example, on the display unit (not shown) of the information processing device 20 as shown. The user can also make a selection, for example, by a radio button (the left column in the figure). Returning to Figure 13 FIG., the information processing device 20 outputs the LD value of the selected lens to the arithmetic unit 15. The arithmetic unit 15 substitutes the LD value obtained from the information processing device 20 as a dynamic parameter into Equation (16) or Equation (17), thereby obtaining the corrected distance Dcr. In other words, the arithmetic unit 15 calculates the distance information to the object OB based on the distance between the irradiation unit 11E and the ToF sensor 13 obtained from the information processing device 20. Figure 12 In addition, in the above example, the LD value of each lens 12E is stored in the information processing device 20, but this LD value can also be stored in a memory such as an EEPROM (not shown) included in the distance information acquisition device 10E. In this case, it can be configured to make a selection as shown from the menu screen of the information processing device 20 or the distance information acquisition device 10E.

[0162] Figure 13

[0163] Figure 14 ​​This is a diagram showing the structure of a modified example of the distance information acquisition device according to Embodiment 5. Referring to this diagram, the structure of the modified example of the distance information acquisition device 10E will be described. The modified example of the distance information acquisition device 10E is different from the distance information acquisition device 10E in that it includes an irradiation unit 114. A modified example of the distance information acquisition device 10E is an example of a case where a plurality of irradiation units 11E are included. In this case, the plurality of irradiation units 11E are provided in the irradiation unit 114. In the illustrated example, as an example of the plurality of irradiation units 11E, it includes irradiation units 11E-1 to 11E-4. The irradiation units 11E-1 to 11E-4 are arranged above, below, left, and right, for example, centered on the optical axis of the lens 12E. As the irradiation unit 114, by sliding the member surrounding the lens, problems such as flexure caused by the individual sliding of the irradiation unit 11E can also be solved. Even in the case of suppressing inclination by arranging above, below, left, and right, the object to be distance-measured may not exist at the center of the optical axis. When the object does not exist at the center of the optical axis, the distance from the irradiation by the irradiation unit to the reflection by the object is different from the distance from the reflection by the object to the incidence on the light-receiving unit, and it can be corrected by Equation (16) or Equation (17).

[0164] [Summary of Embodiment 5]

[0165] According to the embodiment described above, as the irradiation unit 11E moves in the optical axis direction of the lens 12E, the distance between the irradiation surface of the irradiation unit 11E and the light-receiving surface of the ToF sensor 13 changes. The arithmetic unit 15 calculates the distance information to the object OB based on the changed distance between the irradiation surface of the irradiation unit 11E and the light-receiving surface of the ToF sensor 13. Therefore, according to the present embodiment, even when a long lens 12E with a lens barrel is installed as an interchangeable lens, by sliding the irradiation unit 11E, problems such as the irradiation angle of the irradiation unit 11E being blocked by the lens 12E can be solved, and the distance to the object OB can be measured with high accuracy.

[0166] [Embodiment 6]

[0167] Next, referring to Figure 15 and Figure 16 Embodiment 6 will be described. Embodiment 6 is an embodiment in which an RGB image can also be acquired. According to the present embodiment, the ToF sensor 13 and the RGB sensor 18 are arranged on substantially the same optical axis.

[0168] Figure 15This is a diagram for explaining the functional structure of the distance information acquisition device according to Embodiment 6. Referring to this diagram, the functional structure of the distance information acquisition device 10F will be described. The distance information acquisition system 1F shown in this diagram is a modified example of the distance information acquisition system 1D. The distance information acquisition system 1F is different from the distance information acquisition system 1D in that it includes a distance information acquisition device 10F instead of the distance information acquisition device 10D. The distance information acquisition device 10F is different from the distance information acquisition device 10D in that it further includes a prism 16, a visible light reflective dichroic film 17, an RGB sensor 18, and an RGB processing unit 19. In the description of the distance information acquisition system 1F, the description may sometimes be omitted by assigning the same reference numerals to the same structures as those in the distance information acquisition system 1D.

[0169] The prism 16 is disposed between the lens mounting portion 121 and the ToF sensor 13. The light from the lens 12D is incident on the prism 16. The light incident on the prism 16 is emitted to either the ToF sensor 13 or the RGB sensor 18.

[0170] The visible light reflective dichroic film 17 reflects visible light and transmits light with a wavelength in the near-infrared region or above (i.e., infrared light). The visible light in the light incident on the prism 16 is reflected by the visible light reflective dichroic film 17 in the direction including the RGB sensor 18 and is incident on the RGB sensor 18. The infrared light in the light incident on the prism 16 passes through the visible light reflective dichroic film 17 and is incident on the ToF sensor 13. Here, the visible light and the infrared light pass through substantially the same optical axis between the lens 12D and the visible light reflective dichroic film 17.

[0171] Figure 16 This is a diagram showing an example of the reflection spectral characteristics of the visible light reflective dichroic film according to Embodiment 6. Here, an example of the reflection spectral characteristics of the visible light reflective dichroic film 17 will be described. The horizontal axis of this diagram represents the wavelength [nm (nanometers)], and the vertical axis represents the reflectance [%] corresponding to that wavelength. As shown in the figure, the visible light reflective dichroic film 17 has a reflectance of approximately 100% when the wavelength is 700 [nm] or less, and a reflectance of approximately 0% when the wavelength is 700 [nm] or more. That is, the light with a wavelength of approximately 700 [nm] or less (e.g., visible light) in the light incident on the visible light reflective dichroic film 17 is reflected, and the light with a wavelength of approximately 700 [nm] or more (e.g., infrared light) passes through.

[0172] Return Figure 15 , the configuration of the RGB sensor 18 and the RGB processing unit 19 for generating an RGB image will be described.

[0173] The RGB sensor 18 includes a plurality of pixels arranged two-dimensionally, and each pixel receives visible light. Specifically, the RGB sensor 18 may also be an image sensor in which RGB pixels of each color are arranged by a Bayer arrangement. Each pixel included in the RGB sensor 18 outputs information related to the intensity of the received light to the RGB processing unit 19.

[0174] The RGB processing unit 19 acquires information related to the intensity of light in each pixel arranged two-dimensionally from the RGB sensor 18. Based on the acquired information, the RGB processing unit 19 generates an RGB image. The RGB processing unit 19 outputs the generated RGB image to the information processing device 20.

[0175] Here, the distance between the entrance pupil position of the lens and the light receiving surface of the ToF sensor 13 is represented by LI. In addition, the distance from the incident end of the prism 16 to the emission end toward the ToF sensor 13 is set as PI. In this case, the calculation part of Lt in the formula (16) is replaced with the following formula (18) instead of the formula (2).

[0176] Lt = (LI - Lk - PI) / C + Lk / C × Lr + PI / C × Lp…(18)

[0177] Here, Lk represents the total thickness of the glass located inside the lens 12D, Lr represents the reflectivity of the glass located inside the lens 12D, PI represents the optical path length of the prism 16, and Lp represents the reflectivity of the material of the prism 16.

[0178] The correction operation unit 153 in the distance information acquisition device 10F can use Lt obtained by the expression (18) to calculate Dab input to the formula (16) or the formula (17), thereby obtaining the corrected distance Dcr.

[0179] [Summary of Embodiment 6]

[0180] According to the embodiment described above, by further including the prism 16, the visible light reflection dichroic film 17, the RGB sensor 18, and the RGB processing unit 19, an RGB image can be acquired. The visible light for generating the RGB image and the infrared light for generating the distance image in the light incident on the lens 12D pass on the same optical axis. According to this embodiment, by applying the formula (18) for correction operation, even when the prism 16 is used, the distance to the object OB can be measured with high accuracy. Therefore, according to this embodiment, the optical axis offset in the distance space and the RGB space is extremely small, and when synthesizing the distance data and the RGB data images in post-processing, for example, when generating point cloud data with RGB information, complex alignment processing may not be required. Therefore, according to this embodiment, the time required for complex alignment processing of the distance data and the RGB data images in post-processing can be significantly reduced.

[0181] [Embodiment 7]

[0182] Next, refer to Figure 17 to describe Embodiment 7. In distance measurement based on the ToF method, the required light intensity varies according to the measurement distance. If the distance to the measurement object is short, one irradiation unit 11 is sufficient, but when the distance to the measurement object is long, multiple irradiation units 11 may be required depending on the length of the distance to the measurement object. Therefore, in Embodiment 7, the calibration operation in the case of including multiple irradiation units 11 is described.

[0183] Figure 17 FIG. is an example showing the configuration of the lens and the irradiation unit included in the distance information acquisition device 10 of Embodiment 7. In the illustrated example, irradiation units 11-1 to 11-4 are used as the irradiation unit 11. The irradiation units 11-1 to 11-4 are arranged in two columns on the upper side of the optical axis OA. The horizontal interval between the two irradiation units 11 in each column is denoted as SH, and the vertical interval is denoted as SV.

[0184] In this case, the calibration operation unit 153 sets dv and dh in Equation (16) using the midpoint coordinates of the positions of the multiple irradiation units 11. In the illustrated example, the intersection point C of the positions of SH / 2 and SV / 2 at the center of the horizontal and vertical intervals is used as the midpoint coordinates of the four irradiation units 11, and dv and dh in Equation (16) are set.

[0185] [Summary of Embodiment 7]

[0186] According to the embodiment described above, multiple irradiation units 11 are used. The calibration operation unit 153 regards the center point of the multiple irradiation units as the position of the irradiation unit 11, and uses the interval between this center point and the optical axis OA as dv and dh. Therefore, according to this embodiment, even when multiple irradiation units 11 are used, calibration operations can be performed through easy calculations.

[0187] The embodiments of the present invention have been described above, but the present invention is not limited to the above embodiments, and various modifications can be made without departing from the gist of the present invention. In addition, the above embodiments can be appropriately combined.

[0188] Industrial Applicability

[0189] According to the present invention, the distance to an object can be measured with high precision.

[0190] Symbol Explanation

[0191] 1…Distance information acquisition system, 10…Distance information acquisition device, 11…Irradiation unit, 111…Rod, 112…Set screw, 113…Rod, 114…Irradiation unit, 12…Lens, 121…Lens mounting portion, 13…ToF sensor, 14…Distance measurement unit, 15…Arithmetic unit, 151…Distance value acquisition unit, 152…Storage unit, 153…Calibration arithmetic unit, 16…Prism, 17…Visible light reflection dichroic film, 18…RGB sensor, 19…RGB processing unit, 20…Information processing device, 31…Lens control unit, 33…Lens information storage unit, PM…Static parameter, F…Arithmetic expression, LP…Light emission pulse, OB…Object, WC…White board, OA…Optical axis, SB1…First substrate, SB2…Second substrate, FL…Flexible cable, DI…Distance image, 9…ToF camera system, 90…ToF camera device, 91…Irradiation unit, 92…Lens, 93…ToF sensor, 94…Distance measurement unit, 95…Information processing device.

Claims

1. A distance information acquisition device, comprising: an irradiation unit that irradiates light onto an object; and a time-of-flight sensor that receives, via an optical system lens, the light irradiated by the irradiation unit and reflected by the object, the distance information acquisition device being configured to acquire distance information to the object, the distance information acquisition device comprising: a distance value acquisition unit that acquires a distance value obtained based on the timing of irradiation by the irradiation unit, the timing of light reception by the time-of-flight sensor, and the speed of light; a storage unit that stores a prescribed mathematical formula that includes, as a parameter, at least information related to the field of view angle of the optical system lens; a correction operation unit that calculates distance information to the object by applying the acquired distance value to the prescribed mathematical formula stored in the storage unit.

2. The distance information acquisition device according to claim 1, further comprising: a correction value storage unit that stores information related to the difference between the result of actually measuring the distance to the object and the distance information to the object calculated by the correction operation unit, as a correction value, wherein the correction operation unit calculates distance information to the object based on the correction value stored in the correction value storage unit.

3. The distance information acquisition device according to claim 1, wherein the optical system lens is a variable focal length zoom lens, the distance information acquisition device further comprises: a field of view angle information storage unit that stores the focal length of the optical system lens and the field of view angle corresponding to the focal length in association with each other, the correction operation unit calculates distance information to the object based on the field of view angle corresponding to the focal length of the optical system lens.

4. The distance information acquisition device according to claim 1, wherein the optical system lens is a replaceable lens on which any one of a plurality of lenses is mounted, the distance information acquisition device further comprises: a lens identification information acquisition unit that acquires lens identification information for identifying the mounted optical system lens; and a field of view angle information storage unit that stores the focal length of the optical system lens and the field of view angle corresponding to the focal length in association with each other, the field of view angle information storage unit stores, for each of the plurality of lenses, the focal length and the field of view angle corresponding to the focal length in association with each other, the correction operation unit acquires the field of view angle corresponding to the focal length of the lens identified based on the lens identification information acquired by the lens identification information acquisition unit from the field of view angle information storage unit, and calculates distance information to the object based on the acquired field of view angle.

5. The distance information acquisition device according to any one of claims 1 to 4, wherein the irradiation unit changes the distance between the irradiation unit and the time-of-flight sensor by moving in the optical axis direction of the optical system lens, the correction operation unit calculates distance information to the object based on the distance between the irradiation unit and the time-of-flight sensor.

6. The distance information acquisition device according to any one of claims 1 to 4, further comprising: a prism that is provided between the optical system lens and the time-of-flight sensor and through which the light irradiated by the irradiation unit and reflected by the object is incident; and A visible light reflecting dichroic film that reflects visible light in the light incident on the prism; and An RGB sensor that receives the visible light reflected by the visible light reflecting dichroic film.

7. The distance information acquisition device according to claim 6,[ wherein,[ the correction calculation unit further calculates the distance information to the object based on the thickness and reflectivity of the glass of the prism.

Images