Distance measuring device, distance measuring system

Abstract

The present invention relates to a distance measuring device and a distance measuring system, and its purpose is to improve the distance measuring accuracy when the size of the device is limited. The distance measuring device provided by the present invention comprises: a plurality of light projecting units (21) for irradiating light within the range of a measurement object; a plurality of light receiving units (30) for receiving light reflected by an object located within the range of the measurement object; and a distance measuring control unit for calculating the distance to the object based on the time when the plurality of light projecting units project light and the time when the plurality of light receiving units receive light, wherein the number of the plurality of light receiving units is greater than the number of the plurality of light projecting units.

Description

Technical Field

[0001] The invention relates to a distance measuring device and a distance measuring system. Background Art

[0002] The so-called TOF (time of flight) method is one of the distance measurement methods for measuring the distance between the camera device and the subject. This method irradiates the subject with distance measurement light and calculates the distance based on the time difference of the reflected light of the distance measurement light. Specifically, after irradiating the subject with infrared distance measurement light with intensity modulated in a specified irradiation pattern, the infrared camera element receives the distance measurement light reflected by the subject, and the time difference from the irradiation light to the reception light of each pixel is detected by the irradiation pattern to calculate the distance. Then, a graph of the distance value obtained is constructed in the form of a bitmap by pixel and saved as a "distance image". This type of distance image generation device (distance measurement device) is called a TOF camera.

[0003] For example, Patent Document 1 (JP Patent Publication No. 2019-159284) discloses a distance measuring device of a full-sphere imaging device that uses multiple sensors and cameras to obtain all-round three-dimensional information at one time.

[0004] However, the distance measurement accuracy in the technical solution disclosed in Patent Document 1 needs to be improved. Summary of the invention

[0005] The present invention is a technical solution proposed in view of the above-mentioned problem, and its purpose is to improve the ranging accuracy when the size of the device is limited.

[0006] In order to solve the above-mentioned problems and achieve the purpose of the present invention, the present invention provides a distance measuring device, characterized in that it comprises: a plurality of light projecting units for irradiating light within the range of a measurement object; a plurality of light receiving units for receiving light reflected by an object located within the range of the measurement object; and a distance measuring control unit for calculating the distance to the object based on the time when the plurality of light projecting units project light and the time when the plurality of light receiving units receive light, wherein the number of the plurality of light receiving units is greater than the number of the plurality of light projecting units.

[0007] The effect of the present invention is that by having more light receiving parts than projecting parts, the intensity of the projected light of the projecting part can be increased and the focal distance of the light receiving part can be extended under the condition that the size of the device is limited, thereby improving the distance measurement accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1-1 It is a perspective view of the appearance of the imaging device according to the first embodiment.

[0009] Figure 1-2 It is a cross-sectional view of the internal structure of the camera device.

[0010] Figure 1-3 This is an enlarged stereoscopic view of the area around the shutter button in the camera device.

[0011] Figure 2 is a schematic diagram of the inner wall of the camera device.

[0012] Figure 3 This is a diagram of the battery box.

[0013] Figure 4 is a cross-sectional view of the shutter button.

[0014] Figure 5 This is a diagram near the shutter button.

[0015] Figure 6 This is a cross-sectional view near the shutter button.

[0016] Figure 7-1 It is an oblique view of the VCSEL light projection unit.

[0017] Figure 7-2 is a cross-sectional view of the VCSEL optical system.

[0018] Figure 7-3 is a schematic diagram of the VCSEL light projection unit configuration.

[0019] Figure 8 It is a cross-sectional view of a CMOS light receiving unit.

[0020] Figure 9-1 This is a schematic diagram of connecting two CMOS light receiving units through a CMOS substrate.

[0021] Figure 9-2 is a schematic diagram of the CMOS substrate configuration.

[0022] Fig.10 It is a schematic diagram of the configuration of the CMOS light receiving unit and the VCSEL light projecting unit.

[0023] Figure 11-1 This is an oblique view of the appearance of the TOF light receiving unit.

[0024] Figure 11-2 This is a perspective view of the internal structure of the TOF light receiving unit.

[0025] Figure 11-3 This is a schematic diagram with part of the internal structure of the TOF light receiving unit missing.

[0026] Figure 12-1 It is a schematic diagram of the configuration relationship of the TOF optical system.

[0027] Figure 12-2 It is a schematic diagram of the configuration relationship of the TOF optical system.

[0028] Fig.13 It is a plan view of the VCSEL light projection unit and the TOF light receiving unit.

[0029] Fig.14 This is a schematic diagram of the positional relationship between the VCSEL optical system and the TOF optical system.

[0030] Fig.15 It is a schematic diagram of a comparative example of the positional relationship between a VCSEL optical system and a TOF optical system.

[0031] Fig.16 Schematic diagram of the VCSEL optical system viewing angle.

[0032] Fig.17 It is the hardware structure block diagram of the camera device.

[0033] Fig.18 It is a functional structure block diagram of the camera device.

[0034] Fig.19 It is an enlarged schematic diagram of the optical system portion of the imaging device according to the second embodiment.

[0035] Fig. 20 Schematic diagram of the VCSEL optical system viewing angle. DETAILED DESCRIPTION

[0036] The following describes in detail embodiments of the distance measuring device and the distance measuring system with reference to the accompanying drawings.

[0037] <First Embodiment>

[0038] Figure 1-1 is a perspective view of the appearance of the imaging device 100 according to the first embodiment. Figure 1-2 is an internal cross-sectional view of the imaging device 100, Figure 1-3 It is a perspective view showing an enlarged view of the vicinity of the shutter button 62 in the imaging device 100. The imaging device 100 is a distance measuring device of the TOF method.

[0039] like Figure 1-1 to Figure 1-3 As shown, the camera device 100 includes a VCSEL (vertical cavity surface emitting laser) projection unit 21, a TOF light receiving unit 61, a CMOS (complementary metal oxide semiconductor) light receiving unit 30, a substrate (CMOS substrate 35, VCSEL substrates 22F and 22B, main board 41), and a fan 38.

[0040] The VCSEL light projection unit 21 irradiates the object to be measured with distance measurement light (infrared light, etc.) for the purpose of measuring the distance of the object. The VCSEL light projection unit 21 is composed of two VCSEL light projection units 21 (VCSEL light projection units 21F and 21B), and the details will be described later (see Figure 5 ).

[0041] The TOF light receiving unit 61 receives reflected light (scattered light) of the distance measuring light emitted from the VCSEL light projecting unit 21 and reflected by an object, and acquires three-dimensional point group data.

[0042] The CMOS light receiving unit 30 is connected to the CMOS sensor 33 (see Figure 8 ) to obtain a two-dimensional image.

[0043] The substrate (CMOS substrate 35, VCSEL substrates 22F and 22B, main board 41) is a substrate for driving / controlling the VCSEL light projecting unit 21, the TOF light receiving unit 61, and the CMOS light receiving unit 30. The substrate (CMOS substrate 35, VCSEL substrates 22F and 22B, main board 41) is connected to the VCSEL light projecting unit 21, the TOF light receiving unit 61, and the CMOS light receiving unit 30 through cables, FPCs, FFCs, etc.

[0044] The fan 38 is provided inside the imaging device 100 and cools the inside of the imaging device 100 by generating forced convection.

[0045] Here, the arrangement of the plurality of substrates (CMOS substrate 35 , VCSEL substrates 22F and 22B, main board 41 ) included in the imaging device 100 will be described.

[0046] like Figure 1-2 As shown in FIG. 1 , the CMOS substrate 35 is arranged between the two VCSEL substrates 22F and 22B in the Z-axis direction. The main board 41 for overall control / drive of the imaging device 100 is also arranged in parallel with the three substrates. The main board 41 is arranged on the outer side of the VCSEL substrate 22B, i.e., on the -Z side, close to the rear cover 12.

[0047] With the above configuration, a total of four substrates, namely, the VCSEL substrates 22F and 22B, the CMOS substrate 35, and the main board 41, can be arranged reasonably inside the camera device 100, thereby avoiding wasting the space inside the camera device 100. As a result, the dimensions of the four substrates, namely, the Z-axis direction, in the arrangement direction of the VCSEL substrates 22F and 22B, the CMOS substrate 35, and the main board 41 can be miniaturized. Since the four substrates, namely, the VCSEL substrates 22F and 22B, the CMOS substrate 35, and the main board 41, are arranged in parallel with each other, air flow along the substrates can be generated without blocking the natural convection inside the camera device 100 and the forced convection generated by the fan 38, thereby reducing the occurrence of temperature deviation inside the device. Furthermore, since the heat dissipation efficiency of the air inflow / outflow from the vent provided on the cover member described later or the heat dissipation (heat transfer) of the air discharged to the outside through the cover member is improved, the occurrence of temperature rise inside the device can be reduced.

[0048] In addition, if Figure 1-1 to Figure 1-3 As shown, the imaging device 100 includes battery boxes 68a and 68b for storing batteries 18a and 18b (see Figure 3 ), shutter button 62, operation switch unit 17.

[0049] The shutter button 62 is operated by the user to determine the photographing timing of the CMOS light receiving unit 30 .

[0050] The operation switch unit 17 is operated by a user to switch the power supply of the imaging device 100 on / off and switch the operation mode.

[0051] Furthermore, if Figure 1-1 to Figure 1-3 As shown, the camera device 100 includes a front cover 11, a rear cover 12, a left cover 13, a right cover 14, battery covers 15a, 15b, a bottom plate 16, an inner wall 10 and other outer cover components for holding the above components (see Figure 2 ).

[0052] like Figure 1-1 As shown, the camera device 100 is provided with a screw hole 19 for fixing a tripod etc. on the bottom plate 16 (-X side) to prevent shaking during shooting. When the camera device 100 is fixed to a tripod etc. in the screw hole 19, a more stable image can be obtained compared to shooting when the user holds the camera device 100. When the camera device 100 is fixed to a tripod etc., remote operation is more effective.

[0053] Here, the inner wall 10 which is a part of the cover member of the imaging device 100 will be described. Figure 2 Schematic diagram of an example of the inner wall 10.

[0054] like Figure 2As shown, the inner wall 10 connects the front cover 11 and the rear cover 12. As in the past, the rigidity is easily reduced by only relying on the outer cover component (exterior) because it is a single-body structure, but in the camera device 100 of this embodiment, the rigidity can be improved by connecting the front cover 11 and the rear cover 12 with the inner wall 10.

[0055] Next, the battery boxes 68a and 68b will be described. Figure 3 It is a schematic diagram of an example of battery boxes 68a and 68b.

[0056] like Figure 3 As shown in FIG. 1 , the camera device 100 has built-in batteries 18a and 18b. Therefore, the camera device 100 can reduce the burden of carrying and shooting. The battery circuit board 67 with battery boxes 68a and 68b is fixed to the two side surfaces (the +Y side surface and the -Y side surface) of the inner wall 10 of the camera device 100. The battery boxes 68a and 68b store the batteries 18a and 18b.

[0057] Remove the camera device 100 Figure 1-1 The battery covers 15a and 15b shown in the figure can be installed and removed from the batteries 18a and 18b in the Y-axis direction, so the replacement operation of the batteries 18a and 18b is convenient.

[0058] The camera device 100 may also be driven by a power cord (not shown). Preferably, the power cord is detachable. This eliminates the need to build a battery into the main body of the camera device 100, thereby reducing the weight of the camera device 100 and extending the shooting time.

[0059] When the camera device 100 is driven by a power cord, it is preferable to arrange the insertion port of the power cord closer to the bottom (bottom plate 16) than the shutter button 62 (-X side). In this way, the user's finger will find the light blocking before the power cord, and the blind spot caused by the power cord is relatively small compared to the blind spot caused by the user's finger pressing the shutter button 62.

[0060] Here, the shutter button 62 will be described. Figure 4 is a cross-sectional view of the shutter button 62 .

[0061] like Figure 4 As shown in FIG. 1 , a switch 69 is provided on the surface of the -Z side of the main board 41 in the camera device 100. A shutter button 62 coaxial with the switch 69 is provided on the rear cover 12 of the camera device 100. With such a configuration, the switch 69 can be directly pressed by the shutter button 62, thereby reducing the number of parts and simplifying the structure. Since the switch 69 can be directly pressed by the shutter button 62, the switch 69 can be ensured to respond. In addition, when the shutter button 62 and the switch 69 are located separately, the two can be connected by an intermediate component.

[0062] like Figure 4 As shown in FIG. 1 , the imaging device 100 includes a spring member 63 between the main board 41 and the shutter button 62. The spring member 63 is used to push the shutter button 62 back to a predetermined position when the shutter button 62 is released after the user presses the shutter button 62.

[0063] Next, the arrangement of the shutter button 62 will be described. Figure 5 It is a schematic diagram of the structure near the shutter button 62.

[0064] like Figure 5 As shown, the imaging device 100 includes a shutter button 62 in a region where a blind spot is created by various housing members (the rear cover 12 in this embodiment).

[0065] Specifically, in Figure 5 On the XZ section shown, the outermost viewing angle (below; -X direction) of the emitted light from the VCSEL light projection unit 21F is θa, and it is actually ensured that θa is approximately 90°. On the other hand, regarding the emitted light from the VCSEL light projection unit 21B, a portion of the back cover 12 that houses the main board 41 is shaped to protrude toward the -Z direction. For this reason, the outermost viewing angle is θb, and θb<θa. Therefore, as Figure 5 As shown in the figure, when the shutter button 62 is arranged near the part indicated by the arrow V on the rear cover 12, the light of the outermost angle of view θb (outgoing light B) is not blocked by the user's finger. However, if the shutter button 62 is arranged on the opposite side (front cover 11 side), the light of θa (≈90°) (outgoing light A) is blocked by the user's finger, and the angle of view becomes narrow. This arrangement is not selected.

[0066] In addition, the shutter button 62 is disposed between the TOF light receiving unit 61 and the batteries 18a, 18b. Thus, by disposing the shutter button 62 between the TOF light receiving unit 61 and the batteries 18a, 18b, the portion (close to the center of gravity) housing the batteries 18a, 18b can be held by hand, thereby reducing fatigue and preventing shaking.

[0067] Furthermore, the shooting timing may be determined by wired or wireless remote control, rather than by pressing / releasing the shutter button 62 provided on the rear cover 12 of the imaging device 100. The imaging device 100 adopts such a remotely operable structure, and thus can improve the anti-shake effect.

[0068] Figure 6 62 is a cross-sectional view of the vicinity of the shutter button 62. Figure 6 As shown, the camera device 100 includes a plurality of LED elements 65 (LEDs) for indicating the operating status of the camera device 100 on the side of the -Z side of the main board 41. Figure 6The imaging device 100 includes an opening 64 on the rear cover 12 in a manner coaxial with the emission light axis of the LED element 65. By adopting such a configuration, the imaging device 100 can effectively emit the emission light from the LED element 65 from the opening 64 to the outside.

[0069] The imaging device 100 may also be connected between the LED element 65 and the opening 64 by a light guide plate or an optical fiber. Thus, the imaging device 100 can improve light utilization efficiency. The imaging device 100 can further improve the visibility of the light emitted from the LED element 65 by providing a lens system and a diffusion plate in the opening 64.

[0070] Next, the VCSEL light projection unit 21 will be described. Figure 7-1 is a perspective view of the VCSEL light projection unit 21, Figure 7-2 is a cross-sectional view of the VCSEL optical system 23, Figure 7-3 FIG. 2 is a schematic diagram showing an example of the configuration of the VCSEL light projection unit 21 .

[0071] like Figure 7-1 to Figure 7-3 As shown, the VCSEL light projection unit 21 includes a VCSEL substrate 22 , a VCSEL package 24 and a lens unit 26 .

[0072] The VCSEL package 24 is a light source having a VCSEL (surface emitting laser) as a light emitting point 25. The lens unit 26 accommodates the VCSEL optical system 23 composed of a plurality of lenses.

[0073] The VCSEL substrate 22 (VCSEL substrates 22F and 22B) fixes the VCSEL package 24 by soldering. The VCSEL substrate 22 also fixes the VCSEL optical system 23 (i.e., the lens unit 26) by screw fixing or bonding in a state where the VCSEL substrate 22 is aligned with the light emitting point 25 with a predetermined accuracy.

[0074] The VCSEL substrate 22 (VCSEL substrates 22F, 22B) is equipped with a driving circuit for driving the VCSEL package 24. In order to increase the intensity of light emitted from the VCSEL package 24 as a light source, a large current passes through the driving circuit of the VCSEL substrate 22 to generate heat. In response to this, in order to reduce the heat generated by the driving circuit, the VCSEL substrate 22 is equipped with a driving circuit with a large allowable current and a heat dissipation component (such as a heat sink). Therefore, the VCSEL substrate 22 is larger than other substrates (CMOS substrate 35, etc.). By adopting such a structure, the VCSEL substrate 22 is not easy to overheat, and the camera device 100 can project high-intensity light from the VCSEL light projection unit 21.

[0075] like Figure 7-3As shown, the VCSEL light projection units 21 (VCSEL light projection units 21F and 21B) are arranged in a direction in which the emission light axis of the VCSEL optical system 23 is parallel to the Z-axis direction, and the two VCSEL light projection units 21 (VCSEL light projection units 21F and 21B) are arranged opposite to each other.

[0076] The VCSEL optical system 23 has a fisheye lens function and illuminates the entire celestial sphere (4n[sr]) region as a measurement target range through two VCSEL light projection units 21F and 21B as a plurality of light projection units without light being blocked by surrounding components.

[0077] Here, the measurement target range refers to an area that can be illuminated by the two VCSEL light projection units 21F and 21B as a plurality of light projection units in the entire celestial sphere (4π [sr]) area with the camera device 100 as the center. As described above, the measurement target range corresponds to the entire celestial sphere (4π [sr]) in the case where light is not blocked by components around the VCSEL light projection units 21F and 21B. In the case where components around the VCSEL light projection units 21F and 21B block light, the measurement target range is a projection range in the entire celestial sphere that is not blocked by components around the light projection unit.

[0078] The camera device 100 is fixed by screws 66a (see Fig.10 ) are tightened to fix the VCSEL light projection units 21F and 21B to the front cover 11 and the rear cover 12, respectively. When the screws are tightened, the positioning parts provided on the lens units 26 of the VCSEL light projection units 21F and 21B are positioned correspondingly to the positioning parts of one of the front cover 11 and the rear cover 12. Thus, the positioning accuracy of the VCSEL light projection unit 21F relative to the front cover 11 and the positioning accuracy of the VCSEL light projection unit 21B relative to the rear cover 12 can be maintained.

[0079] With such a configuration, it is possible to suppress the deviation in the amount (light amount) of light emitted from the VCSEL light projection units 21F and 21B blocked by the front cover 11 and the rear cover 12, etc., due to the influence of processing errors (dimensional errors or shape errors) and assembly deviations of the components of the imaging device 100. As a result, the imaging device 100 can avoid degradation in the illumination distribution of the emitted light.

[0080] The VCSEL substrates 22F and 22B are described below. Fig.10As shown, the VCSEL substrates 22F and 22B are arranged in parallel with each other. The VCSEL substrates 22F and 22B need to be enlarged to reduce overheating caused by the emission of high-intensity light from the VCSEL light projection units 21F and 21B. In this way, by arranging the VCSEL substrates 22F and 22B with enlarged substrate sizes in parallel, the miniaturization of the camera device 100 can be achieved without mutual interference between the VCSEL substrates 22F and 22B. Since the VCSEL substrates 22F and 22B are configured with the minimum number that can project the entire celestial sphere as the target measurement range, the miniaturization of the camera device 100 can be achieved.

[0081] Next, the CMOS light receiving unit 30 will be described. Figure 8 is a cross-sectional view of the CMOS light receiving unit 30 .

[0082] like Figure 8 As shown, the CMOS light receiving unit 30 includes a CMOS optical system 31 , a CMOS sensor substrate 32 , and a lens holder 34 .

[0083] The CMOS optical system 31 includes a plurality of lenses and prisms, etc. A CMOS sensor 33 is mounted on a CMOS sensor substrate 32. A lens holder 34 holds the CMOS optical system 31 and the CMOS sensor substrate 32 as a whole. The CMOS substrate 32 is fixed to a holding portion 34a of the lens holder 34 by bonding or the like.

[0084] In the imaging device 100, scattered light emitted from the external lighting or VCSEL light projection unit 21 and then reflected by an external object is Figure 8 The light enters the CMOS optical system 31 as shown by the arrow, and is received by the CMOS sensor 33. When a white light source is used for external illumination, the CMOS sensor 33 can capture a brightness image and an RGB image corresponding to the intensity of scattered light reflected by an external object. When scattered light emitted by the VCSEL light projection unit 21 is reflected by an object and enters the CMOS optical system 31, the CMOS sensor 33 can capture a brightness image of the wavelength of the light emitted by the VCSEL light projection unit 21.

[0085] Here, Figure 9-1 It is a schematic diagram of connecting two CMOS light receiving units 30 with a CMOS substrate 35. Figure 9-2 2 is a schematic diagram showing the configuration of the CMOS substrate 35 .

[0086] like Figure 9-1As shown, the CMOS optical system 31 in the camera device 100 is configured so that its optical axis is in a direction parallel to the Y-axis direction, and two CMOS optical systems 31 (CMOS light receiving units 30R, 30L) as multiple camera units are configured opposite to each other, and the CMOS sensor substrates 32 of the CMOS light receiving units 30R, 30L are connected to a common (1) CMOS substrate 35 via cables such as FPC.

[0087] like Figure 9-2 As shown, in the camera device 100, the CMOS substrate 35 is screwed to a bracket member 36L screwed to the left cover 13, a bracket member 36R screwed to the right cover 14 (not shown), and a bracket member 36C fastened to the inner wall 10 through a spacer member 37.

[0088] The CMOS optical system 31 has a fisheye lens function and can receive scattered light from the entire celestial sphere (4π [sr]) region through two CMOS light receiving units 30R and 30L when the surrounding components do not block the light.

[0089] Next, the configuration of the CMOS light receiving unit 30 will be described. Fig.10 2 is a diagram showing an example of arrangement of the CMOS light receiving unit 30 and the VCSEL light projecting unit 21 .

[0090] like Figure 1-2 and Fig.10 As shown, a part (or the whole) of the CMOS light receiving unit 30 (lens holder 34) in the imaging device 100 is arranged between the two VCSEL substrates 22F and 22B. Thus, the imaging device 100 can be miniaturized in the incident light axis direction (Y axis direction) of the CMOS optical system 31.

[0091] like Fig.10 As shown, the CMOS light receiving units 30R and 30L in the camera device 100 are fixed to the right cover 14 and the left cover 13 respectively by tightening the screws 66b. When the screws are tightened, the positioning parts on the lens holders 34 of the CMOS light receiving units 30R and 30L are positioned corresponding to the positioning parts on one side of the right cover 14 and the left cover 13. Thus, the position alignment accuracy of the CMOS light receiving unit 30R relative to the right cover 14 and the position alignment accuracy of the CMOS light receiving unit 30L relative to the left cover 13 can be maintained.

[0092] Next, the TOF light receiving unit 61 will be described. Figure 11-1 is an oblique view of the appearance of the TOF light receiving unit 61, Figure 11-2 is a perspective view of the internal structure of the TOF light receiving unit 61, Figure 11-3This is a schematic diagram in which a part of the internal structure of the TOF light receiving unit 61 is missing.

[0093] like Figure 11-1 to Figure 11-3 As shown, the TOF light receiving unit 61 has a four-eye structure. The TOF light receiving unit 61 includes a TOF optical system 71 (71A, 71B, 71C, 71D), a TOF sensor substrate 74, a relay substrate 77, and a bracket member 78. The TOF optical system 71 is a plurality of light receiving units that receive light reflected by an object within the measurement target range (the entire celestial sphere).

[0094] A TOF sensor 76 is mounted on the TOF sensor substrate 74 .

[0095] The relay substrate 77 is a relay portion between the TOF sensor substrate 74 and the main board 41 .

[0096] The bracket member 78 integrally holds the TOF optical system 71 ( 71A, 71B, 71C, 71D) and the TOF sensor substrate 74 .

[0097] The TOF sensor substrate 74 and the relay substrate 77 , as well as the relay substrate 77 and the main board 41 are connected via cables such as FPC.

[0098] like Figure 11-1 to Figure 11-3 As shown, the imaging device 100 is provided with an integrated structure of the TOF light receiving unit 61, which helps simplify the assembly / adjustment equipment of the factory, shorten the assembly / adjustment time, and ensure the quality of the integrated structure. Moreover, since the TOF light receiving unit 61 is provided with an integrated structure, the imaging device 100 is not limited to being mounted on one model, but can also be mounted on multiple models, thereby helping to reduce costs.

[0099] In the imaging device 100 , light (scattered light) emitted from the VCSEL light projecting unit 21 and reflected by an external object is received by a TOF sensor 76 mounted on a TOF sensor substrate 74 via a TOF optical system 71 .

[0100] Figure 12-1 and Figure 12-2 They are schematic diagrams of the configuration relationship of the TOF optical system 71.

[0101] like Figure 12-1 and Figure 12-2 As shown, the TOF optical system 71A is configured so that its incident light axis faces upward (+X). The incident light axes of the TOF optical systems 71B, 71C, and 71D face horizontally, and the incident light axis of the TOF light receiving system 71B faces rightward (+Y). The TOF optical systems 71C and 71D are configured at an angle of 120° rotated around the X axis relative to the TOF optical system 71B.

[0102] The viewing angle (vertical direction; XY section and XZ section) of the TOF optical system 71A configured in the vertical direction is 65°. The viewing angle of the TOF optical system 71B configured in the horizontal direction (to the right) is 85° in the vertical direction (XY section) and 65° in the horizontal direction (YZ section). TOF optical systems 71C and 71D are the same as the TOF optical system 71B. With such a configuration, the camera device 100 can receive all-round reflected light (scattered light) from the object through the TOF light receiving unit 61 composed of four eyes, except for the area above the viewing angle of 85° below.

[0103] The TOF optical system 71 (TOF optical systems 71A, 71B, 71C, 71D) is configured so that its viewing angle is smaller than that of the VCSEL optical system 23, so the focal length of the TOF optical system 71 is increased. For this reason, the amount of light that can be received per unit viewing angle of the TOF optical system 71, which is related to the amount of light received per unit pixel of the TOF sensor 76, is increased. As a result, the amount of light reflected by a distant object or an object with a low reflectivity that can be received by the TOF optical system 71 is increased.

[0104] Next, the relative positional relationship between the VCSEL light projecting unit 21 and the TOF light receiving unit 61 will be described. Fig.13 is a plan view of the VCSEL light projection unit 21 and the TOF light receiving unit 61, Fig.14 is a schematic diagram of the positional relationship between the VCSEL optical system 23 and the TOF optical system 71, Fig.15 It is a schematic diagram of a comparative example of the positional relationship between the VCSEL optical system 23 and the TOF optical system 71 . Fig.14 Only from Fig.13 The schematic diagram of the outermost lenses of the VCSEL light projection unit 21 and the TOF light receiving unit 61 is extracted from the plan view of the structure of the imaging device 100 of the present embodiment shown in FIG. Fig.15 shown.

[0105] Fig.15 The comparative example shown shows the configuration in which the TOF optical systems 71B, 71C, and 71D are rotated 30° around the X axis. Fig.15 In the comparative example shown, the VCSEL optical system 23F and the TOF optical system 71D are arranged to face the same direction (towards +Z). Fig.15 In the imaging device of the comparative example configuration shown, most of the scattered light emitted from the VCSEL optical system 23F disposed on the +Z side and reflected by an object is received in the TOF optical system 71D.

[0106] Fig.15In the imaging device of the comparative example configuration shown, half of the scattered light emitted from the VCSEL optical system 23B configured on the -Z side and reflected by the object is received by the TOF optical system 71B, and the remaining half is received by the TOF optical system 71C. As a result, Fig.15 In the imaging device of the comparative example configuration shown, the amount of light received by the three TOF light receiving units 71B, 71C, and 71D is prone to large deviations (the light amount ratio is approximately 71B:71C:71D=0.5:0.5:1), which can easily lead to a decrease in detection accuracy.

[0107] In this regard, Fig.14 In the imaging device 100 shown in FIG. 1 , the VCSEL optical systems 23F and 23B are arranged symmetrically with respect to the XY plane, and the TOF optical systems 71B, 71C, and 71D are arranged symmetrically with respect to the XY plane. Fig.15 Compared with the comparative example shown, the variation in the amount of light received by the three TOF optical systems 71B, 71C, and 71D (variation among the three TOF optical systems) can be reduced. As a result, the imaging device 100 can further improve the detection accuracy.

[0108] Next, the viewing angle of the VCSEL optical system 23 will be described. Fig.16 is a schematic diagram of the viewing angle of the VCSEL optical system 23.

[0109] like Fig.16 As shown, the imaging device 100 arranges the TOF optical system 71A in the TOF light receiving unit 61 in the first section from above (+X side), and arranges the TOF optical systems 71B, 71C, and 71D in the second section. The third section of the imaging device 100 arranges the VCSEL optical systems 23F and 23B and the CMOS optical systems 31R and 31L.

[0110] Fig.16 This is a schematic diagram of the light rays at the outermost viewing angle in the vertical section (ZX section) of the VCSEL optical system 23F and the TOF optical systems 71A and 71B. Fig.16 In order to avoid complicated description, TOF optical systems 71B, 71C, and 71D are rotated around the X axis so that the incident light axis of TOF optical system 71B is parallel to the Z axis. In the following description, TOF optical systems 71A to 71D and 171A to 171D are designed to be Figure 12-2 The optical system shown has a larger viewing angle (maximum viewing angle of 90°).

[0111] like Fig.16As shown, the outermost viewing angle of the upper irradiable range of the VCSEL optical system 23F on the +Z side of the camera device 100 is θvcsel2>90°. Therefore, combined with the light projection (irradiation light) of the VCSEL optical system 23B on the rear (-Z side), the irradiable range of the VCSEL light projection unit 21 covers the semi-celestial sphere area (2π[sr]).

[0112] The light acceptance range above the TOF light receiving unit 61 can cover the area of ​​the half celestial sphere based on the light acceptance range (angle of view θtof1) of the upward TOF optical system 71A and the light acceptance range (angle of view θtof2) of the horizontally facing TOF optical system 71B (and 71C, 71D). Therefore, the photography of the camera device 100 can cover the photographic range of the upper half celestial sphere. The "photographic range" of the CMOS light receiving unit 30 is the area where the "irradiation range of the VCSEL light projection unit 21" and the "light acceptance range of the CMOS light receiving unit 30" overlap.

[0113] Next, a description will be given of the hardware configuration of the imaging device 100. Here, a description will be given of the hardware configuration related to distance measurement from the imaging device 100 to a subject.

[0114] Fig.17 FIG. 1 is a block diagram showing an example of a hardware structure of the imaging device 100. Fig.17 In the embodiment, the imaging device 100 includes a distance measurement control unit 230 in addition to the VCSEL light projecting unit 21 , the TOF light receiving unit 61 , and the CMOS light receiving unit 30 .

[0115] The distance measurement control unit 230 is connected to the VCSEL light projection unit 21, the TOF light receiving unit 61, and the CMOS light receiving unit 30, and is built into the cover member. The distance measurement control unit 230 has a CPU (central processing unit) 231, a ROM (read only memory) 232, a RAM (random access memory) 233, an SSD (solid state drive) 234, a light source driving circuit 235, a sensor I / F (interface) 236, an input / output I / F 237, and an RGB sensor I / F 240. These components are electrically connected to each other via a system bus 242.

[0116] The CPU 231 reads the program and data from the storage devices such as the ROM 232 and the SSD 234 to the RAM 233 for processing, thereby realizing the overall control of the ranging control unit 230 and the functions described later. Some or all of the functions of the CPU 231 may also be realized by circuits such as an ASIC (integrated circuit) or an FPGA (field programmable gate array).

[0117] The ROM 232 is a nonvolatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 232 stores programs and data such as BIOS (Basic Input Output System) and OS settings that are executed when the imaging device 100 is activated.

[0118] RAM 233 is a volatile semiconductor memory (storage device) that temporarily stores programs and data.

[0119] The SSD 234 is a nonvolatile memory that stores various data and programs for executing the processing executed by the distance measurement control unit 230. In addition, the SSD may be a HDD (hard disk drive) or the like.

[0120] The light source driving circuit 235 is electrically connected to the VCSEL light projection unit 21, and outputs a driving signal such as a driving voltage to the VCSEL light projection unit 21 according to a control signal input from the CPU 231 or the like. The light source driving circuit 235 drives the plurality of light emitting units in the VCSEL light projection unit 21 to emit light according to the control signal. As the driving signal, a rectangular wave, a sine wave, or a voltage waveform of a predetermined waveform shape can be used. The light source driving circuit 235 can change the frequency of the voltage waveform to modulate the frequency of the driving signal.

[0121] The sensor I / F 236 is electrically connected to the TOF light receiving unit 61 and is an interface for inputting a phase signal output from the TOF light receiving unit 61. The input / output I / F 237 is an interface for connecting to an external device such as a PC (personal computer).

[0122] The RGB sensor I / F 240 is electrically connected to the CMOS light receiving unit 30 , and is an interface for inputting RGB signals output from the CMOS light receiving unit 30 .

[0123] Fig.18 1 is a functional block diagram for explaining the imaging device 100. Fig.18 As shown in FIG. 1 , the imaging device 100 includes a distance measurement control unit 230 as a function of the imaging device 100 .

[0124] The distance measurement control unit 230 includes a light emission control unit 238, a light receiving processing unit 239, and an RGB image processing unit 241. The distance measurement control unit 230 can obtain a distance image and an RGB image at the same time by controlling the light emission of the VCSEL light projection unit 21 via the light emission control unit 238, the light reception of the TOF light receiving unit 61 via the light receiving processing unit 239, and the light reception of the CMOS light receiving unit 30 via the RGB image processing unit 241.

[0125] The light emission control unit 238 includes at least a drive signal output unit 238 a which is a function of the imaging device 100 .

[0126] The driving signal output unit 238a outputs the driving signal to the VCSEL light projection unit 21, so that it emits light simultaneously. The driving signal output unit 238a can also perform time modulation (time control) on the light emission of the VCSEL light projection unit 21 by outputting the driving signal with a predetermined voltage waveform and a predetermined light emission frequency. In this embodiment, as an example, a rectangular wave or a sine wave driving signal is output to the VCSEL light projection unit 21 at a frequency of MHz (megahertz) at a predetermined time.

[0127] The light reception processing unit 239 includes at least a phase signal input unit 239 a , a range image acquisition unit 239 b , a storage unit 239 c , and a range image combining unit 239 d , which are functions of the imaging device 100 .

[0128] The phase signal input unit 239a is implemented by the sensor I / F 236, etc., and inputs the phase signal output by the TOF light receiving unit 61. The phase signal input unit 239a can input the phase signal of each pixel of the two-dimensional arrangement of the TOF light receiving unit 61. The phase signal input unit 239a also outputs the input phase signal to the distance image acquisition unit 239b. In the present embodiment, the phase signal input unit 239a is connected to the TOF light receiving unit 61. Therefore, four groups of phase signals corresponding to the TOF optical system 71 (71A, 71B, 71C, 71D) are output.

[0129] The distance image acquisition unit 239b acquires the distance image data from the imaging device 100 to the object based on the phase signal of each pixel of the TOF light receiving unit 61 input from the phase signal input unit 239a. Here, the distance image refers to an image generated by arranging the distance data acquired for each pixel in two dimensions according to the position of the pixel, for example, an image generated by converting the distance into the brightness of the pixel. The distance image acquisition unit 239b outputs the acquired four distance image data to the storage unit 239c.

[0130] The storage unit 239 c is implemented by the RAM 233 or the like, and temporarily stores the range image data input from the range image acquisition unit 239 b .

[0131] The distance image combining unit 239d reads the four distance image information temporarily stored in the storage unit 239c, combines them and generates one whole celestial sphere distance image data.

[0132] The distance image combining unit 239d is implemented by the CPU 231 executing the control program. However, the present invention is not limited to this example, and a part or all of the distance image combining unit 239d may be implemented by dedicated hardware designed to perform the same functions, such as semiconductor integrated circuits such as ASIC (integrated circuit), DSP (digital signal processor), FPGA (field programmable gate array), or existing circuit modules.

[0133] The RGB image processing unit 241 includes an RGB image input unit 241 a , an RGB image storage unit 241 b , and an RGB image combining unit 241 c .

[0134] The RGB image input unit 241a inputs the RGB image output from the CMOS light receiving unit 30. For example, the RGB image input unit 241a can input the RGB signal of each pixel arranged two-dimensionally in the CMOS light receiving unit 30. The RGB image input unit 241a outputs the input FRGB image to the RGB image storage unit 241b (in this embodiment, since the RGB image processing unit 241 is connected to the two CMOS light receiving units 30R and 30L, two RGB images are output). The RGB image input unit 241a is implemented by the RGB sensor I / F 240 or the like.

[0135] The RGB image storage unit 241 b is implemented by the RAM 233 or the like, and temporarily stores the RGB image data input from the RGB image input unit 241 a .

[0136] The RGB image combining unit 241c reads two RGB images temporarily stored in the RGB image storage unit 241b, and combines them to generate a full celestial RGB image. In addition, the RGB image combining unit 241c is implemented by the CPU 231 executing a control program. However, not limited to this example, part or all of the RGB image combining unit 241c can also be implemented by dedicated hardware designed to perform the same functions, such as semiconductor integrated circuits such as ASIC (integrated circuit), DSP (digital signal processor), FPGA (field programmable gate array), or existing circuit modules.

[0137] Above Fig.18 The configuration shown is described in the example in which the imaging device 100 is built in, but the present invention is not limited to this example. For example, an external information processing device connected to the imaging device 100 may have Fig.18 The structure shown.

[0138] In this way, the number of light projection parts (VCSEL light projection units 21F, 21B) of this embodiment is less than the number of light receiving parts (TOF optical systems 71A, 71B, 71C, 71D), thereby covering the range of the measurement object. Therefore, the light source of the light projection part can be enlarged and the light irradiation intensity can be improved. For this reason, even distant objects and low-reflection objects can receive sufficient light and improve measurement accuracy. In addition, since the number of light receiving parts in this embodiment is greater than the number of light projection parts, the range of the measurement object is covered. Therefore, the viewing angle of a light receiving part can be set narrower, thereby extending the focal distance of the light receiving part, increasing the F value of the lens of the light receiving part, and improving the ranging accuracy of distant objects. That is to say, by making the number of light receiving parts greater than the number of light projection parts, this embodiment can increase the intensity of the light projected by the light projection part, extend the focal distance of the light receiving part, and improve the ranging accuracy with a limited device size.

[0139] <Second Embodiment>

[0140] Next, a second embodiment will be described.

[0141] The difference between the second embodiment and the first embodiment is that the TOF light receiving unit is divided into two sections, the upper and lower sections, and the VCSEL light projection unit and the CMOS light receiving unit are arranged between the two sections. In the following, in the description of the second embodiment, the description of the same parts as the first embodiment is omitted, and only the parts different from the first embodiment are described.

[0142] However, if Fig.16 As shown, in order to ensure the irradiable range of the upper half celestial sphere (2π [sr]) irradiated by the emitted light of the VCSEL light projection unit 21, the imaging device 100 needs to arrange the position of the outermost lens of the VCSEL optical system 23F to the right (+Z direction) of the position of the outermost lens of the TOF optical system 71B. As a result, E1>D1 is inevitable, and the size of the imaging device 100 in the Z-axis direction is likely to increase.

[0143] In addition, Fig.16 As shown, the camera device 100 needs to prevent the incident light of the angle of view θtof3 below the TOF optical system 71B from being blocked by the VCSEL optical system 23F. Therefore, θtof3≤60°, and the blind spot A tends to become larger. As a result, even when the angle of view (irradiable range) θvcsel2 below the VCSEL optical system 23F is greater than 90°, the shooting range is limited by the blind spot A below the TOF optical system 71B. Therefore, in some cases, the photography of the camera device 100 cannot cover the lower half of the celestial sphere.

[0144] Here, Fig.19It is a schematic diagram showing an enlarged optical system portion of an image pickup device 200 according to the second embodiment.

[0145] like Fig.19 As shown, in the imaging device 200 of this embodiment, the TOF light receiving unit is divided into two sections, the upper and lower sections, and the VCSEL light projection unit (VCSEL optical system 23F, 23B) and the CMOS light receiving unit (CMOS optical system 31R, 31L) are arranged between the two. In further detail, in the imaging device 200, the TOF optical system 71A is arranged in the first section, the TOF optical systems 71B, 71C, and 71D are arranged in the third section, and the VCSEL optical systems 23F, 23B and the CMOS optical systems 31R, 31L are arranged between the two (the second section).

[0146] The imaging device 200 receives light (scattered light) emitted from the VCSEL optical systems 23F and 23B and reflected by an external object by TOF sensors mounted on TOF sensor substrates 74A to 74D, respectively, through the TOF optical systems 71A to 71D.

[0147] Fig. 20 is a schematic diagram of the viewing angle of the VCSEL optical system 123. Fig. 20 This is a schematic diagram of light rays at the outermost viewing angle in the vertical cross section (ZX cross section) of the VCSEL optical system 23F and the TOF optical systems 71A and 71B. Fig. 20 The TOF optical systems 71B, 71C, and 71D are rotated around the X-axis so that the incident light axis of the TOF optical system 71B is parallel to the Z-axis.

[0148] like Fig. 20 As shown, by making the outermost viewing angle Ωvcse.2 of the camera device 200 greater than 90°, it is possible to ensure that the irradiable range above the VCSEL light projection unit (VCSEL optical system 23F, 23B) is an area of ​​half the celestial sphere (2π [sr]). The camera device 200 uses the light acceptance range (viewing angle Ωtof2) above the TOF optical system 71B (and 71C, 71D) and the light acceptance range (viewing angle Ωtof1) of the TOF optical system 71A as the light acceptance range above the entire TOF light receiving unit, which can cover the area of ​​half the celestial sphere (2π [sr]). Therefore, the photography of the camera device 200 can cover the shooting range of the upper half of the celestial sphere.

[0149] In the imaging device 200 of this embodiment, only the TOF optical system 71A is disposed above (first section) the VCSEL optical system 23F (second section). Therefore, unlike the imaging device 100 of the first embodiment, D2 = E2 can be set, and the dimension in the Z-axis direction does not need to be increased, which is advantageous for miniaturization.

[0150] In addition, Fig. 20 As shown, the outermost viewing angle Ωvcsel2 below the VCSEL optical system 23F in the camera device 200 can be expanded to a degree that is not blocked by the TOF light receiving system 71B (Ωvcsel2≤about 85°).

[0151] On the other hand, the camera device 200 can ensure that the light acceptance range of the TOF light receiving unit on the ZX section reaches 90° through the light acceptance range (angle of view Ωtof3) below the TOF optical system 71B. Therefore, the camera device 200 can cover almost the shooting range of the lower half of the celestial sphere by including the light receiving range of the TOF optical systems 71C and 71D.

[0152] In this way, the photography of this embodiment can cover almost the entire photographable range.

[0153] In the description of each of the above embodiments, the distance measuring device (camera 100, 200) integrally includes a plurality of VCSEL light projection units 21F, 21B as light projection units, a plurality of TOF optical systems 71 (71A, 71B, 71C, 71D) as light receiving units, a plurality of CMOS light receiving units 30R, 30L as imaging units, and a distance measuring control unit 230 that performs distance measuring calculations, but the present invention is not limited thereto. For example, a distance measuring system may be applied in which the light projection unit including a plurality of VCSEL light projection units 21F, 21B as light projection units, a TOF light receiving unit including a plurality of TOF optical systems 71 (71A, 71B, 71C, 71D) as light receiving units, a CMOS imaging unit including a plurality of CMOS light receiving units 30R, 30L as imaging units, and a distance measuring control unit 230 that performs distance measuring calculations are configured as separate devices, and these devices are connected to each other via a network or the like.

[0154] The above-described embodiments are preferred examples of the present invention, but are not limited thereto, and various modifications are allowed within the scope of the present invention.

[0155] Description of Reference Numerals

[0156] 21F, 21B light projection unit,

[0157] 30R, 30L camera unit,

[0158] 71A, 71B, 71C, 71D light receiving parts,

[0159] 100, 200 Distance measuring device,

[0160] 230 Ranging Control Unit.

Claims

1. A distance measuring device, characterized in that: have: A plurality of light projection parts for irradiating light within the range of the measurement object; A plurality of light receiving parts for receiving light reflected by an object located within the range of the measurement object; as well as a distance measurement control unit for calculating the distance to the object based on the time when the plurality of light projection units project light and the time when the plurality of light receiving units receive light, The number of the plurality of light receiving parts is greater than the number of the plurality of light projecting parts. The optical axis of at least one of the plurality of light receiving parts is oriented in the +X direction and is perpendicular to the optical axes of at least the other two light receiving parts. The optical axes of the plurality of light projection parts are perpendicular to the +X direction, The at least one light receiving portion is arranged at a first section from the +X direction, and the at least two light receiving portions are arranged at a second section from the +X side.

2. The distance measuring device according to claim 1, characterized in that: The viewing angle of each of the plurality of light receiving units is smaller than the irradiation range of each of the plurality of light projecting units.

3. The distance measuring device according to claim 1 or 2, characterized in that: A plurality of imaging units are provided for imaging the range of the measurement object.

4. The distance measuring device according to claim 1, characterized in that: A plurality of camera units are provided for photographing the range of the measurement object. The number of the plurality of imaging units is less than the number of the plurality of light receiving units. The optical axes of the plurality of imaging units are perpendicular to the optical axis of the at least one light receiving unit.

5. The distance measuring device according to claim 1 or 2, characterized in that: The plurality of light projecting portions are arranged between the at least one light receiving portion and the at least other two light receiving portions.

6. The distance measuring device according to claim 5, characterized in that: A plurality of camera units are provided for photographing the range of the measurement object. The number of the plurality of imaging units is less than the number of the plurality of light receiving units. The plurality of imaging units are arranged between the at least one light receiving unit and the at least other two light receiving units.

7. The distance measuring device according to claim 1 or 2, characterized in that: Each of the plurality of light receiving sections receives the light from at least two light projecting sections.

8. The distance measuring device according to claim 1 or 2, characterized in that: The range of the measurement target is a range of the entire celestial sphere where the plurality of light projection units can irradiate light.

9. A distance measurement system, characterized in that: have: a light projection unit having a plurality of light projection sections that irradiate light within a range of a measurement object; A light receiving unit including a plurality of light receiving parts for receiving light reflected by an object located within the range of the measurement object; as well as a distance measurement control unit for calculating the distance to the object based on the time when the plurality of light projection units project light and the time when the plurality of light receiving units receive light, The number of the plurality of light receiving parts is greater than the number of the plurality of light projecting parts. The optical axis of at least one of the plurality of light receiving parts is oriented in the +X direction and is perpendicular to the optical axes of at least the other two light receiving parts. The optical axes of the plurality of light projection parts are perpendicular to the +X direction, The at least one light receiving portion is arranged at a first section from the +X direction, and the at least two light receiving portions are arranged at a second section from the +X side.

10. The distance measurement system according to claim 9, characterized in that: An imaging unit is provided, and the imaging unit has a plurality of imaging units for imaging a range of the measurement object.

11. A camera device, characterized in that: have: A plurality of light projection parts for irradiating light within the range of the measurement object; A plurality of light receiving parts for receiving light reflected by an object located within the range of the measurement object; as well as A plurality of camera units, used for photographing the range of the measurement object, The number of the plurality of light receiving parts is greater than the number of the plurality of light projecting parts. The optical axis of at least one of the plurality of light receiving parts is oriented in the +X direction and is perpendicular to the optical axes of at least the other two light receiving parts. The optical axes of the plurality of light projection parts are perpendicular to the +X direction, The at least one light receiving portion is arranged at a first section from the +X direction, and the at least two light receiving portions are arranged at a second section from the +X side.

12. The imaging device according to claim 11, wherein: The viewing angle of each of the plurality of light receiving units is smaller than the irradiation range of each of the plurality of light projecting units.

13. The imaging device according to claim 11, wherein: The number of the plurality of imaging units is less than the number of the plurality of light receiving units. The optical axes of the plurality of imaging units are perpendicular to the optical axis of the at least one light receiving unit.

14. The imaging device according to claim 11 or 12, characterized in that: The plurality of light projecting portions are arranged between the at least one light receiving portion and the at least other two light receiving portions.

15. The imaging device according to claim 11, characterized in that: The number of the plurality of imaging units is less than the number of the plurality of light receiving units. The plurality of imaging units are arranged between the at least one light receiving unit and the at least other two light receiving units.

16. The imaging device according to claim 11 or 12, characterized in that: Each of the plurality of light receiving sections receives the light from at least two light projecting sections.

17. The imaging device according to claim 11 or 12, characterized in that: The range of the measurement target is a range of the entire celestial sphere where the plurality of light projection units can irradiate light.

Images