Distance measuring device, vehicle-mounted system and mobile device

Abstract

[Problem] To provide a compact and highly accurate distance measuring device. [Solution] A distance measuring device (1) is provided with: an optical system (101) in which the focal length in a peripheral region farther from an optical axis is shorter than the focal length in a central region including the optical axis; an image sensor (102) including a plurality of pixels configured to receive an image of the subject formed by the optical system; and an acquisition unit (103) configured to acquire distance information relating to the subject from the parallax image acquired by the image sensor on the basis of light beams from two different pupil positions in the optical system. A parallax direction of the parallax image is a direction including a component in a direction orthogonal to a mounting surface of the distance measuring device.

Description

Technical Field

[0001] The invention relates to an optical device, a vehicle-mounted system and a movable device. Background Art

[0002] Conventionally, a so-called stereo distance measuring camera is known in which distance information is acquired using parallax of images obtained by a plurality of cameras. Patent document 1 discloses a measuring device configured to measure the distance between vehicles using a parallax image in a vertical direction obtained by vertically arranging two cameras. Patent document 2 discloses an imaging device configured to set the parallax direction to each of the horizontal, vertical, and diagonal directions.

[0003] Prior art documents

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent 3456843

[0006] Patent Document 2: Japanese Patent 6981410 Summary of the invention

[0007] Problem that the invention aims to solve

[0008] The measuring device disclosed in Patent Document 1 separates two cameras to obtain the parallax amount, so the size of the measuring device increases. In the imaging device disclosed in Patent Document 2, the parallax direction can be selected, but when an optical system called a fovea lens is used that increases the focal length near the central angle of view while expanding the peripheral angle of view, the focal length at the peripheral angle of view decreases, so the distance measurement accuracy deteriorates.

[0009] Therefore, an object of the present invention is to provide a small and highly accurate distance measuring device.

[0010] Solutions for solving problems

[0011] A distance measuring device according to an aspect of the present invention includes: an optical system in which a focal length in a peripheral area away from an optical axis is shorter than a focal length in a central area including the optical axis; an image sensor including a plurality of pixels configured to receive an image of a subject formed by the optical system; and an acquisition unit configured to acquire distance information about the subject from a parallax image acquired by the image sensor based on light beams from two different pupil positions in the optical system. The parallax direction of the parallax image is a direction including a component in a direction orthogonal to a mounting surface of the distance measuring device.

[0012] Other objects and features of the present invention will be described in the following examples.

[0013] Effects of the Invention

[0014] The present invention can provide a small and high-precision distance measuring device. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A distance measuring device according to various examples is described.

[0016] Figure 2 Image sensors according to Examples 1 and 3 are described.

[0017] Figure 3 Shown is how a light beam may be directed to a pixel on an image sensor according to various examples.

[0018] Figure 4 The relationship between the distance measurement accuracy and the focal length according to various examples is explained.

[0019] Figure 5 is a cross-sectional view of an optical system according to Example 1.

[0020] Figure 6 The relationship between the angle of view and the focal length of the optical system according to Example 1 is shown.

[0021] Figure 7 The relationship between the angle of view and the focal ratio of the optical system according to each example is shown.

[0022] Figure 8 The relationship between the viewing angle and focal length change of the optical system according to each example is shown.

[0023] Fig. 9 A movable body equipped with the distance measuring device according to Example 2 is described.

[0024] Fig.10 is a cross-sectional view of an optical system according to Example 2.

[0025] Fig.11 An image sensor according to Example 2 is described.

[0026] Fig.12 The relationship between the viewing angle and the image height of the optical system according to each example is shown.

[0027] Fig.13 The installation method of the distance measuring device according to Example 3 is explained.

[0028] Fig.14 is a cross-sectional view of an optical system according to Example 3.

[0029] Fig.15 is a schematic diagram of a movable device according to various examples and illustrates optical characteristics of an optical system.

[0030] Fig.16 is a block diagram showing a configuration example of an in-vehicle system according to various examples. DETAILED DESCRIPTION

[0031] A detailed description will now be given of embodiments according to the present invention with reference to the accompanying drawings.

[0032] (Example 1)

[0033] Reference now Figure 1 , a description will be given of a distance measuring apparatus (imaging apparatus) 1 according to Example 1 of the present invention. Figure 1 A distance measuring device 1 is described. The distance measuring device 1 includes an optical system (imaging optical system) 101, an image sensor 102, and an acquisition unit 103. The distance measuring device 1 is fixed to the ground (mounting surface) 22 using a fixing member 23, and is installed in the horizontal direction (z direction) relative to the ground 22 (zx plane). The distance measuring device 1 acquires a light beam 10 from a subject 21. The image sensor 102 can acquire light beams 11 and 12 from two different pupil positions in the optical system 101, respectively. The direction in which the image sensor 102 splits the light beams 11 and 12 is a direction including a component in a direction (y direction) perpendicular to the ground 22. Here, the direction including a component in a direction perpendicular to the ground 22 is 80≤θ1≤100, where θ1 (degrees) is an angle relative to the ground 22. This conditional expression is preferably 85≤θ1≤95, and more preferably θ1=90.

[0034] In the image sensor 102, the image based on the light beam 11 and the image based on the light beam 12 generate parallax in the direction (y direction) perpendicular to the ground 22 according to the subject distance. Parallax means that the direction (angle) of viewing the subject is different depending on the position of the observation point, and the parallax direction means the direction in which the parallax occurs.

[0035] Reference now Figure 2 , a description will be given of the image sensor 102 according to the present example. Figure 2 The image sensor 102 is illustrated, which shows a planar arrangement on the image sensor 102. The image sensor 102 receives an image of a subject formed by the optical system 101, and includes a pixel 201 configured to detect light corresponding to the light beam 11 and a pixel 202 configured to detect light corresponding to the light beam 12 different from the light beam 11. Each pixel 200 includes a pair of pixels 201 and 202. Figure 2 As shown, the image sensor 102 has a plurality of pixels 200 arranged two-dimensionally on the xy plane, and can acquire two two-dimensional images having parallax in the parallax direction (y direction) according to the object distance. A similar image sensor 102 can be used for Example 3 described later.

[0036] Reference Figure 3 , a description will be given of the relationship between the light beams 11 and 12 and the pixels 201 and 202 in the image sensor 102. Figure 3 It is shown how light beams 11 and 12 are directed to pixels 201 and 202 . Figure 3 One of the plurality of pixels 200 arranged two-dimensionally on the xy plane is shown as a representative. The pixels 202 and 203 are in a conjugate relationship with two pupil positions in the optical system 101 through the microlens 203, and the light beams 11 and 12 are guided to the corresponding pixels 201 and 202, respectively. The acquisition unit 103 acquires (calculates) the distance information to the subject 21 based on the parallax amount of the images acquired by the pixels 201 and 202. Figures 1 to 3 The configuration shown defines the pupil segmentation direction relative to the ground 22 and performs distance measurements.

[0037] Reference now Figure 4 , a description will be given of the relationship between the distance measurement accuracy and the focal length of the optical system 101. Figure 4 The relationship between the distance measurement accuracy and the focal length is described. The distance from the object 24 to the optical system 101 is Z, the focal length of the optical system 101 (the imaging point of the object at infinity) is f, the parallax amount in the image sensor 102 is r, and the offset amount of the pupil position in the optical system 101 is B. In this case, the relationship Z≈B×f / r holds. In the case where the distance change ΔZ and the parallax amount at that time are represented by Δr, the distance measurement accuracy can be expressed as ΔZ≈Δr×Z^2 / (B×f). Therefore, in order to improve the distance measurement accuracy, the focal length f of the optical system 101 is increased.

[0038] Reference now Figure 5 , an optical system 110 according to the present example will be described. The optical system 110 is used as the optical system 101 of the distance measuring device 1. The optical system 110 includes a first group G1, an aperture stop (aperture) STO, and a second group G2 in sequence from the object side to the image side. That is, the optical system 110 includes an aperture stop STO, a first group G1 arranged on the object side of the aperture stop STO, and a second group G2 arranged on the image side of the aperture stop STO. The first group G1 includes lenses L1, L2, and L3 arranged in sequence from the object side to the image side. Lens L1 is a concave meniscus lens having a convex surface facing the object side and having an inflection point. The second group G2 includes lenses L4, L5, L6, and L7 arranged in sequence from the object side to the image side. CG is an optical block, and IMG is an image plane. In the optical system 110, the central focal length is 11.4 mm, the overall viewing angle is 120°, and the Fno (F value) is 1.8.

[0039] The optical system 110 is a fovea lens that increases the focal length of the central region (near the center of the viewing angle) including the optical axis OA while expanding the peripheral region away from the optical axis OA (the focal length of the peripheral region is shorter than that of the central region). Therefore, the distance measurement accuracy of the central region can be improved.

[0040] Reference Figure 6 , a description will be given of the relationship between the angle of view and the focal length of the optical system 110. Figure 6 The relationship between the viewing angle and focal length of the optical system 110 is shown, and the focal length variation is discretely shown in the x and y directions. Now consider the x direction horizontal to the ground 22. Then, for example, when the viewing angle (x) is 60 degrees and the viewing angle (y) is 0 degrees, the focal length in the x direction is 1.4 mm, while the focal length in the y direction is 5.6 mm. For this viewing angle, the focal length in the y direction is 4 times longer. On the contrary, when the viewing angle (x) is 0 degrees and the viewing angle (y) is 60 degrees, the focal length in the x direction is 5.6 mm, while the focal length in the y direction is 1.4. For this viewing angle, the focal length in the x direction is longer. In other words, the focal length direction utilizing parallax is determined based on the viewing angle to be emphasized.

[0041] For example, in the distance measuring device 1 installed on the ground 22, the distance measuring accuracy in the horizontal direction where there are more objects to be processed (such as people) is prioritized to be improved, compared with the distance measuring accuracy in the vertical direction where most objects are located on the ground or in the sky. Therefore, in order to improve the distance measuring accuracy in the x direction which is the horizontal direction, the parallax direction to be used by the distance measuring device 1 is preferably set to the y direction which is the vertical direction.

[0042] Now assume that f_dist is the focal length of the optical system 110 in the y direction which is the direction (azimuth) in which the parallax is acquired (parallax direction), and f_per is the focal length of the optical system 110 in the x direction which is the direction orthogonal to the parallax direction. Then, the following conditional expression (1) is preferably satisfied:

[0043] f_dist / f_per > 2.0 (1)

[0044] Conditional expression (1) is a conditional expression for increasing the focal length near the center of the angle of view while expanding the peripheral angle of view, thereby improving the distance measurement accuracy in the horizontal direction. As the value of f_dist / f_per increases, the focal length used for distance measurement increases, and this is preferred because the distance measurement accuracy in the horizontal direction can be improved. It is not preferred that the value becomes lower than the lower limit of conditional expression (1) because the focal length near the center of the angle of view decreases, or the peripheral angle of view decreases. Since the left side of conditional expression (1) is 4 at the maximum horizontal angle of view, the optical system 110 according to this example satisfies conditional expression (1) and the value is large.

[0045] Reference now Figure 7 , a description will be given of the left side (focal length ratio) of conditional expression (1) in numerical example 1 of the optical system 110 according to the present example and numerical examples 2 and 3 of the optical systems 120 and 130 according to examples 2 and 3 described later. Figure 7 1 shows the relationship between the viewing angle and the focal length ratio of the optical systems 110, 120, and 130 according to various examples, wherein the text ex1 is used for example 1, ex2 is used for example 2, and ex3 is used for example 3. Figure 7 , the horizontal axis indicates the horizontal angle of view (degrees), and the vertical axis indicates the focal length ratio f_dist / f_per (the left side of conditional expression (1)). In each example, from the center of the angle of view to the off-axis position, the focal length in the direction in which the parallax is obtained becomes greater than the focal length in the direction orthogonal to the above direction, and the relationship of conditional expression (1) is satisfied at the maximum horizontal angle of view.

[0046] In each example, the numerical range of the conditional expression (1) is more preferably set to the following conditional expression (1a):

[0047] f_dist / f_per>2.2(1a)

[0048] In each example, the numerical range of the conditional expression (1) is most preferably set to the following conditional expression (1b):

[0049] f_dist / f_per>2.4(1b)

[0050] In each example, it is assumed that f_dist_max is the focal length of the optical system at the maximum viewing angle (maximum horizontal viewing angle) in the direction orthogonal to the parallax direction among the focal length f_dist of the optical system in the parallax direction, and f_AX is the focal length of the optical system at the center viewing angle. Then, it is preferable to satisfy the following conditional expression (2).

[0051] 0.40 < f_dist_max / f_AX < 0.90 (2)

[0052] Conditional expression (2) is a conditional expression for improving the distance measurement performance over the entire screen (entire angle of view). It is not preferable that the value becomes higher than the upper limit of conditional expression (2) because the distance measurement performance at the center of the angle of view decreases. On the other hand, it is not preferable that the value becomes lower than the lower limit of conditional expression (2) because the distance measurement performance in the peripheral area decreases. Since the value of conditional expression (2) is 0.50, the optical system 110 satisfies conditional expression (2).

[0053] Reference now Figure 8, a description will be given of focal length variation in the case of normalization by the focal length f_AX at the center angle of view in each of Numerical Examples 1 to 3. Figure 8 The relationship between the viewing angle and the focal length change of the optical system according to various examples is shown. Figure 8 , the focal lengths of the optical systems according to Examples 1 to 3 in the parallax direction are shown as f_dist1, f_dist2, and f_dist3, respectively, and the focal lengths of the optical systems in the direction orthogonal to the parallax direction are shown as f_per1, f_per2, and f_per3, respectively. Figure 8 In , the horizontal axis represents the horizontal viewing angle (degrees), and the vertical axis shows the standardized focal length. The value on the vertical axis at the maximum viewing angle corresponds to the conditional expression (2). Figure 8 As shown, f_per1 to f_per3 are less than 0.4 at the maximum viewing angle, but the focal lengths f_dist1 to f_dist3 in the parallax direction are greater than or equal to 0.4. Thus, the reduction of the focal length for distance measurement can be suppressed at the peripheral viewing angle, and the distance measurement accuracy can be improved.

[0054] In each example, the numerical range of the conditional expression (2) is more preferably set to the following conditional expression (2a):

[0055] 0.44 <f_dist_max / f_AX<0.87(2a)

[0056] In each example, the numerical range of the conditional expression (2) is most preferably set to the following conditional expression (2b):

[0057] 0.48 <f_dist_max / f_AX<0.84(2b)

[0058] The present example can provide a distance measurement device having a wide viewing angle in the horizontal direction, and suppress a decrease in distance measurement accuracy.

[0059] (Example 2)

[0060] Reference now Fig. 9 , a description will be given of a movable body (movable device) 25 on which the distance measuring device 1 according to Example 2 of the present invention is mounted. Fig. 9A movable body 25 on which a distance measuring device 1 is mounted is described. The distance measuring device 1 according to the present example is mounted facing the moving direction z of the movable body 25. The movable body 25 can move in contact with the ground 22. The present example utilizes the parallax direction of the distance measuring device 1 fixed to the y direction orthogonal to the ground 22 for the movable body 25. As described above, the moving direction of the movable body 25 and the parallax direction of the distance measuring device 1 are in a predetermined relationship (the parallax direction is orthogonal to the moving direction of the movable body 25). Thus, the present example can provide a distance measuring device having a wide viewing angle in the horizontal direction x and suppressing a decrease in the distance measurement accuracy.

[0061] Reference now Fig.10 , an optical system 120 according to this example will be described. The optical system 120 is used as the optical system 101 of the distance measuring device 1. Similar to the optical system 110 of Example 1, the optical system 120 includes a first group G1, an aperture stop (aperture) STO, and a second group G2. The first group G1 includes lenses L1, L2, L3, and L4 arranged in sequence from the object side to the image side. Lens L1 is a concave meniscus lens having a convex surface facing the object side and having an inflection point. The second group G2 includes lenses L5, L6, L7, L8, and L9 arranged in sequence from the object side to the image side. CG is an optical block, and IMG is an image plane. In the optical system 120, the central focal length is 9.0 mm, the overall viewing angle is 120°, and the Fno (F value) is 1.8.

[0062] The optical system 120 is a fovea lens that increases the focal length of the central area (near the center of the angle of view) while expanding the peripheral area (the focal length of the peripheral area is shorter than that of the central area), so that the distance measurement accuracy of the central area can be improved. In addition, in the optical system 120, the left side of the conditional expression (1) is 3.2 at the maximum horizontal angle of view. Using the optical system 120 that satisfies the conditional expression (1) can provide a distance measurement device that has a wide angle of view in the horizontal direction and suppresses a decrease in distance measurement accuracy.

[0063] Reference now Fig.11 , the configuration of the image sensor 102a according to the present example will be described. Fig.11 The image sensor 102a is described. The distance measuring device according to this example includes the image sensor 102a instead of the image sensor 102 according to Example 1. The image sensor 102a is different from the image sensor 102a according to the reference. Figure 2The image sensor 102 described in Example 1 is similar in that the plurality of pixels 200 each include a pixel 201 and a pixel 202, and are arranged on the xy plane. However, the image sensor 102a is different from the image sensor 102 in that the number of pixels in the parallax direction (y direction) is smaller than the number of pixels in the direction (x direction) orthogonal to the parallax direction. In the case where the image sensor 102a is used together with the optical system 101, the direction with a smaller number of pixels is a direction with a smaller viewing angle, and the direction including the component in the direction of the minimum viewing angle of the image sensor 102a coincides with the parallax direction. The direction including the component in the direction of the minimum viewing angle is -10≤θ2≤10, where θ2 (degrees) is an angle relative to the parallax direction. This conditional expression is preferably -5≤θ2≤5, and more preferably θ2=0. Using this combination can suppress the reduction of the focal length in the parallax direction relative to the x-direction side with a wide viewing angle. As a result, the reduction in the distance measurement accuracy relative to the direction with a wider viewing angle can be suppressed.

[0064] When a fovea lens is used, the center of the screen (image) is magnified. Therefore, when an image sensor 102a having a small number of pixels in the y direction is used as in this example, the viewing angle in the vertical direction may not be sufficiently ensured unless the image height of the optical system is appropriately set.

[0065] Fig.12 The relationship between the viewing angle and the image height of the optical system according to various examples is shown. Fig.12 In , example 1 is shown by the text description ex1, example 2 is shown by the text description ex2, and example 3 is shown by the text description ex3. Fig.12 , the horizontal axis indicates the viewing angle (degree), and the vertical axis indicates the normalized image height.

[0066] In each example, it is assumed that h_HA is the image height at a viewing angle that is half the maximum viewing angle, and f_AX is the focal length at the center viewing angle. Then, it is preferable to satisfy the following conditional expression (3):

[0067] 0.25 < h_HA / f_AX < 0.55 (3)

[0068] Conditional expression (3) is a conditional expression for appropriately setting the viewing angle in the y direction when the number of pixels in the y direction is small. It is not preferable that the value becomes higher than the upper limit of conditional expression (3) because the central area is enlarged too much and the viewing angle in the y direction becomes too narrow. On the other hand, it is not preferable that the value becomes lower than the lower limit of conditional expression (3) because the central area is not enlarged and the distance measurement accuracy in the central area is reduced.

[0069] In each example, the numerical range of the conditional expression (3) is more preferably set to the following conditional expression (3a):

[0070] 0.27 <h_HA / f_AX<0.52(3a)

[0071] In each example, the numerical range of the conditional expression (3) is most preferably set to the following conditional expression (3b):

[0072] 0.29 <h_HA / f_AX<0.49(3b)

[0073] The present example can provide a distance measuring device having a wide horizontal viewing angle and suppressing a decrease in distance measurement accuracy.

[0074] (Example 3)

[0075] Reference now Fig.13 , a description will be given of an indoor installation method of the distance measuring device 1 according to Example 3 of the present invention. Fig.13 The installation method of the distance measuring device 1 according to this example is described. In this example, the distance measuring device 1 is fixed to the ceiling 27 via the fixing unit 26. At this time, as shown in FIG. Fig.13 As shown, in order to improve the distance measurement accuracy of the image acquired in the direction of the ground 22 as the direction of interest, the distance measuring device 1 is tilted relative to the zy plane. The vertical direction in the acquired image is limited by the ground 22 and the ceiling 27, so that the viewing angle is not wide. In the horizontal direction x, in order to prepare a wider viewing angle, the parallax direction is set to the y direction side of the zy plane so that the distance measurement accuracy is not reduced. As described above, even when monitoring a fixed point where the distance measuring device 1 is installed obliquely, distance measurement can be performed while suppressing the reduction of the distance measurement accuracy in the desired direction.

[0076] Reference now Fig.14 , an optical system 130 according to this example will be described. The optical system 130 is used as the optical system 101 in the distance measuring device 1. Similar to the optical system 110 according to Example 1, the optical system 130 includes a first group G1, an aperture stop (aperture) STO, and a second group G2. The first group G1 includes lenses L1, L2, and L3 arranged in sequence from the object side to the image side. Lens L1 is a concave meniscus lens having a convex surface toward the object side and having an inflection point. The second group G2 includes lenses L4, L5, and L6 arranged in sequence from the object side to the image side. CG is an optical block, and IMG is an image plane. In the optical system 120, the central focal length is 5.5 mm, the overall viewing angle is 120°, and the Fno (F value) is 1.8.

[0077] The optical system 130 is a fovea lens that increases the focal length of the central area (near the center of the angle of view) while expanding the peripheral area (the focal length of the peripheral area is shorter than that of the central area), thereby improving the distance measurement accuracy of the central area. In the optical system 130, the left side of the conditional expression (1) is 2.7 at the maximum horizontal angle of view. Using the optical system 130 that satisfies the conditional expression (1) can provide a distance measurement device that has a wide angle of view in the horizontal direction and suppresses a decrease in distance measurement accuracy.

[0078] Numerical Examples 1 to 3 corresponding to Examples 1 to 3 will be shown below. In each numerical example, the surface number is the order of each optical surface when counting from the object surface. r [mm] represents the radius of curvature of the i-th optical surface, and d [mm] represents the distance between the i-th surface and the (i + 1)-th surface (the distance on the optical axis). Nd represents the refractive index of the medium between the i-th surface and the (i + 1)-th surface for the d-line (wavelength 587.6nm), and νd represents the Abbe number of the medium based on the d-line. The Abbe number νd is defined by the following formula, where nF, nd and nC are the refractive indices for the F-line, d-line and C-line, respectively:

[0079] νd=(nd-1) / (nF-nC)

[0080] In each numerical example, an asterisk (*) added to a surface number indicates that the surface has an aspherical shape. “E±P” in each numerical value means “×10 ±P The shape of each aspherical surface is expressed by the following formula:

[0081] [Number 1]

[0082]

[0083] Here, z represents the displacement from the surface vertex in the optical axis direction, h represents the height from the optical axis OA in the direction orthogonal to the optical axis direction, c represents the curvature (the inverse of the radius of curvature r), and k represents the cone coefficient. A, B, C, D, E, F, G, ... are aspheric coefficients of the fourth order, sixth order, eighth order, tenth order, twelfth order, fourteenth order, sixteenth order, etc., respectively.

[0084] (Numerical Example 1)

[0085] Area data

[0086]

[0087]

[0088] Aspheric coefficient

[0089]

[0090]

[0091] (Numerical Example 2) Surface Data

[0092]

[0093] Aspheric surface data

[0094]

[0095] (Numerical Example 3) Surface Data

[0096]

[0097] Aspheric surface data

[0098]

[0099]

[0100] Table 1 summarizes the values ​​of each conditional expression used in each of Examples 1 to 3.

[0101] Table 1

[0102]

[0103] In-vehicle system

[0104] Fig.15 (A) is a schematic diagram of a movable device 10 and a distance measuring device (imaging device, vehicle-mounted camera) 1 held by the movable device 10. Fig.15 In (A), the mobile device 10 is a car (vehicle). The mobile device 10 includes an unillustrated on-vehicle (in-vehicle) system (driving support device) for supporting a user (driver, passenger, etc.) 40 of the mobile device 10 using an image acquired by the distance measuring device 1. In this example, the distance measuring device 1 is installed to capture images from the rear of the mobile device 10, but the distance measuring device 1 may be installed to capture images from the front or side of the mobile device 10. In addition, two or more distance measuring devices 1 may be installed at two or more places on the mobile device 10.

[0105] According to various examples, the distance measuring device 1 includes an optical system 101 and an image sensor 102 (102a). The optical system 101 is an optical system (fovea lens, different angle lens) in which the imaging magnification is different between a first angle of view (first field of view) 30 and a second angle of view (second field of view) 31 greater than the first angle of view 30. The imaging surface (light receiving surface) of the image sensor 102 includes a first area for imaging an object included in the first angle of view 30 and a second area for imaging an object included in the second angle of view 31. At this time, the number of pixels per unit angle of view in the first area is greater than the number of pixels per unit angle of view in the second area other than the first area. In other words, in the distance measuring device 1, the resolution of the first angle of view (first area) is higher than the resolution at the second angle of view (second area).

[0106] The optical characteristics of the optical system 101 will be described in detail below. Fig.15 The left diagram in (B) shows the half angle of view θ [degrees] on the imaging plane of the image sensor 102 as contour lines at the image height y [mm]. Fig.15 The right diagram in (B) is a graph showing the relationship between each half angle of view θ and the image height y in the first quadrant of the left diagram (projection characteristics of the optical system 101 ).

[0107] like Fig.15 As shown in (B), the optical system 101 is configured so that the projection characteristic y(θ) differs between a viewing angle less than a predetermined half viewing angle θa and a viewing angle equal to or greater than the half viewing angle θa. Therefore, the increase (resolution) of the image height y per unit half viewing angle θ also differs for each viewing angle. The local resolution of the optical system 101 is expressed as the differential value dy(θ) / dθ of the projection characteristic y(θ) with respect to the half viewing angle θ. Fig.15 The left figure in (B) shows that the longer the distance between the contour lines of the image height y for each half viewing angle θ, the higher the resolution. Fig.15 The right graph in (B) shows that the greater the slope of the graph of the projection characteristic y(θ), the higher the resolution.

[0108] exist Fig.15 In the left figure of (B), the first area 301a as the central area corresponds to a viewing angle less than the half viewing angle θa, and the second area 301b as the peripheral area corresponds to a viewing angle equal to or greater than the half viewing angle θa. Fig.15 The first viewing angle 30 in (A) and the combined viewing angle of the viewing angle less than the half viewing angle θa and the viewing angle equal to or greater than the half viewing angle θa corresponds to Fig.15 The second viewing angle 31 in (A). As described above, the first area 301a is an area of ​​high resolution and low distortion, and the second area 301b is an area of ​​low resolution and high distortion.

[0109] The value θa / θmax of the ratio of the half viewing angle θa to the maximum half viewing angle θmax is preferably greater than or equal to 0.15 and less than or equal to 0.35, and more preferably greater than or equal to 0.16 and less than or equal to 0.25. For example, in this example, since the maximum half viewing angle θmax = 90°, the value of the half viewing angle θa is preferably greater than or equal to 13.5° and less than or equal to 31.5°, and more preferably greater than or equal to 14.4° and less than or equal to 22.5°.

[0110] The optical system 101 is configured so that the projection characteristic y(θ) in the first region 301a is different from f×θ (equidistant projection method) and is also different from the projection characteristic in the second region 301b. In this case, the projection characteristic y(θ) of the optical system 101 can satisfy the following conditional expression (4):

[0111] 1.0 < f×sin(θmax) / y(θmax) ≤ 1.9 (4)

[0112] Satisfying the conditional expression (4) can realize a wide viewing angle of the optical system 101 by reducing the resolution in the second region 301b. The resolution of the first region 301a can be higher than the resolution of the central region of a typical fisheye lens using the orthogonal projection method (y(θ)=f×sinθ). It is not preferable that the value becomes lower than the lower limit of the conditional expression (4) because the resolution in the first region 301a becomes lower than the resolution of the fisheye lens using the orthogonal projection method, the maximum image height becomes larger, or the size of the optical system increases. It is not preferable that the value becomes higher than the upper limit of the conditional expression (4) because the resolution in the first region 301a becomes too high, it becomes difficult to realize a wide viewing angle comparable to the wide viewing angle of the fisheye lens using the orthogonal projection method, or good optical performance cannot be maintained.

[0113] It is preferable to satisfy the following conditional expression (4a), and it is more preferable to satisfy the following conditional expression (4b):

[0114] 1.0 <f×sin(θmax) / y(θmax)≤1.7(4a)

[0115] 1.0 <f×sin(θmax) / y(θmax)≤1.4(4b)

[0116] As described above, in the first area 301a, the distortion of the optical system 101 is small and the resolution is high, so an image with higher definition than that obtained in the second area 301b can be obtained. Therefore, by setting the first area 301a (first viewing angle 30) as the area of ​​interest of the user 40, good visibility can be obtained. For example, Fig.15As shown in (A), in the case where the distance measuring device 1 is placed behind the movable device 10, by displaying an image corresponding to the first viewing angle 30 on the electronic rearview mirror, a natural stereoscopic sense can be obtained when the user 40 looks at the rear vehicle, etc. On the other hand, the second area 301b (second viewing angle 31) corresponds to a wide viewing angle including the first viewing angle 30. Therefore, for example, in the case where the movable device 10 is reversing, an image corresponding to the second viewing angle 31 can be displayed on the display unit in the vehicle to assist the user 40 in driving.

[0117] Fig.16 2 is a functional block diagram for explaining a configuration example of the vehicle-mounted system 2 according to the present example. The vehicle-mounted system 2 is a system for displaying an image obtained by the distance measuring device 1 installed behind the movable device 10 to the user 40. The vehicle-mounted system 2 includes the distance measuring device 1, a processing device 220, and a display device (display unit) 230. The distance measuring device 1 includes the optical system 101 and the image sensor 102 as described above. The image sensor 102 includes a photoelectric conversion element such as a CCD sensor or a CMOS sensor, generates image data by photoelectrically converting an optical image formed by the optical system 101, and outputs the image data to the processing device 220.

[0118] The processing device 220 includes an image processing unit 221, a display viewing angle judgment unit 224 (judgment unit), a user setting change unit 226 (first change unit), a rear vehicle distance detector 223 (first detector), a reverse (backward) gear detector 225 (second detector) and a display viewing angle change unit 222 (second change unit). The processing device 220 is a computer such as a central processing unit (CPU) microcomputer, etc., and serves as a control unit configured to control the operation of each component based on a computer program. At least one component in the processing device 220 can be implemented by hardware such as an application specific integrated circuit (ASIC) or a programmable logic array (PLA).

[0119] The image processing unit 221 generates image data by performing image processing such as wide dynamic range (WDR) correction, gamma correction, lookup table (LUT) processing, and distortion correction on the camera data acquired from the image sensor 102. Distortion correction is performed on at least the camera data corresponding to the second area 301b. As a result, the user 40 can easily visually recognize the image displayed on the display device 230, and the rear vehicle detection rate of the rear vehicle distance detector 223 is improved. Distortion correction may not be performed on the camera data corresponding to the first area 301a. The image processing unit 221 outputs the image data generated by performing the above-mentioned image processing to the display viewing angle changing unit 222 and the rear vehicle distance detector 223.

[0120] The rear vehicle distance detector 223 corresponds to the acquisition unit according to each of the above-mentioned examples, and acquires information about the distance to the rear vehicle included in the image data corresponding to the range of the second viewing angle 31 excluding the first viewing angle 30 using the image data output from the image processing unit 221. In addition to the distance calculated using the parallax, the rear vehicle distance detector 223 can detect the rear vehicle based on the image data corresponding to the second area 301b in the image data, and calculate the distance to his vehicle based on the change in the position or size of the detected rear vehicle, etc. The rear vehicle distance detector 223 outputs the calculated distance information to the display viewing angle judgment unit 224.

[0121] The rear vehicle distance detector 223 can determine the vehicle type of the rear vehicle based on data related to characteristic information (such as shape and color of each vehicle type) output as a result of machine learning (deep learning) based on a large number of vehicle images. At this time, the rear vehicle distance detector 223 can output information related to the vehicle type of the rear vehicle to the display viewing angle judgment unit 224. The reverse gear detector 225 detects whether the transmission of the movable device 10 (his vehicle) is shifted to the reverse gear, and outputs the detection result to the display viewing angle judgment unit 224.

[0122] The display angle of view judgment unit 224 judges whether the angle of view (display angle of view) of the image to be displayed on the display device 230 will be the first angle of view 30 or the second angle of view 31 based on the output of at least one of the rear vehicle distance detector 223 and the reverse gear detector 225. Then, the display angle of view judgment unit 224 provides an output to the display angle of view changing unit 222 according to the judgment result. For example, in the case where the distance value in the distance information is equal to or less than a predetermined threshold value (for example, 3m), the display angle of view judgment unit 224 can judge that the display angle of view will be the second angle of view 31, and in the case where the distance value is greater than the threshold value, it can be judged that the display angle of view will be the first angle of view 30. Alternatively, in the case where the reverse gear detector 225 notifies the user that the transmission of the movable device 10 is shifted to the reverse gear, the display angle of view judgment unit 224 can judge that the display angle of view will be the second angle of view 31. In the case where the vehicle is not shifted to the reverse gear, the display angle of view judgment unit 224 can judge that the display angle of view will be the first angle of view 30.

[0123] In the case where the transmission of the movable device 10 is shifted to the reverse gear, the display angle determination unit 224 can determine that the display angle is the second angle 31 regardless of the result of the rear vehicle distance detector 223. In the case where the transmission of the movable device 10 is not shifted to the reverse gear, the display angle determination unit 224 can determine that the display angle is to be determined based on the detection result of the rear vehicle distance detector 223. The display angle determination unit 224 can change the determination criterion for changing the angle of view according to the vehicle type of the movable device 10 by receiving the vehicle type information from the rear vehicle distance detector 223. For example, in the case where the movable device 10 is a large vehicle such as a truck, the braking distance is longer than that of an ordinary vehicle, so the above threshold value can be set to be longer than that of an ordinary vehicle (for example, 10m).

[0124] The user setting change unit 226 allows the user 40 to change the judgment standard used by the display angle judgment unit 224 to judge whether to change the display angle to the second angle 31. The judgment standard set (changed) by the user 40 is input from the user setting change unit 226 to the display angle judgment unit 224.

[0125] The display angle changing unit 222 generates a display image to be displayed on the display device 230 according to the judgment result of the display angle judging unit 224. For example, in the case where it is judged that the display angle will be the first angle 30, the display angle changing unit 222 cuts out a rectangular narrow-angle image (first image) from the image data corresponding to the first angle 30, and outputs the rectangular narrow-angle image to the display device 230. In the case where a rear vehicle that satisfies a predetermined condition exists in the image data corresponding to the second angle 31, the display angle changing unit 222 outputs an image (second image) including the rear vehicle to the display device 230. The second image may include an image corresponding to the first area 301a. The display angle changing unit 222 serves as a display control unit configured to perform display control to switch between a first display state in which the display device 230 displays the first image and a second display state in which the display device 230 displays the second image.

[0126] The display perspective changing unit 222 cuts out the image by storing the image data output from the image processing unit 221 in a storage unit (memory) such as a RAM, and reading out the image to be cut out from the storage unit. The area corresponding to the first image in the image data is a rectangular area in the first perspective 30 corresponding to the first area 301a. The area corresponding to the second image in the image data is a rectangular area including the rear vehicle in the second perspective 31 corresponding to the second area 301b.

[0127] The display device 230 has a display unit such as a liquid crystal display or an organic EL display, and displays the display image output from the display viewing angle changing unit 222. For example, the display device 230 has a first display unit as an electronic rearview mirror arranged above the windshield (front glass) of the movable device 10 and a second display unit as an operation panel (monitor) arranged below the windshield of the movable device 10. In this configuration, the first image and the second image generated from the above-mentioned image data can be displayed on the first display unit and the second display unit, respectively. The first display unit may include, for example, a half mirror to be used as a reflector when not used as a display unit. The second display unit may also be used as, for example, a display unit of a navigation system or an audio system.

[0128] The movable device 10 is not limited to vehicles such as cars, but may be a movable body such as a ship, an airplane, an industrial robot, or a drone. In addition, the in-vehicle system 2 according to the present example is used to display an image to the user 40, but may also be used for driving assistance such as cruise control (including an adaptive cruise control function) and automatic driving. In addition, the in-vehicle system 2 is not limited to movable devices, and may be applied to various devices using object recognition such as an intelligent transportation system (ITS).

[0129] The vehicle-mounted system 2 may include a judgment unit configured to judge the possibility of collision with an object based on the information related to the distance to the object obtained by the distance measuring device 1. A stereo camera having two image sensors 102 may be used as the distance measuring device 1. In this case, even if an image plane phase difference sensor is not used, image data can be simultaneously obtained by each synchronous imaging unit, and processing similar to the above can be performed by using these two image data. However, if the imaging time difference between these imaging units is known, the imaging units may not be synchronized.

[0130] Each example can provide a distance measuring device having a compact size and improved distance measurement accuracy in the horizontal direction even if the distance measuring device uses a fovea lens. Therefore, each example can provide a distance measuring device, a vehicle-mounted system, and a movable device each having a compact size and high accuracy.

[0131] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various variations and modifications may be made within the scope of the gist.

Claims

1. A distance measuring device, comprising: An optical system, wherein a focal length in a peripheral region away from an optical axis is shorter than a focal length in a central region including the optical axis; an image sensor including a plurality of pixels configured to receive an image of a subject formed by the optical system; and an acquisition unit configured to acquire distance information related to the object from the parallax image acquired by the image sensor based on light beams from two different pupil positions in the optical system, The invention is characterized in that the parallax direction of the parallax image is a direction including a component in a direction orthogonal to the installation surface of the distance measuring device.

2. A distance measuring device, comprising: An optical system, wherein a focal length in a peripheral region away from an optical axis is shorter than a focal length in a central region including the optical axis; an image sensor including a plurality of pixels configured to receive an image of a subject formed by the optical system; and an acquisition unit configured to acquire distance information related to the object from the parallax image acquired by the image sensor based on light beams from two different pupil positions in the optical system, It is characterized in that the parallax direction of the parallax image is a direction of a component including a direction of a minimum viewing angle of the image sensor.

3. A distance measuring device, which is mounted on a movable device, the distance measuring device comprising: An optical system, wherein a focal length in a peripheral region away from an optical axis is shorter than a focal length in a central region including the optical axis; an image sensor including a plurality of pixels configured to receive an image of a subject formed by the optical system; and an acquisition unit configured to acquire distance information related to the object from the parallax image acquired by the image sensor based on light beams from two different pupil positions in the optical system, It is characterized in that the parallax direction of the parallax image is a direction including a component in a direction orthogonal to the moving direction of the movable device.

4. The distance measuring device according to any one of claims 1 to 3, characterized in that A focal length of the optical system in a horizontal direction relative to a mounting surface is shorter than a focal length of the optical system in a direction including a component in a direction orthogonal to the mounting surface.

5. The distance measuring device according to any one of claims 1 to 4, characterized in that The following conditional expressions are met: f_dist / f_per>2.0 wherein f_dist is the focal length of the optical system in the parallax direction, and f_per is the focal length of the optical system in a direction orthogonal to the parallax direction.

6. The distance measuring device according to any one of claims 1 to 5, characterized in that The following conditional expressions are met: 0.40 <f_dist_max / f_AX<0.90 Among them, f_dist_max is the focal length at the maximum angle of view of the optical system in the direction orthogonal to the parallax direction among the focal length f_dist of the optical system in the parallax direction, and f_AX is the focal length at the central angle of view of the optical system.

7. The distance measuring device according to any one of claims 1 to 6, characterized in that In the image sensor, the number of pixels in the parallax direction is smaller than the number of pixels in a direction including a component in a direction orthogonal to the parallax direction.

8. The distance measuring device according to any one of claims 1 to 7, characterized in that The following conditional expressions are met: 0.25 <h_HA / f_AX<0.55 Wherein, h_HA is the image height at a viewing angle that is half of the maximum viewing angle of the optical system, and f_AX is the focal length at the central viewing angle of the optical system.

9. The distance measuring device according to any one of claims 1 to 8, characterized in that The optical system includes an aperture stop, a first group arranged on the object side of the aperture stop, and a second group arranged on the image side of the aperture stop, and Characterized in that the first group includes a concave meniscus lens having a convex surface facing the object side and having an inflection point.

10. A vehicle-mounted system, comprising: The distance measuring device according to any one of claims 1 to 9; as well as A display device is configured to display an image obtained based on the output of the distance measuring device.

11. The vehicle-mounted system according to claim 10, characterized in that: The display device comprises: A first display unit configured to display a first image corresponding to a first viewing angle among the images; and A second display unit is configured to display a second image corresponding to a second viewing angle including the first viewing angle.

12. A movable device comprising: The distance measuring device according to any one of claims 1 to 9, The invention is characterized in that the movable device is configured to move while holding the distance measuring device.

Images