Imaging device and three-dimensional measurement device including the same

Abstract

An imaging device 10 is provided with: a first imaging lens through which light from an object passes; a first solid imaging element receiving light having passed through the first imaging lens; a second imaging lens provided apart from the first imaging lens in a first direction; a second-lens-side shielding part 16 in which first and second second-lens-side transmission parts 16c, 16d are formed at different positions in a second direction perpendicular to the first direction, wherein the second-lens-side shielding part 16 is disposed such that part of light from the object to the second imaging lens passes through the first and second second-lens-side transmission parts 16c, 16d, and the rest of the light is shielded; and a second solid imaging element receiving light from the object having passed through the second imaging lens.

Description

Technical Field

[0001] The present disclosure relates to an imaging device and a three-dimensional measurement device including the same.

Background Art

[0002] Patent Document 1 discloses a three-dimensional measurement device in which a first imaging unit that acquires a first image, a second imaging unit that acquires a second image, and a third imaging unit that acquires a third image are arranged at intervals in a triangular shape. This three-dimensional measurement device specifies a first similarity value related to a pixel of the second image along a first epipolar line for at least one pixel of the first image, and for the at least one pixel of the first image, a second similarity value related to a pixel of the third image is specified along a second epipolar line, the first and second similarity values are combined, a common parallax is determined among the first image, the second image, and the third image for the at least one pixel, and a distance is determined from the common parallax.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, since it is necessary to provide a solid-state imaging device for each of the first to third imaging units, the three-dimensional measurement device is increased in size and cost.

[0005] The present disclosure has been made in view of such a point, and an object thereof is to reduce the size of the three-dimensional measurement device and reduce the cost.

Means for Solving the Problems

[0006] To achieve the above object, the present disclosure provides a first imaging lens that allows light from a subject to pass therethrough, a first solid-state imaging device that receives the light that has passed through the first imaging lens, a second imaging lens that is provided at a distance from the first imaging lens in a first direction and allows the light from the subject to pass therethrough, first and second second-lens-side light-transmitting portions that are formed at different positions in a second direction perpendicular to the first direction, and a second-lens-side shielding portion that is arranged to allow a part of the light from the subject traveling toward the second imaging lens to pass through the first and second second-lens-side light-transmitting portions and block the remaining light, and a second solid-state imaging device that receives the light from the subject that has passed through the second imaging lens. The imaging device is characterized by the above configuration.

[0007] Accordingly, based on the output of the first solid-state imaging device, a first image based on the light from the subject that has passed through the first imaging lens is acquired, and based on the output of the second solid-state imaging device, a second image based on the light from the subject that has passed through the first second-lens-side light-transmitting portion and a third image based on the light from the subject that has passed through the second second-lens-side light-transmitting portion can be acquired. Therefore, it is not necessary to provide three solid-state imaging devices to acquire the first image, the second image, and the third image taken from three points forming a triangular shape. Thus, the three-dimensional measurement device can be miniaturized and the cost can be reduced.

Advantages of the Invention

[0008] According to the present disclosure, the three-dimensional measurement device can be miniaturized and the cost can be reduced.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0011] (Embodiment 1) FIG. 1 shows a three-dimensional measurement device 1 according to Embodiment 1 of the present disclosure. This three-dimensional measurement device 1 includes an imaging device 10, a signal processing unit 20, and a memory 40.

[0012] As shown in FIGS. 2 to 4, the imaging device 10 includes a housing 11, a first imaging lens 12, a first lens-side shielding portion 13, a first solid-state imaging device 14, a second imaging lens 15, a second lens-side shielding portion 16, a second solid-state imaging device 17, and a sub-lens array 19 (see FIG. 5) including a plurality of sub-lenses 18. In FIGS. 3 and 4, the point OP indicates a point included in the subject O.

[0013] In the housing 11, circular first and second openings 11a and 11b that open in a common direction are formed so as to penetrate in a horizontal direction perpendicular to the first direction with a space therebetween in the horizontal first direction. The shape of the housing 11 is not limited to the shape shown in FIG. 2. The housing 11 houses the first imaging lens 12, the first lens-side shielding portion 13, the first solid-state imaging device 14, the second imaging lens 15, the second lens-side shielding portion 16, the second solid-state imaging device 17, and the sub-lens array 19.

[0014] The first imaging lens 12 is disposed so as to face the first opening 11a of the housing 11. The first imaging lens 12 allows light from the subject O to pass through.

[0015] The first lens-side shielding portion 13 is formed in a plate shape and is disposed between the surface of the housing 11 where the first opening 11a is formed and the first imaging lens 12 so as to face the first opening 11a of the housing 11. The first lens-side shielding portion 13 has a glass plate 13a disposed so as to face the first opening 11a of the housing 11. A coating film 13b is formed by coating over the entire region except for the lower half circular region on the surface of the glass plate 13a on the side of the first opening 11a. The non-coated film region of the glass plate 13a constitutes a circular first lens-side light-transmitting portion 13c that allows light to pass through. The first lens-side shielding portion 13 is arranged to allow a part of the light traveling from the subject O toward the first imaging lens 12 to pass through by the first lens-side light-transmitting portion 13c and to block the remaining light. Note that instead of the glass plate 13a, a light-transmitting plate material made of a material other than glass such as resin may be used. Further, instead of the coating film 13b, a light-shielding layer made of a material other than paint may be provided.

[0016] The first solid-state imaging device 14 receives the light L1 from the subject O that has passed through the first lens-side light-transmitting portion 13c of the first lens-side shielding portion 13 and the first imaging lens 12, and outputs the luminance values of a plurality of pixels.

[0017] The second imaging lens 15 is disposed at a distance from the first imaging lens 12 in the first direction so as to face the second opening 11b of the housing 11. That is, the second imaging lens 15 has its thickness direction oriented in the same direction as that of the first imaging lens 12. The second imaging lens 15 also allows the light from the subject O to pass therethrough.

[0018] The second lens-side shielding portion 16 is formed in a plate shape and is disposed between the formation surface of the second opening 11b of the housing 11 and the second imaging lens 15 so as to face the second opening 11b of the housing 11. The second lens-side shielding portion 16 has a glass plate 16a disposed so as to face the second opening 11b of the housing 11. A coating film 16b is formed by coating over the entire region except for the upper semi-circular region and the lower semi-circular region on the surface of the glass plate 16a on the side of the second opening 11b. The non-formed region of the coating film 16b on the upper side of the glass plate 16a constitutes a circular first second lens-side light-transmitting portion 16c that allows light to pass through. The non-formed region of the coating film 16b on the lower side of the glass plate 16a constitutes a circular second second lens-side light-transmitting portion 16d that allows light to pass through. The second lens-side shielding portion 16 is arranged to allow a part of the light traveling from the subject O toward the second imaging lens 15 to pass through the first and second second lens-side light-transmitting portions 16c and 16d, and to block the remaining light. The second second lens-side light-transmitting portion 16d is located at the same position in the vertical direction as the first lens-side light-transmitting portion 13c. The vertical interval between the first and second second lens-side light-transmitting portions 16c and 16d is set to be longer than 0 and shorter than the diameter of the second imaging lens 15. Since the first and second second lens-side light-transmitting portions 16c and 16d are formed at different positions in the vertical direction (the second direction perpendicular to the first direction), two epipolar lines EP1 and EP2 with different inclinations can be formed by the first lens-side light-transmitting portion 13c and the first and second second lens-side light-transmitting portions 16c and 16d. The positional relationship between the first lens-side light-transmitting portion 13c and the first and second second lens-side light-transmitting portions 16c and 16d may be any other positional relationship as long as their respective centers of gravity are not in a straight line in the horizontal direction and two epipolar lines EP1 and EP2 with different inclinations can be formed. A detailed description of the epipolar lines EP1 and EP2 will be given later.

[0019] The second solid-state imaging device 17 receives the light L2 and L3 from the subject O that has passed through the first and second second lens-side light-transmitting portions 16c and 16d of the second lens-side shielding portion 16 and the second imaging lens 15.

[0020] The sub - lenses 18 that make up the sub - lens array 19 are interposed between the second imaging lens 15 and the second solid - state imaging device 17 in parallel with the second imaging lens 15. The sub - lens 18 is located at the focal point on the side of the second lens - side shielding portion 16 opposite to the second imaging lens 15. As shown in FIG. 5, the sub - lens 18 has an elliptical shape in which the length in the vertical direction (hereinafter referred to as the "second direction") is about twice the length in the first direction. The length and position of the sub - lens 18 in the first direction are set such that the irradiation region of the light L2 that has passed through the sub - lens 18 on the second solid - state imaging device 17 fits within one pixel of the second solid - state imaging device 17 in the first direction. The light L2 that has passed through the first second - lens - side light - transmitting portion 16c passes through the lower half (one - half in the second direction) of the sub - lens 18, and the light L3 that has passed through the second second - lens - side light - transmitting portion 16d passes through the upper half (the other half in the second direction) of the sub - lens 18. The lights L2 and L3 that have passed through the sub - lens 18 are incident on regions corresponding to different pixels in the second solid - state imaging device 17. The length and position of the sub - lens 18 in the second direction are set such that the irradiation regions of the lights L2 and L3 that have passed through the sub - lens 18 on the second solid - state imaging device 17 fit within two pixels of the second solid - state imaging device 17 in the second direction. A plurality of sub - lenses 18 are provided adjacent to each other in the first direction. The number of sub - lenses 18 in the first direction corresponds to the number of pixels arranged in the first direction in the second solid - state imaging device 17.

[0021] The signal processing unit 20 includes a first image acquisition unit 21, a second image acquisition unit 22, a first parallax information acquisition unit 23, a second parallax information acquisition unit 24, a distance information acquisition unit 25, and a filter information generation unit 26.

[0022] The first image acquisition unit 21 acquires a first image based on the luminance values of a plurality of pixels output by the first solid-state imaging device 14. The first image acquisition unit 21 includes a first shading correction unit 21a. This first shading correction unit 21a acquires the first image by performing correction corresponding to the position in the second direction on the luminance values of the plurality of pixels output by the first solid-state imaging device 14. Specifically, correction is performed by multiplying the first correction value acquired for each position in the second direction by the calibration process described later and stored in the memory 40 by the luminance values of all the pixels output by the first solid-state imaging device 14. Then, the first image acquisition unit 21 outputs the corrected luminance values of all the pixels output by the first solid-state imaging device 14 as the luminance values of all the pixels of the first image.

[0023] The second image acquisition unit 22 acquires a second image based on the light L2 that has passed through the first second lens-side light-transmitting portion 16c and a third image based on the light L3 that has passed through the second second lens-side light-transmitting portion 16d, based on the luminance values of a plurality of pixels output by the second solid-state imaging device 17. Specifically, the second image acquisition unit 22 includes an information extraction unit 22a, a second shading correction unit 22b, a third shading correction unit 22c, a second image size correction unit 22d, and a third image size correction unit 22e.

[0024] The information extraction unit 22a extracts the luminance values of the pixels based on the light L2 that has passed through the first second lens-side light-transmitting portion 16c and the luminance values of the pixels based on the light L3 that has passed through the second second lens-side light-transmitting portion 16d from the luminance values of all the pixels output by the second solid-state imaging device 17.

[0025] The second shading correction unit 22b performs correction according to the position in the second direction on the luminance values of the pixels based on the light L2 that has passed through the first second lens-side light-transmitting portion 16c extracted by the information extraction unit 22a, thereby obtaining a pre-first-size correction image. Specifically, the second shading correction unit 22b performs correction by multiplying the second correction value obtained for each position in the second direction by the calibration process described later and stored in the memory 40 by the luminance values of all the pixels based on the light L2 that has passed through the first second lens-side light-transmitting portion 16c.

[0026] The third shading correction unit 22c performs correction according to the position in the second direction on the luminance values of the pixels based on the light L3 that has passed through the second second lens-side light-transmitting portion 16d extracted by the information extraction unit 22a, thereby obtaining a pre-second-size correction image. Specifically, the third shading correction unit 22c performs correction by multiplying the third correction value obtained for each position in the second direction by the calibration process described later and stored in the memory 40 by the luminance values of all the pixels based on the light L3 that has passed through the second second lens-side light-transmitting portion 16d.

[0027] The second image size correction unit 22d corrects the image size of the pre-first-size correction image obtained by the second shading correction unit 22b, and outputs the corrected image as the second image. The second image size correction unit 22d corrects the image size based on the information of the first image output from the first shading correction unit 21a so that the second image has the same size (number of pixels) as the first image. At this time, the second image is stretched in the vertical direction but not in the horizontal direction, so that the deterioration of the accuracy in the search on the first epipolar line EP1 described later is not caused.

[0028] The third image size correction unit 22e corrects the size of the second pre-size-correction image obtained by the third shading correction unit 22c, and outputs the corrected image as the third image. The third image size correction unit 22e corrects the image size based on the information of the first image output from the first shading correction unit 21a so that the third image has the same size (number of pixels) as the first image. At this time, the third image is stretched in the vertical direction but not in the horizontal direction, so that the accuracy deterioration in the search on the second epipolar line EP2 described later is not caused.

[0029] The first disparity information acquisition unit 23 searches for a corresponding block (corresponding part) corresponding to each of a plurality of constituent blocks (constituent parts) constituting the first image on the first epipolar line EP1 of the second image. Then, the first disparity information indicating the distance (distance in the direction along the first epipolar line EP1) between the positions of the plurality of constituent blocks in the first image and the positions of the respective corresponding blocks in the second image is acquired. Here, the first epipolar line EP1 used for the search is a line obtained by intersecting the epipolar plane connecting the point OP of the subject O, the first lens-side light-transmitting part 13c of the first lens-side shielding part 13, and the first second lens-side light-transmitting part 16c of the second lens-side shielding part 16, and the second image. The first disparity information is image information indicating a disparity image in which the pixel value of each constituent block of the first image is a pixel value corresponding to the distance between the position of the constituent block in the first image and the position of the corresponding block in the second image. In this image information, the pixel value of the constituent block for which the corresponding block (corresponding part) could not be detected on the first epipolar line EP1 of the second image is a predetermined blank value indicating a blank.

[0030] Note that the first epipolar line EP1 is inclined in the vertical direction with respect to the horizontal direction in the second image.

[0031] As shown in FIG. 6, for example, when the horizontal direction in the second image is the x-axis and the vertical direction is the y-axis, and the first epipolar line EP1 is a straight line of y = x, for the square constituent blocks of 3 pixels in the horizontal direction and 3 pixels in the vertical direction that make up the first image, the corresponding blocks corresponding to each of these constituent blocks are searched for on the first epipolar line EP1 of the second image.

[0032] As shown in FIG. 7, for example, when the horizontal direction in the second image is the x-axis and the vertical direction is the y-axis, and the first epipolar line EP1 is a straight line of y = x / 2, for the rectangular constituent blocks of 3 pixels in the horizontal direction and 6 pixels in the vertical direction that make up the first image, the corresponding blocks corresponding to each of these constituent blocks are searched for on the first epipolar line EP1 of the second image.

[0033] That is, when the horizontal direction in the second image is the x-axis and the vertical direction is the y-axis, and the first epipolar line EP1 is a straight line of y = ax + b, the number of pixels in the vertical direction of the constituent block and the corresponding block is set to 1 / a times the number of pixels in the horizontal direction.

[0034] The second parallax information acquisition unit 24 searches for corresponding blocks corresponding to the plurality of constituent blocks constituting the first image on the second epipolar line EP2 of the third image, respectively. Then, second parallax information indicating the distances between the positions of the plurality of constituent blocks in the first image and the positions of the respective corresponding blocks in the third image is acquired. Here, the second epipolar line EP2 used for the search is a line obtained by intersecting the epipolar plane connecting a point of the subject O, the first lens-side light-transmitting portion 13c of the first lens-side shielding portion 13, and the second lens-side light-transmitting portion 16d of the second lens-side shielding portion 16, and the third image. The second epipolar line EP2 extends in the horizontal direction in the third image and is not inclined in the vertical direction with respect to the horizontal direction in the third image. The second parallax information is image information indicating a parallax image in which the pixel values of each constituent block of the first image are pixel values corresponding to the distance between the position of the constituent block in the first image and the position of the corresponding block in the third image. Also in this image information, the pixel values of the constituent blocks for which the corresponding blocks (corresponding portions) could not be detected on the second epipolar line EP2 of the third image are set to a predetermined blank value indicating a blank.

[0035] The distance information acquisition unit 25 acquires distance information based on the first parallax information acquired by the first parallax information acquisition unit 23 and the second parallax information acquired by the second parallax information acquisition unit 24. The distance information is information indicating the distance between the subject O captured in each constituent block constituting the first image and the imaging device 10. Specifically, the distance information is image information indicating an image in which the pixel values of each constituent block of the first image are pixel values corresponding to the distance between the subject O captured in the constituent block and the imaging device 10.

[0036] The distance information acquisition unit 25 includes a parallax information synthesis unit 25a and a distance image generation unit 25b.

[0037] The parallax information synthesizing unit 25a obtains synthesized parallax information by replacing the pixel values of each pixel of the blank value in the image indicated by the first parallax information with the pixel values of the corresponding pixel in the image indicated by the second parallax information. That is, the synthesized parallax information is image information obtained by complementing the first parallax information with the second parallax information.

[0038] The distance image generating unit 25b generates the distance information based on the synthesized parallax information obtained by the parallax information synthesizing unit 25a.

[0039] Let the distance between the subject O shown in the constituent block and the imaging device 10 (the distance indicated by the pixel value of each pixel in the distance information) be z, the parallax indicated by the synthesized parallax information be d, the interval in the first direction between the first lens-side light transmitting portion 13c and the first and second second lens-side light transmitting portions 16c, 16d be b, and the focal length be f. Then, z can be calculated by the following general formula 1.

[0040] z = b·f / d ···(Formula 1) The filter information generating unit 26 performs calibration processing to obtain the luminance values output by the first solid-state imaging device 14 and the second solid-state imaging device 17 at the time of shipment, calculate the first correction value, the second correction value, and the third correction value based on the luminance values, and store them in the memory 40. The acquisition of the luminance value by the filter information generating unit 26 is performed in a uniform light irradiation state in which uniform light is irradiated from the first or second opening 11a, 11b side to the first lens-side light transmitting portion 13c of the first lens-side shielding portion 13 and the first and second second lens-side light transmitting portions 16c, 16d of the second lens-side shielding portion 16. That is, in the uniform light irradiation state, light is irradiated so that the intensity of the incident light is equal at all positions on the surfaces of the first lens-side light transmitting portion 13c, the first and second second lens-side light transmitting portions 16c, 16d.

[0041] FIG. 8 illustrates the relationship between the luminance values output by the first solid-state imaging device 14 and the second solid-state imaging device 17 in a uniform light irradiation state and the vertical positions of the solid-state imaging devices 14 and 17 when the first lens-side light-transmitting portion 13c is arranged to face the center of the first imaging lens 12. In FIG. 8, the right side in the x-axis direction indicates the upper side.

[0042] When the first lens-side light-transmitting portion 13c is arranged to face the center of the first imaging lens 12, the luminance value of the irradiation region of the light L1 that has passed through the first lens-side light-transmitting portion 13c in the first solid-state imaging device 14 increases at the center in the second direction (vertical direction) in the irradiation region, as shown by the curve C1 in FIG. 8. The luminance value of the irradiation region of the light L2 that has passed through the first second-lens-side light-transmitting portion 16c in the second solid-state imaging device 17 increases on the upper side in the irradiation region, as shown by the curve C2 in FIG. 8. The luminance value of the irradiation region of the light L3 that has passed through the second second-lens-side light-transmitting portion 16d in the second solid-state imaging device 17 increases on the lower side in the irradiation region, as shown by the curve C3 in FIG. 8.

[0043] The filter information generation unit 26 calculates the reciprocal of the luminance value of the irradiation region of the light L1 that has passed through the first lens-side light-transmitting portion 13c in a uniform light irradiation state as the first correction value for each pixel. At this time, the first correction values of the pixels having the same position in the second direction are common.

[0044] Further, the filter information generation unit 26 calculates the reciprocal of the luminance value of the irradiation region of the light L2 that has passed through the first second-lens-side light-transmitting portion 16c in a uniform light irradiation state as the second correction value for each pixel. Therefore, the second correction value for each pixel is the reciprocal of the value shown by the curve C2 in FIG. 8. Also, the second correction values of the pixels having the same position in the second direction are common.

[0045] Further, the filter information generation unit 26 calculates the reciprocal of the luminance value of the irradiation region of the light L3 that has passed through the second second-lens-side light-transmitting portion 16d in a uniform light irradiation state as the third correction value for each pixel. Therefore, the third correction value for each pixel is the reciprocal of the value shown by the curve C3 in FIG. 8. Also, the third correction values of the pixels having the same position in the second direction are common.

[0046] Next, the generation process of the distance information by the three-dimensional measurement device 1 configured as described above will be described with reference to the flowchart of FIG. 9.

[0047] First, in (S101), the first solid-state imaging device 14 and the second solid-state imaging device 17 receive the light L1, L2, and L3 from the subject O at a common timing. The light L1 from the subject O that has passed through the first lens-side light-transmitting portion 13c of the first lens-side shielding portion 13 and the first imaging lens 12 enters the first solid-state imaging device 14. The light L2 from the subject O that has passed through the first second lens-side light-transmitting portion 16c of the second lens-side shielding portion 16, the second imaging lens 15, and the sub-lens 18, and the light L3 from the subject O that has passed through the second second lens-side light-transmitting portion 16d of the second lens-side shielding portion 16, the second imaging lens 15, and the sub-lens 18 enter the second solid-state imaging device 17.

[0048] Next, in (S102), the first shading correction unit 21a corrects the luminance values of all the pixels output by the first solid-state imaging device 14 in (S101) based on the first correction value stored in the memory 40, thereby obtaining a first image. Further, the information extraction unit 22a extracts the luminance values of the pixels based on the light L2 that has passed through the first second lens-side light-transmitting portion 16c and the luminance values of the pixels based on the light L3 that has passed through the second second lens-side light-transmitting portion 16d from the luminance values of all the pixels output by the second solid-state imaging device 17 in (S101). Then, the second shading correction unit 22b corrects the luminance values of all the pixels based on the light L2 that has passed through the first second lens-side light-transmitting portion 16c based on the second correction value stored in the memory 40, thereby obtaining a pre-first-size correction image. Furthermore, the third shading correction unit 22c corrects the luminance values of the pixels based on the light L3 that has passed through the second second lens-side light-transmitting portion 16d based on the third correction value stored in the memory 40, thereby obtaining a pre-second-size correction image.

[0049] Next, in (S103), the second image size correction unit 22d corrects the image size of the first pre-size correction image obtained in (S102), and outputs the corrected image as the second image. Further, the third image size correction unit 22e corrects the image size of the second pre-size correction image obtained in (S102), and outputs the corrected image as the third image.

[0050] Next, in (S104), the first parallax information acquisition unit 23 acquires first parallax information based on the first image acquired in (S102) and the second image acquired in (S103). Further, the second parallax information acquisition unit 24 acquires second parallax information based on the first image acquired in (S102) and the third image acquired in (S103). Details of the acquisition method of the first parallax information and the second parallax information will be described later.

[0051] Next, in (S105), the parallax information synthesis unit 25a of the distance information acquisition unit 25 acquires synthesized parallax information based on the first parallax information and the second parallax information acquired in (S104).

[0052] Finally, in (S106), the distance image generation unit 25b of the distance information acquisition unit 25 generates distance information based on the synthesized parallax information acquired in (S105). Thereby, the generation process of the distance information by the signal processing unit 20 is completed.

[0053] Next, the acquisition process of the first parallax information in (S104) will be described in detail with reference to the flowchart of FIG. 10. Here, the first image acquired in (S102) is used as the reference image, and the second image acquired in (S103) is used as the input image.

[0054] First, in (S201), the first parallax information acquisition unit 23 acquires the reference image and the input image.

[0055] Next, in (S202), the first disparity information acquisition unit 23 divides the reference image into a plurality of constituent blocks. At this time, when the inclination of the first epipolar line EP1 is a, for example, the number of pixels in the horizontal direction of the constituent block is set to 3 pixels, and the number of pixels in the vertical direction of the constituent block is set to 3 pixels multiplied by 1 / a.

[0056] Next, in (S203), the first disparity information acquisition unit 23 selects an unselected constituent block from the plurality of constituent blocks specified by the division in (S202).

[0057] Next, in (S204), the first disparity information acquisition unit 23 calculates the disparity, which is the distance between the position of the constituent block in the reference image and the position of the corresponding block in the input image, by searching for the corresponding block corresponding to the constituent block selected in (S203) on the first epipolar line EP1 of the input image. Details of the processing in (S204) will be described later.

[0058] Next, in (S205), the first disparity information acquisition unit 23 stores the pixel value corresponding to the disparity calculated in (S204) in a disparity image memory (not shown). The storage area of the pixel value in the disparity image memory is the area assigned to the constituent block selected in (S203).

[0059] Next, in (S206), the first disparity information acquisition unit 23 determines whether all of the plurality of constituent blocks specified by the division in (S202) have been selected in (S203). That is, it determines whether there is an unselected constituent block remaining. If all of the plurality of constituent blocks specified by the division in (S202) have been selected in (S203) and there is no unselected constituent block remaining, the acquisition process of the first disparity information is terminated. On the other hand, if a part of the plurality of constituent blocks specified by the division in (S202) has not been selected in (S203) and there is an unselected constituent block remaining, the process returns to (S203).

[0060] Next, the parallax calculation process in (S204) will be described in detail with reference to the flowchart of FIG. 11.

[0061] First, in (S301), the first parallax information acquisition unit 23 sets i to an initial value. As the initial value, for example, 1 is set.

[0062] Next, in (S302), the first parallax information acquisition unit 23 extracts an extraction region AR (see FIGS. 6 and 7) having the same size as the constituent block selected in (S203). Here, first, the extraction region AR is moved so that the center in the vertical direction is located on the first epipolar line EP1 extending from the center of the constituent block selected in (S203), and the center in the horizontal direction is the i-th column. And in this state, when the edge of the extraction region AR is located in the middle of a pixel as indicated by the virtual line in FIG. 7, the first parallax information acquisition unit 23 moves the extraction region AR in the vertical direction by the minimum length so as to coincide with the edge of the pixel.

[0063] Next, in (S303), the first parallax information acquisition unit 23 calculates the similarity between the constituent block selected in (S203) and the extraction region AR extracted in (S302). Here, for example, NCC (Normalized Cross Correlation) is calculated as the similarity. Also, the first parallax information acquisition unit 23 adds 1 to i.

[0064] Next, in (S304), the first parallax information acquisition unit 23 determines whether i is a predetermined value. The predetermined value is set to a value, for example, 1 greater than the total number of pixels in the horizontal direction of the input image. When i is the predetermined value, the first parallax information acquisition unit 23 proceeds to the process of (S305), while when i is not the predetermined value, it returns to the process of (S302).

[0065] In (S305), among the extracted regions AR for which the similarity was calculated in (S303) from the initial value to the predetermined value of i, the extracted region AR with the highest similarity is specified as the corresponding block. Then, the disparity, which is the distance between the position of the corresponding block in the input image and the position of the constituent block in the reference image, is calculated, and the calculation process of the disparity ends.

[0066] Note that the second disparity information acquisition unit 24 acquires the second disparity information by similarly executing the processing of the flowchart in FIG. 10, using the first image acquired in (S102) as the reference image and the third image acquired in (S103) as the input image. In such a case, since the second epipolar line EP2 extends in the horizontal direction, in (S202), the size of the constituent block is always set to be constant. The number of pixels in the horizontal and vertical directions of the constituent block is set to, for example, 3 pixels. Further, the extraction region AR extracted by the second disparity information acquisition unit 24 in (S302) is a region that is located in the same row as the constituent block and has the center in the horizontal direction at the i-th column.

[0067] Therefore, according to the first embodiment, based on the output of the first solid-state imaging device 14, a first image based on the light L1 from the subject O that has passed through the first photographing lens 12 is acquired, and based on the output of the second solid-state imaging device 17, a second image based on the light L2 from the subject O that has passed through the first second-lens-side light-transmitting portion 16c and a third image based on the light L3 from the subject O that has passed through the second second-lens-side light-transmitting portion 16d can be acquired. Therefore, in order to acquire the first image, the second image, and the third image taken from three points forming a triangle, it is not necessary to provide three solid-state imaging devices, so that the three-dimensional measurement device 1 can be miniaturized and the cost can be reduced.

[0068] Further, according to the first embodiment, since the first lens-side light-transmitting portion 13c and the first and second second-lens-side light-transmitting portions 16c, 16d are circular, compared with the case where they are elliptical, the depth of field is increased and image blurring is suppressed, so that the accuracy of the obtained distance information can be improved.

[0069] In addition, since the sub-lenses 18 are provided for each pixel of the second solid-state image sensor 17 in the first direction, the resolution in the first direction of the second image and the third image can be increased more than when fewer sub-lenses are provided.

[0070] (Embodiment 2) FIG. 12 shows a three-dimensional measurement device 1 according to Embodiment 2 of the present disclosure. In Embodiment 2, the first lens-side shielding portion 13 is constituted by a liquid crystal shutter. Further, the signal processing unit 20 further includes a liquid crystal shutter control unit 51 as a first lens-side size adjustment unit. Further, the filter information generation unit 26 causes the memory 40 to store the luminance values output by the first solid-state image sensor 14 and the second solid-state image sensor 17 in the uniform light irradiation state. The liquid crystal shutter control unit 52 adjusts the size of the first lens-side light-transmitting portion 13c of the first lens-side shielding portion 13 based on the luminance values output by the first solid-state image sensor 14 and the second solid-state image sensor 17 and stored in the memory 40. For example, the liquid crystal shutter control unit 51 makes the average of the luminance values of the pixels based on the light L1 that has passed through the first lens-side light-transmitting portion 13c in the uniform light irradiation state equal to the average of the luminance values of the pixels based on the light L2 and L3 that have passed through the first or second second lens-side light-transmitting portions 16c and 16d in the uniform light irradiation state, and adjusts the size of the first lens-side light-transmitting portion 13c of the first lens-side shielding portion 13. By increasing the size of the first lens-side light-transmitting portion 13c, the average of the luminance values of all the pixels output by the first solid-state image sensor 14 can be increased. In Embodiment 2, the signal processing unit 20 does not include the first shading correction unit 21a, the second shading correction unit 22b, and the third shading correction unit 22c. The first image acquisition unit 21 acquires the luminance values of a plurality of pixels output by the first solid-state image sensor 14 as a first image.

[0071] Since the other configurations and operations are the same as those in Embodiment 1, the same reference numerals are given to the same configurations, and the detailed description thereof is omitted.

[0072] (Modification of Embodiment 2) In the above-described Embodiment 2, only the first lens-side shielding portion 13 is configured by a liquid crystal shutter, but only the second lens-side shielding portion 16 may be configured by a liquid crystal shutter. Then, the liquid crystal shutter control unit 51 may adjust the size of at least one of the first and second second lens-side light-transmitting portions 16c and 16d of the second lens-side shielding portion 16 based on the luminance values output by the first solid-state imaging device 14 and the second solid-state imaging device 17 and stored in the memory 40. That is, the liquid crystal shutter control unit 51 may constitute the second lens-side size adjustment unit. Further, both the first lens-side shielding portion 13 and the second lens-side shielding portion 16 may be configured by liquid crystal shutters, and the liquid crystal shutter control unit 51 may adjust the size of the first lens-side light-transmitting portion 13c of the first lens-side shielding portion 13 and the sizes of the first and second second lens-side light-transmitting portions 16c and 16d of the second lens-side shielding portion 16 based on the luminance values stored in the memory 40.

[0073] Also, in Embodiment 1, the light that has passed through the first and second second lens-side light-transmitting portions 16c and 16d is received by the second solid-state imaging device 17 at a common timing. However, when the second lens-side shielding portion 16 is configured by a liquid crystal shutter, imaging is performed on the second solid-state imaging device 17 with only one of the first and second second lens-side light-transmitting portions 16c and 16d formed in the second lens-side shielding portion 16, and then imaging is performed on the second solid-state imaging device 17 with only the other of the first and second second lens-side light-transmitting portions 16c and 16d formed in the second lens-side shielding portion 16, whereby a second image and a third image may be acquired.

[0074] (Embodiment 3) FIG. 13 shows a three-dimensional measuring device 1 according to Embodiment 3. In this Embodiment 3, the first lens-side shielding portion 13 is constituted by a liquid crystal shutter. Further, the signal processing unit 20 further includes a liquid crystal shutter control unit 52 as a position adjustment unit that moves the first lens-side light-transmitting portion 13c in the second direction. The liquid crystal shutter control unit 52 can move the first lens-side light-transmitting portion 13c to the same first position shown in FIG. 2 as the second second lens-side light-transmitting portion 16d in the second direction (vertical direction), and the first and second second lens-side light-transmitting portions 16c, 16d to the second position shown in FIG. 14 that is different from both in the second direction (vertical direction).

[0075] In addition, the parallax information synthesizing unit 25a of the distance information acquiring unit 25 can further synthesize a plurality of synthesized parallax information acquired by photographing with the first lens-side light-transmitting portion 13c arranged at a plurality of different positions in the second direction. For example, the parallax information synthesizing unit 25a can complement the synthesized parallax information acquired by photographing with the first lens-side light-transmitting portion 13c arranged at the first position with the synthesized parallax information acquired by photographing with the first lens-side light-transmitting portion 13c arranged at the second position.

[0076] Since the other configurations and operations are the same as those in Embodiment 1, the same reference numerals are given to the same configurations, and the detailed description thereof is omitted.

[0077] Therefore, according to this Embodiment 3, information corresponding to the distance between the subject O and the imaging device 10 can be more reliably acquired for all the pixels of the first image.

[0078] (Modification Example of Embodiment 3) In the above Embodiment 3, the liquid crystal shutter control unit 52 may move the first lens-side light-transmitting portion 13c in the first direction. Further, the first lens-side shielding portion 13 is constituted by the glass plate 13a and the coating film 13b as in Embodiment 1, and as shown in FIG. 15, by rotating the first lens-side shielding portion 13 automatically or manually in the x direction, a rotation mechanism for moving the first lens-side light-transmitting portion 13c in the first direction and the second direction may be provided.

[0079] According to these modified examples, by moving the first lens-side light-transmitting portion 13c in the first direction, the interval in the first direction between the first lens-side light-transmitting portion 13c and the first and second second lens-side light-transmitting portions 16c, 16d can be changed, and the distance resolution can be adjusted. In the following formula 2, let the distance between the subject O shown in the constituent block and the imaging device 10 be z, the parallax be d, the interval in the first direction between the first lens-side light-transmitting portion 13c and the first and second second lens-side light-transmitting portions 16c, 16d be b, and the focal length be f.

[0080] Δz = z 2 Δd / (fb + zΔd) ≒ z 2 Δd / fb ···(Formula 2) From formula 2, it can be seen that by increasing b, that is, the interval in the first direction between the first lens-side light-transmitting portion 13c and the first and second second lens-side light-transmitting portions 16c, 16d, the distance resolution can be improved. b, that is, the interval in the first direction between the first lens-side light-transmitting portion 13c and the first and second second lens-side light-transmitting portions 16c, 16d, is set to, for example, 10 to 30 cm.

[0081] (Other embodiments) In the above-described first to third embodiments and their modified examples, the first lens-side light-transmitting portion 13c and the first and second second lens-side light-transmitting portions 16c, 16d are circular, but they may be elliptical.

[0082] In the above-described first to third embodiments and their modified examples, the sub-lens 18 is arranged such that the irradiation regions of the light L2, L3 that have passed through the sub-lens 18 on the second solid-state imaging device 17 correspond to one pixel in the first direction and two pixels in the second direction. However, the layout pattern of the sub-lens 18 is not limited to this. For example, the sub-lens 18 may be arranged such that the irradiation regions of the light L2, L3 that have passed through the sub-lens 18 on the second solid-state imaging device 17 correspond to two pixels in the first direction and two pixels in the second direction.

[0083] In the above-described Embodiments 1 to 3 and their modifications, the similarity was calculated in (S303). However, a difference measure such as SAD (sum of absolute differences) or SSD (sum of square differences), or other evaluation values may be calculated. When calculating the difference measure in (S303), in (S305), the extraction region AR with the smallest difference measure may be specified as the corresponding block.

[0084] In the above-described Embodiments 1 to 3 and their modifications, the first disparity information, the second disparity information, and the distance information are used as the image information. However, they may not be the image information. Further, the first disparity information and the second disparity information may be information indicating the distance between the position of one component constituting the first image and the position of its corresponding block in the second image or the third image. The distance information may also be information indicating the distance between the subject O imaged on one component constituting the first image and the imaging device 10.

Industrial Applicability

[0085] The present disclosure is useful as an imaging device that can reduce the size and cost of a three-dimensional measurement device, and a three-dimensional measurement device including the same.

Explanation of Signs

[0086] 1 Three-dimensional measurement device 10 Imaging device 12 First imaging lens 13 First lens-side shielding portion 13c First lens-side light-transmitting portion 14 First solid-state imaging device 15 Second imaging lens 16 Second lens-side shielding portion 16c First second lens-side light-transmitting portion 16d Second second lens-side light-transmitting portion 17 Second solid-state imaging device 18 Sub-lens 21 First image acquisition unit 22 Second image acquisition unit 23 First disparity information acquisition unit 24 Second parallax information acquisition unit 25 Distance information acquisition unit EP1 First epipolar line EP2 Second epipolar line O Subject L1, L2, L3 Light

Claims

1. a first imaging lens that allows light from a subject to pass through; a first solid-state imaging device that receives the light that has passed through the first imaging lens; a second imaging lens that is provided at a distance from the first imaging lens in a first direction and allows light from the subject to pass through; a second lens-side shielding portion in which first and second second-lens-side light-transmitting portions are formed at different positions in a second direction perpendicular to the first direction, and which is arranged to allow a part of the light from the subject traveling toward the second imaging lens to pass through the first and second second-lens-side light-transmitting portions and to block the remaining light; a second solid-state imaging device that receives the light from the subject that has passed through the second imaging lens; an imaging device in which information on the respective lights that have passed through the first and second second-lens-side light-transmitting portions is acquired by one common second solid-state imaging device.

2. The imaging device according to claim 1, further comprising a second lens-side size adjustment portion that adjusts the size of at least one of the first and second second-lens-side light-transmitting portions of the second lens-side shielding portion.

3. The imaging device according to claim 1, a first lens-side shielding portion in which a first lens-side light-transmitting portion is formed and which is arranged to allow a part of the light from the subject traveling toward the first imaging lens to pass through the first lens-side light-transmitting portion and to block the remaining light; further comprising a first lens-side size adjustment portion that adjusts the size of the first lens-side light-transmitting portion of the first lens-side shielding portion.

4. The imaging device according to claim 1, further comprising a sub-lens that is interposed between the second imaging lens and the second solid-state imaging device and that is located at the focal point of the second imaging lens.

5. The imaging device according to claim 1, a first lens-side shielding portion in which a first lens-side light-transmitting portion is formed and which is arranged to allow a part of the light from the subject traveling toward the first imaging lens to pass through the first lens-side light-transmitting portion and to block the remaining light; further comprising a position adjustment portion that moves the first lens-side light-transmitting portion of the first lens-side shielding portion in the second direction.

6. an imaging device according to any one of claims 1 to 5; a first image acquisition portion that acquires a first image based on the luminance values of a plurality of pixels output by the first solid-state imaging device. A second image acquisition unit that acquires a second image based on the light that has passed through the first second-lens-side light-transmitting portion and a third image based on the light that has passed through the second second-lens-side light-transmitting portion, based on the luminance values of a plurality of pixels output by the second solid-state imaging device; A first parallax information acquisition unit that, for each of at least one component part constituting the first image, searches for a corresponding part corresponding to the component part on the epipolar line of the second image, and acquires first parallax information indicating the distance between the position of the at least one component part in the first image and the positions of the respective corresponding parts in the second image; A second parallax information acquisition unit that, for each of at least one component part constituting the first image, searches for a corresponding part corresponding to the component part on the epipolar line of the third image, and acquires second parallax information indicating the distance between the position of the at least one component part in the first image and the positions of the respective corresponding parts in the third image; A three-dimensional measurement device comprising: a distance information acquisition unit that acquires distance information indicating the distance between the subject imaged on the at least one component part constituting the first image and the imaging device, based on the first parallax information and the second parallax information.

Images