Imaging system and distance measurement system

Abstract

To increase resolution and restrain time required for measuring distance by focusing on a subject even if the subject is photographed from an oblique direction in an imaging system and a distance measurement system using a variable focusing lens.SOLUTION: A first imaging device 1 includes: a first light source section 12 for applying pulsed light L1 to a subject A1; an angle adjustment section 121 for changing an angle θ of a light axis direction of the first light source section 12 relative to the subject A1; and a light axis correction section 43. The light axis correction section 43 extracts the light quantity of a regular reflection component and the light quantity of a diffusion reflection component from a first pixel signal outputted from a first imaging section 11 for imaging the subject A1 via a first variable focusing lens 111, and calculates a range θC of an angle θ at which the quantity of a regular reflection component becomes equal to or more than the quantity of light of a diffusion reflection component and the quantity of a regular reflection component becomes close to the maximum value.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an imaging system and a distance measurement system.

Background Art

[0002] Conventionally, an imaging device using a variable focus lens capable of controlling the focal position of a lens within a certain range has been known. In such an imaging device, by changing the focal position of the variable focus lens and imaging an image of a subject, images of the subject at a plurality of focal positions can be captured.

[0003] The variable focal length lens device of Patent Document 1 includes a variable focal length lens (variable focus lens) whose focusing position periodically changes in response to an input drive signal, a light source that irradiates light to an observation object through the variable focal length lens, a light guide unit that guides light passing through a position conjugate to the focusing position by the variable focal length lens after being reflected by the observation object, an illumination optical system that irradiates the light guided by the light guide unit to the observation object, and an imaging unit that images the observation object through the variable focal length lens.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, in the configuration of Patent Document 1, the optical axis of the light irradiated from the light source and the optical axis of the variable focus lens are coaxial. For this reason, when photographing a subject from an oblique direction, the optical axis of the light irradiated from the light source and hitting the subject and returning and the optical axis of the variable focus lens do not coincide, so that the subject cannot be focused.

[0006] The present disclosure aims to provide an imaging system and a distance measurement system that can focus on a subject even when the subject is photographed from an oblique direction in an imaging system and a distance measurement system using a variable focus lens.

Means for Solving the Problems

[0007] In order to solve the above problems, an imaging system according to an embodiment of the present disclosure includes a first light source unit that irradiates light on the subject, a first angle adjustment unit that changes a first angle which is an angle in the optical axis direction of the first light source unit with respect to the subject, and from a first pixel signal output from a first imaging unit that images the subject via a first variable focus lens, extracts a light amount of a first specular reflection component which is a light amount of a specular reflection component and a light amount of a first diffuse reflection component which is a light amount of a diffuse reflection component, and an optical axis correction unit that calculates the first angle at which the light amount of the first specular reflection component is equal to or greater than the light amount of the first diffuse reflection component and the light amount of the first specular reflection component is near a maximum value.

Effects of the Invention

[0008] According to the present disclosure, in an imaging system and a distance measurement system using a variable focus lens, it is possible to focus on a subject even when the subject is photographed from an oblique direction.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following description of the preferred embodiments is merely illustrative in nature and is not intended to limit the present invention, its applications, or its uses.

[0011] (First Embodiment) FIG. 1 is a diagram showing the basic configuration of an imaging device according to the first embodiment. In the following description, in FIG. 1, the left - right direction of the drawing is defined as the X - direction, the depth direction of the drawing is defined as the Y - direction, and the up - down direction of the drawing is defined as the Z - direction for explanation.

[0012] As shown in FIG. 1, the first imaging device 1 (imaging system) includes a first imaging unit 11 and a first light source unit 12. The first imaging device 1 is configured to be communicable with a control unit 4.

[0013] The first imaging unit 11 images the range of the field of view 11a. The imaging direction of the first imaging unit 11 is set to a direction that forms a predetermined angle with the Z - direction. The subject A1 is arranged in a planar shape extending in the X - direction and the Y - direction. That is, the first imaging unit 11 images the subject A1 from an oblique direction.

[0014] The first imaging unit 11 includes a first variable - focus lens 111 and a first imaging element 112.

[0015] The first variable-focus lens 111 condenses the light from the visual field 11a onto the first imaging element 112. The first variable-focus lens 111 is composed of a first lens 111a and a first liquid lens unit 111b. The first lens 111a is, for example, a convex lens and is arranged on the same optical axis as the first liquid lens unit 111b. The refractive index of the first liquid lens unit 111b changes according to the drive signal S1 output by the control unit 4. For example, the drive signal S1 is an alternating-current signal with a frequency that generates a standing wave in the first liquid lens unit 111b.

[0016] The first imaging element 112 is a color image sensor. The first imaging element 112 is, for example, a CMOS image sensor. Note that the first imaging element 112 may be a CCD. The first imaging element 112 may also be a monochrome image sensor. The first imaging element 112 outputs a first pixel signal according to the drive signal S1 output from the first drive control unit 42. That is, the imaging timing of the first imaging unit 11 is determined according to the drive signal S1.

[0017] The first light source unit 12 irradiates the subject A1 with pulsed light L1. For example, the first light source unit 12 is composed of, for example, a light-emitting diode. The pulsed light L1 is light having directivity in the optical axis direction of the first light source unit 12 (in the example of FIG. 1, the arrow direction). The first light source unit 12 irradiates the pulsed light L1 according to the drive signal S1 output by the control unit 4.

[0018] The first light source unit 12 includes an angle adjustment unit 121 (first angle adjustment unit). The angle adjustment unit 121 is a drive mechanism that moves the position of the first light source unit 12. The angle adjustment unit 121 is composed of an actuator or the like. Specifically, the angle adjustment unit 121 changes the irradiation direction (angle θ) of the pulsed light L1 irradiated by the first light source unit 12 according to the angle signal S2 and the correction signal S3 output from the optical axis correction unit 43.

[0019] The control unit 4 is composed of a microcomputer or the like, and controls each part according to a predetermined program stored in the built-in memory. The control unit 4 executes a correction process for correcting the angle of the first light source unit 12.

[0020] The control unit 4 includes a first image generation unit 41, a first drive control unit 42, and an optical axis correction unit 43.

[0021] The first image generation unit 41 generates a first image based on the first pixel signal output from the first imaging element 112.

[0022] The first drive control unit 42 generates a drive signal S1 and outputs it to the first light source unit 12, the first variable focus lens 111, and the first imaging element 112.

[0023] The optical axis correction unit 43 executes an optical axis correction process. Specifically, the optical axis correction unit 43 generates an angle signal S2 and outputs it to the angle adjustment unit 121. The angle signal S2 includes information indicating the angle θ (first angle) formed between the optical axis direction of the first light source unit 12 (the irradiation direction of the pulsed light L1) and the Z direction. The angle adjustment unit 121 moves the first light source unit 12 so that the angle θ formed between the optical axis direction of the first light source unit 12 and the Z direction becomes the angle θ included in the angle signal S2.

[0024] Also, the optical axis correction unit 43 generates a correction signal S3 based on a plurality of first images generated by the first image generation unit 41 and outputs it to the angle adjustment unit 121. The correction signal S3 includes information indicating the angle θ formed between the optical axis direction of the first light source unit 12 (the irradiation direction of the pulsed light L1) and the Z direction. The angle adjustment unit 121 moves the first light source unit 12 so that the angle θ formed between the optical axis direction of the first light source unit 12 and the Z direction becomes the angle θ included in the angle signal S2.

[0025] (Regarding the optical axis correction process of the control unit 4) FIG. 2 is a flowchart for explaining the flow of the optical axis correction process according to the first embodiment.

[0026] The angle adjustment unit 121 moves the first light source unit 12 so that the angle θ formed between the optical axis direction of the first light source unit 12 and the Z direction becomes the angle θ included in the angle signal S2 (step St1). Specifically, the optical axis correction unit 43 generates the angle signal S2 and outputs it to the angle adjustment unit 121.

[0027] The first imaging unit 11 images the subject A (step St2). Specifically, the first drive control unit 42 generates a drive signal S1 and outputs it to the first light source unit 12, the first variable focus lens 111, and the first imaging element 112. Thereby, the first image generation unit 41 generates a first image based on the first pixel signal output from the first imaging element 112. At this time, the pulsed light L1 irradiated from the first light source unit 12 and reflected by the surface of the subject A1 enters the first imaging element 112 through the first variable focus lens 111.

[0028] The optical axis correction unit 43 extracts the light amount of the specular reflection component and the light amount of the diffuse reflection component from the first image generated by the first image generation unit 41 (step St3). In FIG. 1, L1a is the specular reflection component of the pulsed light L1, and L1b is the diffuse reflection component of the pulsed light L1. For example, the optical axis correction unit 43 extracts the light amount of the specular reflection component L1a (the light amount of the first specular reflection component) and the light amount of the diffuse reflection component L1b (the light amount of the first diffuse reflection component) in a predetermined image area of the first image. That is, the optical axis correction unit 43 extracts the light amount of the specular reflection component L1a and the light amount of the diffuse reflection component L1b of the reflected light (return light) by which the pulsed light L1 is reflected by the subject A1 from the first image (the first pixel signal). When multiple imaging operations are performed in step St2, the light amount of the specular reflection component and the light amount of the diffuse reflection component may be extracted for each first image and integrated respectively.

[0029] The optical correction unit 43 adds a predetermined angle (e.g., 1°) to the angle θ (step St4). Then, the optical axis correction unit 43 determines whether the angle θ is within the range of angle change (step St5). When the optical axis correction unit 43 determines that the angle θ is within the range of angle change (Yes in step St4), it generates an angle signal S2 and outputs it to the angle adjustment unit 121. Then, step S1 is executed again. That is, while changing the angle θ little by little, steps St1 to St3 are executed again.

[0030] When the optical axis correction unit 43 determines that the angle θ is outside the range of angle change (No in step St5), it calculates the range θ of the angle θ at which the light quantity of the specular reflection light component L1a is equal to or greater than the light quantity of the diffuse reflection light component L1b. L (step St6).

[0031] FIG. 3 is a graph showing the relationship between the angle θ and the reflected light of the pulsed light according to the first embodiment. In FIG. 3, the light quantity of the specular reflection light component L1a is shown by a solid line, and the light quantity of the diffuse reflection light component L1b is shown by a broken line. In the example of FIG. 3, the optical axis correction unit 43 calculates the range θ as the range of the angle θ at which the light quantity of the specular reflection light component L1a is equal to or greater than the light quantity of the diffuse reflection light component L1b. L

[0032] The optical axis correction unit 43 calculates the range θ of the angle θ within the range θ where the light quantity of the specular reflection light component L1a is in the vicinity of the maximum value. L C (step St7). In FIG. 3, the optical axis correction unit 43 calculates the range of a predetermined angle (e.g., ±2.5°) with respect to the angle θ at which the light quantity of the specular reflection light component L1a is the highest as the range θ. T C

[0033] The optical axis correction unit 43 generates a correction signal S3 and outputs it to the angle adjustment unit 121 (step St8). The correction signal S3 includes the angle included in the range θ as information on the angle θ (here, the angle θ C ). Thereby, the angle adjustment unit 121 makes the angle θ formed between the optical axis direction of the first light source unit 12 and the Z direction within the range θ. T ) is included. CThe first light source unit 12 is moved so as to achieve this. Thereafter, the subject A1 is imaged by the first imaging unit 11 (step St9).

[0034] As described above, the first imaging device 1 includes a first imaging unit 11 that images a subject from an oblique direction via a first variable-focus lens 111, a first light source unit 12 that irradiates the subject A1 with pulsed light L1 having directivity, an angle adjustment unit 121 (first angle adjustment unit) that changes the angle θ in the optical axis direction of the first light source unit 12 with respect to the subject A1, and an optical axis correction unit 43. The optical axis correction unit 43 extracts the light amount of the specular reflection component L1a (light amount of the first specular reflection component) and the light amount of the diffuse reflection component L1b (light amount of the first diffuse reflection component) from the first pixel signal output from the first imaging unit 11, and calculates the range θ of the angle θ in which the light amount of the specular reflection component is equal to or greater than the light amount of the diffuse reflection component and the light amount of the specular reflection component is in the vicinity of the maximum value. C Thereby, even when the first imaging unit 11 is disposed obliquely with respect to the subject A1, by setting the angle θ such that the light amount of the specular reflection component is in the vicinity of the maximum value, it is possible to focus on the subject A1.

[0035] (Modification example) FIG. 4 is a diagram showing the basic configuration of an imaging device according to a modification example of the first embodiment. As shown in FIG. 4, different from FIG. 1, the first light source unit 12 includes a mirror unit 122 as corresponding to the angle adjustment unit 121 in FIG. 1.

[0036] The mirror unit 122 changes the irradiation direction (angle θ) of the pulsed light L1 irradiated from the first light source unit 12.

[0037] Specifically, the mirror unit 122 includes a first mirror 122a and a second mirror 122b. The first mirror 122a is, for example, a planar mirror capable of changing the orientation of the mirror. That is, the first mirror 122a is configured such that the angle formed with the optical axis direction of the first light source unit 12 can be changed. The second mirror 122b is, for example, a curved mirror. As shown in FIG. 4, the pulsed light L1 emitted from the first light source unit 12 is reflected by the first mirror 122a and the second mirror 122b and then irradiates the subject A1.

[0038] Here, in FIG. 4, the mirror unit 122 changes the orientation of the first mirror 122a based on the angle signal S2 and the correction signal S3 output from the optical axis correction unit 43. That is, in this modified example, the mirror angle of the first mirror 122a changes so that the angle θ formed between the optical axis direction of the first light source unit 12 and the Z direction becomes the angle θ included in the angle signal S2 and the correction signal S3.

[0039] Even in this modified example, the same effects as those of the first embodiment can be obtained.

[0040] Note that the number of mirrors included in the mirror unit 122 may be one or three or more. As long as the angle θ formed between the optical axis direction of the first light source unit 12 and the Z direction can be changed according to the angle signal S2 and the correction signal S3, the number of mirrors included in the mirror unit 122 can be any number.

[0041] Further, the change in the mirror orientation of the first mirror 122a and the change in the refractive index of the first variable focus lens 111 may be synchronized by the drive signal S1 and the angle signal S2. That is, the first mirror 122a may be driven in synchronization with the first variable focus lens 111. For example, when the period during which the mirror angle of the first mirror 122a moves from the maximum angle to the minimum angle is defined as period T1, and the period during which the first variable focus lens 111 changes from the maximum to the minimum is defined as period T2, the first mirror 122a and the first variable focus lens 111 may be driven so that period T1 and period T2 coincide. Note that the first mirror 122a and the first variable focus lens 111 may be driven so that period T1 coincides with n times or 1 / n times (n is an integer of 2 or more) of period T2.

[0042] (Second Embodiment) FIG. 5 is a diagram showing the basic configuration of the distance measurement device according to the second embodiment.

[0043] As shown in FIG. 5, the distance measurement device 1a (distance measurement system) includes a first imaging device 1, a second imaging device 2, and a control unit 4. Note that the first imaging device 1 in FIG. 5 has the same configuration as the first imaging device 1 in FIG. 1. The first imaging device 1 and the second imaging device 2 are configured to be communicable with the control unit 4.

[0044] The second imaging device 2 images the range of the visual field 21a. The second imaging unit 21 is set so that the imaging direction is along the Z direction. The second imaging unit 21 is arranged to face the subject A1.

[0045] The second imaging unit 21 includes a second lens 211 and a second imaging element 212.

[0046] The second lens 211 has a focal length and condenses light from the visual field 21a onto the second imaging element 212. The second lens 211 may not be a single lens, and may be configured by combining a plurality of lenses. The second imaging element 212 is, for example, a color image sensor. The second imaging element 212 is, for example, a CMOS image sensor. Note that the second imaging element 212 may be a CCD.

[0047] The control unit 4 includes a first image generation unit 41, a first drive control unit 42, an optical axis correction unit 43, a second image generation unit 44, and a measurement unit 45. Note that the first image generation unit 41, the first drive control unit 42, and the optical axis correction unit 43 in FIG. 5 have the same configuration as the first image generation unit 41, the first drive control unit 42, and the optical axis correction unit 43 in FIG. 1.

[0048] The second image generation unit 44 generates a second image based on the second pixel signal output from the second imaging device 212.

[0049] The measurement unit 45 performs comparison processing on the first image and the second image to perform stereo corresponding point search, and obtains the distance to the surface of the subject A1 for each pixel block on the first image. Note that the first image used is the first image captured in step St9 of FIG. 2.

[0050] The measurement unit 45 transmits the distance information for all the acquired pixel blocks to an external device. That is, the measurement unit 45 sets a pixel block (hereinafter referred to as a "target pixel block") to be the acquisition target of the distance on the first image, and searches for a pixel block corresponding to this target pixel block, that is, a pixel block that best matches the target pixel block (hereinafter referred to as a "matching pixel block") within the search range defined on the second image. Then, the measurement unit 45 obtains the pixel deviation amount between the pixel block at the same position as the target pixel block (hereinafter referred to as a "reference pixel block") on the second image and the matching pixel block extracted from the second image by the above search, and performs a process of calculating the distance from the obtained pixel deviation amount to the surface of the subject A1 at the position of the target pixel block.

[0051] FIGS. 6(a) and (b) are diagrams schematically showing a method of setting the pixel block 102 with respect to the first image 100. FIG. 6(a) shows a method of setting the pixel block 102 with respect to the entire first image 100, and FIG. 6(b) shows an enlarged view of a partial region of the first image 100.

[0052] As shown in FIGS. 6(a) and 6(b), the first image 100 is divided into a plurality of pixel blocks 102 each including a predetermined number of pixel regions 101. The pixel region 101 is an area corresponding to one pixel on the first imaging device 112. That is, the pixel region 101 is the minimum unit of the first image 100. In the example of FIGS. 6(a) and 6(b), one pixel block 102 is constituted by nine pixel regions 101 arranged in three rows and three columns. However, the number of pixel regions 101 included in one pixel block 102 is not limited to this.

[0053] FIG. 7(a) is a diagram schematically showing a state where a target pixel block TB1 is set on the first image 100, and FIG. 7(b) is a diagram schematically showing a search range R0 set on the second image 200 for searching the target pixel block in FIG. 7(a).

[0054] In FIG. 7(b), for convenience, the second image 200 obtained from the second imaging unit 21 is also divided into a plurality of pixel blocks 202, similar to the first image 100. The pixel block 202 includes the same number of pixel regions as the above-described pixel block 102.

[0055] In FIG. 7(a), the target pixel block TB1 is the pixel block 102 to be processed among the pixel blocks 102 on the first image 100. Also, in FIG. 7(b), the reference pixel block TB2 is the pixel block 202 on the second image 200 at the same position as the target pixel block TB1.

[0056] The measurement unit 45 identifies the reference pixel block TB2 at the same position as the target pixel block TB1 on the second image 200. Then, the measurement unit 45 sets the position of the identified reference pixel block TB2 as the reference position P0 of the search range R0, and sets the range extending in the separation direction of the first imaging unit 11 and the second imaging unit 21 from this reference position P0 as the search range R0.

[0057] The extending direction of the search range R0 is set, on the second image 200, as the direction in which the pixel block (matching pixel block MB2) corresponding to the target pixel block TB1 is displaced from the reference position P0 due to parallax. Here, the search range R0 is set within the range of 12 pixel blocks 202 arranged in the right direction (the direction corresponding to the X-axis direction in FIG. 1) from the reference position P0. However, the number of pixel blocks 202 included in the search range R0 is not limited to this.

[0058] Also, the starting point of the search range R0 is not limited to the reference pixel block TB2. For example, a position shifted several blocks to the right from the reference pixel block TB2 may be set as the starting point of the search range R0.

[0059] The measurement unit 45 searches for the pixel block (matching pixel block MB2) corresponding to the target pixel block TB1 within the thus set search range R0. Specifically, the measurement unit 45 calculates the correlation value between the target pixel block TB1 and each search position while shifting the search position by one pixel to the right from the reference pixel block TB2. As the correlation value, for example, SSD (Sum of Square Difference) or SAD (Sum of Absolute Difference) is used. Then, the measurement unit 45 specifies the pixel block at the search position with the highest correlation on the search range R0 as the matching pixel block MB2.

[0060] Furthermore, the measurement unit 45 acquires the pixel displacement amount (i.e., the parallax amount) of the matching pixel block MB2 with respect to the reference pixel block TB2. Then, the measurement unit 45 calculates the distance to the surface of the subject A1 by triangulation from the acquired pixel displacement amount and the separation distance between the first imaging unit 11 and the second imaging unit 21. The measurement unit 45 executes the same process for all pixel blocks 102 (target pixel blocks TB1) on the first image 100. Thus, when the distances at all pixel blocks 102 are acquired, the measurement unit 45 transmits this distance information to an external device.

[0061] The distance measurement device 1a having the above configuration is not only used fixedly, but also installed, for example, on the end effector (gripping part, etc.) of a robot arm that operates in a factory. In this case, the control unit 4 of the distance measurement device 1a receives an instruction to acquire a distance from a robot controller via a communication interface during the work process of the robot arm. In response to this instruction, the control unit 4 causes the measurement unit 45 to measure the distance between the position of the end effector and the surface of the subject A1 to be worked on, and transmits the measurement result to the robot controller via the communication interface. The robot controller performs feedback control on the operation of the end effector based on the received distance information. Thus, when the distance measurement device 1a is installed on the end effector, it is desirable that the distance measurement device 1a be small and lightweight.

[0062] As described above, the distance measurement device 1a is arranged to face the subject A1, and includes a second imaging unit 21 that images the subject A1, and a measurement unit that calculates the distance to the subject A1 based on the first image (first pixel signal) and the second image (second pixel signal output from the second imaging unit). Thereby, even when the first imaging unit 11 is arranged obliquely with respect to the subject A1, the first imaging unit 11 can be focused on the subject A1, so that the distance to the subject A1 can be calculated from the first image and the second image.

[0063] (Third Embodiment) FIG. 8 is a diagram showing the basic configuration of a distance measurement device according to the third embodiment.

[0064] As shown in FIG. 8, the distance measurement device 1b (distance measurement system) includes a first imaging device 1, a third imaging device 3, and a control unit 4. The first imaging device 1 and the third imaging device 3 are configured to be communicable with the control unit 4.

[0065] The first imaging device 1 includes a first imaging unit 11.

[0066] The first imaging unit 11 has substantially the same configuration as the first imaging unit 11 in FIG. 1.

[0067] Specifically, the first imaging unit 11 images the range of the visual field 11a. The imaging direction of the first imaging unit 11 is set to a direction that forms an angle θ1 with the Z direction. That is, the first imaging unit 11 images the subject A1 from an oblique direction (the upper left direction in the drawing).

[0068] The first imaging unit 11 includes a first variable-focus lens 111 and a first imaging element 112.

[0069] The first variable-focus lens 111 condenses the light from the visual field 11a onto the first imaging element 112. The first variable-focus lens 111 is composed of a first lens 111a and a first liquid lens unit 111b. The first lens 111a is arranged on the same optical axis as the first liquid lens unit 111b. The refractive index of the first liquid lens unit 111b changes according to the drive signal S1 output by the control unit 4.

[0070] The first imaging element 112 outputs a first pixel signal according to the drive signal S1 output from the first drive control unit 42. That is, the imaging timing of the first imaging unit 11 is determined according to the drive signal S1.

[0071] A third light source unit 13 is provided in the first imaging device 1. The third light source unit 13 irradiates pulsed light L3. For example, the third light source unit 13 is composed of a light-emitting diode. The pulsed light L3 is light having directivity in the optical axis direction of the third light source unit 13 (in the example of FIG. 1, the arrow direction). The third light source unit 13 irradiates the pulsed light L3 according to the drive signal S4 output by the control unit 4.

[0072] The third imaging device 3 includes a third imaging unit 31. The third imaging unit 31 images the range of the visual field 31a. The imaging direction of the third imaging unit 31 is set to a direction that forms an angle θ2 with the Z direction. That is, the third imaging unit 31 images the subject A1 from an oblique direction (the upper right direction in the drawing).

[0073] The third imaging unit 31 includes a third variable-focus lens 311 and a third imaging element 312.

[0074] The third variable focus lens 311 condenses light from the visual field 31a onto the third imaging element 312. The third variable focus lens 311 is composed of a third lens 311a and a third liquid lens unit 311b. The third lens 311a is, for example, a convex lens and is arranged on the same optical axis as the third liquid lens unit 311b. The refractive index of the third liquid lens unit 311b changes according to the drive signal S4 output by the control unit 4. For example, the drive signal S4 is an alternating current signal with a frequency that generates a standing wave in the third liquid lens unit 311b.

[0075] The third imaging element 312 is a color image sensor. The third imaging element 312 is, for example, a CMOS image sensor. Note that the third imaging element 312 may be a CCD. The third imaging element 312 outputs a third pixel signal according to the drive signal S4 output from the third drive control unit 47. That is, the imaging timing of the third imaging unit 31 is determined according to the drive signal S4.

[0076] The third imaging device 3 is provided with a fourth light source unit 33 (first light source unit). The fourth light source unit 33 irradiates pulsed light L4. For example, the fourth light source unit 33 is composed of a light emitting diode. The pulsed light L4 is light having directivity in the optical axis direction of the fourth light source unit 33 (in the example of FIG. 8, the arrow direction). The fourth light source unit 33 irradiates pulsed light L4 according to the drive signal S1 output by the control unit 4.

[0077] The first lens 111a and the third lens 311a are each provided with a half mirror 14, 34. As shown in FIG. 8, the pulsed light L3 irradiated from the third light source unit 13 is reflected by the half mirror 14 and irradiated onto the subject A1. Then, the pulsed light L3 reflected from the surface of the subject A1 passes through the half mirror 34 (the third variable focus lens 311) and enters the third imaging element 312. Further, the pulsed light L4 irradiated from the fourth light source unit 33 is reflected by the half mirror 34 and irradiated onto the subject A1. Then, the pulsed light L4 reflected from the surface of the subject A1 passes through the half mirror 14 (the first variable focus lens 111) and enters the first imaging element 112. Note that the half mirror 14 is set so that the imaging direction of the first imaging unit 11 with respect to the subject A1 and the irradiation direction of the pulsed light L3 coincide. Specifically, both the imaging direction of the first imaging unit 11 and the irradiation direction of the pulsed light L3 are set in a direction forming an angle θ1 with the Z direction. Also, the half mirror 34 is set so that the imaging direction of the third imaging unit 31 with respect to the subject A1 and the irradiation direction of the pulsed light L4 coincide. Specifically, both the imaging direction of the third imaging unit 31 and the irradiation direction of the pulsed light L4 are set in a direction forming an angle θ2 with the Z direction.

[0078] The first imaging device 1 and the third imaging device 3 are each provided with an angle adjustment unit 15, 35.

[0079] The angle adjustment unit 15 (the third angle adjustment unit) is a drive mechanism that moves the first imaging device 1. Specifically, the angle adjustment unit 15 moves the first imaging device 1 according to the angle signal S5 and the correction signal S3 output from the optical axis correction unit 43. That is, the angle adjustment unit 15 changes the imaging direction of the first imaging unit 11 and the light receiving direction of the return light (angle θ1) in which the pulsed light L4 is reflected by the subject A1 and reaches the first imaging unit 11 side according to the angle signal S5 and the correction signal S3.

[0080] The angle adjustment unit 35 (first angle adjustment unit) is a drive mechanism that moves the third imaging device 3. Specifically, the angle adjustment unit 35 moves the third imaging device 3 in accordance with the angle signal S6 and the correction signal S3 output from the optical axis correction unit 43. That is, the angle adjustment unit 35 changes the imaging direction of the third imaging unit 31 and the light receiving direction (angle θ2) of the return light that the pulsed light L3 is reflected by the subject A1 and reaches the third imaging unit 31 side in accordance with the angle signal S6 and the correction signal S3.

[0081] The control unit 4 includes a first image generation unit 41, a first drive control unit 42, an optical axis correction unit 43, a measurement unit 45, a third image generation unit 46, and a third drive control unit 47.

[0082] The first image generation unit 41 generates a first image based on the first pixel signal output from the first imaging element 112.

[0083] The first drive control unit 42 generates a drive signal S1 and outputs it to the third light source unit 13, the first variable focus lens 111, and the first imaging element 112.

[0084] The third image generation unit 46 generates a third image based on the third pixel signal output from the third imaging element 312.

[0085] The third drive control unit 47 generates a drive signal S4 and outputs it to the fourth light source unit 33, the third variable focus lens 311, and the third imaging element 312.

[0086] The optical axis correction unit 43 executes optical axis correction processing. Specifically, the optical axis correction unit 43 generates angle signals S5 and S6 and outputs them to the angle adjustment units 15 and 35, respectively. The angle signal S5 includes information indicating the angle θ1 (first angle) formed between the imaging direction of the first imaging unit 11 and the irradiation direction of the pulsed light L3, and the Z direction. The angle adjustment unit 15 moves the first imaging unit 11 so that the angle θ1 formed between the imaging direction of the first imaging unit 11 and the irradiation direction of the pulsed light L3, and the Z direction, becomes the angle θ1 included in the angle signal S5. Further, the angle signal S6 includes information indicating the angle θ2 (second angle) formed between the imaging direction of the third imaging unit 31 and the irradiation direction of the pulsed light L4, and the Z direction. The angle adjustment unit 35 moves the third imaging unit 31 so that the angle θ2 formed between the imaging direction of the third imaging unit 31 and the irradiation direction of the pulsed light L4, and the Z direction, becomes the angle θ2 included in the angle signal S6.

[0087] The measurement unit 45 compares the first image and the third image, performs stereo corresponding point search, and obtains the distance to the surface of the subject A1 for each pixel block on the first image. Note that since the stereo corresponding point search in the measurement unit 45 is the same as that in the second embodiment, detailed description thereof is omitted in this embodiment.

[0088] (Regarding the optical axis correction processing of the control unit 4) FIGS. 9 and 10 are flowcharts for explaining the flow of the optical axis correction processing according to the third embodiment.

[0089] The optical axis correction unit 43 generates the angle signals S5 and S6 and arranges the first imaging unit 11 and the third imaging unit 31 at their initial positions (step St11).

[0090] The first imaging unit 11 images the subject A1 (step St12). Specifically, the optical axis correction unit 43 generates a drive signal S1 and outputs it to the fourth light source unit 33, the first variable focus lens 111, and the first imaging element 112. As a result, the first image generation unit 41 generates a first image based on the first pixel signal output from the first imaging element 112. At this time, the pulsed light L4 irradiated from the fourth light source unit 33 and reflected by the surface of the subject A1 enters the first imaging element 112 through the first variable focus lens 111.

[0091] The third imaging unit 31 images the subject A1 (step St13). Specifically, the optical axis correction unit 43 generates a drive signal S4 and outputs it to the third light source unit 13, the third variable focus lens 311, and the third imaging element 312. As a result, the third image generation unit 46 generates a third image based on the third pixel signal output from the third imaging element 312. At this time, the pulsed light L3 irradiated from the third light source unit 13 and reflected by the surface of the subject A1 enters the third imaging element 312 through the third variable focus lens 311.

[0092] The optical axis correction unit 43 extracts the light amount of the specular reflection component and the light amount of the diffuse reflection component from the first image generated by the first image generation unit 41 (step St14). For example, the optical axis correction unit 43 extracts the light amount of the specular reflection component (the light amount of the first specular reflection component) and the light amount of the diffuse reflection component (the light amount of the first diffuse reflection component) in a predetermined image area of the first image. That is, the optical axis correction unit 43 extracts the light amount of the specular reflection component and the light amount of the diffuse reflection component of the reflected light (return light) by which the pulsed light L4 is reflected by the subject A1 based on the first image. In step St14, when imaging is performed multiple times, the light amount of the specular reflection component and the light amount of the diffuse reflection component may be extracted for each first image, and each may be integrated.

[0093] The optical axis correction unit 43 extracts the light quantity of the direct reflection component and the light quantity of the diffuse reflection component from the third image generated by the third image generation unit 46 (step St15). In FIG. 8, L4a is the direct reflection component of the pulsed light L4, and L4b is the diffuse reflection component of the pulsed light L4. For example, the optical axis correction unit 43 extracts the light quantity of the direct reflection component L4a (the light quantity of the third direct reflection component) and the light quantity of the diffuse reflection component L4b (the light quantity of the third diffuse reflection component) in a predetermined image area of the third image. That is, the optical axis correction unit 43 extracts the light quantity of the direct reflection component L4a and the light quantity of the diffuse reflection component L4b of the reflected light (return light) reflected by the subject A1 from the pulsed light L4 based on the third image. In step St15, when imaging is performed multiple times, the light quantity of the direct reflection component L4a and the light quantity of the diffuse reflection component L4b may be extracted for each third image and integrated respectively.

[0094] The optical axis correction unit 43 adds a predetermined angle (for example, 1°) to the angle θ1 (step St16). Then, the optical axis correction unit 43 determines whether the angle θ1 is within the range of angle change (step St17). When the optical axis correction unit 43 determines that the angle θ1 is within the range of angle change (Yes in step St17), it generates an angle signal S5 and outputs it to the angle adjustment unit 15. Thereafter, step St12 is executed again. That is, steps St12 to S15 are executed again while changing the angle θ1 little by little.

[0095] When the optical axis correction unit 43 determines that the angle θ1 is not within the range of angle change (No in step St17), it adds a predetermined angle (for example, 1°) to the angle θ2 (step St18). The optical axis correction unit 43 determines whether the angle θ2 is within the range of angle change (step St19).

[0096] When the optical axis correction unit 43 determines that the angle θ2 is within the range of angle change (Yes in step St19), it generates an angle signal S6 and outputs it to the angle adjustment unit 35. Then, the first imaging unit 11 is arranged at the initial position (step St20). Specifically, the optical axis correction unit 43 generates an angle signal S5, outputs it to the angle adjustment unit 15, and arranges the first imaging unit 11 at the initial position. After that, step St12 is executed again. That is, steps St12 to St17 are executed again while changing the angle θ2 little by little.

[0097] When the optical axis correction unit 43 determines that the angle θ2 included in the angle signal S6 is equal to or greater than the third angle (Yes in step St18), based on a plurality of first images, the range θ of the angle θ1 at which the light quantity of the specular reflection light component L4a is equal to or greater than the light quantity of the diffuse reflection light component L4b L1 is calculated (step St21). The optical axis correction unit 43 calculates the range θ L1 of the angle θ1 in which the light quantity of the specular reflection light component L4a is in the vicinity of the maximum value in the first image among the ranges θ C1 is calculated (step St22). The optical axis correction unit 43 calculates the range θ of the angle θ2 at which the light quantity of the specular reflection light component L3a is equal to or greater than the light quantity of the diffuse reflection light component L3b based on a plurality of second images L2 is calculated (step St23). The optical axis correction unit 43 calculates the range θ L2 of the angle θ2 in which the light quantity of the specular reflection light component L3a is in the vicinity of the maximum value in the second image among the ranges θ C2 is calculated (step St24).

[0098] FIG. 11(a) is a graph showing the relationship between the angle θ1 and the reflected light of the pulsed light L4 according to the third embodiment, and FIG. 11(b) is a graph showing the relationship between the angle θ2 and the reflected light of the pulsed light L3 according to the third embodiment. In FIGS. 11(a) and 11(b), the light quantity of the specular reflection light component is shown by a solid line and the light quantity of the diffuse reflection light component is shown by a broken line, respectively.

[0099] In the example of FIG. 11, the optical axis correction unit 43 has a range θ L1 (θ L2is calculated as the range of the angle θ1 (θ2) where the light quantity of the specular reflection light component L4a (L3a) is equal to or greater than the light quantity of the diffuse reflection light component L4b (L3b). Further, the optical axis correction unit 43 is the angle θ at which the light quantity of the specular reflection light component L4a (L3a) becomes the maximum value. T1 (θ T2 ) is used as a reference to calculate a range of a predetermined angle (for example, ±2.5°) as the range θ C1 (θ C2 ).

[0100] The optical axis correction unit 43 calculates the angles θ1 and θ2 at which the integrated amount of the light quantity of the specular reflection light component L3a of the pulsed light L3 and the light quantity of the specular reflection component L4a of the pulsed light L4 becomes maximum in the region where the ranges θ C1 and the range θ C2 overlap (step St25). In the example of FIG. 10, the angles θ1 and θ2 are calculated as the angles θ T1 , θ T2 . Note that the ranges of the angles θ1 and θ2 at this time may be specified by the user. For example, when |θ1 - θ2| < ε, ε may be set by the user. That is, the optical axis correction unit 43 calculates the angles θ1 and θ2 such that the difference between the angle θ1 and the angle θ2 falls within a predetermined range ε.

[0101] The optical axis correction unit 43 generates a correction signal S3 and outputs it to the angle adjustment units 15 and 35 (step St26). The correction signal S3 includes the angles θ1 and θ2 as the angles θ T1 , θ T2 . As a result, the angle adjustment unit 15 moves the first imaging unit 11 so that the angle formed by the imaging direction of the first imaging unit 11 and the irradiation direction of the pulsed light L3 and the Z direction becomes the angle θ1 included in the angle signal S5. Further, the angle adjustment unit 35 moves the third imaging unit 31 so that the angle formed by the imaging direction of the third imaging unit 31 and the irradiation direction of the pulsed light L4 and the Z direction becomes the angle θ2 included in the angle signal S6.

[0102] Thereafter, the first imaging unit 11 and the third imaging unit 31 image the subject A1 (step St27). Then, the measurement unit 45 performs comparison processing on the first image and the third image respectively generated by the first imaging device 112 and the third imaging device 312, performs stereo corresponding point search, and obtains the distance to the surface of the subject A1 for each pixel block on the first image (step St28).

[0103] As described above, the distance measuring device 1b includes a third imaging unit 31 that images the subject A1 from an oblique direction via the third variable focal lens 311, a third light source unit 13 that irradiates light having directivity to the subject A1, and an angle adjustment unit 15 that changes the angle θ2 (third angle) in the optical axis direction of the third light source unit 13 with respect to the subject A1. The fourth light source unit 33 (first light source unit) and the angle adjustment unit 35 (first angle adjustment unit) are provided in the third imaging unit 31. The third light source unit 13 and the angle adjustment unit 15 (third angle adjustment unit) are provided in the first imaging unit 11. The optical axis correction unit 43 extracts the light amount of the specular reflection component L3a (light amount of the third specular reflection component) and the light amount of the diffuse reflection component L3b (light amount of the third diffuse reflection component) from the third pixel signal output from the third imaging unit 31, and the range θ of the angle θ2 where the light amount of the specular reflection component L3a is greater than or equal to the light amount of the diffuse reflection component L3b and the light amount of the specular reflection component L3a is near the maximum value C2 (third angle) is calculated. Thereby, even when the first imaging unit 11 and the third imaging unit 31 are arranged obliquely with respect to the subject A1, by setting the angles θ1 and θ2 so that the light amount of the specular reflection component is near the maximum value, it is possible to focus on the subject A1. In particular, after providing the third light source unit 13 in the first imaging unit 11 and providing the fourth light source unit 33 in the third imaging unit 31, by moving the first imaging unit 11 and the third imaging unit 31, the first imaging unit 11 and the third imaging unit 31 can be focused simultaneously.

Explanation of Reference Numerals

[0104] 1 First imaging device 1a, 1b Distance measuring device 2 Second imaging device 3 Third imaging device 4 Control unit 11 First imaging unit 111 First variable-focus lens 12 First light source unit 121 Angle adjustment unit (first angle adjustment unit) 122 Mirror unit 122a First mirror 122b Second mirror 13 Third light source unit 15 Angle adjustment unit (third angle adjustment unit) 21 Second imaging unit 31 Third imaging unit 33 Fourth light source unit (first light source unit) 35 Angle adjustment unit (first angle adjustment unit) 43 Optical axis correction unit 46 Measurement unit A1 Subject L1, L3, L4 Pulse light L1a, L3a, L4a Specular reflection component L1b, L3b, L4b Diffuse reflection component

Claims

1. A first light source unit that irradiates a subject with light; A first angle adjustment unit that changes a first angle, which is an angle in the optical axis direction of the first light source unit with respect to the subject; From a first pixel signal output from a first imaging unit that images the subject via a first variable focal lens, the amount of light of a first specular reflection component, which is the amount of light of the specular reflection component, and the amount of light of a first diffuse reflection component, which is the amount of light of the diffuse reflection component, are extracted, and an imaging system comprising an optical axis correction unit that calculates the first angle such that the amount of light of the first specular reflection component is equal to or greater than the amount of light of the first diffuse reflection component and the amount of light of the first specular reflection component is in the vicinity of the maximum value.

2. The imaging system according to claim 1, wherein the first angle adjustment unit changes the optical axis direction of the first light source unit to the first angle calculated by the optical axis correction unit.

3. The first angle adjustment unit includes a first mirror and a second mirror provided between the first light source unit and the subject, The first mirror is a mirror configured such that an angle formed with the optical axis direction of the first light source unit is changeable, The imaging system according to claim 1, wherein the second mirror is a curved mirror.

4. The imaging system according to claim 3, wherein the first mirror is driven in synchronization with the first variable focal lens.

5. An imaging system according to any one of claims 1 to 4, and A second imaging unit that is arranged to face the subject and images the subject, and A distance measurement system comprising a measurement unit that calculates a distance to the subject based on the first pixel signal and a second pixel signal output from the second imaging unit.

6. An imaging system according to any one of claims 1 to 4, and A third light source unit that irradiates the subject with light, and A third angle adjustment unit that changes a third angle, which is the angle in the optical axis direction of the third light source unit with respect to the subject. The first light source unit and the first angle adjustment unit are provided in a third imaging unit that images the subject via a third variable focus lens. The third light source unit and the third angle adjustment unit are provided in the first imaging unit. The optical axis correction unit extracts, from a third pixel signal output from the third imaging unit, the light amount of a third regular reflection component, which is the light amount of the regular reflection component, and the light amount of a third diffuse reflection component, which is the light amount of the diffuse reflection component, and calculates the third angle at which the light amount of the third regular reflection component is greater than or equal to the light amount of the third diffuse reflection component and the light amount of the third regular reflection component is near the maximum value. An imaging system. **Claim 7** The imaging system according to claim 6, wherein the optical axis correction unit calculates the first angle and the third angle so that the difference between the first angle and the third angle falls within a predetermined range. **Claim 8** Further comprising a measurement unit that calculates the distance to the subject. The first angle adjustment unit changes the optical axis direction of the first light source unit to the first angle calculated by the optical axis correction unit. The third angle adjustment unit changes the optical axis direction of the third light source unit to the third angle calculated by the optical axis correction unit. The imaging system according to claim 6, wherein the measurement unit calculates the distance to the subject based on the first pixel signal and the third pixel signal.

Images