Imaging system, electromagnetic wave irradiation system, measurement method, and program

Abstract

An imaging system (100) according to the present invention is provided with: an imaging optical system (110); a light splitting member (120) that splits the light (LT) that has passed through the imaging optical system (110); a first imaging device (140) for imaging a first image (Im1) formed by the light beam split by the light splitting member (120); a second imaging device (150) for imaging a second image (Im2) formed by the other beam of light split by the light splitting member (120); and a distance measurement device (170) that generates distance information of the target object by irradiating measurement light (LM) to the target object via the imaging optical system (110) on the basis of image information of a first image (Im1) including an image of at least a part of the target object captured by the first imaging device (140).

Description

Technical Field

[0001] This invention relates to imaging systems, electromagnetic wave irradiation systems, measurement methods, and procedures. Background Technology

[0002] Previously, as described in Patent Document 1, there was a known imaging system capable of following and photographing an object. However, such imaging systems have difficulty measuring the distance to the object.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: US Patent No. 4,855,838 Summary of the Invention

[0006] The first invention relates to an imaging system comprising: an imaging optical system; a light splitting member that splits at least a portion of the light passing through the imaging optical system; a first imaging device that captures a first image formed by a beam of light split by the light splitting member; a second imaging device that captures a second image formed by another beam of light split by the light splitting member; and a distance measuring device that, based on image information of the first image captured by the first imaging device, which contains an image of at least a portion of the object, illuminates the object with measuring light via the imaging optical system and generates distance information of the object.

[0007] The second invention relates to an imaging system comprising: an imaging optical system; a light splitting member that splits light passing through the imaging optical system; an imaging device that captures a first image formed by a beam of light split by the light splitting member; and a distance measuring device that, based on image information of the first image captured by the imaging device, which contains an image of at least a portion of the object, illuminates the object with measuring light via the imaging optical system and generates distance information of the object.

[0008] The electromagnetic wave irradiation system involved in this invention includes a shooting system; and an electromagnetic wave irradiation device for irradiating electromagnetic waves onto a target object.

[0009] The first measurement method of the present invention uses an imaging optical system and an imaging system comprising an imaging optical system and an imaging component that divides at least a portion of the light passing through the imaging optical system. The measurement method includes: capturing a first image formed by a beam of light divided by the imaging component using a first imaging device; capturing a second image formed by another beam of light divided by the imaging component using a second imaging device; and illuminating the object with measurement light via the imaging optical system based on image information of the first image captured by the first imaging device, which contains an image of at least a portion of the object, and generating distance information of the object.

[0010] The second measurement method of the present invention uses an imaging optical system and an imaging system comprising a light-splitting member for splitting light passing through the imaging optical system. The measurement method includes: capturing a first image formed by a beam of light split by the light-splitting member using an imaging device; and illuminating the object with measurement light via the imaging optical system based on image information of the first image captured by the imaging device, which contains an image of at least a portion of the object, and generating distance information of the object.

[0011] The program involved in this invention enables a computer to perform the above-described measurement method. Attached Figure Description

[0012] Figure 1 This is a three-dimensional diagram illustrating an example of an electromagnetic wave irradiation system.

[0013] Figure 2 This is a simplified structural diagram of the imaging unit involved in Implementation Method 1.

[0014] Figure 3 This diagram illustrates the state in which the first reflective member in the imaging unit according to Embodiment 1 rotates and the light path changes.

[0015] Figure 4 This is the front view of the first camera element.

[0016] Figure 5 This is the front view of the second camera element.

[0017] Figure 6 This is a simplified structural diagram illustrating an example of a distance measuring device.

[0018] Figure 7 This is a flowchart illustrating the measurement method involved in Implementation Method 1.

[0019] Figure 8 This is a schematic diagram representing an example of the first image.

[0020] Figure 9 This is a schematic diagram representing an example of the second image and the intermediate image.

[0021] Figure 10 This is a schematic diagram representing an example of the first image.

[0022] Figure 11 This is a schematic diagram representing an example of the second image.

[0023] Figure 12 This is a simplified structural diagram of a modified example of the imaging unit involved in Embodiment 1.

[0024] Figure 13 This is a front view showing a modified example of the second imaging element.

[0025] Figure 14 This is a schematic diagram representing a variant of the second image.

[0026] Figure 15 This is a simplified structural diagram of the shooting unit involved in Embodiment 2.

[0027] Figure 16 This is a front view of the second imaging element involved in Embodiment 2.

[0028] Figure 17 This is a simplified structural diagram of the imaging unit involved in Embodiment 3.

[0029] Figure 18 This is a simplified structural diagram of the imaging unit involved in Embodiment 4.

[0030] Figure 19 This diagram illustrates the state in which the first reflective member in the imaging unit according to Embodiment 4 rotates and the light path changes.

[0031] Figure 20 This is a flowchart illustrating the measurement method involved in Implementation Method 4.

[0032] Figure 21 This is a schematic diagram illustrating an example of a magnified image based on electronic zoom.

[0033] Figure 22 This is a simplified structural diagram of a modified example of the imaging unit involved in Embodiment 4.

[0034] Figure 23 This is a simplified structural diagram of the imaging unit involved in Embodiment 5.

[0035] Figure 24 This is a flowchart illustrating the measurement method involved in Implementation Method 5.

[0036] Figure 25This is a schematic diagram representing an example of the first image with the region of interest defined.

[0037] Figure 26 This is a three-dimensional diagram illustrating another example of an electromagnetic wave irradiation system.

[0038] Figure 27 This is a simplified structural diagram of the imaging unit involved in Implementation Method 6.

[0039] Figure 28 This is a flowchart illustrating the irradiation method involved in Implementation Method 6.

[0040] Figure 29 This is a simplified structural diagram of the imaging unit involved in Embodiment 7. Detailed Implementation

[0041] The preferred embodiments will be described below. First, using... Figure 1 The electromagnetic wave irradiation system, including the imaging system according to Embodiments 1 to 5, will be described below. In the following description, the directions indicated by arrows in each figure will sometimes be referred to as the X direction, Y direction, and Z direction, respectively. The X direction, Y direction, and Z direction will be explained later.

[0042] [Electromagnetic wave irradiation system]

[0043] like Figure 1 As shown, the electromagnetic wave irradiation system 1 includes an imaging system 100, a housing component 10 housing the imaging system 100, and an electromagnetic wave irradiation device 20. The electromagnetic wave irradiation system 1 can be fixed to a building or similar structure on the ground, or mounted on a movable body. The movable body can be a drone (unmanned aerial vehicle) or a manned aircraft. Furthermore, the movable body can also be a robot such as a multi-jointed robot. The movable body can be an unmanned vehicle (e.g., an unmanned ground vehicle) or a manned vehicle (e.g., a truck). The movable body can be an unmanned vessel or a manned vessel. The movable body can be an artificial satellite. The movable body can be an unmanned spacecraft or a manned spacecraft.

[0044] The electromagnetic wave irradiation device 20 is positioned differently from the imaging system 100 (housing member 10). The electromagnetic wave irradiation device 20 includes a transmitting device 21, a supporting device 23, and an irradiation control device 25. The transmitting device 21 irradiates electromagnetic waves LB onto the target object TG. The supporting device 23 supports the transmitting device 21 in a manner that allows it to rotate about two mutually orthogonal rotation axes (e.g., about a rotation axis extending in the vertical direction (Z direction) and a rotation axis extending in the horizontal direction). By rotating the transmitting device 21 about the two rotation axes, the supporting device 23 can change the irradiation direction of the electromagnetic waves LB emitted from the transmitting device 21. Based on information output from the imaging system 100 (output device 187 described later), the irradiation control device 25 controls at least one of the transmitting device 21 and the supporting device 23 to irradiate the target object TG continuously or intermittently.

[0045] Electromagnetic wave LB is irradiated onto the target object TG from the emitting device 21 of the electromagnetic wave irradiation device 20 to damage, repel, or remove the target object TG. The target object TG to be damaged, repelled, or removed can be at least one of pests, pests, or weeds. Furthermore, the target object TG to be damaged, repelled, or removed can also be space debris (space junk) floating in outer space. The target object TG is not limited to pests, pests, weeds, or space debris; it can be any object that needs to be damaged, repelled, or removed. The electromagnetic wave LB irradiating the target object TG can be light or radio waves, as long as it can damage, repel, or remove the target object TG. For example, when the electromagnetic wave LB irradiating the target object TG is light, it can also be a highly directional beam of light. For example, the beam of light can also be a laser. For example, when the target object TG is a pest (such as a moth), the irradiated electromagnetic wave LB can be light that includes at least the blue wavelength range (e.g., a laser in the blue wavelength range). When the target object TG is a weed, the irradiated electromagnetic wave LB can be light that includes at least the infrared wavelength range (e.g., a laser in the infrared wavelength range). Thus, the electromagnetic wave LB irradiating the target object TG can be light encompassing at least one wavelength range selected from infrared, ultraviolet, and visible light. Furthermore, the electromagnetic wave LB irradiating the target object TG is not limited to the examples described above; it can be light of other wavelength ranges as long as it is capable of destroying or removing the target object TG. Additionally, for example, when the electromagnetic wave LB irradiating the target object TG is an electromagnetic wave, it can be a microwave. Furthermore, the electromagnetic wave irradiating the target object TG is not limited to microwaves; it can be electromagnetic waves of other wavelength ranges (frequency ranges). Furthermore, the output of the electromagnetic wave LB irradiating the target object TG (e.g., the output of light irradiating the target object TG) can be 10 kW or more. Furthermore, the output of the electromagnetic wave LB irradiating the target object TG is not limited to 10 kW or more; it can also be less than 10 kW. Furthermore, the output of the electromagnetic wave LB irradiating the target object TG only needs to be an output capable of destroying, removing, or eliminating the target object TG. Additionally, the output of the electromagnetic wave LB irradiating the target object TG can also be the output of the electromagnetic wave LB emitted from the electromagnetic wave irradiation device 20.

[0046] The housing member 10 is box-shaped. At least a portion of the imaging system 100 is housed inside the housing member 10. The arrangement of the imaging system 100 (housing member 10) and the electromagnetic wave irradiation device 20 is not limited to... Figure 1The positions shown are as follows. For example, the imaging system 100 and the electromagnetic wave irradiation device 20 can be installed on a platform (not shown) in such a way that their positional relationship remains unchanged. For example, the imaging system 100 and the electromagnetic wave irradiation device 20 can be installed on a drivable movable platform (not shown) in such a way that their positional relationship changes. For example, the imaging system 100 (housing member 10) can be mounted on the platform portion below the support device 23 of the electromagnetic wave irradiation device 20. In this case, the imaging system 100 (housing member 10) can rotate about a rotation axis extending in the vertical direction (Z direction) via the support device 23. For example, the imaging system 100 (housing member 10) can also be mounted on the upper part of the emitting device 21 of the electromagnetic wave irradiation device 20. In this case, the imaging system 100 (housing member 10) can rotate together with the emitting device 21 about two rotation axes. In addition, the imaging system 100 can also be mounted on the outside of the housing member 10, such as on the side or upper part. Next, the imaging system 100 according to Embodiment 1 will be described. Alternatively, the shell component 10 may be omitted.

[0047] [The imaging system described in Implementation Method 1]

[0048] like Figure 2 and Figure 3 As shown, the imaging system 100 according to Embodiment 1 includes an imaging optical system 110, a light splitting member 120, a re-imaging optical system 130, a first imaging device 140, a second imaging device 150, a reflector rotating device 160, a distance measuring device 170, a control device 180, and a frame 101. The frame 101 houses and holds the imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the second imaging device 150, the reflector rotating device 160, and the distance measuring device 170.

[0049] In this embodiment, the optical axis of the first optical system 131 in the re-imaging optical system 130 and the optical axis of the imaging optical system 110 coaxial with the first optical system 131 are referred to as the first optical axis AX1. The optical axis of the second optical system 132 in the re-imaging optical system 130 is referred to as the second optical axis AX2. The second optical axis AX2 may intersect the first optical axis AX1. Furthermore, the second optical axis AX2 may intersect the first optical axis AX1. In addition, the direction along the first optical axis AX1 is referred to as the Y direction. The direction along the second optical axis AX2 is referred to as the Z direction. The direction orthogonal to the Y and Z directions is referred to as the X direction. The X, Y, and Z directions are mutually orthogonal. For example, as... Figure 1 As shown, the Z direction can be vertical, while the X and Y directions can be horizontal.

[0050] like Figure 2 and Figure 3 As shown, the imaging optical system 110 includes a front lens 111, a rear lens 112, and an aperture stop 113. Furthermore, the imaging optical system 110 can also be telecentric on the side of the light splitting member 120 (the intermediate image plane Imd described later). The front lens 111 is positioned further away from the light splitting member 120 than the aperture stop 113. The rear lens 112 is positioned closer to the light splitting member 120 than the aperture stop 113. Figure 2 In the diagram, a front lens 111 and a rear lens 112 are schematically represented by a single lens. The front lens 111 and the rear lens 112 can each be composed of a single lens or multiple lenses. In addition to one or more lenses, both the front lens 111 and the rear lens 112 may include existing optical components other than lenses. The front lens 111 may include a focusing lens that can move along the optical axis during focusing.

[0051] Furthermore, the imaging optical system 110 can also be configured to be interchangeable. As an interchangeable imaging optical system 110, for example, at least one of a monofocal lens and a fisheye lens can be used. In other words, as the imaging optical system 110, either a monofocal lens or a fisheye lens can be used. Moreover, as an interchangeable imaging optical system 110, various monofocal lenses with different focal lengths can also be used. Alternatively, the imaging optical system 110 may not be configured to be interchangeable. Even in this case, the imaging optical system 110 can be either a monofocal lens or a fisheye lens. For example, the maximum viewing angle of the imaging optical system 110 (i.e., the maximum viewing angle of the image captured by the first imaging device (the first image PC1 described later) on the object side) can be 170° or more. For example, the maximum viewing angle of the imaging optical system 110 is not limited to 170° or more, and may be less than 170°. For example, the maximum viewing angle of the imaging optical system 110 can be 160°.

[0052] like Figure 2 and Figure 3As shown, a light splitting member 120 is disposed between the imaging optical system 110 and the re-imaging optical system 130. The light splitting member 120 splits the light LT passing through the imaging optical system 110. For example, the light splitting member 120 can be constructed using a semi-reflective mirror to split the amplitude of the light LT passing through the imaging optical system 110. Specifically, the light splitting member 120 can cause a portion of the light LT passing through the imaging optical system 110 to be reflected towards the first imaging device 140 along the +Z direction, while allowing other light to pass through. The semi-reflective mirror can be prism-type or planar-type. The ratio of the transmittance to reflectance of the semi-reflective mirror can be 1:1. Alternatively, the ratio of the transmittance to reflectance of the semi-reflective mirror is not limited to 1:1 and can be other ratios. For example, the ratio can be 2:1.

[0053] Furthermore, the light splitting member 120 can be constructed using a dichroic mirror to split light within a specified wavelength range in the light LT passing through the imaging optical system 110. For example, the light splitting member 120 can cause light in the near-infrared wavelength range of the light LT passing through the imaging optical system 110 to be reflected towards the first imaging device 140 in the +Z direction, while allowing light in other wavelength ranges (visible light) to pass through. The light splitting member 120 can also allow light in the near-infrared wavelength range of the light LT passing through the imaging optical system 110 to pass through, while causing light in other wavelength ranges (visible light) to be reflected towards the first imaging device 140 in the +Z direction.

[0054] Furthermore, the light splitting member 120 can be configured using a polarization beam splitter to polarize the light LT passing through the imaging optical system 110. Alternatively, the light splitting member 120 can be positioned midway through the imaging optical system 110, in which case it can split a portion of the light LT passing through the imaging optical system 110. For example, the rear lens 112 can be positioned between the light splitting member 112 and the first image plane Im1, and between the light splitting member 120 and the intermediate image plane Imd.

[0055] In the light LT passing through at least a portion of the imaging optical system 110, the light reflected by the light splitting member 120 forms a first image Im1. In other words, the first image Im1 is formed by a beam of light split by the light splitting member 120. Additionally, in the light LT passing through the imaging optical system 110, an intermediate image Imd is formed by the light transmitted through the light splitting member 120. In other words, the intermediate image Imd is formed by another beam of light split by the light splitting member 120. Furthermore, the intermediate image Imd can also be referred to as the third image. Here, the position where the intermediate image Imd (the third image) is formed can be conjugate to the position where the first image Im1 is formed. That is, the intermediate image Imd can be an image conjugate to the first image Im1. Furthermore, the first image Im1 formed by a beam of light split by the light splitting member 120 can also be referred to as an intermediate image. Furthermore, since at least a portion of the light LT passing through the imaging optical system 110 is split by the light splitting member 120, and the split light forms the first image Im1 and the intermediate image Imd, it can also be said that the light splitting member 120 contributes to imaging. Therefore, the light splitting member 120 can be part of the imaging optical system 110.

[0056] like Figure 2 and Figure 3 As shown, the re-imaging optical system 130 magnifies and re-images a portion of the intermediate image Imd formed by the light LT passing through the imaging optical system 110, thereby forming a second image Im2. In other words, the second image Im2 is formed by another beam of light split by the light splitting member 120 via the re-imaging optical system 130. As described above, the position where the intermediate image Imd (the third image) is formed can be conjugate with the position where the first image Im1 is formed. Thus, the re-imaging optical system 130 magnifies and re-images a portion of the intermediate image Imd, thereby achieving the same effect as magnifying and re-images a portion of the first image Im1 (the portion at the same relative position as the intermediate image Imd). In other words, the re-imaging optical system 130 magnifies and re-images a portion of the intermediate image Imd corresponding to a portion of the first image Im1.

[0057] Furthermore, the re-imaging optical system 130 can also be telecentric on the intermediate image Imd (light splitting member 120) side. This suppresses the divergence of the principal ray near the intermediate image Imd, thereby reducing the aperture of the re-imaging optical system 130. Therefore, the imaging system 100 can be miniaturized, reducing manufacturing costs. Furthermore, deviations in the incident angle of light incident on the light splitting member 120 (light splitting surface) can be suppressed. However, it is not limited to the imaging optical system 110 and the re-imaging optical system 130 both being telecentric on the intermediate image Imd (light splitting member 120) side. For example, only one of the imaging optical system 110 and the re-imaging optical system 130 may be telecentric on the intermediate image Imd (light splitting member 120) side.

[0058] The re-imaging optical system 130 includes a first optical system 131 and a second optical system 132. Light transmitted through the light splitting member 120 is incident on the first optical system 131. In other words, another beam of light split by the light splitting member 120 is incident on the first optical system 131. Light from the first optical system 131 is incident on the second optical system 132. Figure 2 In the diagram, a first optical system 131 and a second optical system 132 are schematically represented by a single lens. Each of the first optical system 131 and the second optical system 132 may be composed of a single lens or multiple lenses. In addition to one or more lenses, each of the first optical system 131 and the second optical system 132 may also include existing optical components other than lenses.

[0059] The optical axis (second optical axis AX2) of the second optical system 132 is orthogonal to the optical axis (first optical axis AX1) of the first optical system 131. Alternatively, the optical axis of the second optical system 132 may intersect the optical axis of the first optical system 131 at an angle other than 90 degrees. For example, the optical axis of the second optical system 132 may intersect the optical axis of the first optical system 131 within an angle range of 90 degrees ± 5 degrees. At or near the intersection of the optical axes of the first optical system 131 and the second optical system 132, the reflecting surface 162 of the first reflecting member 161 in the reflecting member rotating device 160 is disposed. Furthermore, the intersection of the optical axes of the first optical system 131 and the second optical system 132 may also be referred to as the point where the optical axes of the first optical system 131 and the second optical system 132 intersect. Furthermore, the so-called "near the intersection" position can be, for example, a position where the reflective surface 162 of the first reflective member 161 can reflect light from the first optical system 131 toward the second optical system 132, and which is different from the intersection position. The second optical system 132 uses the light from the first optical system 131 reflected by the reflective surface 162 of the first reflective member 161 to form a second image Im2. Alternatively, it can be said that the second image Im2 is formed by the first optical system 131 and the second optical system 132. It can also be said that the re-imaging optical system 130 is a structure that includes the first reflective member 161 in addition to the first optical system 131 and the second optical system 132.

[0060] Furthermore, the second optical system 132 can also be configured to be replaceable. As a replaceable second optical system 132, for example, at least one of a fixed-magnification optical system and a variable-magnification optical system (zoom lens) can be used. In other words, as the second optical system 132, a fixed-magnification optical system or a variable-magnification optical system (zoom lens) can be used. Moreover, as a replaceable second optical system 132, multiple optical systems with different magnifications can also be used. Alternatively, the second optical system 132 may not be replaceable. Even in this case, the second optical system 132 can be a fixed-magnification optical system or a variable-magnification optical system. Furthermore, since the second optical system 132 is a fixed-magnification optical system or a variable-magnification optical system, it can also be said that the re-imaging optical system 130 is a fixed-magnification optical system or a variable-magnification optical system. Additionally, the re-imaging optical system 130 can be configured to re-image a portion of the intermediate image Imd as a second image Im2 by reducing its size, or it can be configured to re-image a portion of the intermediate image Imd as a second image Im2 of the same size.

[0061] like Figure 2 and Figure 3 As shown, the first imaging device 140 includes a first imaging element 141 for capturing a first image Im1. The first imaging element 141 can be a CMOS (Complementary Metal Oxide Semiconductor) image sensor. Alternatively, the first imaging element 141 can also be an infrared image sensor capable of capturing infrared images. In this case, the first imaging element 141 can be a near-infrared image sensor capable of capturing near-infrared images, or a mid-infrared image sensor capable of capturing mid-infrared images. Furthermore, the first imaging element 141 can also be an event-based vision sensor. For example, when the light reflected by the light splitting member 120 is visible light, a CMOS image sensor or an event-based vision sensor can be used as the first imaging element 141. When the light reflected by the light splitting member 120 contains light in the near-infrared wavelength range, a near-infrared image sensor can be used as the first imaging element 141. When the light reflected by the light splitting member 120 contains light in the mid-infrared wavelength range, a mid-infrared image sensor can be used as the first imaging element 141.

[0062] The first imaging element 141 is disposed at or near the image plane position where the first image Im1 is formed. Furthermore, "near the image plane position where the first image Im1 is formed" can mean, for example, a position where the first imaging element 141 can capture the first image Im1 and identify the object TG in the image of the first image Im1 that differs from the image plane position. Figure 4As shown, in the imaging area 142 of the first imaging element 141, a plurality of first pixels 143 are arranged in a two-dimensional configuration. The first pixels 143 are arranged at a predetermined spacing in the X and Y directions. For example, more than 1000 first pixels 143 can be arranged in both the X and Y directions. Each first pixel 143 of the first imaging element 141 performs photoelectric conversion on the light reflected by the light splitting member 120 and incident on the first imaging element 141. The first imaging device 140 outputs image information of the first image Im1 generated based on the photoelectric conversion in each first pixel 143 of the first imaging element 141 to the control device 180.

[0063] like Figure 2 and Figure 3 As shown, the second imaging device 150 includes a second imaging element 151 for capturing a second image Im2. A CMOS image sensor can be used as the second imaging element 151. Alternatively, the second imaging element 151 can also be an infrared image sensor. In this case, the second imaging element 151 can be a near-infrared image sensor or a mid-infrared image sensor. Furthermore, the second imaging element 151 can also be an event-based vision sensor. For example, if the light transmitted through the light splitting member 120 is visible light, a CMOS image sensor or an event-based vision sensor can be used as the second imaging element 151. If the light transmitted through the light splitting member 120 contains light in the near-infrared wavelength range, a near-infrared image sensor can be used as the second imaging element 151. Moreover, for example, if the light reflected by the light splitting member 120 is visible light, and the light transmitted through the light splitting member 120 is also visible light, an event-based vision sensor can be used as the first imaging element 141, and a CMOS image sensor can be used as the second imaging element 151. When the light reflected by the light splitter 120 is visible light, and the light transmitted through the light splitter 120 includes light in the near-infrared wavelength range, an event-based visual sensor can be used as the first imaging element 141, and a near-infrared image sensor can be used as the second imaging element 151. For example, when both the light reflected by the light splitter 120 and the light transmitted through the light splitter 120 include light in the near-infrared wavelength range, the first imaging element 141 and the second imaging element 151 can be near-infrared image sensors. For example, when both the light reflected by the light splitter 120 and the light transmitted through the light splitter 120 include light in the mid-infrared wavelength range, the first imaging element 141 and the second imaging element 151 can be mid-infrared image sensors.

[0064] The second imaging element 151 is disposed at or near the image plane position where the second image Im2 is formed. Furthermore, "near the image plane position where the second image Im2 is formed" means, for example, the vicinity of the area where the second imaging element 151 can capture the second image Im2 so that the object TG can be identified in the image of the second image Im2. Figure 5 As shown, in the imaging area 152 of the second imaging element 151, a plurality of second pixels 153 are arranged in a two-dimensional configuration. The second pixels 153 are arranged at a predetermined spacing in the X and Y directions. For example, more than 1000 second pixels 153 can be arranged in the X and Y directions respectively. Each second pixel 153 of the second imaging element 151 performs photoelectric conversion on the light incident on the second imaging element 151 after passing through the light splitting member 120, the re-imaging optical system 130, and the light transmitted through the light splitting member 120. The second imaging device 150 outputs image information of the second image Im2 generated based on the photoelectric conversion of each second pixel 153 of the second imaging element 151 to the control device 180. For example, the spacing of the second pixels 153 of the second imaging element 151 can be smaller than the spacing of the first pixels 143 of the first imaging element 141. In other words, the number of second pixels 153 per unit area in the effective area (the area where the second pixels 153 are arranged) of the second imaging element 151 can be greater than the number of first pixels 143 per unit area in the effective area (the area where the first pixels 143 are arranged) of the first imaging element 141 (higher density). That is, the second imaging element 151 can be an imaging element with a higher resolution than the first imaging element 141. Furthermore, the spacing between the second pixels 153 of the second imaging element 151 can be less than or equal to the spacing between the first pixels 143 of the first imaging element 141; for example, the area of ​​the effective area of ​​the second imaging element 151 can be smaller than the area of ​​the effective area of ​​the first imaging element 141. Alternatively, the area of ​​the effective area of ​​the second imaging element 151 can be equal to or greater than the area of ​​the effective area of ​​the first imaging element 141.

[0065] Furthermore, a non-shooting area 154 may be formed in the central portion of the second imaging element 151 (the portion intersecting the optical axis (second optical axis AX2) of the second optical system 132). The shooting area 152 of the second imaging element 151 may also be provided around the non-shooting area 154. In this case, the non-shooting area 154 of the second imaging element 151 is small enough relative to the shooting area 152 to be small enough that the image missing portion caused by the non-shooting area 154 is not obvious when visually confirming the image of the second image Im2. The shooting area 152 can also be described as the effective area where the second pixel 153 is disposed. In addition, the non-shooting area 154 can also be described as the area where the second pixel 153 is not disposed. On the second imaging element 151, a through hole 155 is formed along the direction intersecting the non-shooting area 154 of the second imaging element 151. In this case, the non-shooting area 154 is a through area formed by the through hole 155. In addition, the non-shooting area 154 can also be described as the cross-sectional area of ​​the through hole 155. The through-hole 155 allows the measuring light LM emitted from the distance measuring device 170 to pass through. Furthermore, the cross-sectional shape of the through-hole 155 is not limited to a rectangle; it can also be a circle or other shapes. Additionally, as long as the measuring light LM emitted from the distance measuring device 170 can pass through, the opening size (cross-sectional area) of the through-hole 155 can be any size, but to ensure a larger area of ​​the imaging area 152 (effective area), the opening size is preferably smaller. Furthermore, in Figure 5 In this configuration, the opening size of the through-hole 155 is larger than the size (surface area) of the second pixel 153, but it can also be smaller than the size of the second pixel 153. Furthermore, the position of the through-hole 155 is not limited to the central portion of the second imaging element 151, but can also be formed at a position off-center from the second imaging element 151. Additionally, the through-hole 155 may not be formed in the non-imaging area 154. For example, at least a portion of the distance measuring device 170 (e.g., the measuring light illumination device 171) may be disposed in the non-imaging area 154. In this case, at least a portion of the distance measuring device 170 may be disposed in the non-imaging area 154 of the second imaging element 151, or it may be disposed at a position away from the second imaging element 151 (e.g., between the second imaging element 151 and the re-imaging optical system 130).

[0066] like Figure 2 and Figure 3As shown, the reflective member rotation device 160 includes a first reflective member 161 having a reflective surface 162 and a driving device 163. As described above, the reflective surface 162 of the first reflective member 161 can be disposed at or near the intersection of the optical axis (first optical axis AX1) of the first optical system 131 and the optical axis (second optical axis AX2) of the second optical system 132. Alternatively, the reflective surface 162 of the first reflective member 161 can also be disposed at the pupil position or pupil conjugate position of the optical system composed of the imaging optical system 110 and the first optical system 131. This reduces the area of ​​the reflective surface 162 of the first reflective member 161. The reflective surface 162 of the first reflective member 161 causes at least a portion of the light transmitted through the light splitting member 120 and passing through the first optical system 131 to be reflected toward the second optical system 132. In other words, the reflective surface 162 of the first reflective member 161 causes at least a portion of another beam of light split by the light splitting member 120 to be reflected toward the second imaging device 150. Furthermore, the reflective surface 162 of the first reflective member 161 causes the measuring light LM from the distance measuring device 170, which passes through the second optical system 132, to be reflected toward the first optical system 131. Therefore, it can also be said that the reflective surface 162 of the first reflective member 161 is arranged in the optical path of the measuring light LM from the distance measuring device 170.

[0067] The drive unit 163 is, for example, a voice coil motor (VCM). The drive unit 163 can rotate the first reflecting member 161 about two rotation axes, one extending in the X direction and the other in the Y direction. That is, the reflecting member rotation device 160 can also be a biaxial universal mirror capable of rotating the first reflecting member 161 about the two rotation axes. Thus, the drive unit 163 can rotate the first reflecting member 161 to the following two angles: the angle at which light traveling along the optical axis (first optical axis AX1) of the first optical system 131 is reflected towards the optical axis (second optical axis AX2) of the second optical system 132 (refer to...). Figure 2 ), and the angle at which light traveling off the optical axis of the first optical system 131 is reflected toward the optical axis of the second optical system 132 (e.g., referring to Figure 3 In other words, the drive device 163 can rotate the first reflecting member 161 to the following two angles: the angle at which the measuring light LM reflected by the reflecting surface 162 of the first reflecting member 161 travels along the optical axis of the first optical system 131 (refer to...). Figure 2 ), and the angle by which the measuring light LM reflected by the reflecting surface 162 of the first reflecting member 161 deviates from the optical axis of the first optical system 131 (e.g., referring to Figure 3However, the reflector rotation device 160 is not limited to a two-axis gimbal, but can be other existing devices. For example, the reflector rotation device 160 can also have two first reflectors 161. In this case, the drive device 163 can rotate the two first reflectors 161 about different rotation axes. For example, the drive device 163 can rotate the two first reflectors 161 about mutually orthogonal rotation axes. That is, the reflector rotation device 160 can also be a two-axis galvanometer. In addition, the reflector rotation device 160 is not limited to a device that rotates the first reflector 161 about two rotation axes, but can also be a device that rotates the first reflector 161 about a single rotation axis (e.g., a single-axis gimbal or a single-axis galvanometer) or a device that rotates about three rotation axes (e.g., a three-axis gimbal or a three-axis galvanometer).

[0068] like Figure 2 and Figure 3 As shown, the distance measuring device 170 is positioned on the opposite side of the optical component in the second optical system 132 (re-imaging optical system 130) closest to the second imaging element 151, across the second imaging element 151. The distance measuring device 170 projects onto the target object TG (referencing the image) via the non-imaging area 154 (through-hole 155) of the second imaging element 151, the second optical system 132, the first reflective component 161, the first optical system 131, the light splitting component 120, and the imaging optical system 110. Figure 1 The distance measurement device 170 illuminates the object TG with a measuring light LM and generates distance information for the object TG. This distance information can be related to the distance from the front lens 111 of the imaging optical system 110 (e.g., the lens closest to the object in the front lens 111) to the object TG, or it can be related to the distance from the frame 101 to the object TG. Alternatively, the distance information can also be related to the distance from the distance measurement device 170 to the object TG. In this case, the distance information can be related to the distance from the measuring light illumination device 171 (described later) of the distance measurement device 170 to the object TG, or it can be related to the distance from the light receiving device 172 (described later) of the distance measurement device 170 to the object TG. Furthermore, the distance information can also be related to the distance from the electromagnetic wave illumination device 20 to the object TG. In this case, the distance information can be related to the distance from the electromagnetic wave illumination device 20 to the object TG. In addition, the distance information of the object TG can also be information related to the distance from a specific reference point to the object TG.

[0069] Figure 6 The image shows an example of a portion of the distance measuring device 170. Figure 6The distance measuring device 170 shown includes a measuring light illumination device 171 (partial), a light receiving device 172 (partial), and a lens 173. For example... Figure 6 As shown, the distance measuring device 170 can adopt the following structure: the emitting part of the measuring light Lm of the measuring light illumination device 171 and the receiving part of the light receiving device 171 that receives the light from the object TG illuminated by the measured light LM can transmit and receive light via the lens 173. The lens 173 of the distance measuring device 170 is provided in the aforementioned emitting part (incident part) of the distance measuring device 170. The lens 173 (i.e., the aforementioned emitting part and incident part) of the distance measuring device 170 and the through hole 155 of the second imaging element 151 (see reference) Figure 5 The distance measuring device 170 is configured relatively. However, the distance measuring device 170 is not limited to... Figure 6 The structure shown is as follows. For example, in the distance measuring device 170, at least a portion of the measuring light irradiation device 171 and at least a portion of the light receiving device 171 can be common. In this case, in the distance measuring device 170, the emitting part of the measuring light LM of the measuring light irradiation device 171 and the receiving part of the light receiving device 171 that receives the light from the object TG irradiated by the measured light LM can be common. For example, the distance measuring device 170 can also be structured in which the measuring light irradiation device 171 and the light receiving device 171 are separately arranged. In addition, the distance measuring device 170 may not have a lens 173.

[0070] The measuring light illumination device 171 emits a measuring light LM via a lens 173. The measuring light illumination device 171 can be constructed using a laser diode (LD). In other words, the measuring light LM emitted from the measuring light illumination device 171 can also be a laser. Furthermore, the laser diode (LD) can be a pulsed laser diode or a CW laser diode. The measuring light LM emitted from the measuring light illumination device 171 of the distance measuring device 170 enters the imaging optical system 110 from the non-imaging area 154 (through hole 155) of the second imaging element 151, via the re-imaging optical system 130, the first reflective member 161, and the light splitting member 120.

[0071] The light receiving device 172 receives light from the object TG illuminated by the measured light LM via the lens 173 of the distance measuring device 170. The light receiving device 172 can be constructed using an avalanche photodiode (APD) or a CMOS sensor. Based on the light receiving device 172's reception of the light emitted by the object TG illuminated by the measured light LM from the measurement light illuminating device 171, the distance measuring device 170 generates distance information for the object TG. The distance measuring device 170 outputs the generated distance information for the object TG to the control device 180. However, the distance measuring device 170 is not limited to generating distance information. For example, the control device 180 can acquire the light receiving device 172's reception of the light from the object TG illuminated by the measured light LM and generate distance information based on that reception result.

[0072] The light receiving result of the light receiving device 172 can be the time from when the measurement light LM emitted from the measurement light illumination device 171 is reflected (normal reflection and / or diffuse reflection) by the object TG to when it is received by the light receiving device 172 (time of flight of the measurement light LM). In this case, the distance measuring device 170 measures the time from when the measurement light LM emitted from the measurement light illumination device 171 is reflected by the object TG to when it is received by the light receiving device 172, and generates distance information of the object TG. As such a TOF (Time-of-Flight) distance measuring device 170, a TOF-type LiDAR (Light Detection and Ranging) can be used, for example. In addition, the distance measuring device 170 is not limited to LiDAR, and can also be other TOF-type measuring devices. For example, the distance measuring device 170 can be a TOF-type reflective laser sensor. In addition, the light receiving result of the light receiving device 172 can be the intensity of the light from the object TG illuminated by the measured light LM, received by the light receiving device 172.

[0073] Furthermore, the distance measuring device 170 can be a millimeter-wave sensor (or millimeter-wave radar) that illuminates the object TG with millimeter waves as a measuring light LM. Alternatively, the distance measuring device 170 can be a measuring instrument utilizing an optical comb. Additionally, the distance measuring device 170 can have a structure in which a measuring light illuminating device 171 and a light receiving device 172 are separately disposed within the housing member 10, generating distance information of the object using the triangulation principle. In this case, the measuring light illuminating device 171 can illuminate the object TG with structured light (patterned light), such as a striped pattern with a regular striped intensity distribution or a random dot pattern with a random (irregular) dotted intensity distribution, as a measuring light LM. Furthermore, the distance measuring device 470 may not include the measuring light illuminating device 170. For example, a stereo camera can be used as a distance measuring device that does not include the measuring light illuminating device 170. In this case, distance information of the object TG can be generated based on the parallax of the stereo image acquired by the stereo camera. Furthermore, the output device 187 can output one image information from the stereo image acquired by the stereo camera.

[0074] Alternatively, the light receiving result of the light receiving device 172 can also be the frequency of the light reflected by the object TG from the measurement light LM emitted from the measurement light illumination device 171 while changing its frequency, and then received by the light receiving device 172. In this case, the distance measuring device 170 measures the frequency of the light received by the light receiving device 172 and generates distance information of the object TG. As such an FMCW (Frequency Modulated Continuous Wave) distance measuring device 170, an FMCW-type LiDAR can be used, for example. However, the distance measuring device 170 is not limited to an FMCW-type LiDAR; it can also be other FMCW-type measuring devices. For example, the distance measuring device 170 can be an FMCW-type interferometer.

[0075] In addition, Figure 6 In the distance measuring device 170 shown, the measuring light irradiation device 171 emits a measuring light LM that is converged by the lens 173, but it is not limited to this. The measuring light irradiation device 171 can emit divergent measuring light or parallel measuring light (parallel light). Furthermore, the lens 173 of the distance measuring device 170 (i.e., the aforementioned emitting and incident portions) and the through-hole 155 of the second imaging element 151 (see reference...) Figure 5 The distance measuring device 170 may be configured in a relatively specific manner, but is not limited to this. For example, the lens 173 (emitting part and incident part) of the distance measuring device 170 may be inserted into the through hole 155 of the imaging element 151 and configured in the non-imaging area 154.

[0076] The control device 180 is configured, for example, using a PC (personal computer). The control device 180 operates based on a program stored in the storage unit 181. Figure 2 and Figure 3 As shown, the control device 180 includes a storage unit 181, an image information acquisition unit 182, an image analysis unit 183, a device control unit 184, a distance information acquisition unit 185, an image correction unit 186, and an output device 187. The storage unit 181 stores the aforementioned programs, etc. Alternatively, the control device 180 may not include a storage unit 181. Furthermore, the control device 180 may also be housed within a housing 101. In this case, the housing member 10 may be omitted. Additionally, at least some functional modules of the control device 180 may also be housed within the housing 101.

[0077] The image information acquisition unit 182 acquires the image information of the first image Im1 output from the first imaging device 140. Hereinafter, the image of the first image Im1 captured by the first imaging device 140 (i.e., the image represented by the image information of the first image Im1) is sometimes referred to as the first image PC1 (see reference 140). Figure 10 The first image PC1 can also be referred to as a wide-angle image. The image information of the first image Im1 acquired by the image information acquisition unit 182 can also be stored in the storage unit 181. In addition, the image information acquisition unit 182 acquires the image information of the second image Im2 output from the second imaging device 150. Hereinafter, the image of the second image Im2 captured by the second imaging device 150 (i.e., the image represented by the image information of the second image Im2) is sometimes referred to as the second image PC2 (see reference 181). Figure 11 The second image PC2 can also be referred to as a magnified image. The image information of the second image Im2 acquired by the image information acquisition unit 182 can also be stored in the storage unit 181. Alternatively, the control device 180 may not include the image information acquisition unit 182. In this case, for example, the first imaging device 140 can output the generated image information of the first image Im1 to the image analysis unit 183. For example, the second imaging device 150 can output the image information of the second image Im2 to the image correction unit 186. Furthermore, as described later, when the control device 180 does not include the image correction unit 186, the second imaging device 150 can output the image information of the second image Im2 to the illumination control device 25 of the electromagnetic wave illumination device 20. In this case, the second imaging device 150 can also output the image information of the second image Im2 to the illumination control device 25 of the electromagnetic wave illumination device 20 via the output device 187.

[0078] The image analysis unit 183 generates orientation information of the object TG based on the image information of the first image Im1, which includes an image of the object TG, acquired by the image information acquisition unit 182. The orientation information of the object TG can be information related to the orientation of the object TG relative to the front lens 111 of the imaging optical system 110 (e.g., the lens closest to the object in the front lens 111), or information related to the orientation of the object TG relative to the imaging system 100 (frame 101). Furthermore, the orientation information of the object TG can also be information related to the orientation of the object TG relative to the electromagnetic wave irradiation device 20. In this case, the orientation information of the object TG can be information related to the orientation of the object TG relative to the emitting device 21 of the electromagnetic wave irradiation device 20. Additionally, the orientation information of the object TG can also be information related to the orientation of the object TG relative to a specific reference point. Furthermore, orientation information can also be referred to as azimuth information. Figure 8 An example of the first image Im1 is shown in the figure. Figure 10 An example of an image (first image PC1) of the first image Im1 captured by the first imaging device 140 is shown. The image analysis unit 183 identifies the object TG from the first image PC1, which contains one or more objects, by analyzing the image information of the first image Im1.

[0079] For example, the image analysis unit 183 can identify the object TG from a first image PC1 containing one or more objects through template matching processing. In this case, the object TG can be identified from the first image PC1 through matching processing of the template image of the first image PC1 with the object TG. The image analysis unit 183 is not limited to template matching processing, and can also identify the object TG from the first image PC1 through other processing. For example, the image analysis unit 183 can identify the object TG from the first image PC1 containing one or more objects by performing object detection processing based on machine learning (e.g., deep learning). In this case, the object TG can be identified from the first image PC1 using a learning model learned through machine learning (e.g., deep learning). For example, the image analysis unit 183 can identify the object TG from the first image PC1 containing one or more objects by performing segmentation processing. In addition, the identified object TG is not limited to one, and the image analysis unit 183 can also identify multiple object TGs from the first image PC1 containing multiple objects. Furthermore, the image analysis unit 183 can also identify the object TG from the second image Im2 (second image PC2) containing one or more objects. In this case, the image analysis unit 183 can identify the object TG from the second image Im2 (second image PC2) through the aforementioned processing (e.g., template matching processing). The image analysis unit 183 can also use both the first image PC1 and the second image PC2 to identify the object TG.

[0080] When the image analysis unit 183 identifies the object TG, it generates position information of at least a portion of the object TG in the first image PC1. For example, the position information of at least a portion of the object TG in the first image PC1 may be information related to the coordinate position of the center of the rectangular region containing the object TG in the first image PC1. In the image coordinate system, the position information of at least a portion of the object TG in the first image PC1 may also be information related to the coordinate position of a reference point of the object TG in the first image PC1. Here, the reference point may be a feature point of the object TG. The reference point may also be called a feature point. In addition, the coordinate position may be a two-dimensional coordinate in the image coordinate system or a two-dimensional coordinate in another coordinate system. In addition, the coordinate position may also be a pixel number (pixel address). The position information of at least a portion of the object TG generated by the image analysis unit 183 may be stored in the storage unit 181. If the position of the object TG in the first image PC1 is known, the orientation of the object TG can be determined based on the relationship between its position on the image plane of the first image Im1 and the viewing angle of the imaging optical system 110. In other words, the orientation of the object TG can be uniquely determined from the position of the object TG in the first image PC1 (e.g., the orientation of the object TG relative to the imaging system 100). The image analysis unit 183 can generate orientation information of the object TG based on the position information of at least a portion of the object TG in the generated first image PC1 using a orientation information generation data table. The orientation information generation data table can be a data table representing the relationship between the position information of at least a portion of the object TG in the first image PC1 and the orientation information of the object TG. The orientation information of the object TG generated by the image analysis unit 183 can be stored in the storage unit 181. Alternatively, the image analysis unit 183 can also generate the position information of the object TG in the first image PC1 based on the image information of the first image Im1, which contains a portion of the object TG, acquired by the image information acquisition unit 182, and generate the orientation information of the object TG based on the generated position information of the object TG in the first image PC1.

[0081] The device control unit 184 controls the drive device 163 of the reflector rotation device 160. For example... Figure 9As shown, the device control unit 184 can control the drive device 163 of the reflective member rotation device 160 based on the orientation information of the object TG generated by the image analysis unit 183, so that the area in which the intermediate image Imd (the third image) is formed, i.e., the image forming area of ​​the intermediate image Imd, contains at least a portion of the image of the object TG, and is re-imaged as the second image Im2. Specifically, the device control unit 184 can use the device control data table stored in the storage unit 181 to control the rotation angle of the first reflective member 161 rotated by the drive device 163 based on the orientation information of the object TG, so that the image of the object TG contained in the intermediate image Imd (the first image Im1) is located at the center of the imaging area 152 (the effective area in which the second pixel 153 is formed) of the second imaging element 151 and is re-imaged. In other words, the device control unit 184 can control the rotation angle of the first reflective member 161 based on the orientation information of the object TG, so as to set a region (the region re-imaged by the re-imaging optical system 130) in the image forming region of the intermediate image Imd that contains at least a portion of the image of the object TG, so that a magnified image containing at least a portion of the image of the object TG is re-imaged in the imaging region 152. Alternatively, it can be said that the device control unit 184 controls the rotation angle of the first reflective member 161 based on the orientation information of the object TG, so as to select a region containing at least a portion of the image of the object TG from the image forming region of the intermediate image Imd, so that a magnified image containing at least a portion of the image of the object TG is re-imaged in the imaging region 152. The device control data table can be a data table showing the relationship between the orientation information of the object TG and the rotation angle of the first reflective member 161. Furthermore, in this configuration, the re-imaging optical system 130 can be configured such that, regardless of the rotation angle of the first reflective member 162 (in other words, regardless of which part of the image-forming region of the magnified intermediate image Imd), the second image Im2, re-imaged via the first reflective member 162, is located at the center of the imaging region 152 of the second imaging element 151. Thus, regardless of the location of the image of at least a portion of the object TG within the first image Im1 (i.e., when the first imaging device 140 captures at least a portion of the object TG within the viewing angle of the imaging optical system 130), the second imaging device 150 can capture the second image Im2, which is magnified and re-imaged by the re-imaging optical system 130 and contains at least a portion of the object TG, and generate image information of the second image Im2.

[0082] Furthermore, the device control unit 184, using a device control data table stored in the storage unit 181, controls the rotation angle of the first reflective member 161, which is rotated by the driven device 163, based on the position information of the object TG in the first image PC1. This ensures that at least a portion of the image of the object TG contained in the intermediate image Imd (the first image Im1) is located at the center of the imaging area 152 of the second imaging element 151 and is re-imaged. In other words, the device control unit 184 can control the rotation angle of the first reflective member 161 based on the position information of the object TG in the first image PC1 to set the area in the image forming area of ​​the intermediate image Imd that contains at least a portion of the image of the object TG (i.e., the area re-imaged by the re-imaging optical system 130), so that a magnified image containing at least a portion of the image of the object TG is re-imaged in the imaging area 152. Alternatively, the device control unit 184 can control the rotation angle of the first reflective member 161 based on the position information of the object TG in the first image PC1, so as to select a region containing at least a portion of the image of the object TG from the image forming region of the intermediate image Imd, so that a magnified image containing at least a portion of the image of the object TG is re-imaged in the imaging region 152. The device control data table can also be a data table showing the relationship between the position information of the object TG in the first image PC1 and the rotation angle of the first reflective member 161. Alternatively, the positional relationship of the first imaging element 141, the second imaging element 151, and the first reflective member 161 can be calibrated in advance to create the device control data table. In addition, in this case, the re-imaging optical system 130 can also be configured to be independent of the rotation angle of the first reflective member 162, so that the second image Im2 re-imaged via the first reflective member 162 is located at the center of the imaging region 152 of the second imaging element 151. Therefore, regardless of where the image of at least a portion of the object TG is located in the first image Im1, the second imaging device 150 can capture the second image Im2, which contains at least a portion of the image of the object TG after being magnified and re-imaged by the re-imaging optical system 130, and generate image information of the second image Im2.

[0083] In addition, the device control unit 184 may also control the rotation angle of the first reflective member 161 based on the orientation information of the object TG and / or the position information of the object TG in the first image PC1, so that the image of the object TG contained in the intermediate image Imd (the first image Im1) is located outside the center of the imaging area 152 of the second imaging element 151 and is re-imaged.

[0084] In this case, the re-imaging optical system 130 can also be configured to be independent of the rotation angle of the first reflective member 162, such that the second image Im2 re-imaged via the first reflective member 162 is located outside the center of the imaging area 152 of the second imaging element 151. Alternatively, the device control unit 184 can control the rotation angle of the first reflective member 161 based on the orientation information of the object TG and / or the position information of the object TG in the first image PC1, such that at least a portion of the image of the object TG contained in the intermediate image Imd (the first image Im1) is located in the imaging area 152 of the second imaging element 151 and is re-imaged.

[0085] In this embodiment, based on image information of a first image Im1 containing an image of the object TG, the first reflective member 161 is rotated using the driving device 163 to determine a second image Im2 captured by the second imaging device 150, while simultaneously illuminating the object TG with measurement light LM from the distance measuring device 170. Alternatively, the first image Im1 may also contain an image of a portion of the object TG.

[0086] For example, based on image information of a first image Im1 containing an image of the object TG, the first reflective member 161 can be rotated using the driving device 163 to select a portion of the image forming region of the intermediate image Imd, which is to be re-imaged as the second image Im2, such that a portion of the first optical path of the light forming the image of the object TG contained in the second image Im2 overlaps with a portion of the second optical path of the measuring light LM from the distance measuring device 170. Then, with a portion of the image forming region selected from the image forming region such that a portion of the first optical path overlaps with a portion of the second optical path, the distance measuring device 170 can illuminate the measuring light LM. Alternatively, the first optical path can also be the first optical path of the light forming a portion of the image of the object TG contained in the second image Im2. Furthermore, the entire first optical path can overlap with a portion of the second optical path. Alternatively, a portion of the first optical path can overlap with the entire second optical path. Alternatively, the entire first optical path can overlap with the entire second optical path.

[0087] In other words, a second image Im2 can be formed based on the image information of a first image Im1 containing an image of the object TG captured by the first imaging device 140, such that a portion of the first optical path of the light forming the image of the object TG contained in the second image Im2 overlaps with a portion of the second optical path of the measuring light LM from the distance measuring device 170. Then, in the state where the second image Im2 is formed such that a portion of the first optical path overlaps with a portion of the second optical path, the distance measuring device 170 can illuminate the measuring light LM.

[0088] In other words, based on image information of the first image Im1 containing the image of the object TG, a portion of the image forming region of the intermediate image Imd can be selected from the image forming region such that a portion of the first optical path of the light forming the object TG contained in the second image Im2 overlaps with a portion of the second optical path of the measuring light LM from the distance measuring device 170. Then, with a portion of the image forming region selected from the intermediate image Imd such that a portion of the first optical path overlaps with a portion of the second optical path, the distance measuring device 170 can illuminate the measuring light LM.

[0089] Furthermore, based on the image information of the first image Im1 containing the image of the object TG, the first reflective member 161 can be rotated using the driving device 163, thereby selecting a portion of the image forming region from the image forming region as an intermediate image Imd to be re-imaged by the second image Im2, such that the image of the object TG contained in the second image Im2 overlaps with the non-image area 154 of the second imaging element 151. Then, in the state where a portion of the image forming region is selected from the image forming region such that the image of the object TG contained in the second image Im2 overlaps with the non-image area 154, the distance measuring device 170 can illuminate the measuring light LM. Additionally, the image contained in the second image Im2 can be an image of a portion of the object TG contained in the second image Im2. Furthermore, the image of at least a portion of the object TG contained in the second image Im2 can also overlap with a portion of the non-image area 154 of the second imaging element 151.

[0090] Specifically, as described above, the device control unit 184 can control the rotation angle of the first reflective member 161 based on image information of the first image Im1 containing the image of the object TG (specifically, the orientation information of the object TG generated from the image information, and / or the position information of the object TG in the first image PC1), so that at least a portion of the object TG contained in the intermediate image Imd (the first image Im1) is located at the center of the imaging area 152 of the second imaging element 151 and is re-imaged. Alternatively, in this case, the re-imaging optical system 130 can also be configured such that, regardless of the rotation angle of the first reflective member 162, the second image Im2 re-imaged via the first reflective member 162 is always located at the center of the imaging area 152 of the second imaging element 151. Here, with the through-hole 155 (non-image area 154) formed in the central portion of the second imaging element 151, and the image of at least a portion of the object TG contained in the intermediate image Imd located at the center of the image area 152 of the second imaging element 151 and being re-imaged, if the measuring light LM is emitted from the distance measuring device 170, the measuring light LM passing through the through-hole 155 is transmitted via the re-imaging optical system 130, the first reflecting member 161, the light splitting member 120, and the imaging optical system 110, and illuminates the direction where the object TG is located. At this time, it can be said that at least a portion of the first optical path overlaps with at least a portion of the second optical path. In other words, it can also be said that a portion of the image forming area of ​​the intermediate image Imd is selected such that the image of the object TG contained in the second image Im2 overlaps with the through-hole 155 (non-image area 154).

[0091] Therefore, regardless of the location of the image of at least a portion of the object TG within the first image Im1 (that is, when the first imaging device 140 captures at least a portion of the object TG within the viewing angle of the imaging optical system 110), the measuring light LM emitted from the measuring light illumination device 171 of the distance measuring device 170 will illuminate the object TG via the non-imaging area 154 (through-hole 155) of the second imaging element 151, the re-imaging optical system 130 (the second optical system 132 and the first optical system 131), the first reflective member 161, the light splitting member 120, and the imaging optical system 110. The light from the object TG illuminated by the measured light LM arrives at the light receiving device 172 of the distance measuring device 170 in the reverse order of the measuring light LM from the measuring light illumination device 171. Therefore, the distance measuring device 170 can generate distance information of the object TG based on the light receiving device 172's reception result of the light emitted by the object TG illuminated by the measuring light LM from the measuring light illumination device 171. Furthermore, by simply setting the rotation angle of the first reflecting member 161, while the distance measuring device 170 generates distance information of the object TG, the second imaging device 150 can generate image information of the second image Im2, which contains at least a portion of the image of the object TG, by capturing a second image Im2 that is magnified and re-imaged by the re-imaging optical system 130. Therefore, both the distance information of the object TG and the image information of the second image Im2 containing at least a portion of the image of the object TG can be generated in a short time.

[0092] The distance information acquisition unit 185 acquires the distance information of the object TG output from the distance measuring device 170. The distance information of the object TG acquired by the distance information acquisition unit 185 can be stored in the storage unit 181. Alternatively, the control device 180 may not include the distance information acquisition unit 185. In this case, for example, the distance measuring device 170 can output the generated distance information of the object TG to the irradiation control device 25 of the electromagnetic wave irradiation device 20. Furthermore, the distance measuring device 170 can also output the generated distance information of the object TG to the irradiation control device 25 via the output device 187.

[0093] The image correction unit 186 corrects the image information of the second image Im2 acquired by the image information acquisition unit 182. For example... Figure 9As shown, the orientation of the object TG in the second image Im2 rotates relative to the orientation of the object TG in the intermediate image Imd (or the first image Im1) according to the rotation angle of the first reflecting member 161. The image correction unit 186 can use the correction data table stored in the storage unit 181 to correct the image information of the second image Im2, so that the orientation of the object TG in the second image Im2 is the same as the orientation of the object TG in the intermediate image Imd (or the first image Im1). Figure 11 An example is shown in the image (second image PC2) of the second image Im2 after the orientation of the object TG has been corrected. Alternatively, the correction data table can be a data table showing the relationship between the rotational position of the first reflector 161 and the correction amount of the orientation of the object TG contained in the second image Im2. The image information of the second image Im2 corrected by the image acquisition unit 186 can be stored in the storage unit 181. Furthermore, when the change in the orientation of the object TG in the second image Im2 corresponding to the rotational angle of the first reflector 161 is so small as to not affect the purpose of irradiating the object TG from the electromagnetic wave irradiation device 20 and destroying, repelling, or removing the object TG, the image correction unit 186 may not need to correct the image information of the second image Im2, and the control device 180 may not include the image correction unit 186.

[0094] The output device 187 can output the orientation information of the object TG generated by the image analysis unit 183 and the distance information of the object TG acquired by the distance information acquisition unit 185 (i.e., the distance information of the object TG generated by the distance measurement device 170) to the illumination control device 25 of the electromagnetic wave illumination device 20. At this time, the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 based on the orientation information and distance information of the object TG output from the output device 187, so that electromagnetic waves LB are irradiated onto the object TG. For example, when the orientation information of the object TG is the orientation information of the object TG relative to the imaging system 100, the illumination control device 25 can correct it to the orientation information of the object TG relative to the electromagnetic wave illumination device 20 based on the positional relationship between the imaging system 100 and the electromagnetic wave illumination device 20. For example, when the distance information of the target object TG is the distance information from the imaging system 100 to the target object TG, the illumination control device 25, in addition to or replacing the correction of the direction information of the target object TG, can correct it to the distance information from the electromagnetic wave illumination device 20 to the target object TG based on the positional relationship between the imaging system 100 and the electromagnetic wave illumination device 20. Furthermore, the illumination control device 25 is not limited to correcting at least one of the direction information and the distance information of the target object TG. For example, the control device 180 can generate at least one of the direction information of the target object TG relative to the electromagnetic wave illumination device 20 and the distance information from the electromagnetic wave illumination device 20 to the target object TG based on the positional relationship between the imaging system 100 and the electromagnetic wave illumination device 20. Additionally, the positional relationship between the imaging system 100 and the electromagnetic wave illumination device 20 can be predetermined. The predetermined positional relationship between the imaging system 100 and the electromagnetic wave illumination device 20 can be stored in the storage unit 181.

[0095] For example, the irradiation control device 25 controls the support device 23 based on the direction information of the target object TG (the direction information of the target object TG relative to the electromagnetic wave irradiation device 20), so that electromagnetic waves LB are emitted from the transmitting device 21 in the direction of the target object TG (i.e., adjusting the emission direction of the electromagnetic waves LB from the transmitting device 21). For example, in addition to or instead of controlling the support device 23, the irradiation control device 25 can adjust the output of the electromagnetic waves LB emitted from the transmitting device 21 based on the distance information of the target object TG (the distance information from the electromagnetic wave irradiation device 20 to the target object TG). In this case, the irradiation control device 25 can increase the output of the electromagnetic waves LB emitted from the transmitting device 21 as the distance from the electromagnetic wave irradiation device 20 to the target object TG increases.

[0096] Furthermore, the illumination control device 25 can generate the position coordinate information of the object TG based on the orientation information and distance information of the object TG. Based on the generated position coordinate information of the object TG, the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 to illuminate the object TG with electromagnetic waves LB. Alternatively, the illumination control device 25 may not generate the position coordinate information of the object TG, but the control device 180 may generate the position coordinate information of the object TG and output it to the illumination control device 25. The position coordinate information of the object TG can be the three-dimensional coordinates of the object TG in a coordinate system based on the electromagnetic wave illumination device 20, or it can be the three-dimensional coordinates of the object TG in other coordinate systems (e.g., a global coordinate system, or a coordinate system based on the imaging system 100). In this case, the position coordinate information of the object TG can be generated by calibrating a pre-calculated transformation matrix between coordinate systems.

[0097] Furthermore, the output device 187 may not output at least one of the direction information and distance information of the target object TG to the illumination control device 25. In this case, the control device 180 may also not generate at least one of the direction information and distance information of the target object TG that will not be output to the illumination control device 25. For example, if the direction of the target object TG in the electromagnetic wave illumination device 20 (illumination control device 25) is known, the output device 187 may not output the direction information of the target object TG to the illumination control device 25. Additionally, if the distance to the target object TG in the electromagnetic wave illumination device 20 is known, the output device 187 may not output the distance information of the target object TG to the illumination control device 25. Furthermore, for example, if the direction information of the target object TG output by the output device 187 is related to the direction of the target object TG relative to the electromagnetic wave illumination device 20 (or, if the illumination control device 25 is capable of generating the direction information of the target object TG relative to the electromagnetic wave illumination device 20), the output device 187 may not output the distance information of the target object TG. At this time, the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 based on the direction information of the target object TG, so that electromagnetic waves LB are irradiated onto the target object TG. Thus, the output device 187 can output at least one of the direction information and the distance information of the target object TG to the electromagnetic wave illumination device 20. The electromagnetic wave illumination device 20 can control at least one of the transmitting device 21 and the supporting device 23 based on at least one of the direction information and the distance information of the target object TG output from the output device 187, so that electromagnetic waves LB are irradiated onto the target object TG. Alternatively, the control device 180 may not include the output device 187. In this case, for example, the image analysis unit 183 can output the generated direction information of the target object TG to the illumination control device 25. Furthermore, the distance information acquisition unit 185 can output the generated distance information of the target object TG to the illumination control device 25.

[0098] In addition, besides / alternatively outputting at least one of the direction information and distance information of the target object TG to the illumination control device 25, the output device 187 can also output the image information of the second image Im2 corrected by the image correction unit 186 to the illumination control device 25 of the electromagnetic wave illumination device 20. Furthermore, as described above, the control device 180 may not include the image correction unit 186. In this case, the output device 187 can output the image information of the second image Im2, which is a magnified image containing at least a portion of the target object TG generated by the second imaging device 150, to the illumination control device 25. In this case, for example, the illumination control device 25 can use the image information of the second image Im2, which is a magnified image containing at least a portion of the target object TG, to compare (determine) whether the target object TG is an object that should be destroyed, removed, or eliminated by irradiating electromagnetic waves LM (i.e., whether there is a case of misidentification as the target object TG). If the comparison (determination) result based on image information determines that the object TG is an object that should be destroyed, driven away or removed by irradiating electromagnetic waves LM (i.e., object TG), the irradiation control device 25 can control at least one of the transmitting device 21 and the supporting device 23 to irradiate electromagnetic waves LB onto the object TG based on at least one of the direction information of the object TG and the distance information of the object TG.

[0099] In this way, the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 based on at least one of the direction information and distance information of the object TG output from the output device 187 and the image information of the second image Im2, so that electromagnetic wave LB is irradiated onto the object TG.

[0100] Here, we describe the situation where the positional relationship between the object TG and the imaging system 100 changes. The image information of the first image Im1 captured by the first imaging device 140 before the positional relationship between the object TG and the imaging system 100 changes is referred to as first image information. Furthermore, the positional information of the object TG in the image (first image PC1) represented by the first image information generated by the image analysis unit 183 is referred to as first position information. The orientation information of the object TG generated by the image analysis unit 183 before the positional relationship between the object TG and the imaging system 100 changes is referred to as first orientation information. The distance information of the object TG generated by the distance measuring device 170 before the positional relationship between the object TG and the imaging system 100 changes is referred to as first distance information. The image information of the second image Im2, which contains at least a portion of the object TG, captured by the second imaging device 150 before the positional relationship between the object TG and the imaging system 100 changes, is referred to as third image information. When the positional relationship between the object TG and the imaging system 100 changes (e.g., when the position of the object TG changes relative to the imaging system 100), the position of at least a portion of the object TG contained in the first image Im1 changes, and the position of at least a portion of the object TG contained in the intermediate image Imd also changes. Furthermore, based on the change in the position of at least a portion of the object TG contained in the intermediate image Imd, the position of at least a portion of the object TG contained in the second image Im2 also changes. The first imaging device 140 captures the first image Im1 after the position of at least a portion of the object TG has changed, and outputs the second image information of the first image Im1 to the control device 180.

[0101] The image information acquisition unit 182 of the control device 180 acquires second image information of the first image Im1 output from the first imaging device 140. As described above, the image analysis unit 183 generates second position information of the object TG in the image (first image PC1) represented by the second image information based on the second image information of the first image Im1 acquired by the image information acquisition unit 182. As described above, the image analysis unit 183 generates second orientation information of the object TG in the image (first image PC1) represented by the generated second image information based on the second position information of the object TG.

[0102] As described above, the device control unit 184 can control the drive device 163 of the reflective member rotation device 160 based on at least one of the second direction information of the object object TG generated by the image analysis unit 183 and the second position information of the object object TG in the image (first image PC1) represented by the second image information, so that the region in the image forming area of ​​the intermediate image Imd that contains at least a portion of the object object TG after its position has changed is re-imaged as the second image Im2. Specifically, the device control unit 184 can control the rotation angle of the first reflective member 161 rotated by the drive device 163 based on at least one of the second direction information of the object object TG and the second position information of the object object TG in the image (first image PC1) represented by the second image information, so that the image of at least a portion of the object object TG after its position has changed, contained in the intermediate image Imd (first image Im1), is located at the center of the imaging area 152 of the second imaging element 151 and is magnified and re-imaged. Furthermore, as described above, when the image of at least a portion of the object TG whose position has changed, contained in the intermediate image Imd, is located at the center of the imaging area 152 of the second imaging element 151 and is re-imaged, if the measuring light LM is emitted from the distance measuring device 170 (measuring light illumination device 171), the measuring light LM passing through the through-hole 155 illuminates the direction where the object TG whose position has changed is located via the re-imaging optical system 130, the first reflecting member 161, the light splitting member 120, and the imaging optical system 110. The light from the object TG illuminated by the measured light LM reaches the light receiving device 172 in the reverse order of the measuring light LM from the measuring light illumination device 171. The distance measuring device 170 can generate second distance information of the object TG whose position has changed based on the light receiving result of the light receiving device 172. Thus, even if the positional relationship between the object TG and the imaging system 100 changes, the second imaging device 150 can still capture a second image Im2 containing an image of at least a portion of the object TG whose position has changed. The second imaging device 150 generates a fourth image information, a second image Im2, which includes a magnified image of at least a portion of the object TG after its position has changed. That is, even if the positional relationship between the object TG and the imaging system 100 changes, the second imaging device 150 can still follow the object TG after its positional relationship has changed and generate a fourth image information, a second image Im2, which includes a magnified image of at least a portion of the object TG after its position has changed.

[0103] Furthermore, even if the positional relationship between the object TG and the imaging system 100 changes, a second distance information for the object TG after the positional change can be generated. In other words, even if the positional relationship between the object TG and the imaging system 100 changes, the distance measuring device 170 can follow the object TG after the positional change and generate the second distance information for the object TG after the positional change. In this case, the device control unit 184 can control the rotation angle of the first reflecting member 161 according to the change in the positional relationship of the object TG, so that the measuring light LM from the distance measuring device 170 continuously or intermittently illuminates the object TG.

[0104] The output device 187 can output at least one of the second direction information of the object TG generated by the image analysis unit 183 and the second distance information of the object TG acquired by the distance information acquisition unit 185 (i.e., the second distance information of the object TG generated by the distance measurement device 170) to the illumination control device 25 of the electromagnetic wave illumination device 20. At this time, the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 to continuously or intermittently irradiate the object TG with electromagnetic waves LB, based on at least one of the direction change information generated based on the first direction information and the second direction information output from the output device 187 and the distance change information generated based on the first distance information and the second distance information output from the output device 187. Here, the direction change information may be information indicating the change of the direction of the object TG relative to the imaging system 100 or the electromagnetic wave illumination device 20 as the positional relationship between the object TG and the imaging system 100 changes. Furthermore, the distance change information can represent the change in distance from the imaging system 100 or the electromagnetic wave irradiation device 20 to the object TG as the positional relationship between the object TG and the imaging system 100 changes. Additionally, the control device 180 can generate direction change information based on the first direction information and the second direction information, or it can generate distance change information based on the first distance information and the second distance information. In this case, the control device 180 (output device 187) can output at least one of the generated direction change information and distance change information to the irradiation control device 25.

[0105] Furthermore, in addition to / alternatively outputting at least one of the second direction information of the object TG generated by the image analysis unit 183 and the second distance information of the object TG acquired by the distance information acquisition unit 185, the output device 187 may also output the fourth image information of the magnified image of the second image Im2, which includes at least a portion of the object TG, corrected by the image correction unit 186, to the illumination control device 25 of the electromagnetic wave illumination device 20. Alternatively, as described above, the control device 180 may not include the image correction unit 186. In this case, the output device 187 may output the fourth image information of the second image Im2 acquired by the image information acquisition unit 182 (i.e., the fourth image information of the second image Im2 captured by the second imaging device 150) to the illumination control device 25 of the electromagnetic wave illumination device 20. In this scenario, for example, the illumination control device 25 can use fourth image information—a second image Im2 containing at least a portion of the magnified image of the target object TG—to compare (determine) whether the target object TG is an object that should be destroyed, driven away, or removed by irradiating electromagnetic waves LM (i.e., whether there is a case of misidentification as the target object TG). If, based on the comparison (determination) result of the image information, it is determined that the target object TG is an object that should be destroyed, driven away, or removed by irradiating electromagnetic waves LM (i.e., the target object TG), the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 to irradiate electromagnetic waves LB onto the target object TG based on at least one of the direction information and distance information of the target object TG. Alternatively, the illumination control device 25 can also use image change information generated based on the third and fourth image information to compare (determine) whether the target object TG is an object that should be destroyed, driven away, or removed by irradiating electromagnetic waves LM.

[0106] For example, image change information can also be information representing the brightness change of the pixel on the image corresponding to the object TG. In this case, the illumination control device 25 can use the image change information to determine that the object TG compared with the third image information before the position change is less than a threshold, and thus determine that the object TG is the same object TG.

[0107] Therefore, even if the positional relationship between the object TG and the imaging system 100 changes, the electromagnetic wave irradiation device 20 can reliably irradiate the object TG with electromagnetic waves LB after the positional relationship changes.

[0108] Furthermore, the imaging system 100 (output device 187) controls the rotation angle of the first reflective member 161 at a predetermined rate based on the image information of the first image Im1, which contains at least a portion of the object TG generated by the first imaging device 140, so that the image of at least a portion of the object TG contained in the intermediate image Imd is located at the center position of the imaging area 152 of the second imaging element 151 and is magnified and re-imaged. As described above, at least one of the image information of the second image Im2, which contains at least a portion of the magnified image of the object TG generated by the second imaging device 150, the orientation information of the object TG generated by the image analysis unit 183, and the distance information of the object TG generated by the distance measuring device 170, can be sequentially output to the illumination control device 25 of the electromagnetic wave illumination device 20. In this case, the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 based on at least one of the image information of the second image Im2, the orientation information of the object TG, and the distance information of the object TG, which are sequentially output at a predetermined rate, so that electromagnetic waves LB are continuously or intermittently irradiated onto the object TG. Therefore, even if the positional relationship between the object TG and the shooting system 100 changes continuously or intermittently, the electromagnetic wave irradiation device 20 can reliably irradiate the object TG with electromagnetic waves LB. Furthermore, the prescribed rate can be at least one of the shooting rate (frame rate) of the first shooting device 140 and the shooting rate (frame rate) of the second shooting device 150. In this case, the shooting rate can be 500 fps or higher (e.g., 1000 fps) or less than 500 fps. Moreover, the prescribed rate can also be set based on at least one of the moving speed and moving frequency of the object TG. For example, the faster the moving speed of the object TG, the higher the prescribed rate can be set. In this case, at least one of the shooting rate of the first shooting device 140 and the shooting rate of the second shooting device 150 can be set to the prescribed rate.

[0109] Additionally, the output device 187 may also output at least one of the generated image information of the first image Im1, the image information of the second image Im1, the orientation information of the object TG, and the distance information of the object TG to a display device (not shown). The imaging system 100 may also include this display device. Based on the information output from the output device 187, the display device displays at least one of the following: the image represented by the image information of the first image Im1 (first image PC1), the image represented by the image information of the second image Im1 (second image PC2), the direction represented by the orientation information of the object TG (e.g., the direction of the object TG relative to at least one of the imaging system 100 and the electromagnetic wave irradiation device 20), and the distance represented by the distance information of the object TG (e.g., the distance of the object TG relative to at least one of the imaging system 100 and the electromagnetic wave irradiation device 20). The imaging system 100 may also include this display device. Furthermore, the image represented by the image information of the first image Im1 output to the display device (first image PC1) may also include an image of at least a portion of the object TG. In addition, the image represented by the image information of the second image Im2 output to the display device (the second image PC2) may also include an image of at least a portion of the object TG.

[0110] [Measurement Method]

[0111] Next, the measurement method using the imaging system 100 according to Embodiment 1 will be summarized. Figure 7 This is a flowchart illustrating the measurement method according to Embodiment 1. Furthermore, each process in the measurement method is executed based on a program stored in the storage unit 181. For example... Figure 7 As shown, firstly, the imaging system 100 acquires the orientation information of the object TG (step ST10). At this time, the first imaging device 140 captures a first image Im1 containing at least a portion of the object TG, and outputs the image information of the generated first image Im1 to the control device 180. The image information acquisition unit 182 acquires the image information of the first image Im1 output from the first imaging device 140.

[0112] As described above, the image analysis unit 183 generates position information of the object TG in the first image PC1 (the image represented by the image information of the first image Im1) based on the image information of the first image Im1, which is acquired by the image information acquisition unit 182 and contains at least a portion of the object TG. Based on the position information of the object TG in the generated first image PC1, the image analysis unit 183 generates orientation information of the object TG. Furthermore, at this time, the device control unit 184 controls the drive device 163 of the reflective member rotation device 160 based on the orientation information of the object TG generated by the image analysis unit 183, so that the region of the image forming area of ​​the intermediate image Imd containing at least a portion of the object TG is magnified and re-imaged by the re-imaging optical system 130 as the second image Im2. For example, the device control unit 184 controls the rotation angle of the first reflective member 161, which is rotated by the driven device 163, based on at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1, so that the image of at least a portion of the object TG contained in the intermediate image Imd (the first image Im1) is located at the center of the imaging area 152 of the second imaging element 151 and is re-imaged.

[0113] Next, the imaging system 100 acquires the distance information of the object TG (step ST20). At this time, with the rotation angle of the first reflective member 161 set such that at least a portion of the object TG contained in the intermediate image Imd is located at the center of the imaging area 152 of the second imaging element 151 and is re-imaged, the distance measuring device 170 illuminates the object TG with measuring light LM through the non-imaging area 154 (through hole 155) of the second imaging element 151, the second optical system 132, the first reflective member 161, the first optical system 131, the light splitting member 120, and the imaging optical system 110, and generates the distance information of the object TG. The distance measuring device 170 outputs the generated distance information of the object TG to the distance information acquisition unit 185. The distance information acquisition unit 185 acquires the distance information of the object TG output from the distance measuring device 170. Additionally, in step ST10, in order to control the rotation angle of the first reflecting member 161, step ST10 can also be performed between steps ST20 and ST40 (described later) without using the orientation information of the object TG. In this case, in step ST20, the first imaging device 140 captures a first image Im1 containing at least a portion of the object TG, and the image analysis unit 183 generates position information of the object TG in the first image PC1 based on the image information of the first image Im1 containing at least a portion of the object TG.

[0114] Next, the imaging system 100 acquires image information of the second image Im2 (step ST30). At this time, with the rotation angle of the first reflective member 161 set such that at least a portion of the object TG contained in the intermediate image Imd is located at the center of the imaging area 152 of the second imaging element 151 and is re-imaged, the second imaging device 150 captures the second image Im2 containing at least a portion of the object TG and outputs the generated image information of the second image Im2 to the image information acquisition unit 182. The image information acquisition unit 182 acquires the image information of the second image Im2 output from the second imaging device 150. In addition, the image correction unit 186 can use the correction data table stored in the storage unit 181 to correct the image information of the second image Im2 so that the orientation of the object TG in the image of the second image Im2 is the same as the orientation of the object TG in the intermediate image Imd (or the first image Im1). In addition, step ST30 can also be performed between steps ST10 and ST20.

[0115] Then, the imaging system 100 outputs information (step ST40). At this time, the output device 187 can output at least one of the direction information of the object TG generated by the image analysis unit 183 and the distance information of the object TG obtained by the distance information acquisition unit 185 (i.e., the distance information of the object TG generated by the distance measurement device 170) to the illumination control device 25 of the electromagnetic wave illumination device 20. At this time, the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 based on at least one of the direction information and distance information of the object TG output from the output device 187, so that electromagnetic waves LB are irradiated onto the object TG. In addition, the output device 187 can also output the image information of the second image Im2 after correction by the image correction unit 186 to the illumination control device 25. At this time, the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 based on at least one of the direction information of the object TG and the distance information of the object TG output from the output device 187 and the image information of the second image Im2, so that electromagnetic wave LB is irradiated onto the object TG.

[0116] Alternatively, steps ST10 to ST40 can be repeated. For example, when the positional relationship between the object TG and the imaging system 100 changes, by repeating steps ST10 to ST40, as described above, the distance measuring device 170 can continuously or intermittently illuminate the measuring light LM onto the object TG whose positional relationship has changed. Furthermore, by repeating steps ST10 to ST40, as described above, each time based on at least one of the direction information of the object TG, the distance information of the object TG, and the image information of the second image Im2 output in ST40, the illumination control device 25 emits electromagnetic wave LB onto the object TG, thereby allowing the electromagnetic wave illumination device 20 to continuously or intermittently illuminate the electromagnetic wave LB onto the object TG whose positional relationship has changed. Additionally, even if the positional relationship between the object TG and the imaging system 100 does not change, steps ST10 to ST40 can be repeated.

[0117] According to Embodiment 1, the imaging system 100 includes: a light splitting member 120 that splits at least a portion of the light LT passing through the imaging optical system 110; a first imaging device 140 that captures a first image Im1 formed by a beam of light split by the light splitting member 120; a second imaging device 150 that captures a second image Im2 formed by another beam of light split by the light splitting member 120; and a distance measuring device 170 that, based on image information of the first image Im1 captured by the first imaging device 140, which contains an image of at least a portion of the object, illuminates a measuring light LM onto the object via the imaging optical system 110 and generates distance information of the object. Thus, the measuring light LM can be accurately illuminated onto the object TG within the viewing angle of the imaging optical system 110, and the distance information of the object TG can be reliably generated. Furthermore, even when the positional relationship between the object TG and the imaging system 100 changes, the measuring light LM can be continuously or intermittently illuminated onto the object TG following the change in positional relationship. Therefore, even if the distance to the target object TG changes due to changes in positional relationships, the change in distance can be measured accurately and reliably. Thus, the electromagnetic wave irradiation device 20 can accurately and reliably irradiate electromagnetic waves LB onto the target object TG.

[0118] Furthermore, a reflective member rotation device 160 is included, comprising a first reflective member 161 having a reflective surface 162 that reflects another beam of light split by the light splitting member 120 toward the second imaging device 150; and a drive device 163 that rotates the first reflective member 161, the reflective surface 162 of the first reflective member 161 being arranged in the optical path of the measuring light LM from the distance measuring device 170. Thus, by rotating the first reflective member 161 via the drive device 163, the measuring light LM from the distance measuring device 170 can be directed toward the target object in a short time. Therefore, after identifying the target object TG through the image (image information of the first image Im1), the distance to the target object can be measured in a short time. Furthermore, by utilizing the distance information measured and generated by the distance measuring device 170 in a short time, electromagnetic waves LB can be immediately irradiated onto the appearing target object TG.

[0119] Furthermore, by rotating the first reflective member 161 via the drive device 163, the light from the object TG illuminated by the measured light LM can be directed toward the second imaging device 150 (the second imaging element 151) for a short period of time. Therefore, image information of a second image Im2 containing at least a portion of the image of the object TG can be generated in a short time. By utilizing the image information of the second image Im2, which contains at least a portion of the image of the object TG and is generated by the second imaging device 150 in a short time, electromagnetic waves LB can be immediately irradiated onto the appearing object TG.

[0120] Furthermore, the image analysis unit 183 can generate the orientation information of the target object TG in a short time. Therefore, by utilizing the orientation information of the target object TG generated by the image analysis unit 183 in a short time, electromagnetic waves LB can be immediately irradiated onto the appearing target object TG.

[0121] Furthermore, a re-imaging optical system 130 is included, which forms a second image Im2 by magnifying and re-imaging a portion of the intermediate image Imd formed by the imaging optical system 110. The image of the second image Im2, optically magnified by the re-imaging optical system 130, has higher image quality compared to a case where a portion of the first image Im1 is magnified using image processing such as digital zoom. Therefore, by using the second image Im2, which is optically magnified by the re-imaging optical system 130 and contains at least a portion of the object, the object TG can be identified with high precision (distinguished from other similar objects). This reliably prevents false illumination by electromagnetic waves LB.

[0122] In Embodiment 1 described above, the light-receiving device 172 of the distance measuring device 170 may also be located at a different position from the measuring light irradiation device 171. For example, a semi-reflective mirror may be provided in the optical path of the re-imaging optical system 130 (second optical system 132) between the re-imaging optical system 130 and the second imaging element 151. The light-receiving device may also be disposed offset from the semi-reflective mirror in a direction (e.g., the Y direction) intersecting the optical axis (second optical axis AX2) of the second optical system 132 in the re-imaging optical system 130. In this case, the semi-reflective mirror allows a portion of the measuring light LM emitted from the measuring light irradiation device 171 to pass through, while reflecting other light along the -Y direction. The semi-reflective mirror allows a portion of the light emitted from the object illuminated by the measuring light LM to be reflected toward the light-receiving device along the +Y direction, while allowing other light to pass through. Furthermore, the semi-reflective mirror can also allow a portion of the light passing through the re-imaging optical system 130 to pass through, while reflecting other light. The semi-reflective mirror may be prism-type or planar-type. The ratio of transmittance to reflectance of the semi-reflective mirror can be 1:1. Furthermore, in this case, the configurations of the measuring light irradiation device and the light receiving device can be interchanged. Additionally, as long as the light receiving device 172 can receive light from the object TG illuminated by the measured light LM, it does not need to be coaxially configured with the optical axis of the imaging system 100 (e.g., the optical axis of the re-imaging optical system 130). For example, the light receiving device 172 can be disposed on the outer surface of the frame 101 or on the aforementioned movable body.

[0123] [Modification of Implementation Method 1]

[0124] In Embodiment 1 described above, a reflective member rotation device 160 is provided, but the system is not limited to this. Therefore, a modified example of the imaging system according to Embodiment 1 will be described. In the imaging system according to the modified example of Embodiment 1, apart from the structures of the re-imaging optical system 230, the second imaging device 250, the holding part 260, the driving part 265, the distance measuring device 270, the control device 280, and the frame 201, the main parts have the same structure as the imaging system 100 according to Embodiment 1. Therefore, for structures that are the same as those in Embodiment 1, the same reference numerals as in Embodiment 1 are used, and detailed descriptions are omitted. Figure 12 As shown, the imaging system 200 involved in the modified example of Embodiment 1 includes an imaging optical system 110, a light splitting member 120, a re-imaging optical system 230, a first imaging device 140, a second imaging device 250, a holding part 260, a driving part 265, a distance measuring device 270, a control device 280, and a frame 201.

[0125] The frame 201 houses and holds the imaging optical system 110, the light splitting member 120, the re-imaging optical system 230, the first imaging device 140, the holding part 260 (including the second imaging device 250 and the distance measuring device 270), and the driving part 265. The imaging optical system 110, the light splitting member 120, and the first imaging device 140 have the same structure as those described in Embodiment 1, therefore detailed descriptions are omitted. In this modified example, the light splitting member 120 is disposed between the imaging optical system 110 and the re-imaging optical system 230. In this modified example, the light splitting member 120 may also be disposed midway through the imaging optical system 110, in which case the light splitting member 120 can split the light LT passing through a portion of the imaging optical system 110. The light splitting member 120 may be part of the imaging optical system 110. For example, the rear lens 112 can be disposed between the light splitting member 112 and the first image plane Im1, and between the light splitting member 120 and the intermediate image plane Imd.

[0126] In this modified example, the optical axis of the imaging optical system 110 and the optical axis of the re-imaging optical system 230, which is coaxial with the imaging optical system 110, are sometimes collectively referred to as the optical axis AX11. Furthermore, the direction along the optical axis AX11 of the imaging optical system 110 and the re-imaging optical system 230 is referred to as the Y direction. The direction opposite to the first imaging device 140 is referred to as the Z direction. The direction orthogonal to the Y and Z directions is referred to as the X direction. The X, Y, and Z directions are mutually orthogonal. For example, as... Figure 1 As shown, the Z direction can be vertical, while the X and Y directions can be horizontal.

[0127] like Figure 12As shown, the re-imaging optical system 230 re-images the intermediate image Imd formed by the light LT passing through the imaging optical system 110 as the second image Im2. In other words, another beam of light split by the light splitting member 120 forms the second image Im2 via the re-imaging optical system 230. The re-imaging optical system 230 can also re-image the intermediate image Imd formed by the light LT passing through the imaging optical system 110 as the second image Im2 at the same size. The re-imaging optical system 230 can also reduce the size of the intermediate image Imd formed by the light LT passing through the imaging optical system 110 and re-image it as the second image Im2. The re-imaging optical system 230 can also form the second image Im2 by magnifying a portion of the intermediate image Imd and re-imageing it. The position where the intermediate image Imd is formed can be conjugate to the position where the first image Im1 is formed. Therefore, the re-imaging optical system 230 re-images at least a portion of the intermediate image Imd as the second image Im2, thereby achieving the same effect as re-imaging at least a portion of the first image Im1 (the portion at the same relative position as the intermediate image Imd). In other words, the re-imaging optical system 230 re-images a portion of the intermediate image Imd that corresponds to a portion of the first image Im1. Furthermore, the position forming the intermediate image Imd (the third image) can be conjugate to the position forming the first image Im1.

[0128] Furthermore, the re-imaging optical system 230 can also be telecentric on the intermediate image Imd (light splitting member 120) side. This suppresses the divergence of the principal ray near the intermediate image Imd, thereby reducing the aperture of the re-imaging optical system 230. Therefore, the imaging system 200 can be miniaturized, reducing manufacturing costs. Furthermore, deviations in the incident angle of light incident on the light splitting member 120 (light splitting surface) can be suppressed. However, it is not limited to the imaging optical system 110 and the re-imaging optical system 230 both being telecentric on the intermediate image Imd (light splitting member 120) side. For example, only one of the imaging optical system 110 and the re-imaging optical system 230 may be telecentric on the intermediate image Imd (light splitting member 120) side.

[0129] The re-imaging optical system 230 includes a lens 231. In Figure 12 In the diagram, two lenses 231 are used to schematically represent the lenses 231 of the re-imaging optical system 230. The number of lenses 231 in the re-imaging optical system 230 can be one or more. In addition to the lenses 231, the re-imaging optical system 230 may also include existing optical components other than lenses.

[0130] like Figure 12As shown, the second imaging device 250 includes a second imaging element 251 for capturing a second image Im2. The second imaging element 251 is configured similarly to the second imaging element 151 described in Embodiment 1, except that it is held by the holding portion 260 together with the distance measuring device 270. Furthermore, as... Figure 13 As shown, in the imaging area 252 of the second imaging element 251, a plurality of second pixels 253 are arranged in a two-dimensional configuration. The second pixels 253 are arranged at a predetermined spacing in the X and Z directions. Each second pixel 253 of the second imaging element 251 performs photoelectric conversion on the light transmitted through the light splitting member 120, passing through the re-imaging optical system 230 and incident on the second imaging element 251. The second imaging device 250 outputs image information of the second image Im2 generated based on the photoelectric conversion in each second pixel 253 of the second imaging element 251 to the control device 280. For example, the spacing of the second pixels 253 of the second imaging element 251 may be smaller than the spacing of the first pixels 143 of the first imaging element 141. In other words, the number of second pixels 253 per unit area in the effective area (the area where the second pixels 253 are arranged) of the second imaging element 251 can be greater than the number of first pixels 143 per unit area in the effective area (the area where the first pixels 143 are arranged) of the first imaging element 141 (higher density). That is, the second imaging element 251 can be an imaging element with a higher resolution than the first imaging element 141. Furthermore, the spacing between the second pixels 253 of the second imaging element 251 can be less than or equal to the spacing between the first pixels 143 of the first imaging element 141; for example, the area of ​​the effective area of ​​the second imaging element 251 can be smaller than the area of ​​the effective area of ​​the first imaging element 141.

[0131] Furthermore, similar to Embodiment 1, a non-image area 254 may be formed in the central portion of the second imaging element 251. A through-hole 255 is formed on the second imaging element 251 along a direction intersecting the non-image area 254. In this case, the non-image area 254 is a through-area formed by the through-hole 255. The non-image area 254 can also be considered the cross-sectional area of ​​the through-hole 255. The through-hole 255 allows the measuring light LM emitted from the distance measuring device 270 to pass through. Furthermore, the cross-sectional shape of the through-hole 255 is not limited to a rectangle; it can also be a circle or other shapes. Additionally, as long as the measuring light LM emitted from the distance measuring device 270 can pass through, the opening size (cross-sectional area) of the through-hole 255 can be any size, but to ensure a larger area of ​​the image area 252 (effective area), the opening size is preferably smaller. Figure 14As shown, the second imaging element 251 (imaging area 252) is smaller than the second image Im2 and captures a portion of the second image Im2. For example, the diagonal length of the second imaging element 251 (imaging area 252) may be smaller than the diameter of the second image Im2, or it may be smaller than the radius of the second image Im2. Furthermore, the position of the through-hole 255 is not limited to the central portion of the second imaging element 251, but may also be formed at a position off-center from the center of the second imaging element 251. Additionally, the through-hole 255 may not be formed in the non-imaging area 254. For example, at least a portion of the distance measuring device 270 (e.g., a measuring light illumination device (171)) may be disposed in the non-imaging area 254. In this case, at least a portion of the distance measuring device 270 may be disposed in the non-imaging area 254 of the second imaging element 251, or it may be disposed at a position away from the second imaging element 251 (e.g., disposed between the second imaging element 251 and the re-imaging optical system 230).

[0132] like Figure 12 As shown, the distance measuring device 270 is configured the same as the distance measuring device 170 in Embodiment 1, except that it is held together with the second imaging element 251 by the holding part 260. The distance measuring device 270 is disposed on the side opposite to the optical component of the re-imaging optical system 230 that is closest to the second imaging element 251, separated by the second imaging element 251. The distance measuring device 270 measures the distance from the object TG (refer to the image) via the non-imaging area 254 (through hole 255) of the second imaging element 251, the re-imaging optical system 230, the light splitting component 120, and the imaging optical system 110. Figure 1 Illuminate the measurement light LM and generate distance information of the object TG.

[0133] like Figure 12 As shown, the holding unit 260 holds the second imaging element 251 and the distance measuring device 270. The driving unit 265 is configured, for example, using a linear motor or a stepper motor. The driving unit 265 can move the holding unit 260, which holds the second imaging element 251 and the distance measuring device 270, in parallel in two directions (e.g., the X direction and the Z direction) intersecting the optical axis AX11 of the re-imaging optical system 230. Driven by the driving unit 265, the second imaging element 251 moves in the X and Z directions, thereby changing the imaging range captured by the second imaging element 251 in the intermediate image Imd (the third image). As a result, the field of view of the object side (target side) captured by the second imaging element 251 also changes. In addition, driven by the driving unit 265, the distance measuring device 270 moves in the X and Z directions, thereby changing the illumination direction of the measuring light LM emanating from the distance measuring device 270.

[0134] The control device 280 is configured, for example, using a PC (personal computer). The control device 280 operates based on a program stored in the storage unit 281. Figure 12 As shown, the control device 280 includes a storage unit 281, an image information acquisition unit 282, an image analysis unit 283, a device control unit 284, a distance information acquisition unit 285, and an output device 287. The storage unit 281, image information acquisition unit 282, image analysis unit 283, distance information acquisition unit 285, and output device 287 are configured similarly to the storage unit 181, image information acquisition unit 182, image analysis unit 183, distance information acquisition unit 185, and output device 187 according to Embodiment 1.

[0135] The device control unit 284 controls the drive unit 265. The device control unit 284 can control the direction information of the object TG generated by the image analysis unit 283, such as... Figure 14 The drive unit 265 is controlled as shown to capture images of at least a portion of the image of the object TG within the region where the second image Im2 (intermediate image Imd) is formed, i.e., the image forming region of the second image Im2. Specifically, the device control unit 284 can use a device control data table stored in the storage unit 281 to control the position of the second imaging element 251 driven by the drive unit 265 based on the orientation information of the object TG, so that the image of the object TG contained in the second image Im2 is located at the center of the imaging region 252 (the effective region where the second pixel 253 is formed) of the second imaging element 251. The device control data table can be a data table representing the relationship between the orientation information of the object TG and the position of the second imaging element 251. Thus, the second imaging device 250 can capture images of the portion of the image of the object TG contained in the second image Im2, which is re-imaged by the re-imaging optical system 230. Alternatively, the portion of the second image Im2 captured by the second imaging element 251 (containing the image of the object TG) can be renamed as the second image Im2#.

[0136] Furthermore, the device control unit 284 can utilize the device control data table stored in the storage unit 281 to control the position of the second imaging element 251 driven by the drive unit 265 based on the position information of the object TG in the first image Im1, so that at least a portion of the image of the object TG contained in the second image Im2 is located at the center of the imaging area 252 of the second imaging element 251. The device control data table may also be a data table representing the relationship between the position information of the object TG in the first image Im1 and the position of the second imaging element 251. Alternatively, the positional relationship between the first imaging element 141, the second imaging element 251, and the holding unit 260 can be calibrated in advance to create the device control data table.

[0137] In this modified example, a second image Im2# can be formed based on image information of a first image Im1 containing an image of the object TG captured by the first imaging device 140, such that a portion of the first optical path of the light forming the image of the object TG contained in the second image Im2# overlaps with a portion of the second optical path of the measuring light LM from the distance measuring device 270. Then, with the second image Im2# formed such that a portion of the first optical path overlaps with a portion of the second optical path, the distance measuring device 270 can illuminate the measuring light LM. Alternatively, the first image Im1 may also contain an image of a portion of the object TG. The image contained in the second image Im2# may also be an image of a portion of the object TG contained in the second image Im2#. Furthermore, the entire first optical path may overlap with a portion of the second optical path. Alternatively, a portion of the first optical path may overlap with the entire second optical path. Alternatively, the entire first optical path may overlap with the entire second optical path.

[0138] In other words, based on the image information of the first image Im1 containing the image of the object TG, a portion of the image forming region of the intermediate image Imd can be selected from the image forming region, such that a portion of the first optical path of the light forming the object TG contained in the second image Im2# overlaps with a portion of the second optical path of the measuring light LM from the distance measuring device 270. Furthermore, with a portion of the image forming region of the intermediate image Imd selected from the image forming region, such that a portion of the first optical path overlaps with a portion of the second optical path, the distance measuring device 270 can illuminate the measuring light LM.

[0139] Therefore, regardless of the location of the image of at least a portion of the object TG within the first image Im1 (that is, when the first imaging device 140 captures at least a portion of the object TG within the viewing angle of the imaging optical system 110), the measuring light LM emitted from the distance measuring device 270 (measuring light illumination device (171)) will illuminate the object TG via the non-imaging area 254 (through-hole 255) of the second imaging element 251, the re-imaging optical system 230, the light splitting member 120, and the imaging optical system 110. The light from the object TG illuminated by the measured light LM arrives at the distance measuring device 270 (light receiving device) in the reverse order of the measuring light LM from the distance measuring device 270. Therefore, the distance measuring device 270 can generate distance information of the object TG based on the light receiving result of the object TG illuminated by the measured light LM. Furthermore, by simply setting the position of the second imaging element 251, while the distance measuring device 270 generates distance information of the object TG, the second imaging device 250 can generate image information of the second image Im2#, which is at least a portion of the image of the object TG and is re-imaged by the re-imaging optical system 230. Therefore, both the distance information of the object TG and the image information of the second image Im2#, which is at least a portion of the image of the object TG, can be generated in a short time.

[0140] [Measurement Method]

[0141] Next, the measurement method of the imaging system 200 according to the modified example of Embodiment 1 will be summarized. The measurement method according to this modified example is the same as that according to Embodiment 1. Therefore, the same measurement method as that according to Embodiment 1 is used. Figure 7 The flowchart shown will be used for explanation. Furthermore, each process in the measurement method is executed based on a program stored in the storage unit 281. First, the imaging system 200 acquires the orientation information of the object TG, similar to the case in Embodiment 1 (step ST10). In this modified example, the device control unit 284 controls the drive unit 265 based on the orientation information of the object TG generated by the image analysis unit 283, enabling the imaging of a portion of the image forming area of ​​the second image Im2 (intermediate image Imd) containing at least a portion of the object TG. For example, the device control unit 284 controls the position of the second imaging element 251 driven by the drive unit 265 based on at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1, such that the image of at least a portion of the object TG contained in the second image Im2 is located at the center of the imaging area 252 of the second imaging element 251.

[0142] Next, the imaging system 200 acquires distance information of the target object TG (step ST20). At this time, with the position of the second imaging element 251 set such that at least a portion of the image of the target object TG contained in the second image Im2 is located at the center of the imaging area 252 of the second imaging element 251, the distance measuring device 270 illuminates the target object TG with measuring light LM through the non-imaging area 254 (through hole 255) of the second imaging element 251, the re-imaging optical system 230, the light splitting member 120, and the imaging optical system 110, and generates distance information of the target object TG. The distance measuring device 270 outputs the generated distance information of the target object TG to the distance information acquisition unit 285. The distance information acquisition unit 285 acquires the distance information of the target object TG output from the distance measuring device 270. In addition, in step ST10, in order to control the position of the second imaging element 251, step ST10 can also be performed between steps ST20 and ST40 (described later) without using the orientation information of the target object TG. In this case, in step ST20, the first imaging device 140 captures a first image Im1 containing at least a portion of the object TG, and the image analysis unit 283 generates position information of the object TG in the first image PC1 based on the image information of the first image Im1 containing at least a portion of the object TG.

[0143] Next, the imaging system 200 acquires image information of the second image Im2# (step ST30). At this time, with the position of the second imaging element 251 set such that at least a portion of the object TG contained in the second image Im2 is located at the center of the imaging area 252 of the second imaging element 251, the second imaging device 250 captures the second image Im2# containing the object TG and outputs the generated image information of the second image Im2# to the image information acquisition unit 282. The image information acquisition unit 282 acquires the image information of the second image Im2# output from the second imaging device 250. Alternatively, the correction performed by the image correction unit 186 as described in Embodiment 1 may not be performed. Step ST30 may also be performed between steps ST10 and ST20.

[0144] Then, the imaging system 200 outputs information in the same manner as in Embodiment 1 (step ST40). Alternatively, steps ST10 to ST40 can be repeated in the same manner as in Embodiment 1.

[0145] According to this variation, similar to Embodiment 1, the distance to the object can be measured by the distance measuring device 270 while the object is being photographed and followed by the first shooting device 140.

[0146] In a variation of Embodiment 1, the re-imaging optical system 230 may not be provided. In this case, the second imaging element 251 may also be held by the holding unit 260, and positioned at or near the image plane where the intermediate image Imd is formed. Furthermore, "near the image plane where the intermediate image Imd is formed" means, for example, the area near which the object TG can be identified in the image of the intermediate image Imd, and where the second imaging element 251 can capture a portion of the intermediate image Imd.

[0147] [Implementation Method 2]

[0148] Next, the imaging system according to Embodiment 2 will be described. In the imaging system according to Embodiment 2, apart from the structures of the second imaging device 350, the second reflective member 368, the distance measuring device 370, and the frame 301, the main parts have the same structure as the imaging system 100 according to Embodiment 1. Therefore, for structures identical to those in Embodiment 1, the same reference numerals as in Embodiment 1 are used, and detailed descriptions are omitted. Figure 15 As shown, the imaging system 300 according to Embodiment 2 includes an imaging optical system 110, a light splitting component 120, a re-imaging optical system 130, a first imaging device 140, a second imaging device 350, a reflective component rotation device 160, a second reflective component 368, a distance measuring device 370, a control device 180, and a frame 301.

[0149] The frame 301 houses and holds the imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the second imaging device 350, the reflector rotating device 160, the second reflector 368, and the distance measuring device 370. The imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the reflector rotating device 160, and the control device 180 have the same structure as those of the imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the reflector rotating device 160, and the control device 180 according to Embodiment 1, and detailed descriptions are omitted.

[0150] In Embodiment 2, similar to Embodiment 1, the optical axis of the first optical system 131 in the re-imaging optical system 130 and the optical axis of the imaging optical system 110 coaxial with the first optical system 131 are referred to as the first optical axis AX1. The optical axis of the second optical system 132 in the re-imaging optical system 130 is referred to as the second optical axis AX2. Furthermore, the direction along the first optical axis AX1 is referred to as the Y direction. The direction along the second optical axis AX2 is referred to as the Z direction. The direction orthogonal to the Y and Z directions is referred to as the X direction.

[0151] like Figure 15 As shown, the second imaging device 350 includes a second imaging element 351 for capturing a second image Im2. Except for the absence of a non-imaging area and a through-hole, the second imaging element 351 is constructed identically to the second imaging element 151 described in Embodiment 1. Furthermore, as... Figure 16 As shown, in the imaging area 352 of the second imaging element 351, a plurality of second pixels 353 are arranged in a two-dimensional configuration. The second pixels 353 are arranged at a predetermined spacing in the X and Z directions. Each second pixel 353 of the second imaging element 351 performs photoelectric conversion on the light transmitted through the light splitting member 120, passing through the re-imaging optical system 130 and incident on the second imaging element 351. The second imaging device 350 outputs image information of the second image Im2 generated based on the photoelectric conversion in each second pixel 353 of the second imaging element 351 to the control device 380.

[0152] like Figure 15 As shown, the second reflective member 368 is disposed on at least a portion of the optical path of the re-imaging optical system 130 (second optical system 132) between the re-imaging optical system 130 and the second imaging element 351. The second reflective member 368 can be constructed, for example, using a semi-reflective mirror that performs light amplitude segmentation. The semi-reflective mirror can be prism-type or planar-type. The ratio of the transmittance to reflectance of the semi-reflective mirror can be 1:1. The second reflective member 368 can cause a portion of the measuring light LM emitted from the distance measuring device 370 to be reflected toward the re-imaging optical system 130 in the -Z direction, while allowing other light to pass through. The second reflective member 368 can also cause a portion of the light from the object TG (refer to the object TG) illuminated by the measured light LM to be reflected. Figure 1 A portion of the light passing through the re-imaging optical system 130 is reflected towards the distance measuring device 370 along the +Y direction, while other light passes through. Furthermore, the second reflective member 368 can also allow a portion of the light passing through the re-imaging optical system 130 to pass through, while reflecting other light. The second reflective member 368 can be smaller than the area through which the light from the re-imaging optical system 130 passes (the cross-sectional area of ​​the optical path of the re-imaging optical system 130). In other words, light traveling away from the second optical axis AX2 can be prevented from incident on the second reflective member 368, passing outside the second reflective member 368, while a portion of the light traveling near the second optical axis AX2 passes through the second reflective member 368.

[0153] Furthermore, the second reflective member 368 can be larger than the area through which light from the re-imaging optical system 130 passes (the cross-sectional area of ​​the optical path of the re-imaging optical system 130), or it can be the same size as the area through which light from the re-imaging optical system 130 passes. In this case, the transmittance of the second reflective member 368 at a position near the second optical axis AX2 can be designed to be higher than that at a position far from the second optical axis AX2, or the transmittance of the second reflective member 368 at a position near the second optical axis AX2 can be designed to be lower than that at a position far from the second optical axis AX2. Additionally, when the second optical system 132 of the re-imaging optical system 130 is composed of two or more lenses, the second reflective member 368 can be disposed in the middle portion of the second optical system 132 (e.g., between lenses). Furthermore, the second reflective member 368 can be disposed in the optical path of the re-imaging optical system 130 between the second optical system 132 and the first reflective member 161. Thus, the second reflective member 368 can be disposed on at least a portion of the optical path of the re-imaging optical system 130. Alternatively, the second reflective member 368 can be constructed using a dichroic mirror that performs wavelength division of light.

[0154] like Figure 15 As shown, the distance measuring device 370 can be configured similarly to the distance measuring device 170 described in Embodiment 1, except for a different configuration. The distance measuring device 370 is positioned opposite the second reflective member 368 in a direction (e.g., the Y direction) that intersects the optical axis (second optical axis AX2) of the second optical system 132 in the re-imaging optical system 130. The distance measuring device 370 projects onto the target object TG (refer to the image) via the second reflective member 368, the second optical system 132, the first reflective member 161, the first optical system 131, the light splitting member 120, and the imaging optical system 110. Figure 1 The distance measuring device 370 illuminates the measuring light LM and generates distance information for the target object TG. As described in Embodiment 1, the emitting portion of the measuring light LM and the incident portion of the light from the target object TG in the distance measuring device 370 may be of a common structure. Alternatively, the distance measuring device 370 may also have a structure in which the emitting portion of the measuring light LM and the incident portion of the light from the target object TG are separately arranged.

[0155] In addition, in Embodiment 2, the distance measuring device 370 can be, for example, a TOF-type LiDAR, a TOF-type reflective laser sensor, or a TOF-type distance image sensor. As an FMCW-type distance measuring device 370, an FMCW-type LiDAR can be used, for example. Furthermore, at least a portion of the distance measuring device 370 (e.g., the measuring light illumination device (171)) can be configured at a position conjugate to the position where the second image Im2 is formed. For example, when using a LiDAR or a reflective laser sensor, the emitting portion of the measuring light LM in the distance measuring device 370 can be configured at a position conjugate to the position where the second image Im2 is formed. Furthermore, when using a TOF-type distance image sensor, all the pixel arrangement portions in the TOF-type distance image sensor (distance measuring device 370) can be configured at a position conjugate to the position where the second image Im2 is formed (the position of the second imaging element 351). Furthermore, a portion of the pixel arrangement in the TOF-based distance image sensor (distance measuring device 370) can also be configured at a position conjugate to the position where the second image Im2 is formed.

[0156] Similarly to Embodiment 1 described above, the light-receiving device (172) of the distance measuring device 370 can also be located at a different position than the measuring light illumination device (171). For example, two second reflective members 368 can be provided at different positions on the optical path of the re-imaging optical system 130. In this case, one second reflective member 368 is provided on the optical path of the re-imaging optical system 130, and the other second reflective member 368 can be provided on the optical path of the re-imaging optical system 130 at a position further away from the second imaging element 351 than the position of the first second reflective member 368. For example, one second reflective member 368 can cause a portion of the measuring light LM from the measuring light illumination device (171) to be reflected toward the first reflective member 161 in the -Z direction, while allowing other light to pass through. Furthermore, the other second reflective member 368 can cause a portion of the light from the object TG illuminated by the measured light LM to be reflected toward the light-receiving device (172) in the +Y direction, while allowing other light to pass through. Furthermore, as long as the light receiving device (172) can receive light from the object TG illuminated by the measured light LM, it does not need to be coaxially configured with the optical axis of the imaging system 100 (e.g., the optical axis of the re-imaging optical system 130). For example, the light receiving device (172) can be provided on the outer surface of the frame 301 or on the aforementioned movable body.

[0157] [Measurement Method]

[0158] Next, the measurement method using the imaging system 300 according to Embodiment 2 will be summarized. The measurement method according to Embodiment 2 is the same as the measurement method according to Embodiment 1. Therefore, the same method as in Embodiment 1 is used. Figure 7 The flowchart shown will be used for explanation. Furthermore, each process in the measurement method is executed based on a program stored in the storage unit 181. First, the imaging system 300 acquires the orientation information of the object TG, similar to that in Embodiment 1 (step ST10).

[0159] Next, the imaging system 300 acquires distance information of the target object TG (step ST20). At this time, with the rotation angle of the first reflective member 161 set such that at least a portion of the target object TG contained in the intermediate image Imd is located at the center of the imaging area 352 of the second imaging element 351 and is re-imaged, the distance measuring device 370 illuminates the target object TG with measuring light LM via the second reflective member 368, the second optical system 132, the first reflective member 161, the first optical system 131, the light splitting member 120, and the imaging optical system 110, and generates distance information of the target object TG. The distance measuring device 170 outputs the generated distance information of the target object TG to the distance information acquisition unit 185. The distance information acquisition unit 185 acquires the distance information of the target object TG output from the distance measuring device 370. In Embodiment 2, at this time, the control device 180 can correct for errors in the distance information based on changes in the optical path length corresponding to the rotation of the first reflective member 161. For example, the control device 180 can use a calibration data table stored in the storage unit 181 to calibrate the distance information of the object TG output from the distance measuring device 370. The calibration data table can be a data table showing the relationship between the rotation angle of the first reflector 161 and the calibration amount of the distance information of the object TG. In addition, in step ST10, in order to control the rotation angle of the first reflector 161, step ST10 can also be performed between steps ST20 and ST40 (described later) without using the orientation information of the object TG. In this case, in step ST20, the first imaging device 140 captures a first image Im1 containing an image of at least a portion of the object TG, and the image analysis unit 383 generates the position information of the object TG in the first image PC1 based on the image information of the first image Im1 containing at least a portion of the object TG.

[0160] In addition, in step ST10, in order to control the rotation angle of the first reflective member 161, without using the orientation information of the object TG, step ST10 can also be performed between steps ST20 and ST40 (described later), similar to the case of embodiment 1.

[0161] Next, the imaging system 300 acquires image information of the second image Im2, similar to the case in Embodiment 1 (step ST30). Alternatively, step ST30 can be performed between steps ST10 and ST20. Then, the imaging system 300 outputs information, similar to the case in Embodiment 1 (step ST40). Alternatively, steps ST10 to ST40 can be repeated as in Embodiment 1.

[0162] According to Embodiment 2, since the non-image area 154 (e.g., through-hole 155) formed on the second image element 151 in Embodiment 1 is not formed on the second image element 351 in this embodiment, a portion of the second image Im2 (i.e., the portion corresponding to the non-image area 154) in the image captured by the second image device 350 containing at least a portion of the object TG is not missing. Therefore, more accurate image information of the object TG can be generated. Consequently, the object TG can be identified more accurately (more accurately distinguished from other similar objects), thus more reliably preventing false illumination by electromagnetic waves LB.

[0163] The other effects of Implementation 2 are the same as those of Implementation 1.

[0164] [Implementation Method 3]

[0165] Next, the shooting system according to Embodiment 3 will be described. In the shooting system according to Embodiment 3, apart from the structures of the second shooting device 450, the distance measuring device 470, the control device 480, and the frame 401, its main parts have the same structure as the shooting system 100 according to Embodiment 1. Therefore, for structures identical to those in Embodiment 1, the same reference numerals as in Embodiment 1 are used, and detailed descriptions are omitted. Figure 17 As shown, the imaging system 400 according to Embodiment 3 includes an imaging optical system 110, a light splitting component 120, a re-imaging optical system 130, a first imaging device 140, a second imaging device 450, a reflective component rotation device 160, a distance measuring device 470, a control device 480, and a frame 401.

[0166] The housing 401 houses and holds the imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the second imaging device 450, and the reflective member rotating device 160. The imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the second imaging device 450, and the reflective member rotating device 160 housed and held in the housing 401 are collectively referred to as optical devices. The imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, and the reflective member rotating device 160 have the same structure as those described in Embodiment 1, and detailed descriptions are omitted.

[0167] In Embodiment 3, similar to Embodiment 1, the optical axis of the first optical system 131 in the re-imaging optical system 130 and the optical axis of the imaging optical system 110 coaxial with the first optical system 131 are referred to as the first optical axis AX1. The optical axis of the second optical system 132 in the re-imaging optical system 130 is referred to as the second optical axis AX2. Furthermore, the direction along the first optical axis AX1 is referred to as the Y direction. The direction along the second optical axis AX2 is referred to as the Z direction. The direction orthogonal to the Y and Z directions is referred to as the X direction.

[0168] like Figure 17 As shown, the second imaging device 450 includes a second imaging element 451 for capturing a second image Im2. The second imaging element 451 is configured similarly to the second imaging element 351 described in Embodiment 2.

[0169] like Figure 17 As shown, the distance measuring device 470 can be configured similarly to the distance measuring device 170 described in Embodiment 1, except for a different configuration. The distance measuring device 470 is positioned differently from the shooting system 400 (frame 401). For example, the distance measuring device 470 can be located in a portion of the housing member 10 that differs from the position of the shooting system 400 (frame 401). The distance measuring device 470 measures the distance towards the target object TG (refer to...). Figure 1The distance measuring device 470 illuminates the measuring light LM onto the object TG and generates distance information for the object TG. In other words, the distance measuring device 470 illuminates the measuring light LM onto the object TG without passing through the second optical system 132, the first reflecting member 161, the first optical system 131, the light splitting member 120, and the imaging optical system 110, and generates distance information for the object TG. Alternatively, the distance measuring device 470 illuminates the measuring light LM onto the object TG non-coaxially with the optical axis of the imaging system 400, and generates distance information for the object TG. In addition, the distance measuring device 470 may also be provided on the movable body described above. Furthermore, similar to Embodiment 1 described above, the light receiving device (172) of the distance measuring device 370 may also be provided at a different position than the measuring light illuminating device (171). For example, the measuring light illuminating device (171) may be fixedly provided, and the light receiving device (172) may be provided on the movable body described above. In addition, at least one functional module of the storage unit 481, image information acquisition unit 482, image analysis unit 483, device control unit 484, distance information acquisition unit 485, image correction unit 486, and output device 487 may also be provided in the distance measuring device 470.

[0170] In embodiment 3, the distance measuring device 470 can be, for example, a TOF-type LiDAR, a TOF-type reflective laser sensor, or a TOF-type distance image sensor. As an FMCW-type distance measuring device 470, an FMCW-type LiDAR can be used. Furthermore, the LiDAR used in the distance measuring device 470 can also have a structure that allows the illumination direction of the measuring light LM to be changed using a multifaceted mirror, galvanometer, or similar means. In this case, scanning can be performed by changing the illumination direction of the measuring light LM based on the orientation information of the object TG generated by the image analysis unit 483 of the control device 480, thereby searching for the object TG. For example, the distance measuring device 470 can also scan around the direction of the object TG, as indicated by the orientation information of the object TG, by changing the illumination direction of the measuring light LM, thereby searching for the object TG. The LiDAR used in the distance measuring device 470 can also have a structure where the illumination direction of the measuring light LM is fixed. When using a LiDAR or reflective laser sensor with a fixed illumination direction of the measuring light LM, the distance measuring device 470 may also include a rotary stage capable of rotating to hold the LiDAR or reflective laser sensor and thus change the illumination direction of the measuring light LM. In this case, at least one of the measuring light illumination device (171) and the light receiving device (172) may also be mounted on the rotary stage.

[0171] Furthermore, the distance measuring device 470 can be a millimeter-wave sensor (or millimeter-wave radar) that illuminates the object TG with millimeter waves as a measuring light LM. Alternatively, the distance measuring device 470 can be a measuring instrument utilizing an optical comb. Here, when using a millimeter-wave sensor (or millimeter-wave radar) and an optical comb-based measuring instrument, the distance measuring device 470 may also include a rotating stage capable of rotatably holding the millimeter-wave sensor (or millimeter-wave radar) and measuring instrument, thereby changing the illumination direction of the measuring light LM. The distance measuring device 470 can be structured with the measuring light illuminating device and the light receiving device separately disposed in the housing member 10, and the distance information of the object generated using the triangulation principle. In this case, the measuring light illuminating device 171 can illuminate the object TG with structured light (pattern light), such as striped patterns with regular striped intensity distribution or random dot patterns with random (irregular) dotted intensity distribution, as the measuring light LM. Alternatively, the distance measuring device 470 may not include a measuring light illuminating device. For example, a stereo camera can be used as a distance measuring device that does not include a measuring light illuminating device. In this case, distance information of the object TG can be generated based on the parallax of the stereo image acquired by the stereo camera. Furthermore, the output device 487 can output one side of the image information from the stereo image acquired by the stereo camera.

[0172] The control device 480 is configured, for example, using a PC (personal computer). The control device 480 operates based on a program stored in the storage unit 481. Figure 17 As shown, the control device 480 includes a storage unit 481, an image information acquisition unit 482, an image analysis unit 483, a device control unit 484, a distance information acquisition unit 485, an image correction unit 486, and an output device 487. The storage unit 481, image information acquisition unit 482, image analysis unit 483, distance information acquisition unit 485, image correction unit 486, and output device 487 are configured similarly to the storage unit 181, image information acquisition unit 182, image analysis unit 183, distance information acquisition unit 185, image correction unit 186, and output device 187 according to Embodiment 1.

[0173] The device control unit 484 controls the drive device 163 of the reflector rotation device 160 and the distance measuring device 470. Similar to the device control unit 184 in Embodiment 1, the device control unit 484 controls the drive device 163 of the reflector rotation device 160. Furthermore, the device control unit 484 can set the illumination direction of the measuring light LM based on at least one of the orientation information of the object TG generated by the image analysis unit 483 and the position information of the object TG in the first image PC1, as well as the position information of the distance measuring device 470. Here, the position information of the distance measuring device 470 can also be information about the positional relationship between the optical device (imaging optical system 110, light splitting member 120, re-imaging optical system 130, first imaging device 140, second imaging device 450, and reflector rotation device 160) and the distance measuring device 470. Additionally, the information about the positional relationship between the optical device and the distance measuring device 470 can be pre-determined information or stored in the storage unit 481. The device control unit 484 outputs a control signal containing information about the set illumination direction to the distance measuring device 470, thereby controlling the measuring light LM to illuminate in that direction. Additionally, the position information of the distance measuring device 470 can also be referred to as the positional relationship information between the optical device and the distance measuring device 470.

[0174] Furthermore, when the light from the target object TG is not received by the distance measuring device 470 (light receiving device (172)), the device control unit 484 can reset the illumination direction. Specifically, the device control unit 484 can use information about the positional relationship between the optical devices (imaging optical system 110, light splitting member 120, re-imaging optical system 130, first imaging device 140, second imaging device 450, and reflector rotating device 160) and the distance measuring device 470 to correct the direction information of the target object TG, and reset the illumination direction based on the corrected direction information. In addition, the situation where the light from the target object TG is not received by the distance measuring device 470 (light receiving device (172)) may also be due to the distance information acquisition unit 485 being unable to properly acquire the distance information from the distance measuring device 470.

[0175] Therefore, in the illumination direction set based on the position and orientation information of the distance measuring device 470, the measuring light LM emitted from the distance measuring device 470 (measuring light illumination device (171)) is irradiated onto the object TG. The light from the object TG irradiated by the measuring light LM reaches the distance measuring device 470 (light receiving device (172)). Therefore, the distance measuring device 470 can generate distance information of the object TG based on the light receiving result of the object TG irradiated by the measuring light LM.

[0176] [Measurement Method]

[0177] Next, the measurement method using the imaging system 400 according to Embodiment 3 will be summarized. The measurement method according to Embodiment 3 is the same as the measurement method according to Embodiment 1. Therefore, the same method as in Embodiment 1 is used. Figure 7 The flowchart shown will be used for explanation. Furthermore, each process in the measurement method is executed based on a program stored in the storage unit 481. First, the imaging system 400 acquires the orientation information of the object TG, similar to the case in Embodiment 1 (step ST10). At this time, in Embodiment 3, the device control unit 484 controls the drive device 163 of the reflector rotation device 160 based on at least one of the orientation information of the object TG generated by the image analysis unit 483 and the position information of the object TG in the first image PC1, so that the region of the image forming area of ​​the intermediate image Imd containing at least a portion of the object TG is magnified and re-imaged as the second image Im2. Additionally, the device control unit 484 sets the illumination direction of the measurement light LM based on at least one of the orientation information of the object TG generated by the image analysis unit 483 and the position information of the object TG in the first image PC1, and the position information of the distance measuring device 470.

[0178] Next, the imaging system 400 acquires distance information of the target object TG (step ST20). At this time, the distance measuring device 470 irradiates the target object TG with measuring light LM by emitting a measuring light LM in the irradiation direction set by the device control unit 484, and generates distance information of the target object TG. The distance measuring device 470 outputs the generated distance information of the target object TG to the control device 480. The distance information acquisition unit 485 of the control device 480 acquires the distance information of the target object TG output from the distance measuring device 470. In addition, in step ST10, in order to control the rotation angle of the first reflective member 161, without using the direction information of the target object TG, similar to the case of Embodiment 1, step ST10 can also be performed between steps ST20 and ST40 (described later).

[0179] Next, the imaging system 400 acquires image information of the second image Im2, similar to the case in Embodiment 1 (step ST30). Alternatively, step ST30 can be performed between steps ST10 and ST20. Then, the imaging system 400 outputs information, similar to the case in Embodiment 1 (step ST40). Alternatively, steps ST10 to ST40 can be repeated as in Embodiment 1.

[0180] According to embodiment 3, the same effect as that of embodiment 1 can be obtained.

[0181] [Implementation Method 4]

[0182] Next, the imaging system according to Embodiment 4 will be described. In the imaging system according to Embodiment 4, apart from the structures of the first optical system 531, the second optical system 532, the imaging device 540, the reflective member rotation device 160, the distance measuring device 570, the control device 580, and the frame 501, its main parts have the same structure as the imaging system 100 according to Embodiment 1. Therefore, for structures identical to those in Embodiment 1, the same reference numerals as in Embodiment 1 are used, and detailed descriptions are omitted. Figure 18 and Figure 19 As shown, the imaging system 500 according to Embodiment 4 includes an imaging optical system 110, a light splitting member 120, a first optical system 531, a second optical system 532, an imaging device 540, a reflective member rotation device 160, a distance measuring device 570, a control device 580, and a housing 501. Alternatively, the imaging system 500 may not include the second optical system 532. Furthermore, the imaging system 500 may include an optical system for converting (collimating) the measurement light LM from the distance measuring device 570 into parallel light, in place of the second optical system 532.

[0183] The frame 501 houses and holds the imaging optical system 110, the light splitting member 120, the first optical system 531, the second optical system 532, the imaging device 540, the reflector rotating device 160, and the distance measuring device 570. The imaging optical system 110, the light splitting member 120, and the reflector rotating device 160 have the same structure as those described in Embodiment 1, therefore detailed descriptions are omitted. Furthermore, in the reflector rotating device 160, the reflective surface 162 of the first reflector 161 reflects the measuring light LM from the distance measuring device 570 after the second optical system 532 toward the first optical system 531 (imaging optical system 110). Furthermore, the drive device 163 can rotate the first reflecting member 161 to the following two positions: the position where the measuring light LM reflected by the reflecting surface 162 of the first reflecting member 161 travels along the optical axis (first optical axis AX1) of the first optical system 531 (refer to...). Figure 18 ); and the position of the measurement light LM reflected by the reflecting surface 162 of the first reflecting member 161 from the optical axis of the first optical system 531 (e.g., referring to Figure 19 ).

[0184] In embodiment 4, the optical axis of the first optical system 531 and the optical axis of the imaging optical system 110 coaxial with the first optical system 531 are referred to as the first optical axis AX1. The optical axis of the second optical system 532 is referred to as the second optical axis AX2. The second optical axis AX2 may intersect the first optical axis AX1. Furthermore, the second optical axis AX2 may be orthogonal to the first optical axis AX1. Additionally, the direction along the first optical axis AX1 is referred to as the Y direction. The direction along the second optical axis AX2 is referred to as the Z direction. The direction orthogonal to both the Y and Z directions is referred to as the X direction. The X, Y, and Z directions are mutually orthogonal. For example, as... Figure 1 As shown, the Z direction can be vertical, while the X and Y directions can be horizontal.

[0185] Light passing through the light splitter 120 enters the first optical system 531. In other words, another beam of light split by the light splitter 120 enters the first optical system 531. Light from the first optical system 531 enters the second optical system 532. Figure 18 In the diagram, a first optical system 531 and a second optical system 532 are schematically represented by a single lens. Each of the first optical system 531 and the second optical system 532 may be composed of a single lens or multiple lenses. In addition to one or more lenses, each of the first optical system 531 and the second optical system 532 may also include existing optical components other than lenses.

[0186] The optical axis (second optical axis AX2) of the second optical system 532 is orthogonal to the optical axis (first optical axis AX1) of the first optical system 531. Alternatively, the optical axis of the second optical system 532 may intersect the optical axis of the first optical system 531 at an angle other than 90 degrees. For example, the optical axis of the second optical system 532 may intersect the optical axis of the first optical system 531 within an angle range of 90 degrees ± 5 degrees. At or near the intersection of the optical axes of the first optical system 531 and the second optical system 532, the reflecting surface 162 of the first reflecting member 161 in the reflecting member rotating device 160 is disposed. The intersection of the optical axes of the first optical system 531 and the second optical system 532 is also the point where the optical axes of the first optical system 531 and the second optical system 532 intersect. Furthermore, the so-called intersection location is, for example, the vicinity of the range in which the reflecting surface 162 of the first reflecting member 161 can reflect light from the first optical system 531 toward the second optical system 532.

[0187] Furthermore, the first optical system 531 can also be telecentric on the intermediate image Imd (light splitting member 120) side. This suppresses the divergence of the principal ray near the intermediate image Imd, thereby reducing the aperture of the first optical system 531. Therefore, the imaging system 500 can be miniaturized, reducing manufacturing costs. Furthermore, deviations in the incident angle of light incident on the light splitting member 120 (light splitting surface) can be suppressed. Therefore, the imaging system 500 can be miniaturized, reducing manufacturing costs. However, it is not limited to the imaging optical system 110 and the first optical system 531 both being telecentric on the intermediate image Imd (light splitting member 120) side. For example, either the imaging optical system 110 or the first optical system 531 can also be telecentric on the intermediate image Imd (light splitting member 120) side.

[0188] like Figure 18 and Figure 19 As shown, the imaging device 540 includes an imaging element 541 for capturing the first image Im1. The imaging device 540 and the imaging element 541 are configured in the same way as the first imaging device 140 and the first imaging element 141 according to Embodiment 1.

[0189] The distance measuring device 570 is positioned such that the measuring light LM can be irradiated onto the light splitting member 120 via the first reflector 161 of the reflector rotation device 160. Alternatively, the distance measuring device 570 can be positioned conjugate to the position where the intermediate image Imd is formed. The position where the intermediate image Imd is formed can be conjugate to the position where the first image Im1 is formed. The distance measuring device 570 irradiates the target object TG (refer to the image LM) via the second optical system 531, the first reflector 161, the first optical system 131, the light splitting member 120, and the imaging optical system 110. Figure 1The distance measuring device 570 can also be configured similarly to the distance measuring device 170 described in Embodiment 1. Furthermore, as described above, the imaging system 500 may not include the second optical system 532. In this case, the distance measuring device 570 can be positioned closer to the first reflective member 161 than the position of the second optical system 532. In this case, the imaging system 500 (frame 501) can be further miniaturized. Additionally, the light receiving device (172) of the distance measuring device 570 can be positioned differently from the measuring light irradiation device (171). For example, a semi-reflective mirror can be provided in the optical path of the second optical system 532. The light receiving device (172) can also be positioned opposite the semi-reflective mirror in a direction (e.g., the Y direction) that intersects the optical axis (second optical axis AX2) of the second optical system 532. In this case, the semi-reflective mirror allows a portion of the measuring light LM emitted from the measuring light irradiation device (171) to pass through, while reflecting the remaining light along the -Y direction. A semi-reflective mirror can cause a portion of the light from the object illuminated by the measured light LM to be reflected toward the light-receiving device along the +Y direction, while allowing the rest of the light to pass through. The semi-reflective mirror can be prism-type or planar-type. The ratio of the transmittance to the reflectance of the semi-reflective mirror can be 1:1. Furthermore, in this case, the configurations of the measuring light illumination device (171) and the light-receiving device (172) can be interchanged. In addition, as long as the light-receiving device (172) can receive the light from the object TG illuminated by the measured light LM, it does not need to be coaxially configured with the optical axis of the imaging system 500 (e.g., the optical axis of the second optical system 532). For example, the light-receiving device (172) can be provided on the outer surface of the frame 501 or on the aforementioned movable body.

[0190] The control device 580 is configured, for example, using a PC (personal computer). The control device 580 operates based on a program stored in the storage unit 581. Figure 18 and Figure 19 As shown, the control device 580 includes a storage unit 581, an image information acquisition unit 582, an image analysis unit 583, a device control unit 584, a distance information acquisition unit 585, and an output device 587. The storage unit 581, the distance information acquisition unit 585, and the output device 587 are configured similarly to the storage unit 181, the distance information acquisition unit 185, and the output device 187 according to Embodiment 1.

[0191] The image information acquisition unit 582 acquires the image information of the first image Im1 output from the imaging device 540. Thereafter, the image of the first image Im1 captured by the imaging device 540 is sometimes referred to as the first image PC1 (see reference). Figure 21The first image PC1 can also be referred to as a wide-angle image. The image information of the first image Im1 acquired by the image information acquisition unit 582 can also be stored in the storage unit 581.

[0192] The image analysis unit 583 generates orientation information of the object TG based on image information of a first image Im1, which is acquired by the image information acquisition unit 582 and contains at least a portion of the object TG. Similar to the image analysis unit 183 of Embodiment 1, the image analysis unit 583 identifies the object TG from a first image PC1 containing one or more objects by analyzing the image information of the first image Im1. When identifying the object TG, the image analysis unit 583 generates position information of the object TG in the first image PC1, similar to the image analysis unit 183 of Embodiment 1. Similarly to the image analysis unit 183 of Embodiment 1, the image analysis unit 583 generates orientation information of the object TG based on the generated position information of the object TG in the first image PC1.

[0193] Furthermore, when the image analysis unit 583 identifies the object TG, it performs an enlargement process to magnify the image of the object TG in the first image PC1, generating an enlarged image PC12 (see reference). Figure 21 The image information of the magnified image PC12 generated by the image analysis unit 583 can also be stored in the storage unit 581. Furthermore, the magnification process performed by the image analysis unit 583 can also be referred to as digital zoom (electronic zoom). The image analysis unit 583 can also perform magnification processing that magnifies a portion of the image of the object TG.

[0194] The device control unit 584 controls the drive device 163 of the reflector rotation device 160. Based on at least one of the orientation information of the object TG generated by the image analysis unit 583 and the position information of the object TG in the first image PC1, the device control unit 584 controls the drive device 163 of the reflector rotation device 160, such that the image forming area of ​​the intermediate image Imd contains at least a portion of the image of the object TG. Specifically, the device control unit 584 uses a device control data table stored in the storage unit 581 to control the rotation angle of the first reflector 161 rotated by the drive device 163, based on at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1, such that the image forming area of ​​the intermediate image Imd contains at least a portion of the image of the object TG. As described above, the device control data table may also be a data table representing the relationship between at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1 and the rotation angle of the first reflector 161.

[0195] Therefore, based on the image information of the first image Im1 containing the image of the object TG (specifically, at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1), the measurement optical path of the measurement light LM from the distance measuring device 570 is set such that the measurement light LM is emitted from the imaging optical system 110 toward the object TG. In other words, the measurement optical path is set such that the measurement light LM passes through a portion of the image forming region of the intermediate image Imd containing at least a portion of the object TG. Specifically, a region of the image containing at least a portion of the object TG is selected from the image forming region of the intermediate image Imd. Furthermore, in other words, the measurement optical path of the measurement light LM from the distance measuring device 570 is set such that it overlaps with at least a portion of the optical path of the light forming the image of at least a portion of the object TG contained in the intermediate image Imd.

[0196] The device control unit 584 sets the measurement optical path such that at least a portion of the optical path of the light forming the image of the object TG overlaps with at least a portion of the measurement optical path. Thus, the measurement light LM from the distance measuring device 570 (measurement light illumination device (171)) illuminates the object TG via the second optical system 532, the first reflective member 161, the first optical system 531, the light splitting member 120, and the imaging optical system 110. The light from the object TG illuminated by the measured light LM arrives at the distance measuring device 570 (light receiving device (172)) in the reverse order of the measurement light LM from the distance measuring device 570. Therefore, the distance measuring device 570 can generate distance information of the object TG based on the light reception result of the object TG illuminated by the measured light LM. Furthermore, in other words, the measurement optical path of the measurement light LM is selected from multiple optical paths formed by the imaging optical system 110 based on at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1. Specifically, by using the drive device 163 to rotate the first reflective member 161, a measurement optical path is selected from a plurality of optical paths formed by the imaging optical system 110.

[0197] The output device 587 can output the orientation information of the object TG generated by the image analysis unit 583 and the distance information of the object TG acquired by the distance information acquisition unit 585 (i.e., the distance information of the object TG generated by the distance measurement device 570) to the irradiation control device 25 of the electromagnetic wave irradiation device 20. At this time, the irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the supporting device 23 based on the orientation information and distance information of the object TG output from the output device 587, so that electromagnetic waves LB are irradiated onto the object TG. Alternatively, when the distance information of the object TG is information related to the distance from the transmitting device 21 of the electromagnetic wave irradiation device 20 to the object TG, the output device 587 may not output the orientation information of the object TG. In this case, the irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the supporting device 23 based on the distance information of the object TG output from the output device 587, so that electromagnetic waves LB are irradiated onto the object TG. Therefore, the output device 587 can output at least one of the direction information and distance information of the target object TG to the irradiation control device 25 of the electromagnetic wave irradiation device 20. Based on at least one of the direction information and distance information of the target object TG output from the output device 587, the irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the supporting device 23 to irradiate the target object TG with electromagnetic waves LB.

[0198] Furthermore, the output device 587 can output image information of the magnified image PC12 generated by the image analysis unit 583 to the irradiation control device 25 of the electromagnetic wave irradiation device 20. The irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the supporting device 23 to irradiate electromagnetic waves LB onto the object TG based on the direction information and distance information of the object TG output from the output device 587 and the image information of the magnified image PC12. Additionally, the irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the supporting device 23 to irradiate electromagnetic waves LB onto the object TG based on the distance information of the object TG output from the output device 587 and the image information of the magnified image PC12. The irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the supporting device 23 to irradiate electromagnetic waves LB onto the object TG based on at least one of the direction information and distance information of the object TG output from the output device 587 and the image information of the magnified image PC12.

[0199] Here, we describe the situation where the positional relationship between the object TG and the imaging system 500 changes. The image information of the first image Im1 captured by the imaging device 540 before the positional relationship between the object TG and the imaging system 500 changes is referred to as the first image information. Additionally, the positional information of the object TG in the first image PC1 generated by the image analysis unit 583 is referred to as the first position information. The orientation information of the object TG generated by the image analysis unit 583 is referred to as the first orientation information. The distance information of the object TG generated by the distance measuring device 570 is referred to as the first distance information. The image information of the magnified image PC12 generated by the image analysis unit 583 is referred to as the third image information. After the positional relationship between the object TG and the imaging system 500 changes, the position of the image of the object TG contained in the first image Im1 changes, and the position of the image of the object TG contained in the intermediate image Imd also changes. The imaging device 540 captures the first image Im1 after the position of the image of the object TG changes, and outputs the second image information of the first image Im1 to the control device 580.

[0200] The image information acquisition unit 582 of the control device 580 acquires the second image information of the first image Im1 output from the imaging device 540. The image analysis unit 583 generates second position information of the object TG based on the second image information of the first image Im1 acquired by the image information acquisition unit 582. Based on the generated second position information of the object TG, the image analysis unit 583 generates second orientation information of the object TG. Furthermore, based on the second image information of the first image Im1, the image analysis unit 583 generates fourth image information of the magnified image PC12.

[0201] Based on the second direction information of the object TG generated by the image analysis unit 583, the device control unit 584 controls the drive device 163 of the reflector rotation device 160 so that the measurement light LM passes through a portion of the image area containing the image of the object TG in the image forming area of ​​the intermediate image Imd. Therefore, even if the positional relationship between the object TG and the imaging system 500 changes, the measurement light LM from the distance measuring device 570 (measurement light illumination device (171)) will still illuminate the object TG via the second optical system 532, the first reflector 161, the first optical system 531, the light splitting member 120, and the imaging optical system 110. The light from the object TG illuminated by the measured light LM arrives at the distance measuring device 570 (light receiving device (172)) in the reverse order of the measurement light LM from the distance measuring device 570. Therefore, the distance measuring device 570 can generate second distance information of the object TG based on the light receiving result of the light from the object TG illuminated by the measured light LM.

[0202] The output device 587 can output the second direction information of the object TG generated by the image analysis unit 583 and the second distance information of the object TG acquired by the distance information acquisition unit 585 (i.e., the second distance information of the object TG generated by the distance measurement device 570) to the irradiation control device 25 of the electromagnetic wave irradiation device 20. At this time, the irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the supporting device 23 to continuously or intermittently irradiate the object TG with electromagnetic waves LB, based on the direction change information generated based on the first direction information and the second direction information output from the output device 587 and the distance change information generated based on the first distance information and the second distance information output from the output device 587.

[0203] Furthermore, when the second distance information of the target object TG is information related to the distance from the transmitting device 21 of the electromagnetic wave irradiation device 20 to the target object TG, the output device 587 may not output the second direction information of the target object TG. In this case, the irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the support device 23 based on the distance change information, so that electromagnetic waves LB are continuously or intermittently irradiated onto the target object TG. Thus, the output device 587 can output at least one of the second direction information and the second distance information of the target object TG to the irradiation control device 25 of the electromagnetic wave irradiation device 20. The irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the support device 23 based on at least one of the direction change information and the distance change information, so that electromagnetic waves LB are continuously or intermittently irradiated onto the target object TG. Furthermore, the output device 587 can generate direction change information and output it to the irradiation control device 25 of the electromagnetic wave irradiation device 20, and it can also generate distance change information and output it to the irradiation control device 25 of the electromagnetic wave irradiation device 20.

[0204] Furthermore, the output device 587 can output the fourth image information of the magnified image PC12 generated by the image analysis unit 583 to the irradiation control device 25 of the electromagnetic wave irradiation device 20. The irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the supporting device 23 based on direction change information, distance change information, and image change information generated based on the third and fourth image information output from the output device 587, so that electromagnetic waves LB are continuously or intermittently irradiated onto the target object TG. Additionally, the irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the supporting device 23 based on distance change information and image change information, so that electromagnetic waves LB are continuously or intermittently irradiated onto the target object TG. The irradiation control device 25 of the electromagnetic wave irradiation device 20 can control at least one of the transmitting device 21 and the supporting device 23 based on at least one of direction change information, distance change information, and image change information, so that electromagnetic waves LB are continuously or intermittently irradiated onto the target object TG. In addition, the output device 587 can generate image change information and output it to the irradiation control device 25 of the electromagnetic wave irradiation device 20.

[0205] Additionally, the output device 587 may also output at least one of the generated image information of the first image Im1, the image information of the magnified image PC12, the orientation information of the object TG, and the distance information of the object TG to a display device (not shown). The imaging system 500 may also include this display device. This display device can display at least one of the following based on the information output from the output device 587: the image represented by the image information of the first image Im1 (first image PC1), the magnified image PC12, the direction represented by the orientation information of the object TG (e.g., the direction of the object TG relative to at least one of the imaging system 500 and the electromagnetic wave irradiation device 20), and the distance represented by the distance information of the object TG (e.g., the distance of the object TG relative to at least one of the imaging system 500 and the electromagnetic wave irradiation device 20). The imaging system 500 may also include this display device. Furthermore, the image represented by the image information of the first image Im1 output to the display device (first image PC1) may also include an image of at least a portion of the object TG. Additionally, the magnified image PC12 output to the display device may also include an image of at least a portion of the object TG.

[0206] [Measurement Method]

[0207] Next, the measurement method using the imaging system 500 according to Embodiment 4 will be summarized. Figure 20 This is a flowchart illustrating the measurement method according to Embodiment 4. Furthermore, each process in the measurement method is executed based on a program stored in the storage unit 581. For example... Figure 20 As shown, firstly, the imaging system 500 acquires the orientation information of the object TG (step ST110). At this time, the imaging device 540 captures a first image Im1 containing at least a portion of the object TG, and outputs the image information of the generated first image Im1 to the control device 580. The image information acquisition unit 582 acquires the image information of the first image Im1 output from the imaging device 540.

[0208] As described above, the image analysis unit 583 generates position information of the object TG in the first image PC1 (the image represented by the image information of the first image Im1) based on the image information of the first image Im1, which is acquired by the image information acquisition unit 582 and contains at least a portion of the object TG. Based on the generated position information of the object TG in the first image PC1, the image analysis unit 583 generates orientation information of the object TG.

[0209] Next, the imaging system 500 acquires image information of the magnified image (step ST120). At this time, the image analysis unit 583 generates image information of the magnified image PC12 based on the image information of the first image Im1, which contains at least a portion of the object TG. Alternatively, in step ST110, in order to control the rotation angle of the first reflecting member 161, step ST110 can also be performed between steps ST120 and ST140 (described later) without using the orientation information of the object TG. In this case, in step ST120, the imaging device 540 captures the first image Im1, which contains at least a portion of the object TG, and the image analysis unit 583 generates position information of the object TG in the first image PC1 based on the image information of the first image Im1, which contains at least a portion of the object TG.

[0210] Next, the imaging system 500 acquires distance information of the target object TG (step ST130). At this time, the device control unit 584, based on the orientation information of the target object TG generated by the image analysis unit 583, controls the drive device 163 of the reflector rotation device 160, causing the measuring light LM to pass through a portion of the image forming area of ​​the intermediate image Imd containing the image of the target object TG. The distance measuring device 570 illuminates the measuring light LM onto the target object TG via the second optical system 531, the first reflector 161, the first optical system 531, the light splitting member 120, and the imaging optical system 110, and generates distance information of the target object TG. The distance measuring device 570 outputs the generated distance information of the target object TG to the distance information acquisition unit 585. The distance information acquisition unit 585 acquires the distance information of the target object TG output from the distance measuring device 570. Alternatively, step ST130 can also be performed between steps ST110 and ST120.

[0211] Then, the imaging system 500 outputs information (step ST140). At this time, the output device 587 can output at least one of the direction information of the object TG generated by the image analysis unit 583 and the distance information of the object TG acquired by the distance information acquisition unit 585 (i.e., the distance information of the object TG generated by the distance measurement device 570) to the irradiation control device 25 of the electromagnetic wave irradiation device 20. At this time, the irradiation control device 25 can control at least one of the transmitting device 21 and the supporting device 23 based on at least one of the direction information and distance information of the object TG output from the output device 587, so that electromagnetic waves LB are irradiated onto the object TG. In addition, the output device 587 can output the image information of the magnified image PC12 generated by the image analysis unit 583 to the irradiation control device 25. At this time, the irradiation control device 25 can control at least one of the direction information and distance information of the object TG output from the output device 587 and the image information of the magnified image PC12, so that electromagnetic waves LB are irradiated onto the object TG.

[0212] Alternatively, steps ST110 to ST140 can be repeated. For example, when the positional relationship between the object TG and the imaging system 500 changes, by repeating steps ST110 to ST140, as described above, the distance measuring device 570 can continuously or intermittently irradiate the measuring light LM onto the object TG whose positional relationship has changed. Furthermore, by repeating steps ST110 to ST140, as described above, each time based on at least one of the direction information of the object TG, the distance information of the object TG, and the image information of the magnified image PC12 output in ST140, the irradiation control unit 25 emits electromagnetic waves LB onto the object TG, thereby allowing the electromagnetic wave irradiation device 20 to continuously or intermittently irradiate electromagnetic waves LB onto the object TG whose positional relationship has changed. Additionally, even if the positional relationship between the object TG and the imaging system 500 does not change, steps ST110 to ST140 can be repeated.

[0213] According to embodiment 4, the imaging system 500 includes: a light splitting member 120 for splitting the light LT passing through the imaging optical system 110; an imaging device 540 for capturing a first image Im1 formed by a beam of light split by the light splitting member 120; and a distance measuring device 570, which, based on image information of the first image Im1 containing an image of the object captured by the imaging device 540, illuminates a measuring light LM onto the object via the imaging optical system 110 and generates distance information of the object. Thus, while capturing and following the object by the imaging device 540, the distance to the object can be measured by the distance measuring device 570.

[0214] Furthermore, the system includes a reflective member rotation device 160, which has a first reflective member 161 having a reflective surface 162 that reflects the measurement light LM from the distance measuring device 570 toward the imaging optical system 110; and a driving device 163 that rotates the first reflective member 161. Thus, by rotating the first reflective member 161 via the driving device 163, the measurement light LM from the distance measuring device 570 can be directed toward the object in a short time. Therefore, the distance to the object can be measured using the distance measuring device 570 in a short time. Furthermore, using the distance information obtained by the distance measuring device 570 in a short time, the three-dimensional position of the object can be measured in a short time, thus allowing the object to be irradiated with electromagnetic waves LB in a short time.

[0215] In addition, since there is no need to set up a second shooting device, the manufacturing cost can be reduced.

[0216] [Modification of Implementation Method 4]

[0217] In the above-described embodiment 4, a reflective member rotation device 160 is provided, but the system is not limited thereto. Therefore, a modified example of the imaging system according to embodiment 4 will be described. In the imaging system according to the modified example of embodiment 4, apart from the structure of the imaging device 640, holding part 660, driving part 665, distance measuring device 670, control device 680, and frame 601, its main parts have a structure common to the imaging system 100 according to embodiment 1. Therefore, for structures the same as those in embodiment 1, the same reference numerals as in embodiment 1 are used, and detailed descriptions are omitted. Figure 22 As shown, the imaging system 600 involved in the modified example of Embodiment 4 includes an imaging optical system 110, a light splitting member 120, an imaging device 640, a holding part 660, a driving part 665, a distance measuring device 670, a control device 680, and a frame 601.

[0218] The frame 601 houses and holds the imaging optical system 110, the light splitting member 120, the imaging device 640, the holding part 660 (distance measuring device 670), and the driving part 665. The imaging optical system 110 and the light splitting member 120 have the same structure as those of the imaging optical system 110 and the light splitting member 120 according to Embodiment 1, and detailed description is omitted. In addition, in this variation, the light splitting member 120 is disposed between the imaging optical system 110 and the distance measuring device 670. In this variation, the light splitting member 120 may also be disposed in the middle of the imaging optical system 110, in which case the light splitting member 120 can split the light LT passing through a part of the imaging optical system 110. The light splitting member 120 may be a part of the imaging optical system 110. For example, the rear lens 112 may be disposed between the light splitting member 112 and the first image plane Im1, and between the light splitting member 120 and the intermediate image plane Imd.

[0219] In this modified example, the direction along the optical axis AX11 of the imaging optical system 110 is referred to as the Y direction. The direction opposite to the imaging device 640 is referred to as the Z direction. The direction orthogonal to the Y and Z directions is referred to as the X direction. The X, Y, and Z directions are mutually orthogonal. For example, as Figure 1 As shown, the Z direction can be vertical, while the X and Y directions can be horizontal.

[0220] like Figure 22 As shown, the imaging device 640 includes an imaging element 641 for capturing the first image Im1. The imaging device 640 and the imaging element 641 are configured in the same way as the imaging device 540 and the imaging element 541 according to Embodiment 4.

[0221] like Figure 22 As shown, the distance measuring device 670 is configured similarly to the distance measuring device 570 according to Embodiment 4, except that it is held by the holding part 660. The distance measuring device 670 is positioned at a location where the measuring light LM can be irradiated onto the light splitting member 120. Alternatively, the distance measuring device 670 can also be positioned at a location where an intermediate image Imd is formed. The distance measuring device 670, via the light splitting member 120 and the imaging optical system 110, irradiates the target object TG (see reference 120). Figure 1 Illuminate the measurement light LM and generate distance information of the object TG.

[0222] like Figure 22As shown, the holding part 660 holds the distance measuring device 670. The driving part 665 is configured, for example, using a linear motor or a stepper motor. The driving part 665 is capable of moving the holding part 660 of the distance measuring device 670 in two directions (e.g., the X direction and the Z direction) that intersect the optical axis AX11 of the imaging optical system 110. Driven by the driving part 665, the distance measuring device 670 moves in the X and Z directions as an example, thereby changing the illumination direction of the measuring light LM that is irradiated from the distance measuring device 670 via the imaging optical system 110.

[0223] The control device 680 is configured, for example, using a PC (personal computer). The control device 680 operates based on a program stored in the storage unit 681. Figure 22 As shown, the control device 680 includes a storage unit 681, an image information acquisition unit 682, an image analysis unit 683, a device control unit 684, a distance information acquisition unit 685, and an output device 687. The storage unit 681, image information acquisition unit 682, image analysis unit 683, distance information acquisition unit 685, and output device 687 are configured similarly to the storage unit 581, image information acquisition unit 582, image analysis unit 583, distance information acquisition unit 585, and output device 587 according to Embodiment 4.

[0224] The device control unit 684 controls the drive unit 665. Based on at least one of the orientation information of the object TG generated by the image analysis unit 683 and the position information of the object TG in the first image PC1, the device control unit 684 controls the drive unit 665 such that the image forming area of ​​the intermediate image Imd contains a portion of the image of the object TG. Specifically, the device control unit 684 uses a device control data table stored in the storage unit 681 to control the drive unit 665 based on at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1, such that the image forming area of ​​the intermediate image Imd contains at least a portion of the image of the object TG. The device control data table may also be a data table representing the relationship between at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1 and the position of the distance measuring device 670.

[0225] Therefore, based on the image information of the first image Im1 containing the image of the object TG (specifically, at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1), the measurement optical path of the measurement light LM from the distance measuring device 670 is set such that the measurement light LM is emitted from the imaging optical system 110 toward the object TG. In other words, the measurement optical path is set such that the measurement light LM passes through the region of the image forming area of ​​the intermediate image Imd that contains at least a portion of the image of the object TG. Specifically, a portion of the image containing at least a portion of the object TG is selected from the image forming area of ​​the intermediate image Imd. Furthermore, in other words, the measurement optical path of the measurement light LM from the distance measuring device 670 is set such that it overlaps with at least a portion of the optical path of the light forming the image of at least a portion of the object TG contained in the intermediate image Imd.

[0226] The measurement optical path is set by the device control unit 684 such that at least a portion of the optical path of the light forming the image of the object TG overlaps with at least a portion of the measurement optical path, thereby illuminating the object TG with the measurement light LM from the distance measuring device 670 (measurement light illumination device (171)) via the light splitting member 120 and the imaging optical system 110. The light from the object TG illuminated by the measured light LM arrives at the distance measuring device 670 (light receiving device (172)) in the reverse order of the measurement light LM from the distance measuring device 670. Therefore, the distance measuring device 670 can generate distance information of the object TG based on the light receiving result of the object TG illuminated by the measured light LM. In other words, the measurement optical path of the measurement light LM is selected from multiple optical paths formed by the imaging optical system 110 based on at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1.

[0227] Furthermore, the light-receiving device (172) of the distance measuring device 670 can also be located at a different position from the measuring light irradiation device (171). For example, as long as the light-receiving device (172) can receive light from the object TG irradiated by the measured light LM, it does not need to be coaxially configured with the optical axis of the imaging system 600. For example, the light-receiving device (172) can be provided on the outer surface of the frame 601 or on the aforementioned movable body.

[0228] [Measurement Method]

[0229] Next, the measurement method of the imaging system 600 according to the modified example of Embodiment 4 will be summarized. The measurement method according to this modified example is the same as that according to Embodiment 4. Therefore, the same measurement method as that according to Embodiment 4 is used. Figure 20The flowchart shown will be used for explanation. Furthermore, each process in the measurement method is executed based on a program stored in the storage unit 681 of the control device 680. First, the imaging system 600, similar to the case in Embodiment 4, acquires the orientation information of the target object TG (step ST110).

[0230] Next, the imaging system 600 acquires image information of the magnified image in the same manner as in Embodiment 4 (step ST120). In addition, in step ST110, if the orientation information of the object TG is not used, step ST110 can also be performed between steps ST120 and ST140 (described later).

[0231] Next, the imaging system 600 acquires distance information of the target object TG (step ST130). At this time, the device control unit 684 controls the drive unit 665 based on at least one of the orientation information of the target object TG generated by the image analysis unit 683 and the position information of the target object TG in the first image PC1, so that the measuring light LM passes through a portion of the image forming area of ​​the intermediate image Imd containing the image of the target object TG. The distance measuring device 670 illuminates the measuring light LM onto the target object TG via the light splitting member 120 and the imaging optical system 110, and generates distance information of the target object TG. The distance measuring device 670 outputs the generated distance information of the target object TG to the distance information acquisition unit 685. The distance information acquisition unit 685 acquires the distance information of the target object TG output from the distance measuring device 670. Alternatively, step ST130 can also be performed between steps ST110 and ST120.

[0232] Then, the imaging system 600 outputs information in the same manner as in Embodiment 4 (step ST140). Alternatively, steps ST110 to ST140 described above can be repeated. Furthermore, the processing order of step ST110, which acquires the TG direction information of the object, and step ST120, which acquires the image information, can also be interchanged.

[0233] According to this modification, the imaging system 600 includes: a light splitting member 120 that splits the light LT passing through the imaging optical system 110; an imaging device 640 that captures a first image Im1 formed by a beam of light split by the light splitting member 120; and a distance measuring device 670 that, based on image information of the first image Im1 containing an image of the object captured by the imaging device 640, illuminates a measuring light LM onto the object via the imaging optical system 110 and generates distance information of the object TG. Thus, while capturing and following the object by the imaging device 640, the distance to the object can be measured by the distance measuring device 670.

[0234] [Implementation Method 5]

[0235] Next, the shooting system according to Embodiment 5 will be described. In the shooting system according to Embodiment 5, apart from the structures of the shooting device 740, distance measuring device 770, control device 780, and frame 701, its main parts have the same structure as the shooting system 100 according to Embodiment 1. Therefore, for structures identical to those in Embodiment 1, the same reference numerals as in Embodiment 1 are used, and detailed descriptions are omitted. Figure 23 As shown, the imaging system 700 according to Embodiment 5 includes an imaging optical system 110, a light splitting member 120, a first optical system 731, a second optical system 732, an imaging device 740, a reflective member rotation device 160, a distance measuring device 770, a control device 780, and a frame 701.

[0236] The frame 701 houses and holds the imaging optical system 110, the light splitting member 120, the first optical system 731, the second optical system 732, the imaging device 740, the reflector rotating device 160, and the distance measuring device 770. The imaging optical system 110, the light splitting member 120, and the reflector rotating device 160 have the same structure as those described in Embodiment 1, therefore detailed descriptions are omitted. Furthermore, in the reflector rotating device 160, the reflective surface 162 of the first reflector 161 reflects the measuring light LM from the distance measuring device 770 after the second optical system 732 towards the first optical system 631 (imaging optical system 110). Alternatively, the imaging system 700 may not include the second optical system 732. In this case, the distance measuring device 770 may be positioned closer to the first reflector 161 than the position of the second optical system 732. In this case, the imaging system 700 (frame 701) can be further miniaturized. Alternatively, the imaging system 700 may include an optical system for converting (collimating) the measurement light LM from the distance measuring device 770 into parallel light, instead of the second optical system 732. Furthermore, the light-receiving device (172) of the distance measuring device 770 may be positioned differently from the measurement light illumination device (171). For example, a semi-reflective mirror may be provided in the optical path of the second optical system 732. The light-receiving device (172) may also be positioned offset from the semi-reflective mirror in a direction (e.g., the Y direction) intersecting the optical axis (optical axis AX2) of the second optical system 732. In this case, the semi-reflective mirror allows a portion of the measurement light LM emitted from the measurement light illumination device (171) to pass through, while reflecting the remaining light along the -Y direction. The semi-reflective mirror allows a portion of the light from the object illuminated by the measurement light LM to be reflected towards the light-receiving device along the +Y direction, while allowing the remaining light to pass through. Furthermore, in this case, the configurations of the measuring light irradiation device (171) and the light receiving device (172) can be interchanged. Additionally, as long as the light receiving device (172) can receive light from the object TG illuminated by the measured light LM, it does not need to be coaxially configured with the optical axis of the imaging system 700 (e.g., the optical axis of the second optical system 732). For example, the light receiving device (172) can be provided on the outer surface of the frame 701 or on the aforementioned movable body.

[0237] In Embodiment 5, similar to Embodiment 4, the optical axis of the first optical system 731 and the optical axis of the imaging optical system 110 coaxial with the first optical system 731 are referred to as the first optical axis AX1. The optical axis of the second optical system 732 is referred to as the second optical axis AX2. Furthermore, the direction along the first optical axis AX1 is referred to as the Y direction. The direction along the second optical axis AX2 is referred to as the Z direction. The direction orthogonal to the Y direction and the Z direction is referred to as the X direction.

[0238] like Figure 23 As shown, the first optical system 731 and the second optical system 732 are configured in the same way as the first optical system 531 and the second optical system 532 according to Embodiment 4. The imaging device 740 includes an imaging element 741 for capturing the first image Im1. The imaging device 740 and the imaging element 741 are configured in the same way as the imaging device 540 and the imaging element 541 according to Embodiment 4. The distance measuring device 770 may also be configured in the same way as the distance measuring device 570 according to Embodiment 4.

[0239] The control device 780 is configured, for example, using a PC (personal computer). The control device 780 operates based on a program stored in the storage unit 781. Figure 23 As shown, the control device 780 includes a storage unit 781, an image information acquisition unit 782, an image analysis unit 783, a device control unit 784, a distance information acquisition unit 785, and an output device 787. The storage unit 781, the image information acquisition unit 782, the distance information acquisition unit 785, and the output device 787 are configured similarly to the storage unit 581, the image information acquisition unit 582, the distance information acquisition unit 585, and the output device 587 according to Embodiment 4.

[0240] The image analysis unit 783 generates orientation information of the object TG based on image information of a first image Im1, which is acquired by the image information acquisition unit 782 and contains at least a portion of the object TG. Similar to the image analysis unit 183 described in Embodiment 1, the image analysis unit 783 identifies the object TG from a first image PC1 containing one or more objects by analyzing the image information of the first image Im1. When identifying the object TG, the image analysis unit 783 sets a region of interest (ROI) in the first image PC1 (refer to...). Figure 25The image analysis unit 783 sets multiple measurement points MP for the object TG in the region of interest (ROI) at a predetermined measurement interval. When setting the ROI, the image analysis unit 783 generates position information of the object TG in the first image PC1 for the multiple measurement points MP, similar to the image analysis unit 183 in Embodiment 1. Based on the generated position information of the object TG in the first image PC1, the image analysis unit 783 also generates orientation information of the object TG for the multiple measurement points MP, similar to the image analysis unit 183 in Embodiment 1.

[0241] Furthermore, when setting the region of interest (ROI), the image analysis unit 783 performs magnification processing on a portion of the ROI in the first image PC1 (i.e., an image containing at least a portion of the object TG), generating image information of a magnified image of the object TG after magnifying the portion of the ROI in the first image PC1. The image information of the magnified image generated by the image analysis unit 783 can also be stored in the storage unit 781. Moreover, the magnification processing performed by the image analysis unit 783 can also be referred to as digital zoom (electronic zoom).

[0242] The device control unit 784 controls the drive device 163 of the reflector rotation device 160. Based on at least one of the orientation information of the object TG generated by the image analysis unit 783 and the position information of the object TG in the first image PC1, the device control unit 784 controls the drive device 163 of the reflector rotation device 160, such that the image forming area of ​​the measurement light LM through the intermediate image Imd contains an image region containing at least a portion of the object TG. Specifically, the device control unit 784 uses a device control data table stored in the storage unit 781 to control the rotation position of the first reflector 161 rotated by the drive device 163 based on at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1, such that the image forming area of ​​the measurement light LM through the intermediate image Imd contains an image region containing at least a portion of the object TG. The device control data table may also be a data table representing the relationship between at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1 and the rotation position of the first reflector 161.

[0243] Therefore, based on the image information of the first image Im1 containing the image of the object TG (specifically, at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1), the measurement optical path of the measurement light LM from the distance measuring device 770 is set such that the measurement light LM is emitted from the imaging optical system 110 toward the object TG. In other words, the measurement optical path is set such that the measurement light TG passes through a portion of the image forming region of the intermediate image Imd containing at least a portion of the object TG. Specifically, a portion of the image forming region containing at least a portion of the object TG is selected from the image forming region of the intermediate image Imd. Furthermore, in other words, the measurement optical path of the measurement light LM from the distance measuring device 770 is set such that it overlaps with at least a portion of the optical path of the light forming the image of at least a portion of the object TG contained in the intermediate image Imd.

[0244] The device control unit 784 sets the measurement optical path so that at least a portion of the optical path of the light forming the image of the object TG overlaps with at least a portion of the measurement optical path. Thus, the measurement light LM from the distance measuring device 770 (measurement light illumination device (171)) illuminates the object TG via the second optical system 732, the first reflective member 161, the first optical system 731, the light splitting member 120, and the imaging optical system 110. The light from the object TG illuminated by the measured light LM arrives at the distance measuring device 770 (light receiving device (172)) in the reverse order of the measurement light LM from the distance measuring device 770. Therefore, the distance measuring device 770 can generate distance information of the object TG based on the light received from the object TG illuminated by the measured light LM. Furthermore, the measurement optical path of the measurement light LM is selected from multiple optical paths formed by the imaging optical system 110 based on the orientation information of the object TG. Specifically, by using the drive device 163 to rotate the first reflective member 161, a measurement optical path is selected from a plurality of optical paths formed by the imaging optical system 110.

[0245] Here, the device control unit 784 controls the drive device 163 to rotate the first reflective member 161 based on the orientation information of the object TG generated from multiple measurement points MP for the region of interest (ROI). This causes the measurement light LM from the distance measuring device 770 to sequentially illuminate the multiple measurement points MP on the object TG. As a result, the distance measuring device 770 can sequentially generate distance information for the object TG from the multiple measurement points MP. In this case, distance information for multiple points (points corresponding to the multiple measurement points MP) of the object TG can be generated. The distance information for the multiple points of the object TG represents information indicating at least a portion of the shape of the object TG. Therefore, the distance information of the object TG (specifically, the distance information for the multiple points of the object TG) can also be considered the shape information of the object TG. Furthermore, the orientation information of the object TG generated from the multiple measurement points MP for the region of interest (ROI) can also be referred to as the orientation information of the measurement points MP.

[0246] [Measurement Method]

[0247] Next, the measurement method using the imaging system 700 according to Embodiment 5 will be summarized. Figure 24 This is a flowchart illustrating the measurement method according to Embodiment 5. Furthermore, each process in the measurement method is executed based on a program stored in the storage unit 781. For example... Figure 24 As shown, firstly, the imaging system 700 sets the region of interest (ROI) and acquires the orientation information of the object TG (step ST210). At this time, the imaging device 740 captures a first image Im1 containing at least a portion of the object TG and outputs the image information of the generated first image Im1 to the control device 780. The image information acquisition unit 782 acquires the image information of the first image Im1 output from the imaging device 740.

[0248] As described above, the image analysis unit 783 sets a region of interest (ROI) based on image information of a first image Im1, which contains at least a portion of the object TG, acquired by the image information acquisition unit 782. If the ROI is set, the image analysis unit 783 generates position information of the object TG in the first image PC1 for multiple measurement points MP. Based on the generated position information of the object TG in the first image PC1, the image analysis unit 783 generates orientation information of the object TG for the multiple measurement points MP.

[0249] Next, the imaging system 700 acquires image information of the magnified image (step ST220). At this time, the image analysis unit 783 generates image information of the magnified image of the object TG after magnifying a portion of the region of interest (ROI) based on the image information of the first image Im1 containing the image of the object TG.

[0250] Next, the imaging system 700 acquires distance information of the target object TG (step ST230). At this time, the device control unit 784, based on the orientation information of the target object TG generated by the image analysis unit 783, controls the drive device 163 of the reflector rotation device 160, so that the measuring light LM passes through a portion of the image forming area of ​​the intermediate image Imd containing the image of the target object TG. The distance measuring device 770 illuminates the measuring light LM onto the target object TG via the second optical system 732, the first reflector 161, the first optical system 731, the light splitting member 120, and the imaging optical system 110, and generates distance information of the target object TG.

[0251] Additionally, at this time, the device control unit 784 controls the drive device 163 to rotate the first reflective member 161 based on the orientation information of the object TG generated from multiple measurement points MP for the region of interest (ROI), causing the measurement light LM from the distance measuring device 770 to sequentially illuminate the multiple measurement points MP on the object TG. As a result, the distance measuring device 770 sequentially generates distance information for the object TG for the multiple measurement points MP. The distance measuring device 770 outputs the generated distance information of the object TG to the distance information acquisition unit 785. The distance information acquisition unit 785 acquires the distance information of the object TG output from the distance measuring device 770. Furthermore, step ST230 can also be performed between steps ST210 and ST220.

[0252] Then, the imaging system 700 outputs information (step ST240). At this time, the output device 787 can output at least one of the direction information of the object TG for multiple measurement points MP generated by the image analysis unit 783 and the distance information of the object TG for multiple measurement points MP acquired by the distance information acquisition unit 785 (i.e., the distance information of the object TG generated by the distance measurement device 770) to the illumination control device 25 of the electromagnetic wave illumination device 20. At this time, the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 based on at least one of the direction information and distance information of the object TG for multiple measurement points MP output from the output device 787, so that electromagnetic waves LB are irradiated onto the object TG. In addition, the output device 787 can output the image information of the magnified image generated by the image analysis unit 783 to the illumination control device 25. At this time, the illumination control device 25 can control at least one of the transmitting device 21 and the supporting device 23 to irradiate electromagnetic waves LB onto the object TG based on at least one of the direction information and distance information of the object TG for multiple measurement points MP output from the output device 787, as well as the image information of the magnified image.

[0253] Alternatively, steps ST110 to ST140 can be repeated in the same manner as in Embodiment 4. Furthermore, in step ST140, the output device 787 can output information related to multiple measurement points MP of the region of interest (ROI) instead of the orientation information of the object TG.

[0254] According to embodiment 5, the same effect as that of embodiment 4 can be obtained.

[0255] In the above-described embodiment 5, a reflective member rotation device 160 is provided, but it is not limited to this. For example, similar to the imaging system involved in the modified example of embodiment 4, a holding part of the holding distance measuring device 770 and a drive part that moves the holding part parallel to each other in two directions (e.g., the X direction and the Z direction) that intersect the optical axis of the imaging optical system 110 may also be provided.

[0256] [Implementation Method 6]

[0257] Next, the imaging system according to Embodiment 6 will be described. First, using... Figure 26 The electromagnetic wave irradiation system, including the imaging systems described in Embodiments 6 and 7, will be described. For example... Figure 26 As shown, the electromagnetic wave irradiation system 51 includes an imaging system 800, a housing member 60 housing the imaging system 800, and a rotation drive device 61. The electromagnetic wave irradiation system 51 can be fixed to the ground or to a building or similar structure. Furthermore, the electromagnetic wave irradiation system 51 can also be mounted on the aforementioned movable body. When the electromagnetic wave irradiation system 51 is mounted on a movable body, the rotation drive device 61 may not be included. Additionally, the electromagnetic wave irradiation system 51 may also omit the housing member 60.

[0258] The housing member 60 is box-shaped. At least a portion of the imaging system 800 is housed inside the housing member 60. A rotation drive device 61 rotatably supports the housing member 60. The rotation drive device 61 enables the housing member 60 to rotate about a rotation axis extending in the vertical direction (Z direction). The imaging system 800 is capable of irradiating electromagnetic waves LB in the same way as the electromagnetic wave irradiation device 20 described above. Furthermore, the electromagnetic waves LB can be the same as those described in Embodiment 1. Additionally, the imaging system 800 can also be mounted on the outside of the housing member 60, such as on its side or top. Next, the imaging system 800 according to Embodiment 6 will be described.

[0259] [The imaging system described in Implementation Method 6]

[0260] Next, the imaging system according to Embodiment 6 will be described. In the imaging system according to Embodiment 6, apart from the structures of the electromagnetic wave irradiation device 865, the second reflective member 868, the distance measuring device 870, the control device 880, and the frame 801, its main parts have a structure common to the imaging system 100 according to Embodiment 1. Therefore, for structures identical to those in Embodiment 1, the same reference numerals as in Embodiment 1 are used, and detailed descriptions are omitted. Figure 27 As shown, the imaging system 800 according to Embodiment 6 includes an imaging optical system 110, a light splitting member 120, a re-imaging optical system 130, a first imaging device 140, a second imaging device 150, a reflective member rotation device 160, an electromagnetic wave irradiation device 865, a second reflective member 868, a distance measuring device 870, a control device 880, and a frame 801.

[0261] The frame 801 houses and holds the imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the second imaging device 150, the reflector rotation device 160, the electromagnetic wave irradiation device 865, the second reflector 868, and the distance measuring device 870. The imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the second imaging device 150, and the reflector rotation device 160 have the same structure as those of the imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the second imaging device 150, and the reflector rotation device 160 according to Embodiment 1, and detailed descriptions are omitted. In addition, in the reflective member rotation device 160, the reflective surface 162 of the first reflective member 161 reflects the electromagnetic wave LB from the electromagnetic wave irradiation device 865 after passing through the second optical system 132 (re-imaging optical system 130) toward the first optical system 131 (imaging optical system 110).

[0262] In Embodiment 6, similar to Embodiment 1, the optical axis of the first optical system 131 in the re-imaging optical system 130 and the optical axis of the imaging optical system 110 coaxial with the first optical system 131 are referred to as the first optical axis AX1. The optical axis of the second optical system 132 in the re-imaging optical system 130 is referred to as the second optical axis AX2. Furthermore, the direction along the first optical axis AX1 is referred to as the Y direction. The direction along the second optical axis AX2 is referred to as the Z direction. The direction orthogonal to the Y direction and the Z direction is referred to as the X direction.

[0263] like Figure 27As shown, the electromagnetic wave irradiation device 865 is positioned on the opposite side of the optical component in the second optical system 132 (re-imaging optical system 130) closest to the second imaging element 151, across the second imaging element 151. The electromagnetic wave irradiation device 865 projects light onto the target object TG (refer to the image) via the non-imaging area 154 (through-hole 155) of the second imaging element 151, the second reflective component 868, the second optical system 132, the first reflective component 161, the first optical system 131, the light splitting component 120, and the imaging optical system 110. Figure 1 The electromagnetic wave LB is irradiated. For example, the electromagnetic wave irradiation device 865 may include an optical fiber (not shown) that emits the electromagnetic wave LB from the laser source. That is, the electromagnetic wave LB can also be a laser. In addition, the laser source can be located outside the imaging system 800 or inside the imaging system 800.

[0264] like Figure 27 As shown, the second reflective member 868 is disposed on at least a portion of the optical path of the re-imaging optical system 130 (second optical system 132) between the re-imaging optical system 130 and the second imaging element 151. The second reflective member 868 is configured similarly to the second reflective member 368 according to Embodiment 2. Furthermore, when the second optical system 132 of the re-imaging optical system 130 is composed of two or more lenses, the second reflective member 868 may also be disposed in the middle portion of the second optical system 132. Thus, the second reflective member 868 can be disposed on at least a portion of the optical path of the re-imaging optical system 130.

[0265] like Figure 27 As shown, the distance measuring device 870 can be configured similarly to the distance measuring device 370 according to Embodiment 2. The distance measuring device 870 is offset from the second reflective member 868 in a direction (e.g., the Y direction) intersecting the optical axis (second optical axis AX2) of the second optical system 132 in the re-imaging optical system 130. The distance measuring device 870 illuminates the object TG with measuring light LM via the second reflective member 868, the second optical system 132, the first reflective member 161, the first optical system 131, the light splitting member 120, and the imaging optical system 110, and generates distance information of the object TG.

[0266] The control device 880 is configured, for example, using a PC (personal computer). The control device 880 operates based on a program stored in the storage unit 881. Figure 27As shown, the control device 880 includes a storage unit 881, an image information acquisition unit 882, an image analysis unit 883, a device control unit 884, a distance information acquisition unit 885, and an image correction unit 886. The storage unit 881, image information acquisition unit 882, image analysis unit 883, distance information acquisition unit 885, and image correction unit 886 are configured similarly to the storage unit 181, image information acquisition unit 182, image analysis unit 183, distance information acquisition unit 185, and image correction unit 186 according to Embodiment 1.

[0267] Similar to the device control unit 184 in Embodiment 1, the device control unit 884 controls the drive device 163 of the reflector rotation device 160. Based on at least one of the orientation information of the object TG generated by the image analysis unit 883 and the position information of the object TG in the first image PC1, the device control unit 884 controls the drive device 163 of the reflector rotation device 160, such that the electromagnetic wave LB passes through a portion of the image forming area of ​​the intermediate image Imd containing at least a portion of the object TG. Specifically, the device control unit 884 uses a device control data table stored in the storage unit 881 to control the rotation angle of the first reflector 161 rotated by the drive device 163, based on at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1, such that the electromagnetic wave LB passes through a portion of the image forming area of ​​the intermediate image Imd containing at least a portion of the object TG. Furthermore, the portion of the image containing at least a portion of the object TG can be any portion of the image containing at least a portion of the object TG. The data table for device control can also be a data table showing the relationship between at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1 and the rotation angle of the first reflective member 161.

[0268] Therefore, similarly to Embodiment 1, based on the image information of the first image Im1 containing at least a portion of the object TG (specifically, at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1), the illumination optical path through which the measurement light LM from the electromagnetic wave illumination device 865 passes is set, such that the electromagnetic wave LB is emitted from the imaging optical system 110 toward the object TG. In other words, the illumination optical path is set such that the electromagnetic wave LB passes through a portion of the image forming region of the intermediate image Imd containing at least a portion of the object TG. Specifically, a portion of the image containing at least a portion of the object TG is selected from the image forming region of the intermediate image Imd. Furthermore, in other words, the illumination optical path of the electromagnetic wave LB from the electromagnetic wave illumination device 865 is set such that it overlaps with at least a portion of the optical path of the light forming the image of at least a portion of the object TG contained in the intermediate image Imd.

[0269] The illumination light path is set by the device control unit 884 such that at least a portion of the light path forming an image of at least a part of the object TG overlaps with at least a portion of the illumination light path through which the electromagnetic wave LB passes. Thus, the electromagnetic wave LB from the electromagnetic wave illumination device 865 illuminates the object TG via the non-imaging area 154 (through-hole 155) of the second imaging element 151, the second reflective member 868, the second optical system 132, the first reflective member 161, the first optical system 131, the light splitting member 120, and the imaging optical system 110. Furthermore, the illumination light path is selected from multiple light paths formed by the imaging optical system 110 based on at least one of the orientation information of the object TG and the position information of the object TG in the first image PC1. Specifically, the first reflective member 161 is rotated by the drive device 163, thereby selecting the illumination light path from the multiple light paths formed by the imaging optical system 110. Furthermore, the measuring light LM from the distance measuring device 870 (measuring light illumination device (171)) illuminates the object TG via the second reflector 868, the second optical system 132, the first reflector 161, the first optical system 131, the light splitter 120, and the imaging optical system 110. The light from the object TG illuminated by the measured light LM arrives at the distance measuring device 870 (light receiving device (172)) in the reverse order of the measuring light LM from the distance measuring device 870. Therefore, the distance measuring device 870 can generate distance information of the object TG based on the light receiving result of the light from the object TG illuminated by the measured light LM.

[0270] [Irradiation Method]

[0271] Next, the illumination method using the imaging system 800 according to Embodiment 6 will be summarized. Figure 28 This is a flowchart illustrating the irradiation method according to Embodiment 6. Furthermore, each process in the irradiation method is executed based on a program stored in the storage unit 881. For example... Figure 28 As shown, firstly, the imaging system 800 acquires the orientation information of the object TG (step ST310). At this time, the first imaging device 140 captures a first image Im1 containing at least a portion of the object TG, and outputs the image information of the generated first image Im1 to the control device 880. The image information acquisition unit 882 acquires the image information of the first image Im1 output from the first imaging device 140.

[0272] Similar to Embodiment 1, the image analysis unit 883 generates position information of the object TG in the first image PC1 (the image represented by the image information of the first image Im1) based on the image information of the first image Im1, which is an image containing at least a portion of the object TG, acquired by the image information acquisition unit 882. Based on the generated position information of the object TG in the first image PC1, the image analysis unit 883 generates orientation information of the object TG. At this time, the device control unit 884 controls the drive device 163 of the reflecting member rotation device 160 based on at least one of the orientation information of the object TG generated by the image analysis unit 883 and the position information of the object TG in the first image PC1, so that the electromagnetic wave LB passes through a portion of the image forming area of ​​the intermediate image Imd containing at least a portion of the object TG.

[0273] Next, the imaging system 800 acquires the distance information of the target object TG (step ST320). At this time, the distance measuring device 870 illuminates the target object TG with measuring light LM via the second reflector 868, the second optical system 132, the first reflector 161, the first optical system 131, the light splitting member 120, and the imaging optical system 110, and generates the distance information of the target object TG. The distance measuring device 870 outputs the generated distance information of the target object TG to the distance information acquisition unit 885. The distance information acquisition unit 885 acquires the distance information of the target object TG output from the distance measuring device 870. In addition, in step ST310, in order to control the rotation angle of the first reflector 161, step ST310 can also be performed between steps ST320 and ST340 (described later) without using the direction information of the target object TG. In this case, in step ST320, the first imaging device 140 captures a first image Im1 containing at least a portion of the object TG, and the image analysis unit 883 generates position information of the object TG in the first image PC1 based on the image information of the first image Im1 containing at least a portion of the object TG.

[0274] Next, the imaging system 800 acquires image information of the second image Im2 in the same manner as in Embodiment 1 (step ST330). Alternatively, step ST330 can also be performed between steps ST310 and ST320.

[0275] Then, the imaging system 800 irradiates electromagnetic waves LB onto the target object TG (step ST340). At this time, the electromagnetic wave irradiation device 865 irradiates the target object TG (reference) via the non-imaging area 154 (through hole 155) of the second imaging element 151, the second reflective member 868, the second optical system 132, the first reflective member 161, the first optical system 131, the light splitting member 120, and the imaging optical system 110. Figure 1 The laser source is irradiated with electromagnetic waves LB. At this time, the control device 880 can control the output of the laser source based on the distance information of the target object TG.

[0276] Alternatively, steps ST310 to ST340 can be repeated. For example, when the positional relationship between the object TG and the imaging system 800 changes, by repeating steps ST310 to ST340, as described above, the distance measuring device 870 can continuously or intermittently illuminate the measuring light LM onto the object TG whose positional relationship has changed. Furthermore, when the positional relationship between the object TG and the imaging system 800 changes, by repeating steps ST310 to ST340, as described above, the electromagnetic wave irradiation device 865 can continuously or intermittently irradiate the object TG whose positional relationship has changed. Additionally, even if the positional relationship between the object TG and the imaging system 800 does not change, steps ST310 to ST340 can be repeated.

[0277] According to Embodiment 6, the imaging system 800 includes: a light splitting member 120 that splits the light LT passing through the imaging optical system 110; an imaging device 140 that captures a first image Im1 formed by a beam of light split by the light splitting member 120; and an electromagnetic wave irradiation device 865 that irradiates electromagnetic waves LB onto the object TG via the imaging optical system 110 based on image information of the first image Im1, which contains an image of at least a portion of the object captured by the imaging device 140. Thus, while capturing and following the object by the imaging device 140, electromagnetic waves LB can be irradiated onto the object by the electromagnetic wave irradiation device 865. Furthermore, by including a distance measuring device 870 that irradiates a measurement light LM onto the object via the imaging optical system 110 and generates distance information of the object TG, the distance to the object TG can be measured by the distance measuring device 870 while capturing and following the object TG by the imaging device 140.

[0278] Furthermore, it includes a reflective member rotation device 160, which comprises: a first reflective member 161 having a reflective surface 162 that reflects electromagnetic waves LB from the electromagnetic wave irradiation device 865 toward the imaging optical system 110; and a driving device 163 that rotates the first reflective member 161. Thus, by rotating the first reflective member 161 via the driving device 163, electromagnetic waves LB from the electromagnetic wave irradiation device 865 can be directed toward the target object for a short period of time. Therefore, electromagnetic waves LB can be irradiated onto the target object for a short period of time.

[0279] In the above embodiment 6, the shooting system 800 includes a distance measuring device 870, but is not limited thereto, and may not include a distance measuring device 870.

[0280] In the above-described embodiment 6, the shooting system 800 includes a second shooting device 150, but is not limited thereto, and may not include the second shooting device 150.

[0281] [Implementation Method 7]

[0282] Next, the imaging system according to Embodiment 7 will be described. In the imaging system according to Embodiment 7, apart from the structures of the electromagnetic wave irradiation device 965, the second reflective member 968, the distance measuring device 970, the control device 980, and the frame 901, its main parts have a structure common to the imaging system 100 according to Embodiment 1. Therefore, for structures identical to those in Embodiment 1, the same reference numerals as in Embodiment 1 are used, and detailed descriptions are omitted. Figure 29 As shown, the imaging system 900 according to Embodiment 7 includes an imaging optical system 110, a light splitting member 120, a re-imaging optical system 130, a first imaging device 140, a second imaging device 150, a reflective member rotation device 160, an electromagnetic wave irradiation device 965, a second reflective member 968, a distance measuring device 970, a control device 980, and a frame 901.

[0283] The frame 901 houses and holds the imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the second imaging device 150, the reflector rotation device 160, the electromagnetic wave irradiation device 965, the second reflector 968, and the distance measuring device 970. The imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the second imaging device 150, and the reflector rotation device 160 have the same structure as those of the imaging optical system 110, the light splitting member 120, the re-imaging optical system 130, the first imaging device 140, the second imaging device 150, and the reflector rotation device 160 according to Embodiment 1, and detailed descriptions are omitted. In addition, in the reflective member rotation device 160, the reflective surface 162 of the first reflective member 161 reflects the electromagnetic wave LB from the electromagnetic wave irradiation device 865 after passing through the second optical system 132 (re-imaging optical system 130) toward the first optical system 131 (imaging optical system 110).

[0284] In Embodiment 7, similar to Embodiment 1, the optical axis of the first optical system 131 in the re-imaging optical system 130 and the optical axis of the imaging optical system 110 coaxial with the first optical system 131 are referred to as the first optical axis AX1. The optical axis of the second optical system 132 in the re-imaging optical system 130 is referred to as the second optical axis AX2. Furthermore, the direction along the first optical axis AX1 is referred to as the Y direction. The direction along the second optical axis AX2 is referred to as the Z direction. The direction orthogonal to the Y direction and the Z direction is referred to as the X direction.

[0285] like Figure 29 As shown, the distance measuring device 970 can be configured similarly to the distance measuring device 170 according to Embodiment 1. The distance measuring device 970 is disposed on the opposite side of the optical component closest to the second imaging element 151 in the second optical system 132 (re-imaging optical system 130), separated by the second imaging element 151. The distance measuring device 970 illuminates the measuring light LM onto the object TG via the non-imaging area 154 (through hole 155) of the second imaging element 151, the second reflective component 968, the second optical system 132, the first reflective component 161, the first optical system 131, the light splitting component 120, and the imaging optical system 110, and generates distance information of the object TG.

[0286] like Figure 29As shown, the second reflective member 968 is disposed on a portion of the optical path of the re-imaging optical system 130 (second optical system 132) between the re-imaging optical system 130 and the second imaging element 151. The second reflective member 968 is configured similarly to the second reflective member 368 according to Embodiment 2. Furthermore, when the second optical system 132 of the re-imaging optical system 130 is composed of two or more lenses, the second reflective member 968 may also be disposed in the middle portion of the second optical system 132. Thus, the second reflective member 968 can be disposed on at least a portion of the optical path of the re-imaging optical system 130.

[0287] like Figure 29 As shown, the electromagnetic wave irradiation device 965 is offset from the second reflective member 968 in a direction (e.g., the Y direction) intersecting the optical axis (second optical axis AX2) of the second optical system 132 in the re-imaging optical system 130. The electromagnetic wave irradiation device 965 irradiates electromagnetic waves LB onto the target object TG via the second reflective member 968, the second optical system 132, the first reflective member 161, the first optical system 131, the light splitting member 120, and the imaging optical system 110. For example, the electromagnetic wave irradiation device 965 may include an optical fiber (not shown) that emits electromagnetic waves LB from a laser source. That is, the electromagnetic waves LB can also be laser light. Furthermore, the laser source can be located outside or within the imaging system 900.

[0288] The control device 980 is configured, for example, using a PC (personal computer). The control device 980 operates based on a program stored in the storage unit 981. Figure 29 As shown, the control device 980 includes a storage unit 981, an image information acquisition unit 982, an image analysis unit 983, a device control unit 984, a distance information acquisition unit 985, and an image correction unit 986. The storage unit 981, image information acquisition unit 982, image analysis unit 983, device control unit 984, distance information acquisition unit 985, and image correction unit 986 are configured similarly to those in Embodiment 6.

[0289] [Irradiation Method]

[0290] Next, the illumination method using the imaging system 900 according to Embodiment 7 will be summarized. The illumination method according to Embodiment 7 is the same as the illumination method according to Embodiment 6. Therefore, the same method as in Embodiment 6 is used. Figure 28The flowchart shown will be used for explanation. Furthermore, each process in the measurement method is executed based on a program stored in the storage unit 981. First, the imaging system 900, similar to the case in Embodiment 6, acquires the orientation information of the object TG (step ST310). At this time, the device control unit 984 of the control device 980 controls the drive device 163 of the reflector rotation device 160 based on at least one of the orientation information of the object TG generated by the image analysis unit 983 and the position information of the object TG in the first image PC1, so that the electromagnetic wave LB passes through a portion of the image forming area of ​​the intermediate image Imd containing at least a portion of the object TG.

[0291] Next, the imaging system 900 acquires the distance information of the target object TG in the same manner as in Embodiment 6 (step ST320). In addition, in step ST310, in order to control the rotation angle of the first reflective member 161, without using the direction information of the target object TG, step ST310 can also be performed between steps ST320 and ST340 (described later), just as in Embodiment 6.

[0292] Next, the imaging system 900 acquires image information of the second image Im2 in the same manner as in Embodiment 6 (step ST330). Alternatively, step ST330 can also be performed between steps ST310 and ST320.

[0293] Then, similarly to the case in Embodiment 6, the imaging system 900 irradiates electromagnetic waves LB onto the target object TG (step ST340). At this time, the control device 980 can control the output of the laser light source based on the distance information of the target object TG.

[0294] Alternatively, steps ST310 to ST340 can be repeated in the same manner as in embodiment 6.

[0295] According to embodiment 7, the same effect as that of embodiment 6 can be obtained.

[0296] In the above embodiment 7, the shooting system 900 includes a distance measuring device 970, but is not limited thereto, and may not include the distance measuring device 970.

[0297] In the above embodiment 7, the shooting system 900 includes a second shooting device 150, but is not limited thereto, and may not include the second shooting device 150.

[0298] In embodiments 1 to 3, 5, and 6 described above, the re-imaging optical system 130 forms a second image Im2 by magnifying and re-imaging a portion of the intermediate image Imd formed by the light LT passing through the imaging optical system 110, but is not limited thereto. For example, the re-imaging optical system may also re-image the intermediate image Imd formed by the light LT passing through the imaging optical system 110 as the second image Im2 at the same size. Furthermore, the re-imaging optical system may also reduce the size of the intermediate image Imd formed by the light LT passing through the imaging optical system 110 and re-image it as the second image Im2. In addition, the re-imaging optical system 130 may be configured to re-image a portion of the intermediate image Imd as the second image Im2 by reducing its size, or it may be configured to re-image the second image Im2 at the same size as a portion of the intermediate image Imd.

[0299] In embodiments 1 to 3 described above, the output device outputs the orientation information of the object TG, the distance information of the object TG, and the image information of the second image Im2, but is not limited thereto. For example, the output device may also output three-dimensional information of the object TG generated based on the orientation information of the object TG, the distance information of the object TG, and the image information of the second image Im2.

[0300] In embodiments 1 to 3 described above, similar to the case in embodiment 5, if the image analysis unit identifies an object TG, it sets a region of interest (ROI) in the first image PC1 such that it contains at least a portion of the identified object TG. At this time, the image analysis unit may also set multiple measurement points MP arranged at predetermined measurement intervals for the object TG within the ROI. If the image analysis unit sets the ROI, it can generate position information of the object TG in the first image PC1 for the multiple measurement points MP, similar to the image analysis unit 183 in embodiment 1. Similarly, for the multiple measurement points MP, the image analysis unit can generate orientation information of the object TG based on the generated position information of the object TG in the first image PC1, also similar to the image analysis unit 183 in embodiment 1. Based on the orientation information of the object TG generated for the multiple measurement points MP of the ROI, the device control unit controls the drive device 163 to rotate the first reflective member 161, causing the measuring light LM from the distance measuring device to sequentially illuminate the multiple measurement points MP on the object TG. The output device can also output information related to multiple measurement points MP of the region of interest (ROI) to replace the orientation information of the object TG.

[0301] In embodiments 1 to 3, 6, and 7 described above, the control device may also include a distance measuring unit that uses machine learning to generate distance information of the target object based on image information from a second image captured by the second imaging device. In this case, since the control device (distance measuring unit) functions as a distance measuring device, a distance measuring device that illuminates the measuring light may not be necessary.

[0302] The following notes further describe the implementation methods described above.

[0303] [Note 1] A shooting system includes: Imaging optical systems; A light splitting component that splits at least a portion of the light that has passed through the imaging optical system; The first imaging device captures a first image formed by a beam of light split by the light splitting member; A second imaging device, which captures a second image formed by another beam of light split by the light splitting member; and A distance measuring device that, based on image information of a first image captured by the first imaging device that contains an image of at least a portion of the object, illuminates the object with measuring light and generates distance information of the object.

[0304] [Appendix 2] The shooting system as described in Appendix 1, wherein, The distance measuring device illuminates the measuring light onto the object based on the position information of the object in the first image generated according to the image information of the first image, and generates the distance information.

[0305] [Note 3] The shooting system as described in Note 1 or 2, wherein, The distance measuring device illuminates the measuring light onto the object based on the orientation information of the object generated from the image information of the first image, and generates the distance information.

[0306] [Appendix 4] The imaging system as described in any one of Appendices 1 to 3, wherein, The distance measuring device illuminates measuring light onto the object without passing through the imaging optical system and the light splitting member, based on image information of the first image containing at least a portion of the object, and generates distance information of the object.

[0307] [Appendix 5] The imaging system as described in any one of Appendices 1 to 4, wherein, The distance measuring device illuminates the object by emitting the measuring light in an illumination direction set based on image information of the first image containing at least a portion of the object, and generates distance information of the object.

[0308] [Appendix 6] The imaging system as described in any one of Appendices 1 to 5, wherein, The distance measuring device illuminates the object with measuring light by emitting measuring light in an illumination direction set based on image information of the first image containing at least a portion of the object and position information of the distance measuring device, thereby generating distance information of the object.

[0309] [Appendix 7] The imaging system as described in any one of Appendices 1 to 6, wherein, By emitting the measuring light toward the object based on image information of the first image containing at least a portion of the object and the positional relationship between the optical device containing the imaging optical system, the light splitting member, the first imaging device, and the second imaging device and the distance measuring device, the measuring light is emitted toward the object, thereby illuminating the object and generating distance information of the object.

[0310] [Appendix 8] The imaging system as described in any one of Appendices 1 to 7, wherein, The second image is formed based on the image information of the first image, which contains at least a portion of the object.

[0311] [Appendix 9] The imaging system as described in any one of Appendices 1 to 8, wherein, The second image is a part of the third image formed by the imaging optical system.

[0312] [Note 10] The shooting system as described in Note 9, wherein, It also includes a re-imaging optical system that forms the second image by magnifying and re-imaging a portion of the third image formed by the imaging optical system.

[0313] [Note 11] The shooting system as described in Note 10, wherein, The re-imaging optical system is telecentric on the side of the third image.

[0314] [Note 12] The imaging system as described in any one of Notes 9 to 11, wherein, Based on the image information of the first image containing at least a portion of the object, a region is selected as a part of the image forming region of the third image to be re-imaged as the second image, such that the second image contains at least a portion of the object.

[0315] [Note 13] The imaging system as described in any one of Notes 1 to 12, wherein, It also includes a reflective member rotation device having at least one first reflective member and a driving device, the first reflective member having a reflective surface that reflects at least a portion of the other beam of light split by the light splitting member toward the second imaging device, and the driving device rotating the at least one first reflective member.

[0316] [Note 14] The shooting system as described in Note 13, wherein, It also includes a re-imaging optical system that re-images at least a portion of the third image formed by the other beam of light split by the light splitting member through the imaging optical system as the second image.

[0317] [Note 15] The imaging system as described in Note 14, wherein, The re-imaging optical system magnifies a portion of the third image formed by the other beam of light split by the light splitting member and re-images it as the second image.

[0318] [Note 16] The imaging system as described in Notes 14 or 15, wherein, The position where the third image is formed is conjugate to the position where the first image is formed.

[0319] [Note 17] The imaging system as described in any one of Notes 14 to 16, wherein, The re-imaging optical system includes: The first optical system into which the other beam of light, split by the light-splitting component, is incident; and The second optical system is incident on the light from the first optical system. The reflective surface of the at least one first reflective member is disposed at or near the intersection of the optical axis of the first optical system and the optical axis of the second optical system. The second optical system uses light from the first optical system reflected by the reflective surface to form the second image.

[0320] [Note 18] The imaging system as described in Note 17, wherein, The first optical system is telecentric on the side of the third image.

[0321] [Note 19] The imaging system as described in Note 17 or 18, wherein, The reflective surface is positioned at the pupil position or pupil conjugate position of the optical system consisting of the imaging optical system and the first optical system.

[0322] [Note 20] The imaging system as described in any one of Notes 13 to 19, wherein, Based on image information of the first image containing at least a portion of the object, the driving device is used to rotate the at least one first reflecting member, thereby determining the second image captured by the second imaging device.

[0323] [Note 21] The imaging system as described in any one of Notes 13 to 20, wherein, The image information of the second image captured by the second imaging device is corrected based on the rotation of the first reflective member performed by the driving device.

[0324] [Note 22] The imaging system as described in any one of Notes 1 to 21, wherein, The distance measuring device includes: A measuring light irradiation device that emits the measuring light; and a light receiving device that receives light from the object irradiated by the measuring light. Based on the light received by the light receiving device on the object being illuminated by the measuring light from the measuring light irradiation device, the distance information of the object is generated, wherein the object is illuminated by the measuring light from the measuring light irradiation device.

[0325] [Note 23] The imaging system as described in any one of Notes 1 to 22, wherein, It also includes a frame that holds the imaging optical system, the light splitting component, the first imaging device, and the second imaging device. The measuring light illumination device is positioned at a different location from the frame.

[0326] [Note 24] The imaging system as described in any one of Notes 1 to 23, wherein, Based on the image information of the first image, which contains images of multiple objects, captured by the first imaging device, at least one of the multiple objects is identified as the target object. The measuring light from the distance measuring device is shone onto the identified object.

[0327] [Note 25] The imaging system as described in any one of Notes 1 to 23, wherein, Based on the image information of the first image containing the image of the object captured by the first imaging device, the object is identified as the target object. The measuring light from the distance measuring device is shone onto the identified object.

[0328] [Note 26] The imaging system as described in any one of Notes 1 to 25, wherein, It also includes an output device that outputs at least one of the orientation information of the object generated based on image information of the first image containing at least a portion of the object, and the distance information generated by the distance measuring device.

[0329] [Note 27] The imaging system as described in Note 26, wherein, The second image is formed based on the image information of the first image, which contains an image of at least a portion of the object, in a manner that includes an image of at least a portion of the object. The output device outputs image information of the second image, which is an image of at least a portion of the object captured by the second imaging device.

[0330] [Note 28] The imaging system as described in Note 26, wherein, Based on at least one of the distance information and the direction information output from the output device, the electromagnetic wave irradiation device is controlled so that an electromagnetic wave different from the measuring light is irradiated onto the object.

[0331] [Note 29] The imaging system as described in Note 27, wherein, Based on at least one of the distance information and direction information output from the output device and the image information of the second image, the electromagnetic wave irradiation device is controlled so that electromagnetic waves different from the measurement light are irradiated onto the object.

[0332] [Note 30] The imaging system as described in Note 26, wherein, The image information of the first image is used as the first image information. The aforementioned direction information is used as the first direction information. When the distance information is used as the first distance information When the positional relationship between the object and the imaging system changes. The output device outputs at least one of the second distance information and the second direction information of the object, wherein the second distance information of the object is generated by illuminating the measuring light from the distance measuring device onto the object based on the second image information of the first image containing at least a part of the object captured by the first imaging device after the positional relationship changes, and the second direction information of the object is generated based on the second image information of the first image containing at least a part of the object captured by the first imaging device after the positional relationship changes.

[0333] [Note 31] The imaging system as described in Note 27, wherein, The image information of the first image is used as the first image information. The image information of the second image, which contains at least a portion of the object, is used as the third image information. The aforementioned direction information is used as the first direction information. When the distance information is used as the first distance information When the positional relationship between the object and the imaging system changes. The output device outputs at least one of the following: second distance information of the object, second direction information of the object, and fourth image information of the second image. The second distance information of the object is generated by illuminating the object with measuring light from the distance measuring device, based on the second image information of the first image containing at least a portion of the object, captured by the first imaging device after the change in positional relationship. The second orientation information of the object is generated based on the second image information of the first image, which contains at least a portion of the object, captured by the first imaging device after the positional relationship has changed. The fourth image information of the second image is generated by capturing the second image using the second imaging device, wherein the second image is formed based on the second image information of the first image, which contains at least a part of the object, captured by the first imaging device after the positional relationship has changed, in a manner that contains at least a part of the object.

[0334] [Note 32] The shooting system as described in Note 30, wherein, Based on at least one of the distance change information of the first distance information and the second distance information output from the output device, and the direction change information of the first direction information and the second direction information, the electromagnetic wave irradiation device is controlled so that electromagnetic waves different from the measuring light are continuously or intermittently irradiated onto the object as the positional relationship changes.

[0335] [Note 33] The shooting system as described in Note 31, wherein... Based on at least one of the distance change information of the first distance information and the second distance information, the direction change information of the first direction information and the second direction information, and the image change information of the third image information and the fourth image information output from the output device, the electromagnetic wave irradiation device is controlled such that electromagnetic waves different from the measuring light are continuously or intermittently irradiated onto the object as the positional relationship changes.

[0336] [Note 34] The imaging system as described in any one of Notes 1 to 25, wherein, When the positional relationship between the object and the imaging system changes, the measuring light is continuously or intermittently irradiated onto the object based on the image information of the first image, which is captured by the first imaging device and contains an image of at least a part of the object.

[0337] [Note 35] The shooting system as described in Note 34, wherein... When the positional relationship between the object and the shooting system changes, the second image is formed based on the image information of the first image, which contains an image of at least a part of the object, captured by the first shooting device, in a manner that contains an image of at least a part of the object.

[0338] [Note 36] The shooting system as described in Notes 34 or 35, wherein... When the positional relationship between the object and the imaging system changes, the electromagnetic wave irradiation device is controlled based on the image information of the first image, which contains an image of at least a part of the object, captured by the first imaging device, so that electromagnetic waves different from the measuring light are continuously or intermittently irradiated onto the object.

[0339] [Note 37] The imaging system as described in any one of Notes 28, 29, 32, 33 and 36, wherein, The electromagnetic wave irradiation device is located in a different position from the imaging system.

[0340] [Note 38] The imaging system as described in any one of Notes 28, 29, 32, 33, 36 and 37, wherein, The electromagnetic waves are irradiated from the electromagnetic wave irradiation device onto the target object to repel or destroy the target object.

[0341] [Note 39] The imaging system as described in any one of Notes 28, 29, 32, and 33 or any one of Notes 36 to 38, wherein, The electromagnetic wave is light.

[0342] [Note 40] The shooting system as described in Note 39, wherein... The output of the light irradiated by the electromagnetic wave irradiation device is 10kW or more.

[0343] [Appendix 41] An electromagnetic wave irradiation system, wherein... Including a photographing system as described in any one of notes 28, 29, 32, 33 or any one of notes 36 to 40; and an electromagnetic wave irradiation device for irradiating an object with electromagnetic waves.

[0344] [Appendix 42] A measurement method, This measurement method uses an imaging system comprising an imaging optics system and a light-splitting component that splits at least a portion of the light passing through the imaging optics system, including: The first imaging device is used to capture a first image formed by a beam of light split by the light splitting member; The second imaging device is used to capture a second image formed by another beam of light split by the light splitting member; Based on the image information of the first image captured by the first imaging device, which contains an image of at least a portion of the object, measurement light is irradiated onto the object, and distance information of the object is generated.

[0345] [Note 43] A program, The computer is made to perform the distance measurement method as described in Appendix 42.

[0346] [Appendix 44] A shooting system, comprising: Imaging optical systems; A light splitting component that splits at least a portion of the light that has passed through the imaging optical system; A first imaging device, which captures a first image formed by a beam of light split by the light splitting member; and The second imaging device captures a second image formed by another beam of light split by the light splitting member. Based on the image information of the first image captured by the first imaging device, which contains an image of at least a portion of the object, measuring light from the distance measuring device is irradiated onto the object, and distance information of the object is generated.

[0347] [Note 45] A shooting system, comprising: Imaging optical systems; A light splitting component that splits the light passing through the imaging optical system; An imaging device that captures a first image formed by a beam of light split by the light-splitting member; and An electromagnetic wave irradiation device that irradiates electromagnetic waves onto the object via the imaging optical system based on image information of a first image captured by the imaging device, which contains an image of at least a portion of the object.

[0348] [Note 46] The imaging system as described in Note 45, wherein, The electromagnetic wave irradiation device irradiates electromagnetic waves onto the object via the imaging optical system based on the position information of the object in the first image generated according to the image information of the first image.

[0349] [Note 47] The imaging system as described in Notes 45 or 46, wherein, The electromagnetic wave irradiation device irradiates electromagnetic waves onto the object via the imaging optical system based on the orientation information of the object generated from the image information of the first image.

[0350] [Note 48] The imaging system as described in any one of Notes 45 to 47, wherein, Based on the image information of the first image containing at least a portion of the object, the irradiation path of the electromagnetic wave from the electromagnetic wave irradiation device is set.

[0351] [Note 49] The imaging system as described in Note 48, wherein, Based on image information of the first image containing at least a portion of the object, the illumination optical path is configured such that the electromagnetic wave is emitted from the imaging optical system toward the object. The electromagnetic waves are irradiated by the electromagnetic wave irradiation device through the set irradiation optical path, and the electromagnetic waves emitted from the imaging optical system are irradiated onto the object.

[0352] [Note 50] The imaging system as described in Note 49, wherein, Based on image information of the first image containing at least a portion of the object, the illumination optical path is selected from a plurality of optical paths formed by the imaging optical system.

[0353] [Note 51] The shooting system as described in Note 50, wherein, Based on the image information of the first image containing at least a portion of the object, a portion region of the image containing at least a portion of the object is selected from the image forming region of the image formed by another beam of light segmented by the imaging optical system and the light segmentation member, thereby selecting the illumination light path from the plurality of light paths.

[0354] [Note 52] The imaging system as described in any one of Notes 48 to 50, wherein, Based on the image information of the first image containing at least a portion of the object, the illumination optical path is configured such that the electromagnetic wave passes through a portion of the image region containing at least a portion of the object in the image forming region of the image formed by another beam of light segmented by the imaging optical system and the light segmentation member.

[0355] [Note 53] The imaging system as described in any one of Notes 48 to 50, wherein, Based on image information of the first image containing at least a portion of the object, the illumination optical path is configured such that the illumination optical path overlaps with at least a portion of the optical path of the light forming the image of at least a portion of the object, wherein the image of at least a portion of the object is contained in an image formed by another beam of light segmented by the light splitting member using the imaging optical system. By setting the illumination optical path, the electromagnetic waves from the electromagnetic wave illumination device are irradiated onto the object via the imaging optical system.

[0356] [Note 54] The imaging system as described in any one of Notes 45 to 53, wherein, It also includes a reflector rotation device having at least one first reflector and a driving device, wherein the at least one first reflector has a reflective surface that reflects the electromagnetic waves from the electromagnetic wave irradiation device toward the imaging optical system, and the driving device rotates the at least one first reflector.

[0357] [Note 55] The shooting system as described in Note 54 further includes: The first optical system into which another beam of light, split by the light-splitting component, is incident; and The second optical system is incident on the light from the first optical system. The reflective surface of the at least one first reflective member is disposed at or near the intersection of the optical axis of the first optical system and the optical axis of the second optical system.

[0358] [Note 56] The shooting system as described in Note 55, wherein, The first optical system is telecentric on one side of the optical splitting member.

[0359] [Note 57] The imaging system as described in Notes 55 or 56, wherein, The reflective surface is positioned at the pupil position or pupil conjugate position of the optical system consisting of the imaging optical system and the first optical system.

[0360] [Note 58] The imaging system as described in any one of Notes 54 to 57, wherein, The electromagnetic waves from the electromagnetic wave irradiation device are irradiated onto the object via the at least one first reflecting member, the light splitting member, and the imaging optical system.

[0361] [Note 59] The imaging system as described in any one of Notes 54 to 58, wherein, At least a portion of the electromagnetic wave irradiation device is configured at a position where the electromagnetic wave can be irradiated toward the light splitting member via the at least one first reflective member.

[0362] [Note 60] The imaging system as described in any one of Notes 54 to 59, wherein, Based on image information of the first image containing at least a portion of the object, the driving device is used to rotate the at least one first reflecting member, thereby causing the electromagnetic wave to irradiate the object.

[0363] [Note 61] The imaging system as described in any one of Notes 54 to 60, wherein, Based on image information of the first image containing at least a portion of the object, the driving device is used to rotate the at least one first reflecting member, thereby setting the illumination optical path of the electromagnetic waves from the electromagnetic wave illumination device, so that the electromagnetic waves are emitted from the imaging optical system toward the object. The electromagnetic wave irradiation device irradiates the electromagnetic wave via the set irradiation optical path and the at least one first reflective member, thereby causing the electromagnetic wave emitted from the imaging optical system to irradiate the object.

[0364] [Note 62] The shooting system as described in Note 61, wherein, Based on image information of the first image containing at least a portion of the object, the driving device is used to rotate the at least one first reflecting member, thereby selecting the illumination optical path from a plurality of optical paths formed by the imaging optical system.

[0365] [Note 63] The imaging system as described in Note 61 or 62, wherein, Based on image information of the first image containing at least a portion of the object, the driving device is used to rotate the at least one first reflecting member, thereby setting the illumination optical path such that the electromagnetic wave passes through a portion of the image region containing at least a portion of the object in the image forming region of the image formed by another beam of light segmented by the imaging optical system and the light segmenting member.

[0366] [Note 64] The imaging system as described in any one of Notes 54 to 63, wherein, Based on image information of the first image containing at least a portion of the object, the at least one first reflecting member is rotated using the driving device, thereby setting the illumination path of the electromagnetic waves from the electromagnetic wave illumination device such that the illumination path overlaps with at least a portion of the light path forming the image of at least a portion of the object, wherein the image of at least a portion of the object is contained in an image formed by another beam of light segmented by the light splitting member using the imaging optical system. By setting the illumination optical path such that at least a portion of the optical path of the light forming the image of the part overlaps with at least a portion of the illumination optical path, the electromagnetic waves from the electromagnetic wave illumination device are irradiated onto the object via the reflective surface and the imaging optical system.

[0367] [Note 65] The imaging system as described in any one of Notes 45 to 64, wherein, The aforementioned shooting device is designated as the first shooting device. It also includes a second imaging device that captures a second image formed by another beam of light split by the light splitting member.

[0368] [Note 66] The shooting system as described in Note 65, wherein, The second image is formed based on the image information of the first image, which contains at least a portion of the object.

[0369] [Note 67] The imaging system as described in Note 65 or 66, wherein, The second image is a part of the third image formed by the imaging optical system.

[0370] [Note 68] The imaging system as described in Note 67, wherein, It also includes a re-imaging optical system that magnifies and re-images a portion of the third image formed by the imaging optical system, thereby forming the second image.

[0371] [Note 69] The imaging system as described in Note 68, wherein, The re-imaging optical system is telecentric on the side of the third image.

[0372] [Note 70] The imaging system as described in any one of Notes 67 to 69, wherein, Based on the image information of the...

Claims

1. A shooting system, characterized in that, include: Imaging optical systems; A light splitting component that splits at least a portion of the light that has passed through the imaging optical system; The first imaging device captures a first image formed by a beam of light split by the light splitting member; The second imaging device captures a second image formed by another beam of light split by the light splitting member; as well as A distance measuring device that, based on image information of a first image captured by the first imaging device that contains an image of at least a portion of the object, illuminates the object with measuring light via the imaging optical system and generates distance information of the object.

2. The shooting system as described in claim 1, characterized in that, The distance measuring device illuminates the object with measuring light via the imaging optical system based on the position information of the object in the first image generated according to the image information of the first image, and generates the distance information.

3. The shooting system as described in claim 1 or 2, characterized in that, The distance measuring device illuminates the measuring light onto the object via the imaging optical system based on the orientation information of the object generated from the image information of the first image, and generates the distance information.

4. The shooting system as described in any one of claims 1 to 3, characterized in that, Based on image information of the first image captured by the first imaging device, which contains an image of at least a portion of the object, the second image is formed such that at least a portion of the first optical path of the light forming the image of at least a portion of the object contained in the second image overlaps with at least a portion of the second optical path of the measuring light from the distance measuring device. When the second image is formed such that at least a portion of the first optical path overlaps with the second optical path, the measuring light is illuminated by the distance measuring device, thereby the measuring light from the distance measuring device is illuminated onto the object via the imaging optical system.

5. The shooting system as described in any one of claims 1 to 4, characterized in that, The second image is formed based on the image information of the first image, which contains at least a portion of the object.

6. The shooting system as described in any one of claims 1 to 5, characterized in that, The second image is a part of the third image formed by the imaging optical system.

7. The shooting system as described in claim 6, characterized in that, It also includes a re-imaging optical system that forms the second image by magnifying and re-imaging a portion of the third image formed by the imaging optical system.

8. The shooting system as described in claim 7, characterized in that, The re-imaging optical system is telecentric on the side of the third image.

9. The shooting system as described in any one of claims 6 to 8, characterized in that, Based on the image information of the first image containing at least a portion of the object, a region is selected as part of the image forming region of the third image, which is re-imaged as the second image, such that the second image contains an image of at least a portion of the object.

10. The shooting system as described in claim 9, characterized in that, Based on image information of the first image containing at least a portion of the object, the partial region is selected from the image forming region such that at least a portion of the first optical path of the light forming the image of at least a portion of the object contained in the second image overlaps with at least a portion of the second optical path of the measuring light from the distance measuring device. With the portion of the image forming region selected such that at least a portion of the first optical path overlaps with at least a portion of the second optical path, the measuring light is illuminated by the distance measuring device, thereby the measuring light from the distance measuring device is illuminated onto the object via the imaging optical system.

11. The shooting system as described in any one of claims 1 to 10, characterized in that, It also includes a reflective member rotation device having at least one first reflective member and a driving device, wherein the at least one first reflective member has a reflective surface that reflects at least a portion of the other beam of light split by the light splitting member toward the second imaging device, and the driving device rotates the at least one first reflective member. The reflective surface is positioned on the optical path of the measuring light from the distance measuring device.

12. The shooting system as described in claim 11, characterized in that, It also includes a re-imaging optical system that re-images at least a portion of the third image formed by the other beam of light split by the light splitting member through the imaging optical system as the second image.

13. The shooting system as described in claim 12, characterized in that, The re-imaging optical system magnifies a portion of the third image formed by the other beam of light split by the light splitting member and re-images it as the second image.

14. The shooting system as described in claim 12 or 13, characterized in that, The position where the third image is formed is conjugate to the position where the first image is formed.

15. The shooting system as described in any one of claims 12 to 14, characterized in that, The re-imaging optical system includes: The first optical system into which the other beam of light, split by the light-splitting component, is incident; and The second optical system is incident on the light from the first optical system. The reflective surface of the at least one first reflective member is disposed at or near the intersection of the optical axis of the first optical system and the optical axis of the second optical system. The second optical system uses light from the first optical system reflected by the reflective surface to form the second image.

16. The shooting system as described in claim 15, characterized in that, The first optical system is telecentric on the side of the third image.

17. The shooting system as described in claim 15 or 16, characterized in that, The reflective surface is positioned at the pupil position or pupil conjugate position of the optical system consisting of the imaging optical system and the first optical system.

18. The imaging system as described in any one of claims 11 to 17, characterized in that, The measuring light from the distance measuring device is projected onto the object via the at least one first reflecting member, the light splitting member, and the imaging optical system.

19. The imaging system as described in any one of claims 12 to 17, characterized in that, The second imaging device has a second imaging element for capturing the second image. The shooting area of ​​the second imaging element is set around the non-shooting area. The measuring light from the distance measuring device is incident on the imaging optical system from the non-shooting area via the re-imaging optical system, the at least one first reflecting member, and the light splitting member.

20. The shooting system as described in claim 19, characterized in that, A through hole is formed on the second imaging element in a direction intersecting with the non-imaging area of ​​the second imaging element. At least a portion of the distance measuring device is positioned, across the second imaging element, on the opposite side of the optical component in the re-imaging optical system that is closest to the second imaging element. The measuring light from the non-shooting area includes measuring light emitted from at least a portion of the distance measuring device and passing through the through-hole.

21. The imaging system as described in any one of claims 12 to 17, characterized in that, It also includes a second reflective member disposed on at least a portion of the optical path of the re-imaging optical system. The measuring light from the distance measuring device is incident on the re-imaging optical system via the second reflecting member.

22. The shooting system as described in claim 21, characterized in that, The second reflective member is disposed on at least a portion of the optical path of the re-imaging optical system between the re-imaging optical system and the second imaging element.

23. The shooting system as described in claim 21 or 22, characterized in that, At least a portion of the distance measuring device is configured at a position conjugate to the position where the second image is formed.

24. The shooting system as described in any one of claims 11 to 23, characterized in that, Based on image information of the first image containing at least a portion of the object, the driving device is used to rotate the at least one first reflecting member, thereby determining the second image captured by the second imaging device, and causing the measuring light to illuminate the object.

25. The shooting system as described in claim 24, characterized in that, Based on image information of the first image containing at least a portion of the object, the driving device is used to rotate the at least one first reflecting member, thereby selecting a region as part of the image forming region of the third image in which the second image is re-imaged, such that at least a portion of the first optical path of the light forming the image of at least a portion of the object contained in the second image overlaps with at least a portion of the second optical path of the measuring light from the distance measuring device. With the portion of the image forming region selected such that at least a portion of the first optical path overlaps with at least a portion of the second optical path, the measuring light is illuminated by the distance measuring device, thereby causing the measuring light from the distance measuring device to illuminate the object via at least a portion of the re-imaging optical system, the at least one first reflecting member, and the imaging optical system.

26. The shooting system as described in claim 19, characterized in that, Based on image information of the first image containing at least a portion of the object, the at least one first reflecting member is rotated using the driving device, thereby selecting a region from the image forming region as a part of the image forming region of the third image to be re-imaged by the second image, such that the image of at least a portion of the object contained in the second image overlaps with at least a portion of the non-image area of ​​the second imaging element. With the selected portion of the image forming region such that at least a portion of the object contained in the second image overlaps with at least a portion of the non-imaged region, the measuring light is illuminated by the distance measuring device, thereby the measuring light from the distance measuring device is illuminated onto the object via the re-imaging optical system, the at least one first reflecting member, and the imaging optical system.

27. The imaging system as described in any one of claims 11 to 26, characterized in that, The image information of the second image captured by the second imaging device is corrected based on the rotation of the first reflective member performed by the driving device.

28. The imaging system as described in any one of claims 1 to 27, characterized in that, The distance measuring device includes: A measuring light irradiation device that emits the measuring light; and a light receiving device that receives light from the object irradiated by the measuring light. Based on the light receiving result of the light from the object by the light receiving device, the distance information of the object is generated, wherein the object is illuminated by the measuring light from the measuring light irradiation device.

29. The imaging system as described in any one of claims 1 to 28, characterized in that, Based on the image information of the first image, which contains images of multiple objects, captured by the first imaging device, at least one of the multiple objects is identified as the target object. The measuring light is irradiated onto the identified object via the imaging optical system.

30. The imaging system as described in any one of claims 1 to 28, characterized in that, Based on the image information of the first image containing the image of the object captured by the first imaging device, the object is identified as the target object. The measuring light is irradiated onto the identified object via the imaging optical system.

31. The shooting system as described in any one of claims 1 to 30, characterized in that, It also includes an output device that outputs at least one of the orientation information of the object generated based on image information of the first image containing at least a portion of the object, and the distance information generated by the distance measuring device.

32. The shooting system as described in claim 31, characterized in that, The second image is formed based on the image information of the first image, which contains an image of at least a portion of the object, in a manner that includes an image of at least a portion of the object. The output device outputs image information of the second image, which is an image of at least a portion of the object captured by the second imaging device.

33. The shooting system as described in claim 31, characterized in that, Based on at least one of the distance information and the direction information output from the output device, the electromagnetic wave irradiation device is controlled so that an electromagnetic wave different from the measuring light is irradiated onto the object.

34. The shooting system as described in claim 32, characterized in that, Based on at least one of the distance information and direction information output from the output device and the image information of the second image, the electromagnetic wave irradiation device is controlled so that electromagnetic waves different from the measurement light are irradiated onto the object.

35. The shooting system as described in claim 31, characterized in that, The image information of the first image is used as the first image information. The aforementioned direction information is used as the first direction information. When the distance information is used as the first distance information When the positional relationship between the object and the imaging system changes. The output device outputs at least one of the second distance information and the second direction information of the object, wherein the second distance information of the object is generated by illuminating the object with measuring light from the distance measuring device via the imaging optical system, based on the second image information of the first image containing at least a portion of the object captured by the first imaging device after the change in positional relationship, and the second direction information of the object is generated based on the second image information of the first image containing at least a portion of the object captured by the first imaging device after the change in positional relationship.

36. The shooting system as described in claim 32, characterized in that, The image information of the first image is used as the first image information. The image information of the second image, which contains at least a portion of the object, is used as the third image information. The aforementioned direction information is used as the first direction information. When the distance information is used as the first distance information When the positional relationship between the object and the imaging system changes. The output device outputs at least one of the following: second distance information of the object, second direction information of the object, and fourth image information of the second image. The second distance information of the object is generated based on the second image information of the first image, which contains at least a portion of the object and was captured by the first imaging device after the change in positional relationship. This second image is generated by illuminating the object with measuring light from the distance measuring device via the imaging optical system. The second orientation information of the object is generated based on the second image information of the first image, which contains at least a portion of the object, captured by the first imaging device after the positional relationship has changed. The fourth image information of the second image is generated by capturing the second image using the second imaging device, wherein the second image is formed based on the second image information of the first image, which contains at least a part of the object, captured by the first imaging device after the positional relationship has changed, in a manner that contains at least a part of the object.

37. The shooting system as described in claim 35, characterized in that, Based on at least one of the distance change information of the first distance information and the second distance information output from the output device, and the direction change information of the first direction information and the second direction information, the electromagnetic wave irradiation device is controlled so that electromagnetic waves different from the measuring light are continuously or intermittently irradiated onto the object as the positional relationship changes.

38. The shooting system as described in claim 36, characterized in that, Based on at least one of the distance change information of the first distance information and the second distance information, the direction change information of the first direction information and the second direction information, and the image change information of the third image information and the fourth image information output from the output device, the electromagnetic wave irradiation device is controlled such that electromagnetic waves different from the measuring light are continuously or intermittently irradiated onto the object as the positional relationship changes.

39. The shooting system as described in any one of claims 1 to 30, characterized in that, When the positional relationship between the object and the imaging system changes, the measuring light is continuously or intermittently irradiated onto the object based on the image information of the first image, which is captured by the first imaging device and contains an image of at least a part of the object.

40. The shooting system as described in claim 39, characterized in that, When the positional relationship between the object and the shooting system changes, the second image is formed based on the image information of the first image, which contains an image of at least a part of the object, captured by the first shooting device, in a manner that contains an image of at least a part of the object.

41. The shooting system as described in claim 39 or 40, characterized in that, When the positional relationship between the object and the imaging system changes, the electromagnetic wave irradiation device is controlled based on the image information of the first image, which contains an image of at least a part of the object, captured by the first imaging device, so that electromagnetic waves different from the measuring light are continuously or intermittently irradiated onto the object.

42. A shooting system, characterized in that, include: Imaging optical systems; A light splitting component that splits the light passing through the imaging optical system; An imaging device that captures a first image formed by a beam of light split by the light splitting member; as well as A distance measuring device that, based on image information of a first image captured by the imaging device that contains an image of at least a portion of the object, illuminates the object with measuring light via the imaging optical system and generates distance information of the object.

43. The shooting system as described in claim 42, characterized in that, The distance measuring device, based on the position information of the object in the first image generated according to the image information of the first image, illuminates the object with measuring light via the imaging optical system and generates the distance information.

44. The shooting system as described in claim 42 or 43, characterized in that, The distance measuring device, based on the orientation information of the object generated according to the image information of the first image, illuminates the object with measuring light via the imaging optical system and generates the distance information.

45. The imaging system as described in any one of claims 42 to 44, characterized in that, Based on the image information of the first image containing at least a portion of the object, the measurement optical path through which the measurement light from the distance measuring device passes is set.

46. ​​The shooting system as described in claim 45, characterized in that, Based on image information of the first image containing at least a portion of the object, the measurement optical path is configured such that the measurement light is emitted from the imaging optical system toward the object. The distance measuring device illuminates the measuring light via the established measuring optical path, thereby illuminating the object with the measuring light emitted from the imaging optical system.

47. The shooting system as described in claim 46, characterized in that, Based on image information of the first image containing at least a portion of the object, the measurement optical path is selected from a plurality of optical paths formed by the imaging optical system.

48. The shooting system as described in claim 47, characterized in that, Based on image information of the first image containing at least a portion of the object, a portion region of the image containing at least a portion of the object is selected from the image forming region of the image formed by another beam of light segmented by the light segmentation member using the imaging optical system, thereby selecting the measurement optical path from the plurality of optical paths.

49. The imaging system as described in any one of claims 45 to 47, characterized in that, Based on image information of the first image containing at least a portion of the object, the measurement optical path is configured such that the measurement light passes through a portion of the image forming region of the image formed by another beam of light segmented by the light segmentation member using the imaging optical system, which contains at least a portion of the object.

50. The imaging system as described in any one of claims 45 to 47, characterized in that, Based on image information of the first image containing at least a portion of the object, the measurement optical path is configured such that the measurement optical path overlaps with at least a portion of the optical path of the light forming the image of at least a portion of the object, wherein the image of at least a portion of the object is contained in an image formed by another beam of light segmented by the light splitting member using the imaging optical system. By setting the measurement optical path, the measurement light from the distance measuring device is projected onto the object via the imaging optical system.

51. The imaging system as described in any one of claims 42 to 50, characterized in that, It also includes a reflector rotation device having at least one first reflector and a driving device, wherein the at least one first reflector has a reflective surface that reflects the measuring light from the distance measuring device toward the imaging optical system, and the driving device rotates the at least one first reflector.

52. The shooting system as described in claim 51, characterized in that, Also includes: The first optical system into which the other beam of light, split by the light-splitting component, is incident; as well as The second optical system is incident on the light from the first optical system. The reflective surface of the at least one first reflective member is disposed at or near the intersection of the optical axis of the first optical system and the optical axis of the second optical system.

53. The shooting system as described in claim 52, characterized in that, The first optical system is telecentric on one side of the optical splitting member.

54. The shooting system as described in claim 52 or 53, characterized in that, The reflective surface is positioned at the pupil position or pupil conjugate position of the optical system consisting of the imaging optical system and the first optical system.

55. The imaging system as described in any one of claims 51 to 54, characterized in that, The measuring light from the distance measuring device is projected onto the object via the at least one first reflecting member, the light splitting member, and the imaging optical system.

56. The imaging system as described in any one of claims 51 to 55, characterized in that, At least a portion of the distance measuring device is configured at a position where the measuring light can be irradiated onto the light splitting member via the at least one first reflecting member.

57. The imaging system as described in any one of claims 51 to 56, characterized in that, Based on image information of the first image containing at least a portion of the object, the driving device is used to rotate the at least one first reflecting member, thereby illuminating the object with the measuring light.

58. The imaging system as described in any one of claims 51 to 57, characterized in that, Based on image information of the first image containing at least a portion of the object, the driving device is used to rotate the at least one first reflecting member, thereby setting the measurement optical path of the measurement light from the distance measuring device, such that the measurement light is emitted from the imaging optical system toward the object. The distance measuring device illuminates the object by illuminating the measuring light emitted from the imaging optical system through the set measuring optical path and the at least one first reflecting member.

59. The shooting system as described in claim 58, characterized in that, Based on image information of the first image containing at least a portion of the object, the at least one first reflecting member is rotated using the driving device, thereby selecting the measurement optical path from a plurality of optical paths formed by the imaging optical system.

60. The shooting system as described in claim 58 or 59, characterized in that, Based on image information of the first image containing at least a portion of the object, the at least one first reflecting member is rotated using the driving device, thereby setting the measurement optical path such that the measurement light passes through a portion of the image forming region of the image formed by another beam of light segmented by the light segmenting member using the imaging optical system, which contains at least a portion of the object.

61. The shooting system as described in any one of claims 51 to 60, characterized in that, Based on image information of the first image containing an image of at least a portion of the object, the driving device is used to rotate the at least one first reflecting member, thereby setting the measuring optical path of the measuring light from the distance measuring device such that the measuring optical path overlaps with at least a portion of the optical path of the light forming the image of at least a portion of the object, wherein the image of at least a portion of the object is contained in an image formed by another beam of light segmented by the light splitting member using the imaging optical system. By setting the measurement optical path such that at least a portion of the optical path of the light forming the image of the portion overlaps with at least a portion of the measurement optical path, the measurement light from the distance measuring device illuminates the object via the reflective surface and the imaging optical system.

62. The imaging system as described in any one of claims 51 to 61, characterized in that, Based on image information of the first image containing at least a portion of the object, the driving device is used to rotate the at least one first reflecting member in such a way that the measuring light sequentially illuminates a plurality of points of at least a portion of the object, thereby generating the distances of the plurality of points of at least a portion of the object as the distance information.

63. The imaging system as claimed in any one of claims 48 to 50 or any one of claims 60 and 61, characterized in that, The position where the image formed by the other beam of light split by the light splitting member is formed is conjugate to the position where the first image is formed.

64. The shooting system as described in any one of claims 42 to 63, characterized in that, The distance measuring device includes: A measuring light irradiation device that emits the measuring light; and a light receiving device that receives light from the object irradiated by the measuring light. Based on the light receiving result of the light from the object by the light receiving device, the distance information of the object is generated, wherein the object is illuminated by the measuring light from the measuring light irradiation device.

65. The imaging system as described in any one of claims 42 to 64, characterized in that, Based on the image information of the first image, which contains images of multiple objects, captured by the first imaging device, at least one of the multiple objects is identified as the target object. The measuring light is irradiated onto the identified object via the imaging optical system.

66. The imaging system as described in any one of claims 42 to 64, characterized in that, Based on the image information of the first image containing the image of the object captured by the first imaging device, the object is identified as the target object. The measuring light is irradiated onto the identified object via the imaging optical system.

67. The imaging system as described in any one of claims 42 to 66, characterized in that, It also includes an output device that outputs at least one of the orientation information of the object generated based on image information of the first image containing at least a portion of the object, and the distance information generated by the distance measuring device.

68. The shooting system as described in claim 67, characterized in that, The output device outputs image information of an enlarged image, generated based on image information of the first image containing at least a portion of the object, which is an image of at least a portion of the object in the first image.

69. The shooting system as described in claim 67, characterized in that, Based on at least one of the distance information and the direction information output from the output device, the electromagnetic wave irradiation device is controlled so that an electromagnetic wave different from the measuring light is irradiated onto the object.

70. The shooting system as described in claim 68, characterized in that, Based on at least one of the distance information and direction information output from the output device and the image information of the magnified image, the electromagnetic wave irradiation device is controlled so that electromagnetic waves different from the measurement light are irradiated onto the object.

71. The shooting system as described in claim 67, characterized in that, The image information of the first image is used as the first image information. The aforementioned direction information is used as the first direction information. When the distance information is used as the first distance information When the positional relationship between the object and the imaging system changes. The output device outputs at least one of the second distance information and the second direction information of the object, wherein the second distance information of the object is generated by illuminating the object with measuring light from the distance measuring device via the imaging optical system, based on the second image information of the first image containing at least a portion of the object captured by the first imaging device after the change in positional relationship, and the second direction information of the object is generated based on the second image information of the first image containing at least a portion of the object captured by the first imaging device after the change in positional relationship.

72. The shooting system as described in claim 68, characterized in that, The image information of the first image is used as the first image information. The image information of the magnified image is used as the third image information. The aforementioned direction information is used as the first direction information. When the distance information is used as the first distance information When the positional relationship between the object and the imaging system changes. The output device outputs at least one of the following: second distance information of the object, second direction information of the object, and fourth image information of the magnified image, wherein... The second distance information of the object is generated based on the second image information of the first image, which contains at least a portion of the object and was captured by the first imaging device after the change in positional relationship, by the distance measuring device illuminating the object with measuring light via the imaging optical system. The second orientation information of the object is generated based on the second image information of the first image, which contains at least a portion of the object, captured by the first imaging device after the positional relationship has changed. The fourth image information of the magnified image is generated based on the second image information of the first image, which contains at least a part of the object, captured by the first imaging device after the change in positional relationship. It is the image information of the magnified image obtained by magnifying the image of at least a part of the object in the first image.

73. The shooting system as described in claim 71, characterized in that, Based on at least one of the distance change information of the first distance information and the second distance information output from the output device, and the direction change information of the first direction information and the second direction information, the electromagnetic wave irradiation device is controlled so that electromagnetic waves different from the measuring light are continuously or intermittently irradiated onto the object as the positional relationship changes.

74. The shooting system as described in claim 72, characterized in that, Based on at least one of the distance change information of the first distance information and the second distance information, the direction change information of the first direction information and the second direction information, and the image change information of the third image information and the fourth image information output from the output device, the electromagnetic wave irradiation device is controlled such that electromagnetic waves different from the measuring light are continuously or intermittently irradiated onto the object as the positional relationship changes.

75. The imaging system as described in any one of claims 42 to 66, characterized in that, When the positional relationship between the object and the imaging system changes, the measuring light is continuously or intermittently irradiated onto the object based on the image information of the first image, which is captured by the first imaging device and contains an image of at least a part of the object.

76. The shooting system as described in claim 75, characterized in that, When the positional relationship between the object and the imaging system changes, the electromagnetic wave irradiation device is controlled based on the image information of the first image, which contains an image of at least a part of the object, captured by the first imaging device, so that electromagnetic waves different from the measuring light are continuously or intermittently irradiated onto the object.

77. The imaging system as described in any one of claims 33, 34, 37, 38, 41, 69, 70, 73, 74, and 76, characterized in that, The electromagnetic wave irradiation device is located in a different position from the imaging system.

78. The imaging system as described in any one of claims 33, 34, 37, 38, 41, 69, 70, 73, 74, 76, and 77, characterized in that, The electromagnetic waves are irradiated from the electromagnetic wave irradiation device onto the target object to repel or destroy the target object.

79. The imaging system according to any one of claims 33, 34, 37, 38, 41, 69, 70, 73, 74 or any one of claims 76 to 78, characterized in that, The electromagnetic wave is light.

80. The shooting system as described in claim 79, characterized in that, The output of the light irradiated by the electromagnetic wave irradiation device is 10kW or more.

81. An electromagnetic wave irradiation system, characterized in that, The system includes a photographing system as described in any one of claims 33, 34, 37, 38, 41, 69, 70, 73, 74 or any one of claims 76 to 80; and an electromagnetic wave irradiation device for irradiating electromagnetic waves onto a target object.

82. A measurement method, This measurement method uses an imaging system comprising an imaging optical system and a light-splitting component that splits at least a portion of the light passing through the imaging optical system, characterized in that... include: The first imaging device is used to capture a first image formed by a beam of light split by the light splitting member; The second imaging device is used to capture a second image formed by another beam of light split by the light splitting member; as well as Based on the image information of the first image captured by the first imaging device, which contains an image of at least a portion of the object, measurement light is irradiated onto the object via the imaging optical system, and distance information of the object is generated.

83. A measurement method, This measurement method uses an imaging system comprising an imaging optical system and a light-splitting component that splits the light passing through the imaging optical system, characterized in that... include: The first image formed by a beam of light split by the light splitting member is captured using an imaging device; as well as Based on the image information of the first image captured by the imaging device, which contains an image of at least a portion of the object, measurement light is irradiated onto the object via the imaging optical system, and distance information of the object is generated.

84. A program, characterized in that, The computer is made to perform the measurement method as described in claim 82 or 83.

Images