Imaging device and imaging system

Abstract

This imaging device comprises: a first optical system for forming an intermediate image of a subject; a second optical system for re-imaging at least a portion of the intermediate image to form a final image, in which the optical system allows for modification of the magnification of the final image; an imaging element for capturing the final image; and a drive unit for causing the second optical system and the imaging element to move in a direction intersecting with the optical axis of the first optical system.

Description

[Technical Field]

[0001] The present invention relates to an imaging device and an imaging system. [Background technology]

[0002] A camera is attached to a rotating platform device that allows panning and tilting, and the camera is pointed at an object (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 4,855,838 Summary of the Invention

[0004] According to a first aspect, an imaging device includes a first optical system that forms an intermediate image of a subject, a second optical system that re-images at least a portion of the intermediate image to form a final image and that is capable of changing the magnification of the final image, an imaging element that captures the final image, and a drive unit that moves the second optical system and the imaging element in a direction intersecting the optical axis of the first optical system. According to a second aspect, an imaging system includes a plurality of imaging devices according to the first aspect, and at least one of the plurality of imaging devices images the subject using information from an imaging device other than the at least one imaging device. According to a third aspect, the imaging device includes a first optical system that forms an intermediate image of a subject, and a second optical system that captures at least a part of the intermediate image. Zoom in Re-imaging By doing Forming the final image The The optical system includes a second optical system, an imaging element that captures the final image, and a drive unit that moves the second optical system and the imaging element in a direction that intersects with the optical axis of the first optical system. According to a fourth aspect, an imaging system includes a plurality of imaging devices according to the third aspect, and at least one of the plurality of imaging devices images the subject using information from an imaging device other than the at least one imaging device. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 1 is a diagram showing an overview of an imaging apparatus according to a first embodiment. [Figure 2] 4A and 4B are diagrams for explaining an intermediate image and an imaging region, which is a portion of the intermediate image that is re-imaged on an imaging element by a second optical system. [Figure 3] FIG. 2 is an enlarged view of a part of the first optical system and the second optical system. [Figure 4] FIG. 2 is a diagram showing an example of a usage state of the imaging device according to the first embodiment. [Figure 5] FIG. 2 is a diagram showing an example of an image captured by an imaging device. [Figure 6] FIG. 10 is a diagram showing another example of an image captured by the imaging device. [Figure 7] FIG. 10 is a diagram showing an overview of an imaging apparatus according to a second embodiment. [Figure 8] FIG. 10 is a diagram showing an example of a usage state of the imaging device according to the second embodiment. [Figure 9] 10A and 10B are diagrams showing examples of an image captured by an imaging device and an image generated by a virtual image generating unit according to a second embodiment. [Figure 10] 10A and 10B are views showing another example of an image captured by the imaging device of the second embodiment and an image generated by the virtual image generation unit. [Figure 11] FIG. 2 is a diagram showing a distance measurement unit in which part of the optical system is shared by the first optical system and the second optical system. DETAILED DESCRIPTION OF THE INVENTION

[0006] In this specification, "image data" refers to data representing a moving image or a still image. In this specification, the term "image" refers to image data that is visually displayed on a display or the like.

[0007] (Imaging device of the first embodiment) FIG. 1 is a diagram showing an overview of an imaging device 1 of a first embodiment. The X, Y, and Z directions indicated by arrows in FIG. 1 are positive directions. The X, Y, and Z directions are mutually orthogonal. In the following, a position in the X direction will be referred to as the X position, a position in the Y direction as the Y position, and a position in the Z direction as the Z position. The X, Y, and Z directions in the drawings referred to hereinafter indicate the same directions as the X, Y, and Z directions shown in FIG. 1, respectively.

[0008] The imaging device 1 of the first embodiment includes a first optical system 10, a second optical system 20, an imaging element 36, and a control unit 50. The first optical system 10 forms an intermediate image 18 of an object (not shown). The second optical system 20 re-images at least a portion of the intermediate image 18 formed by the first optical system 10 to form a final image 35 of the object.

[0009] An imaging element 36 having an imaging surface is disposed at a position where a final image 35 of the subject is formed. The imaging surface of the imaging element 36 coincides with the final image 35, and the center of the imaging surface in the X and Y directions roughly coincides with the optical axis AX2 of the second optical system 20. The imaging element 36 is held by the second lens barrel 21.

[0010] The imaging surface of the imaging element 36 and the image plane of the final image 35 formed by the second optical system 20 do not have to coincide with each other. For example, the imaging surface of the imaging element 36 and the image plane of the final image 35 formed by the second optical system 20 may be shifted in a direction along the optical axis AX2, as long as, for example, the image of the subject can be visually recognized on the image generated by imaging by the imaging element 36, or the image data of the subject can be recognized on the image data by existing image processing.

[0011] Furthermore, the center of the imaging surface of the imaging element 36 in the X and Y directions does not have to coincide with the optical axis AX2 of the second optical system 20. For example, the center of the imaging surface of the imaging element 36 in the X and Y directions may be shifted from the optical axis AX2 of the second optical system 20. For example, it is sufficient that the image of the subject can be visually recognized on the image generated by imaging with the imaging element 36, or that image data of the subject can be recognized on the image data by existing image processing.

[0012] As an example, the optical axis AX1 of the first optical system 10 and the optical axis AX2 of the second optical system 20 are both parallel to the Z direction. Also, as an example, the intermediate image plane on which the intermediate image 18 is formed is parallel to the XY plane perpendicular to the Z direction, and the final image plane on which the final image 35 is formed is also parallel to the XY plane.

[0013] The lenses 12 to 15 constituting the first optical system 10 are fixed to the first lens barrel 11, which is fixed to a housing 38 that is the outer frame of the imaging device 1. Note that one or more of the lenses 12 to 15 of the first optical system 10 do not have to be fixed to the first lens barrel 11, but may be movable relative to the first lens barrel 11. Also, the first lens barrel 11 does not have to be fixed to the housing 38. Note that when a subject is at a finite distance from the imaging device 1 (first optical system 10), the first optical system 10 may have a reduction magnification. Note that when a subject is at infinity relative to the imaging device 1, the size of the intermediate image 18 (intermediate image formation area 19) formed by the first optical system 10 may be smaller than the size in the radial direction (direction perpendicular to the optical axis) of the optical member constituting the first optical system 10 that is closest to the subject. In these cases, there is an advantage in that the movement stroke of the second optical system 20 can be reduced. As will be described later, lenses 25 to 28 that make up second optical system 20 are held in second lens barrel 21. Second lens barrel 21 is held in housing 38 via first drive unit 24.

[0014] As an example, first drive unit 24 includes stator 23 fixed to housing 38 and mover 22 fixed to second lens barrel 21 and movable in the XY plane relative to stator 23, and holds second lens barrel 21 while moving it in the XY plane relative to housing 38. That is, first drive unit 24 moves second optical system 20 and image sensor 36, which are supported by second lens barrel 21, in the XY plane relative to first optical system 10, which is fixed to housing 38. When first drive unit 24 drives second optical system 20, the center position of the area (i.e., the image capture area) to be re-imaged as final image 35 on image sensor 36 changes in intermediate image 18 formed on the intermediate image plane by first optical system 10.

[0015] As an example, first drive unit 24 may be a linear motor having stator 23 and slider 22, or may have a configuration including a stepping motor and a lead screw. Note that first drive unit 24 may also have a configuration including a stepping motor and a guide bar. Note that second lens barrel 21 is moved in the XY plane by first drive unit 24 while supporting second optical system 20 and image sensor 36 so that the optical positional relationship between second optical system 20 and image sensor 36 (for example, the relative positional relationship between second optical system 20 and image sensor 36) does not change.

[0016] The direction of movement of second lens barrel 21 by first driver 24 is not necessarily limited to the XY plane direction, but may be any direction intersecting the Z direction, i.e., any direction intersecting the optical axis AX1 of first optical system 10. In other words, first driver 24 may move second optical system 20 and image sensor 36 in a direction intersecting the optical axis AX1 of first optical system 10. The direction of movement of second lens barrel 21 may be a rotational direction, for example, at least one of the rotational direction about the X axis (θx direction), the rotational direction about the Y axis (θy direction), and the rotational direction about the Z axis (θz direction).

[0017] Of lenses 25 to 28 that constitute second optical system 20, lenses 25 and 28 are fixed to second barrel 21. In contrast, lens 26 is supported by movable support part 29, which is supported by drive mechanism 31, such as a linear motor, so that it can move in the optical axis direction (for example, ±Z directions) relative to second barrel 21. Lens 27 is supported by movable support part 30, which is supported by drive mechanism 32, such as a linear motor, so that it can move in the optical axis direction (for example, ±Z directions) relative to second barrel 21.

[0018] That is, the driving mechanisms 31 and 32 allow the lenses 26 and 27 to move in the optical axis direction relative to the lenses 25 and 28. The driving mechanisms 31 and 32 are not limited to the mechanisms including the linear motors described above, and may be movement mechanisms including a stepping motor and a lead screw.

[0019] Lens 26 may be configured to be movable in either the +Z direction or the −Z direction by drive mechanism 31, or may be configured to be movable in only one of the +Z direction or the −Z direction. Lens 27 may be configured to be movable in either the +Z direction or the −Z direction by drive mechanism 32, or may be configured to be movable in only one of the +Z direction or the −Z direction.

[0020] Hereinafter, the drive mechanisms 31 and 32, either together or individually, will also be referred to as a "second drive unit" 33. The second drive unit 33 drives the lenses 26 and 27, causing the lenses 26 and 27 to move in the Z direction, and the imaging magnification of the second optical system 20 (the magnification of the final image 35 relative to the intermediate image 18, i.e., the magnification of the final image 35) changes. Hereinafter, second lens barrel 21 that holds second optical system 20 and image sensor 36 will also be referred to as a "holding portion." Second lens barrel 21 may integrally hold second optical system 20 and image sensor 36. Second lens barrel 21 may also be configured to be integrally formed, or may be configured to have multiple components joined together.

[0021] 1, lenses 12 to 15 and lenses 25 to 28 are each shown as a single lens, but each may be a lens group consisting of multiple lenses, or may include a mirror, a diffractive optical element, etc. Furthermore, the number of lens groups constituting first optical system 10 is not limited to the four shown in the figure, and may be any other number.

[0022] The number of lens groups constituting the second optical system 20 is not limited to the four shown in the figure, and may be any number equal to or greater than 1. Furthermore, among the lens groups constituting the second optical system 20, the number of lens groups (or lenses) moved in the Z direction by the second drive unit 33 is not limited to the two groups described above, and may be any other number.

[0023] The imaging position of the final image 35 may be aligned with the imaging surface of the imaging element 36 by moving some of the lenses 26, 27, etc. in the optical axis direction by the second driving unit 33. Alternatively, a driving mechanism may be provided to move lenses 25, 28, etc., other than the lenses 26, 27, which are moved in the optical axis direction by the second driving unit 33, in the optical axis direction, and this driving mechanism may be used to match the imaging position of the final image 35 with the imaging surface of the imaging element 36. Alternatively, a driving mechanism may be provided to move the imaging element 36 in the optical axis direction, and this driving mechanism may be used to match the imaging surface of the imaging element 36 with the imaging position of the final image 35.

[0024] As described above, in the imaging device 1 of the first embodiment, at least a portion of the intermediate image 18 of the subject formed by the first optical system 10 is re-imaged by the second optical system 20 to form a final image 35 of the subject on the imaging element 36.

[0025] Figure 2 is a diagram showing the relationship between the intermediate image 18 and an imaging area IA1, which is a part of the intermediate image 18 that is re-imaged by the second optical system 20 and formed as the final image 35 on the imaging element 36, and is a view of the intermediate image 18 from the +Z direction. An intermediate image 18 of the subject is formed by the first optical system 10 in an intermediate image formation area 19, which is an image formation area of ​​the first optical system 10 and is, for example, substantially circular. As described above, the imaging area IA1 is the portion of the intermediate image 18 that is re-imaged on the imaging element 36 as the final image 35, and therefore corresponds to the field of view of the imaging device 1. Therefore, the imaging area IA1 may also be referred to as the field of view area of ​​the second optical system 20.

[0026] When the optical axis AX2 of the second optical system 20 coincides with the optical axis AX1 of the first optical system 10 and the imaging magnification of the second optical system 20 (the magnification of the final image 35) is minimum, the second optical system 20 re-images the intermediate image 18 within the rectangular wide-field imaging area IA0 indicated by the dashed line onto the imaging element 36 as the final image 35.

[0027] The center position CT of imaging area IA1, which is part of intermediate image 18 and is re-imaged on image sensor 36 as final image 35, changes when second lens barrel 21, which holds second optical system 20 and image sensor 36, is moved by first drive unit 24. The distance DX in the X direction and the distance DY in the Y direction to center position CT of imaging area IA1, which are based on optical axis AX1 of first optical system 10, match the distances in the X direction and Y direction from optical axis AX1 of first optical system 10 to optical axis AX2 of second optical system 10, respectively.

[0028] Furthermore, the size of the imaging area IA1, that is, the width WX in the X direction and the width WY in the Y direction, changes with a change in the imaging magnification of the second optical system 20 (the magnification of the final image 35) caused by driving the second driving unit 33. The second driving unit 33 changes the imaging magnification of the imaging device 1 as a whole by changing the imaging magnification of the second optical system 20.

[0029] As described above, the imaging area IA1 corresponds to the field of view of the imaging device 1, and therefore, it can be said that the field of view of the imaging device 1 changes when the first driving unit 24 or the second driving unit 33 is driven. More specifically, the center of the field of view of the imaging device 1 changes when the first driving unit 24 is driven. Furthermore, the width of the field of view of the imaging device 1 changes when the second driving unit 33 is driven.

[0030] It should be noted that the size relationship between the wide-field imaging area IA0 and the intermediate image formation area 19 of the first optical system 10 described above is not necessarily limited to the relationship shown in Fig. 2. For example, the wide-field imaging area IA0 may include the entire intermediate image formation area 19, in which case the entire intermediate image 18 of the first optical system 10 can be re-imaged as the final image 35 on the image sensor 36 by the second optical system 20. Alternatively, conversely, the entire wide-field imaging area IA0 may be included in the intermediate image formation area 19. It should be noted that the wide-field imaging area IA0 may be inscribed in a circle that defines the outer edge of the intermediate image formation area 19.

[0031] Note that first driver 24 may be configured to move first lens barrel 11 including first optical system 10 in the XY plane direction, instead of moving second lens barrel 21 including second optical system 20 and image sensor 36 in the direction intersecting with optical axis AX1 of first optical system 10 (for example, in the XY plane direction). Even in this configuration, first driver 24 can change the positional relationship between first optical system 10, second optical system 20, and image sensor 36 in the direction intersecting with optical axis AX2 of second optical system 20, by moving first optical system 10.

[0032] Note that first drive unit 24 may be configured to move first lens barrel 11 including first optical system 10 in a direction intersecting optical axis AX2 (e.g., in the XY plane) in addition to moving second lens barrel 21 including second optical system 20 and image sensor 36 in a direction intersecting optical axis AX1 (e.g., in the XY plane). Even in this configuration, first drive unit 24 can move first optical system 10, second optical system 20, and image sensor 36 relative to each other in a direction intersecting optical axis AX1 of first optical system 10 by moving first optical system 10, second optical system 20, and image sensor 36.

[0033] It should be noted that the second optical system 20 and the image sensor 36 do not have to be held integrally by a holding unit (second lens barrel 21). For example, the second optical system 20 may be held by the second lens barrel 21, and the image sensor 36 may be held by a holding mechanism separate from the second lens barrel 21. The first driver 24 may then move the second lens barrel 21 and the other holding mechanism in a direction intersecting the optical axis AX1 of the first optical system 10 so that the optical positional relationship between the second optical system 20 and the image sensor 36 (the relative positional relationship between the second optical system 20 and the image sensor 36) does not change. In this case, the second lens barrel 21 and the other holding mechanism move synchronously.

[0034] The imaging magnification of the second optical system 20 (i.e., the magnification of the final image 35) may be changed by a method other than moving the lenses 26 and 27 in the Z direction as described above. For example, a so-called liquid lens may be provided in the second optical system 20, and the imaging magnification of the second optical system 20 may be changed by changing the voltage applied to the liquid lens. In this case, the second driver 33 serves as a voltage controller that controls the voltage applied to the liquid lens.

[0035] The imaging magnification of the second optical system 20 may be changed by inserting or removing a predetermined optical system into or from the second optical system 20. In this case, the second drive unit 33 serves as an inserting / removing unit that inserts or removes the predetermined optical system. The optical system arranged on the optical path of the light from the first optical system 10 (i.e., at least a part of the second optical system 20) may be configured to be interchangeable between an optical system with a first magnification and an optical system with a second magnification, which have different imaging magnifications.

[0036] In this case, the magnification of the final image 35 changes depending on whether the optical system arranged on the optical path of the light from the first optical system 10 is an optical system with the first magnification or an optical system with the second magnification. In this case, the second drive unit 33 serves as a switching unit that switches the optical system arranged on the optical path of the light from the first optical system 10 between an optical system with the first magnification and an optical system with the second magnification. Also, the imaging magnification of the optical system (second optical system 20) arranged on the optical path of the light from the first optical system 10 may be changed by inverting at least a portion of this optical system.

[0037] Fig. 3 is an enlarged view of a portion of the first optical system 10 and the second optical system 20. Two light beams LB1 and LB2 shown in Fig. 3 are light beams that enter the first optical system 10 from mutually different subjects (not shown), form part of the intermediate image 18, and then enter the second optical system 20.

[0038] Light beam LB1 travels from the first optical system 10 toward a first location Q1 in an intermediate image formation area 19 where an intermediate image 18 is formed. Light beam LB2 travels from the first optical system 10 toward a second location Q2 in the intermediate image formation area 19 where an intermediate image 18 is formed. A distance D1 from the optical axis AX1 of the first optical system 10 to the first location Q1 is different from a distance D2 from the optical axis AX1 of the first optical system 10 to the second location Q2. The chief ray PR1 of the light beam LB1 is a ray that passes through the center of the light beam LB1, and the chief ray PR2 of the light beam LB2 is a ray that passes through the center of the light beam LB2.

[0039] As described above, in the imaging device 1 of the first embodiment, the second optical system 20 re-images at least a part of the intermediate image 18 of the subject formed by the first optical system 10, to form a final image 35 of the subject on the imaging element 36. Generally, in such a configuration, if the optical axis AX2 of the second optical system 20 is misaligned in a direction intersecting with the optical axis AX1 of the first optical system 10, the angle of incidence of the light beam incident on the second optical system 20 changes, and the imaging performance of the second optical system 20 deteriorates.

[0040] In contrast, in the imaging device 1 of the first embodiment, chief rays PR1, PR2, etc. of each light beam (light beams LB1, LB2, etc.) from the subject that is imaged by the first optical system 10 are incident on the intermediate image formation region 19 substantially parallel to the optical axis AX1. In other words, the first optical system 10 is so-called telecentric on the intermediate image 18 side. The angles of the traveling directions of the chief rays (chief rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) that enter the intermediate image formation region 19 relative to the optical axis AX1 may be, for example, within 1°. In this case, it can be said that the difference between the angle, relative to the optical axis AX1, of the chief ray PR1 of the light beam LB1 that travels from the first optical system 10 toward the first location Q1 of the intermediate image formation region 19 and the angle, relative to the optical axis AX1, of the chief ray PR2 of the light beam LB2 that travels from the first optical system 10 toward the second location Q2 of the intermediate image formation region 19 is within 1°.

[0041] The angle of the traveling direction of the chief ray (chief ray PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident on the intermediate image formation region 19 with respect to the optical axis AX1 may be within 0.5° or may be within 0.3°. In this case, the difference between the angle of the chief ray PR1 of the light beam LB1 traveling from the first optical system 10 to the first location Q1 of the intermediate image formation region 19 with respect to the optical axis AX1 and the angle of the chief ray PR2 of the light beam LB2 traveling from the first optical system 10 to the second location Q2 of the intermediate image formation region 19 with respect to the optical axis AX1 may be within 0.5° or may be within 0.3°. Furthermore, the angles of the traveling direction of the chief ray of all the light beams incident on the intermediate image formation region 19 with respect to the optical axis AX1 may be within 0.5° or may be within 0.3°. Furthermore, the angle of the traveling directions of two chief rays of light incident at different positions on the intermediate image forming area 19 with respect to the optical axis AX1 may be within 0.5° or 0.3°.

[0042] Therefore, the chief rays (chief rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) that enter the second optical system 20 via the intermediate image 18 also travel approximately parallel to the optical axis AX2 of the second optical system 20 and enter the second optical system 20. For this reason, even if the second optical system 20 is moved by the first drive unit 24 in a direction perpendicular to the optical axis AX1 of the first optical system 10 (in the XY plane), the incident angles of the light beams (light beams LB1, LB2, etc.) that enter the second optical system 20 hardly change. Therefore, even if the second optical system 20 is moved in the XY plane relative to the first optical system 10, the second optical system 20 maintains good imaging performance and forms a high-resolution final image 35 on the image sensor 36.

[0043] For example, an aperture stop 16 may be provided at a predetermined position to make the first optical system 10 telecentric on the side of the intermediate image 18. Alternatively, the effective diameter of a predetermined lens, such as the lens 14, may be limited to a predetermined size to make the first optical system 10 telecentric on the side of the intermediate image 18.

[0044] Note that by making the second optical system 20 telecentric on the intermediate image 18 side, i.e., on the first optical system 10 side, it is possible to prevent a change in the angle of incidence of the incident light beams (light beams LB1, LB2, etc.) on the second optical system 20 due to movement of the second optical system 20 in the XY plane. To achieve this, for example, an aperture stop (not shown) may be provided inside the second optical system 20, and this aperture stop may be set so that the chief rays PR1, PR2, etc. of the light beams (light beams LB1, LB2, etc.) passing through the second optical system 20 travel substantially parallel to the optical axis AX2 of the second optical system 20. Instead of an aperture stop, the effective diameters of the lenses 25 to 28, etc. constituting the second optical system 20, may be limited to a predetermined system, thereby making the second optical system 20 telecentric as described above.

[0045] The angle of the traveling direction of the chief ray (chief ray PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident on the second optical system 20 from the intermediate image formation region 19 with respect to the optical axis AX2 may be, for example, within 1°. In this case, it can be said that the difference between the angle of the chief ray PR1 of the light beam LB1 traveling from the first location Q1 of the intermediate image formation region 19 to the second optical system 20 with respect to the optical axis AX2 and the angle of the chief ray PR2 of the light beam LB2 traveling from the second location Q2 of the intermediate image formation region 19 to the second optical system 20 with respect to the optical axis AX2 is within 1°.

[0046] The angle of the traveling direction of the chief ray (chief ray PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident on the second optical system 20 from the intermediate image formation region 19 with respect to the optical axis AX2 may be within 0.5° or within 0.3°. In this case, the difference between the angle of the chief ray PR1 of the light beam LB1 traveling from the first location Q1 of the intermediate image formation region 19 to the second optical system 20 with respect to the optical axis AX2 and the angle of the chief ray PR2 of the light beam LB2 traveling from the second location Q2 of the intermediate image formation region 19 to the second optical system 20 with respect to the optical axis AX2 may be within 0.5° or within 0.3°.

[0047] The angles of the traveling directions of the chief rays of all light beams incident on the second optical system 20 from the intermediate image forming area 19 with respect to the optical axis AX2 may be within 0.5° or within 0.3°. The angles of the traveling directions of two chief rays from different positions among the light beams incident on the second optical system 20 from the intermediate image forming area with respect to the optical axis AX1 may be within 0.5° or within 0.3°.

[0048] Both the first optical system 10 and the second optical system 20 may be telecentric on the side of the intermediate image 18. In this case, the difference between the angle of the traveling direction of the chief rays (chief rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident from the first optical system 10 to the intermediate image formation region 19 relative to the optical axis AX1 and the angle of the traveling direction of the chief rays (chief rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident from the intermediate image formation region 19 to the second optical system 20 relative to the optical axis AX2 may be, for example, within 1°.

[0049] The optical axis AX1 of the first optical system 10 and the optical axis AX2 of the second optical system 20 in the intermediate image formation region 19 may be substantially parallel. Not limited to the intermediate image formation region 19, the optical axis AX1 of the entire first optical system 10 and the optical axis AX2 of the entire second optical system 20 may be substantially parallel.

[0050] The difference between the angle of the traveling direction of the chief rays (chief rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident on the intermediate image formation region 19 from the first optical system 10 relative to the optical axis AX1 and the angle of the traveling direction of the chief rays (chief rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident on the second optical system 20 from the intermediate image formation region 19 relative to the optical axis AX2 may be within 0.5° or within 0.3°. These conditions may be satisfied for all light beams passing through the intermediate image formation region 19, or may be satisfied for one light beam among the light beams passing through the intermediate image formation region 19.

[0051] It is not necessary that both the first optical system 10 and the second optical system 20 are telecentric on the side of the intermediate image 18. If either the first optical system 10 or the second optical system 20 is telecentric on the side of the intermediate image 18, the other optical system may be an optical system with a large numerical aperture (or a bright optical system) that passes through one optical system and the imaging area IA1 without blocking the light beams (light beams LB1, LB2, etc.).

[0052] 3 indicate the chief rays of the light beams that form an image on the outer edge of the intermediate image 18. Here, the outer edge of the intermediate image 18 corresponds to the outer periphery of the intermediate image formation region 19 of the intermediate image 18 shown in FIG. 2. The first optical system 10 has a half angle of view of angle θ and an angle of view of angle 2θ, both of which are centered in the +Z direction, which is parallel to the optical axis AX1.

[0053] The angle of view of the first optical system 10 may be, for example, 170° or more. It should be noted that the angle of view of the subject captured by the imaging device 1 is narrower than the angle of view of the first optical system 10, and therefore the angle of view of the first optical system 10 can also be called the maximum angle of view.

[0054] Hereinafter, the control unit 50 of the imaging device 1 will be described with reference to FIG. The control unit 50 controls various operations of the imaging device 1. The control unit 50 includes an imaging control unit 51, an image generation unit 52, an analysis unit 53, a field of view control unit 54, a storage unit 55, and an interface unit 56. The imaging control unit 51, the image generation unit 52, the analysis unit 53, the field of view control unit 54, the storage unit 55, and the interface unit 56 communicate signals with each other via wiring 58 such as a bus.

[0055] The imaging control unit 51 controls the imaging element 36 by sending a control signal S2 to the imaging element 36. Specifically, the imaging control unit 51 controls the imaging element 36 to start and end output of low-resolution moving images (through images), start and end output of high-resolution moving images, capture still images, set the light receiving time (exposure time) for the final image 35, set the gain of photoelectric conversion, and the like.

[0056] The image generation unit 52 receives the imaging signal S1 from the imaging element 36 and generates image data of the final image 35 captured by the imaging element 36. The image generation unit 52 may have a function to correct distortion of the final image 35 caused by distortion of at least one of the first optical system 10 and the second optical system 20 when generating the image data. The image generation unit may also have a function to correct degradation of the final image 35 caused by aberrations other than distortion of at least one of the first optical system 10 and the second optical system 20. The analysis unit 53 analyzes information about the subject included in the image data of the final image 35 generated by the image generation unit 52. Details of the information about the subject analyzed by the analysis unit 53 will be described later.

[0057] The field of view control unit 54 sends a drive signal S3 to the first drive unit 24, drives a drive member such as a linear motor included in the first drive unit 24 (moves the movable element 22 relative to the stator 23), and moves the second optical system 20 and the image sensor 36 in a direction intersecting the optical axis AX1 of the first optical system (for example, in the XY plane). The field of view control unit 54 sends a drive signal S4 to the second drive unit 24, and drives drive members such as linear motors included in the drive mechanisms 31 and 32 of the second drive unit 24, thereby moving the lenses 26 and 27 in the Z direction and changing the magnification of the second optical system 20, i.e., the magnification of the final image 35.

[0058] The field of view control unit 54 drives at least one of the first driving unit 24 and the second driving unit 24 based on the information of the subject analyzed by the analysis unit 53, and changes at least one of the center position and size (range) of the imaging area IA1 (the field of view of the imaging device 1). It can also be said that the field of view control unit 54 changes at least one of the center position and size of the field of view of the imaging device 1 by driving at least one of the first driving unit 24 and the second driving unit 24.

[0059] In addition, the field of view control unit 54 may send a drive signal S3 to the first drive unit 24 and move the second optical system 20 and the image sensor 36 based on a signal regarding a change in the center position of the field of view input from an operation unit (not shown) via the interface unit 56, regardless of the information on the subject analyzed by the analysis unit 53.

[0060] In addition, the field of view control unit 54 may send a drive signal S4 to the second drive unit 33 to move the lenses 26 and 27 based on a signal regarding a change in the size of the field of view input from an operation unit (not shown) via the interface unit 56.

[0061] In addition, the field of view control unit 54 may send a drive signal S3 to the first drive unit 24 and a drive signal S4 to the second drive unit 33 based on a signal regarding a change in the center position of the field of view and a signal regarding a change in the size of the field of view input from an operation unit (not shown) via the interface unit 56.

[0062] In addition, the operator may input at least one of a signal regarding a change in the center position of the field of view and a signal regarding a change in the size of the field of view to the field of view control unit 54 via the interface unit 56 by operating an operation unit not shown. The storage unit 55 includes a storage member such as a memory element or a magnetic disk, and stores the image data of the subject captured by the imaging element 36 and generated by the image generation unit 52 as needed.

[0063] The interface unit 56 outputs image data of the subject captured by the image sensor 36 and generated by the image generation unit 52, or image data stored in the storage unit 55, to an external device via the network line NW. The interface unit 56 may also receive commands for the imaging device 1 from an external device. The interface unit 56 may include a wireless transmission / reception mechanism, and may output image data and input commands wirelessly.

[0064] The imaging control unit 51, image generation unit 52, analysis unit 53, field of view control unit 54, storage unit 55, and interface unit 56 may be independent, i.e., mechanically separated, hardware components, or some of the imaging control unit 51, image generation unit 52, analysis unit 53, field of view control unit 54, storage unit 55, and interface unit 56 may be integrated into one or more pieces of hardware.

[0065] Furthermore, at least one of the imaging control unit 51, image generation unit 52, analysis unit 53, field of view control unit 54, memory unit 55, and interface unit 56 may be configured by hardware and software that controls it.

[0066] Fig. 4 is a diagram showing an example of how the imaging device 1 of the first embodiment is used. In the example shown in Fig. 4, a plurality of imaging devices 1a to 1c of the first embodiment are installed, for example, on a wall surface of a living room LM of a home where one or more users live. The plurality of imaging devices 1a to 1c of the first embodiment constitute an imaging system 2, which will be described later. The number of imaging devices 1 constituting the imaging system 2 is not limited to three as shown in Fig. 4, and may be any number of two or more.

[0067] When using the imaging device 1, it is not necessary to use a plurality of imaging devices 1a to 1c in combination, and it is also possible to use only one of the imaging devices 1a to 1c. In this example, the imaging device 1 (1a to 1c) generally always captures images of subjects in the living room LM, capturing suitable shutter opportunities to record, for example, the daily life of a family.

[0068] In the initial state or normal state, the imaging device 1 captures an image of a wide range of subject by, for example, setting the imaging magnification of the second optical system 20 to a low magnification, that is, by setting the size (width WX, width WY) of the imaging area IA1 relative to the intermediate image formation area 19 shown in Fig. 2 to a large value. As described above, the position and size of the imaging area IA1 relative to the intermediate image formation area 19 are set by the field of view control unit 54 driving the first drive unit 24 and the second drive unit 22.

[0069] The image generation unit 52 receives the imaging signal S1 from the imaging element 36 and generates image data (wide-area image data) of the final image 35. In the initial state or normal state, the imaging device 1 may set the imaging magnification of the second optical system 20 to the minimum magnification, in other words, widen the field of view of the imaging device 1 to the maximum, and capture an image of the wide-field imaging area IA0 shown in FIG.

[0070] 5(a) is a diagram showing a wide-area image Im1 based on wide-area image data of the living room LM as an example of an image captured by the imaging device 1, and as described above, a wide range of subjects in the living room LM are captured. The analysis unit 53 analyzes information about the subjects included in the wide-area image Im1, which is an image of the final image 35 generated by the image generation unit 52.

[0071] For example, the analysis unit 53 detects whether or not there is a region of interest IA in the wide-area image Im1 that should be particularly imaged or recorded, by analyzing the wide-area image data including information about the subject generated by the image generation unit 52. The region of interest IA is, for example, a region in the image data that includes a person, a part of a person such as a person's face, an animal, a part of an animal such as an animal's face, etc.

[0072] The area of ​​interest IA is not limited to at least a portion of a person or animal, but may be any area that includes an object of interest (e.g., a subject to be imaged or recorded). For example, the area of ​​interest IA may be an area that includes at least a portion of an object worn or carried by a person (clothing, shoes, a bag, a cane, etc.), a dangerous object such as a gun or explosive, a moving object such as a vehicle, a ship, or an aircraft (drone, etc.), or a building. For example, the area of ​​interest IA may be an area that includes at least a portion of the interior of a machining chamber (a space where a workpiece is machined with a machining tool) of a machine tool, and the interior of a machining tool exchanger that exchanges a machining tool attached to the spindle of the machining chamber with a different type of machining tool.

[0073] For example, the region of interest IA may be the interior of a machining chamber of a machine tool, including at least a portion of the machining tool attached to the spindle of the machining chamber, the workpiece placed on the stage of the machining chamber, the machining point of the workpiece with the machining tool, chips produced by machining the workpiece with the machining tool, and cutting fluid applied to the portion machined by the machining tool.For example, the region of interest IA may be the interior of an exchange device that exchanges the machining tool attached to the spindle of the machining chamber with a different type of machining tool, including at least a portion of the stored machining tool.

[0074] For example, the analysis unit 53 may detect the region of interest IA from the wide area image Im1 by performing object detection processing using machine learning to analyze wide area image data including information about the subject generated by the image generation unit 52. As an example, the analysis unit 53 may detect the region of interest IA by performing object detection processing using deep learning.

[0075] More specifically, the analysis unit 53 may detect the region of interest IA from the wide area image Im1 using an object detection algorithm that uses a convolutional neural network (CNN). Note that the analysis unit 53 may also detect the region of interest IA from the wide area image Im1 by other object detection processing using deep learning.

[0076] For example, the analysis unit 53 may use an object detection algorithm that uses R-CNN (Region with CNN features), Faster R-CNN (Faster Region with CNN features), Mask R-CNN (Mask Region with CNN features), etc. as a region proposal type. Furthermore, the analysis unit 53 is not limited to the region proposal type, and may use other deep learning object detection algorithms such as YOLO (You Only Look Once) and SSD (Single Shot Multibox Detector).

[0077] The analysis unit 53 is not limited to deep learning, and may detect the region of interest IA using an algorithm such as linear regression, a decision tree, or a support vector machine (SVM). The analysis unit 53 is not limited to machine learning, and may detect the region of interest IA using an existing template matching process.

[0078] For example, since the wide-area image Im1 shown in Fig. 5(a) includes images of multiple people, the analysis unit 53 detects the area including the multiple people indicated by the dashed lines as the region of interest IA. Note that the region of interest IA to be detected is not limited to one, and the analysis unit 53 may detect multiple regions of interest IAs from the image. When the analysis unit 53 detects the region of interest IA, it calculates the position and size of the region of interest IA in the wide area image Im1, and then transmits the position and size of the region of interest IA in the wide area image Im1 to the field of view control unit 54.

[0079] Upon receiving a signal from the analysis unit 53, the field of view control unit 54 sends a drive signal S3 to the first drive unit 24 to move the second optical system 20 and the image sensor 36 in a direction intersecting the optical axis AX1 (for example, in the XY plane) so that the center position CT (see FIG. 2) of the imaging area IA1 approximately coincides with the center position of the area of ​​interest IA. The field of view control unit 54 sends a drive signal S4 to the second drive unit 24 to change the magnification of the second optical system 20, i.e., the magnification of the final image 35, so that the size of the imaging area IA1 approximately coincides with the size of the area of ​​interest IA.

[0080] As a result, an image corresponding to the region of interest IA is formed on the imaging element 36, and the imaging device 1 captures the enlarged image Im2 shown in FIG. 5(b), which corresponds to an enlarged image of the portion of the region of interest IA in the wide-area image Im1. In the imaging device 1, the center position CT (see FIG. 2) of the imaging area IA1 can be changed only by moving the second optical system 20 in the XY plane, without rotating the entire imaging device 1 or the entire first optical system 10 and second optical system 20. This allows the imaging device 1 to quickly change the center position CT of the imaging area IA1, i.e., change the field of view of the imaging device 1.

[0081] The analysis unit 53 may also analyze the enlarged image Im2, i.e., the subject of the area of ​​interest IA, and may detect, for example, whether the subject, such as a person (i.e., a person included in the area of ​​interest), shows a particular facial expression such as a smile, or whether it makes a particular gesture or hand movement.

[0082] For example, the analysis unit 53 may detect a specific facial expression such as a smile, or a specific gesture or hand movement of a person, etc. (a person, etc. included in the region of interest), by executing the above-mentioned object detection process, an existing tracking process (described later), etc. Note that the analysis unit 53 may detect a specific facial expression such as a smile, or a specific gesture or hand movement of a person, etc. (a person, etc. included in the region of interest), by other image processing, not limited to the above-mentioned object detection process or existing tracking process.

[0083] In addition, not limited to the enlarged image Im2, when analyzing the wide-area image data generated by the image generation unit 52 in the initial state or normal state, the analysis unit 53 may detect whether a person, etc. included in the area of ​​interest shows a specific facial expression such as a smile, or whether or not a specific gesture or hand movement is made, and may perform one or more of the following actions A to G. When the above-mentioned facial expression, gesture, or hand movement of the subject is detected, the analysis unit 53 may perform one or more of the following actions A to G.

[0084] A: A command is issued to the imaging control unit 51 to cause the imaging element 36 to capture a still image. B: A command is issued to the imaging control unit 51 to cause the imaging element 36 to start or stop outputting low-resolution video data. C: A command is issued to the imaging control unit 51 to cause the imaging element 36 to start or stop outputting high-resolution video data.

[0085] D: A command is issued to the storage unit 55 to start or end storage of image data of the subject captured by the imaging element 36 and generated by the image generation unit 52. E: A command is issued to the storage unit 55 to start or end the addition of a predetermined identification signal (flag) to the image data being stored.

[0086] F: A command is issued to the interface unit 56 to start or end transmission of image data of the subject captured by the imaging element 36 and generated by the image generation unit 52 to an external device. G: Issue a command to the field of view control unit 54 to make the imaging area IA1 have a wide field of view, i.e., to reduce the imaging magnification of the second optical system 20, or issue a command to make the imaging area IA1 have a narrow field of view, i.e., to increase the imaging magnification of the second optical system 20.

[0087] In addition, when the analysis unit 53 detects the above-mentioned facial expressions, gestures, or hand movements of a subject (for example, a person included in the area of ​​interest) and starts any of the above-mentioned actions A to G, the analysis unit 53 may end those actions after a predetermined time has elapsed.

[0088] Furthermore, the analysis unit 53 may start any of the above-mentioned operations A to G when it detects the area of ​​interest IA from the wide-area image Im1, regardless of the above-mentioned facial expressions, gestures, or hand movements of the subject (for example, a person included in the area of ​​interest), or when the field of view control unit 54 subsequently determines that the size of the imaging area IA1 approximately matches the size of the area of ​​interest IA.

[0089] The analysis unit 53 may detect whether or not the area of ​​interest IA in the enlarged image Im2 has moved (the movement of a subject (person, animal, etc.) to be imaged or recorded that is included in the area of ​​interest IA). If the area of ​​interest IA has moved, the analysis unit 53 may issue a command to the field of view control unit 54 to move the imaging area IA1 (the field of view of the imaging device 1) in accordance with the movement of the area of ​​interest IA.

[0090] Upon receiving this command, the field of view control unit 54 sends a drive signal S3 to the first drive unit 24 to move the positions of the second optical system and the image sensor 36. Therefore, even if the subject to be imaged or recorded that is included in the region of interest IA moves relative to the image capture device 1, the image capture device 1 can continue to image the region of interest IA without the region of interest IA moving out of the image capture area IA1 of the image capture device 1.

[0091] The analysis unit 53 may issue a command to the field of view control unit 54 to change the size of the imaging area IA1 (the field of view of the imaging device 1) together with a command to move the imaging area IA1 (the field of view of the imaging device 1). The field of view control unit 54 may then send a drive signal S3 to the first control unit 24 and a drive signal S4 to the second control unit 33, thereby moving the positions of the second optical system 20 and the imaging element 36 and the lenses 26 and 27.

[0092] In this case, at least a portion of the period during which the positions of the second optical system 20 and the image sensor 36 move overlaps with at least a portion of the period during which the positions of the lenses 26 and 27 move. Note that the entire period during which the positions of the second optical system 20 and the image sensor 36 move does not have to completely overlap with the entire period during which the positions of the lenses 26 and 27 move. In other words, at least a portion of the period during which the imaging area IA1 (field of view of the imaging device 1) moves may overlap with at least a portion of the period during which the size of the imaging area IA1 (field of view of the imaging device 1) is changed.

[0093] For example, the analysis unit 53 may execute an existing tracking process to detect movement of the region of interest IA between multiple enlarged images Im2 captured at different times by the image sensor 36. As an example, the analysis unit 53 may use image data of the region of interest IA detected from the wide-area image Im1 by the above-described object detection process as a template to detect the region of interest IA from each of the enlarged images Im2 captured at different times by an existing template matching process, and calculate the amount and direction of movement of the region of interest IA on the enlarged image Im2 (i.e., temporal displacement).

[0094] Then, the analysis unit 53 calculates the amount of movement and direction of movement of the imaging area IA1 based on the calculated amount of movement and direction of movement of the area of ​​interest IA, and issues a command to the field of view control unit 54 to move the imaging area IA1. In response to this command, the field of view control unit 54 sends a drive signal S3 corresponding to the amount of movement and direction of movement of the imaging area IA1 to the first drive unit 24, and moves the positions of the second optical system 20 and the image sensor 36.

[0095] Note that the analysis unit 53 does not have to use the image data of the region of interest IA detected from the wide-area image Im1 as a template. For example, the analysis unit 53 may use the image data of the region of interest IA detected from the enlarged image Im2 captured at a first time by the above-described object detection process as a template, detect the region of interest IA from the enlarged image Im2 captured at a second time a predetermined time later, and calculate the amount and direction of movement of the region of interest IA between the first time and the second time.

[0096] The analysis unit 53 may calculate the amount of movement and the direction of movement of the region of interest IA by other existing tracking processing, not limited to the template matching processing. The analysis unit 53 may calculate the amount of movement and the direction of movement of the region of interest IA by other existing image processing, not limited to the existing tracking processing.

[0097] The analysis unit 53 may use existing deep learning to predict the amount and direction of movement of the region of interest IA per predetermined time period, based on the amount and direction of movement of the region of interest IA between different times calculated in the tracking process described above. In this case, the analysis unit 53 calculates (predicts) the amount and direction of movement of the imaging area IA1 per predetermined time period, based on the predicted amount and direction of movement of the region of interest IA per predetermined time period, and issues a command to the field of view control unit 54 to move the imaging area IA1.

[0098] The analysis unit 53 may detect whether or not the region of interest IA has moved in any of the image data captured by the image sensor 36, not limited to the enlarged image Im2, and issue a command to the field of view control unit 54 to move the imaging region IA1 (field of view of the imaging device 1) in accordance with the movement of the region of interest IA. For example, the analysis unit 53 may detect whether or not the region of interest IA has moved in the wide-area image Im1.

[0099] For example, when a subject to be imaged or recorded that is included in the area of ​​interest IA moves relative to the imaging device 1 in the initial or normal state, the analysis unit 53 may detect the movement of the area of ​​interest IA in the wide-area image Im1 detected by analyzing the wide-area image data generated by the image generation unit 52, and issue a command to the field of view control unit 54 to move the imaging area IA1 in accordance with the movement of the area of ​​interest IA.

[0100] In this case, the analysis unit 53 may also calculate the amount of movement and the direction of movement (displacement) of the area of ​​interest IA by the tracking process described above. Alternatively, the analysis unit 53 may use existing deep learning to predict the amount of movement and the direction of movement of the area of ​​interest IA per predetermined time period, based on the amount of movement and the direction of movement of the area of ​​interest IA between different times calculated by the tracking process described above.

[0101] In addition, if the imaging device 1 and the subject move relatively, not just the subject to be imaged or recorded that is included in the area of ​​interest IA, the analysis unit 53 detects the movement of the area of ​​interest IA on the image and issues a command to the field of view control unit 54 to move the imaging area IA1 in accordance with the movement of the area of ​​interest IA, thereby allowing the area of ​​interest IA to continue to be imaged without moving out of the imaging area IA1 of the imaging device 1.

[0102] Note that even if the analysis unit 53 executes the above-described object detection process, if the area of ​​interest IA cannot be recognized in the enlarged image Im1, it may determine that the subject to be imaged or recorded that is included in the area of ​​interest IA has moved out of the imaging area IA1, and may issue a command to the field of view control unit 54 to reduce the imaging magnification of the second optical system 20 in order to re-detect the subject to be imaged or recorded. By reducing the imaging magnification of the second optical system 20, the field of view of the imaging device 1 becomes wider, and therefore the search range for the subject to be imaged or recorded can be expanded.

[0103] Furthermore, when the distance between the subject to be imaged or recorded and the imaging device 1 changes, the analysis unit 53 may detect whether or not the size of the region of interest IA in the enlarged image Im2 (the size of the image of the subject to be imaged or recorded (person, animal, etc.) included in the region of interest IA) has changed. If the size of the region of interest IA has changed, the analysis unit 53 may issue a command to the field of view control unit 54 to change the size of the imaging region IA1 in accordance with the change in the size of the region of interest IA. Upon receiving this command, the field of view control unit 54 sends a drive signal S3 to the second drive unit 33 to change the imaging magnification of the second optical system.

[0104] For example, when the analysis unit 53 detects that the size of the region of interest IA has changed, it may issue a command to the field of view control unit 54 to change the imaging region IA1 so as to offset the change in the size of the region of interest IA. For example, when the analysis unit 53 detects that the region of interest IA has increased in size, it may issue a command to the field of view control unit 54 to increase the imaging region IA1. In response to this command, the field of view control unit 54 reduces the imaging magnification of the second optical system.

[0105] On the other hand, when the analysis unit 53 detects that the region of interest IA has become smaller, it may issue a command to the field of view control unit 54 to reduce the imaging region IA1. In response to this command, the field of view control unit 54 increases the imaging magnification of the second optical system. Therefore, even if the distance between the imaging device 1 and a subject to be imaged or recorded and included in the region of interest IA changes, the imaging device 1 can continue to image the region of interest IA at a generally constant size relative to the imaging region IA1 (the field of view of the imaging device 1).

[0106] The analysis unit 53 may issue a command to the field of view control unit 54 to change the position of the imaging area IA1 together with a command to change the imaging magnification (the size of the imaging area IA1). The field of view control unit 54 may then send a drive signal S3 to the first control unit 24 while sending a drive signal S4 to the second control unit 33, thereby moving the positions of the lenses 26 and 27 and the positions of the second optical system 20 and the image sensor 36.

[0107] For example, the analysis unit 53 may perform the above-described object detection process to detect a change in the size of the region of interest IA on the image between multiple enlarged images Im2 captured at different times by the image sensor 36. For example, the analysis unit 53 may use the above-described object detection process to detect the region of interest IA from each of enlarged images Im2 captured at a first time and a second time a predetermined time after the first time, and calculate the amount of change in the size of the region of interest IA at the second time relative to the size of the region of interest IA at the first time.

[0108] Then, the analysis unit 53 may calculate the amount of change in the size of the imaging area IA1 based on the calculated amount of change in the size of the region of interest IA, and issue a command to the field of view control unit 54 to change the size of the imaging area IA1. In response to this command, the field of view control unit 54 sends a drive signal S3 corresponding to the amount of change in the size of the imaging area IA1 to the second drive unit 33, and changes the imaging magnification of the second optical system 20.

[0109] In addition, the analysis unit 53 may detect changes in the image size of the area of ​​interest IA between multiple enlarged images Im2 captured by the image sensor 36 at different times by performing other existing image processing, not limited to the above-mentioned object detection processing.

[0110] The analysis unit 53 may use existing deep learning to predict the amount of change in size of the region of interest IA per predetermined time period in the future, based on the amount of change in size of the region of interest IA between different times calculated by the object detection device described above. In this case, the analysis unit 53 calculates (predicts) the amount of change in size of the imaging region IA1 per predetermined time period, based on the predicted amount of change in size of the region of interest IA per predetermined time period, and issues a command to the field of view control unit 54 to change the size of the imaging region IA1.

[0111] The analysis unit 53 may detect whether or not there has been a change in the size of the region of interest IA in any image captured by the image sensor 36, not limited to the enlarged image Im2, and issue a command to the field of view control unit 54 to move the size of the imaging region IA1 (field of view of the imaging device 1) in accordance with the change in the size of the region of interest IA. For example, the analysis unit 53 may detect whether or not there has been a change in the size of the region of interest IA in the wide-area image Im1.

[0112] For example, in the initial or normal state, if the distance between the subject to be imaged or recorded contained in the area of ​​interest IA and the imaging device 1 changes, the change in size of the area of ​​interest IA in the wide-area image Im1 detected by analyzing the wide-area image data generated by the image generation unit 52 can be detected, and a command can be issued to the field of view control unit 54 to change the size of the imaging area IA1 in accordance with the change in the size of the area of ​​interest IA.

[0113] 4, the imaging devices 1 (1a to 1c) are installed in the living room LM, but this is not limiting. For example, one or more imaging devices 1 may be installed in a predetermined location indoors in a building to capture and record images of any object (subject to be captured or recorded). As an example, they may be installed in a predetermined location indoors in a nursery school, school, medical facility, nursing home, conference room, shop, train station, etc. to capture and record images of any object, such as a person. One or more imaging devices 1 may be installed in a predetermined location inside a moving body to capture and record images of any object (subject to be captured or recorded).

[0114] As an example, the imaging device 1 may be installed in a predetermined location inside a vehicle, ship, aircraft, etc., to capture and record images of any object. Note that, for example, one or more imaging devices 1 may be installed in a predetermined location outdoors to capture and record images of any object (subject to be captured or recorded), such as a person or animal. As an example, one or more imaging devices 1 may be installed in a predetermined location on a building, such as a vending machine, streetlight, telephone pole, bridge, or entrance to a store or station, to capture and record images of any object.

[0115] In addition, one or more imaging devices 1 may be installed at a predetermined location on a moving object such as a vehicle, ship, or aircraft (drone, etc.), and images and records any object (subject to be imaged or recorded) such as a person or animal from the moving object.

[0116] FIG. 6 is a diagram showing an example of an image based on image data generated by imaging by the imaging device 1 in another example of the usage state of the imaging device 1 of the first embodiment. In this example, the imaging device 1 is installed outdoors as an example and used as a surveillance camera that monitors the surroundings of the imaging device 1. Note that even when used as a surveillance camera, one or more imaging devices 1 may be installed indoors. Note that one or more imaging devices 1 may be installed as surveillance cameras in predetermined locations that require surveillance, such as shops, train stations, airports, medical facilities, nursing homes, prisons, military facilities, borders, roads, and parks, to capture and record any object (subject to be captured or recorded). Note that one or more imaging devices 1 may be installed on a mobile object such as a vehicle, ship, or aircraft (drone, etc.) to capture and record any object that requires surveillance from the mobile object.

[0117] 6(a) is a diagram showing a wide-area image Im3 based on wide-area image data captured of a scene around the imaging device 1 installed outdoors, as an example of an image captured by the imaging device 1. The analysis unit 53 analyzes information about the subject included in the wide-area image Im3, and detects whether or not there is a region of interest IA that should be captured or recorded. Note that the analysis unit 53 may detect the region of interest IA by executing the object detection process described above, similar to the usage state of the imaging device 1 shown in FIGS. 4 and 5 above.

[0118] 6(a) includes an image of a person, and as an example, the analysis unit 53 detects the area including the image of the person indicated by the dashed line as the region of interest IA. The analysis unit 53 calculates the position and size of the region of interest IA in the wide-area image Im3 and transmits them to the field of view control unit 54.

[0119] Upon receiving the signal from the analyzer 53, the field-of-view controller 54 sends a drive signal S3 to the first driver 24, which moves the second optical system 20 and the image sensor 36 in the XY plane, causing the center position CT (see FIG. 2) of the image sensing area IA1 to approximately coincide with the center position of the area of ​​interest IA. As a result, an image of the subject corresponding to the area of ​​interest A2 is formed on the image sensor 36, and the imaging device 1 captures an enlarged image Im4 shown in FIG. 6(b), which corresponds to an enlarged image of the portion of the area of ​​interest A2 in the wide-area image Im3.

[0120] The analysis unit 53 may detect whether the region of interest IA in the enlarged image Im4 has moved. If a subject (such as a person or an animal) to be imaged or recorded and included in the region of interest IA moves relative to the imaging device 1 (for example, if the subject to be imaged or recorded and included in the region of interest IA moves), the analysis unit 53 may issue a command to the field of view control unit 54 to move the imaging region IA1 in accordance with the movement of the region of interest IA.

[0121] Upon receiving this command, the field of view control unit 54 sends a drive signal S3 to the first drive unit 24 to move the positions of the second optical system 20 and the image sensor 36. Therefore, even if the subject to be imaged or recorded that is included in the region of interest IA moves relative to the image capture device 1, the image capture device 1 can continue to image the region of interest IA without the region of interest IA moving out of the image capture area IA1 of the image capture device 1.

[0122] 4 and 5, the analysis unit 53 may execute the tracking process described above to detect movement of the area of ​​interest IA between multiple enlarged images Im2 captured by the image sensor 36 at different times. The analysis unit 53 may detect movement of the area of ​​interest IA by other existing image processing, not limited to the existing tracking process. The analysis unit 53 may issue a command to the field of view control unit 54 to change the size of the imaging area IA1, along with a command to move the imaging area IA1.

[0123] 4 and 5, the analysis unit 53 may use existing deep learning to predict the amount and direction of movement of the region of interest IA per predetermined time period, based on the amount and direction of movement of the region of interest IA between different times calculated by the tracking process described above. In this case, the analysis unit 53 calculates (predicts) the amount and direction of movement of the imaging area IA1 per predetermined time period, based on the predicted amount and direction of movement of the region of interest IA per predetermined time period, and issues a command to the field of view control unit 54 to move the imaging area IA1.

[0124] Furthermore, the analysis unit 53 may analyze the size of the region of interest IA in the enlarged image Im4 in accordance with the relative movement (e.g., movement of the subject) between the subject to be imaged or recorded and the imaging device 1, which is included in the region of interest IA. If the proportion of the region of interest IA in the enlarged image Im4 becomes higher than a predetermined proportion, the analysis unit 53 may issue a command to the field of view control unit 54 to reduce the imaging magnification of the second optical system 20.

[0125] On the other hand, if the proportion of the region of interest IA in the enlarged image Im4 becomes lower than a predetermined proportion, a command may be issued to the field of view control unit 54 to increase the imaging magnification of the second optical system 20. Note that a case where the proportion of the region of interest IA in the enlarged image Im4 becomes higher than a predetermined proportion is, for example, a case where the distance between the imaging device 1 and a subject to be imaged or recorded, which is included in the region of interest IA, becomes smaller, causing at least a part of the region of interest IA to fall outside the enlarged image Im4.

[0126] In addition, when the proportion of the area of ​​interest IA in the enlarged image Im4 becomes lower than a predetermined proportion, for example, it is when the distance between the subject to be imaged or recorded contained in the area of ​​interest IA and the imaging device 1 becomes too large, making it impossible to recognize the area of ​​interest IA on the enlarged image Im4.

[0127] The analysis unit 53 may calculate the size of the area of ​​interest IA in the image by the object detection process described above. The analysis unit 53 may issue a command to the field of view control unit 54 to change the position of the imaging area IA1 together with a command to change the imaging magnification.

[0128] The analysis unit 53 may calculate the size of the region of interest IA in the enlarged image Im4, and issue a command to the field of view control unit 54 to change the imaging magnification of the second optical system 20 so that the proportion of the region of interest IA in the enlarged image Im4 becomes a constant value. Note that the analysis unit 53 may calculate the size of the region of interest IA in the image by the object detection process described above.

[0129] Furthermore, even if the analysis unit 53 executes the object detection process described above, if the region of interest IA cannot be recognized within the enlarged image Im4, the analysis unit 53 may issue a command to the field of view control unit 54 to reduce the imaging magnification of the second optical system 20 in order to re-detect the subject to be imaged or recorded, assuming that the subject to be imaged or recorded, which is included in the region of interest IA, has moved out of the imaging area IA1 for capturing the enlarged image Im4. By reducing the imaging magnification of the second optical system 20, the field of view of the imaging device 1 is widened, and therefore the search range for the subject to be imaged or recorded can be expanded.

[0130] In addition, if the analysis unit 53 cannot recognize the area of ​​interest IA in the enlarged image Im4, it may issue a command to the field of view control unit 54 to increase the imaging magnification of the second optical system 20 in order to re-detect the subject to be imaged or recorded, assuming that the subject to be imaged or recorded contained in the area of ​​interest IA has become smaller than the imaging area IA1 that captures the enlarged image Im4.

[0131] By increasing the imaging magnification of the second optical system 20, the area of ​​interest IA including the subject to be imaged or recorded can be detected again on the image data even if the subject to be imaged or recorded is located far away from the imaging device 1. The analysis unit 53 may issue a command to the field of view control unit 54 to change the position of the imaging area IA1 together with a command to change the imaging magnification.

[0132] Note that, if the analysis unit 53 cannot recognize the region of interest IA in the enlarged image Im4 even after executing the above-described object detection process, the analysis unit 53 may issue a command to the field of view control unit 54 to change the position of the imaging region IA1 (or may drive the first drive unit 24) in order to re-detect the subject to be imaged or recorded that is included in the region of interest IA. In this case, the analysis unit 53 may issue a command to the field of view control unit 54 to change the position of the imaging region IA1 while changing the imaging magnification of the second optical system 20.

[0133] In the example shown in Fig. 6, the imaging device 1 may perform the above-described operations A to G in the same manner as in the example shown in Fig. 5. Furthermore, the analysis unit 53 may analyze whether or not a person in the region of interest A2 is carrying a dangerous object such as a gun or a bomb, and if a person is carrying a predetermined dangerous object, may start any of the above-described operations C, D, E, F, and G.

[0134] (Effects of the imaging device of the first embodiment) (1) The imaging device 1 of the first embodiment includes a first optical system 10 that forms an intermediate image 18 of a subject, a second optical system 20 that is an optical system that re-images at least a portion of the intermediate image 18 to form a final image 35 and that is capable of changing the magnification of the final image 35, an imaging element 36 that images the final image 35, and a first drive unit 24 that moves the second optical system 20 and the imaging element 36 in a direction that intersects with the optical axis AX1 of the first optical system 10. With this configuration, the imaging device 1 can quickly extract a desired imaging area IA1 from a wide-field imaging area (such as wide-field imaging area IA0) and capture an image with high resolution.

[0135] (2) The first optical system 10 may be telecentric on the side of the intermediate image 18. In this case, even if the second optical system 20 moves in a direction intersecting the optical axis AX1 of the first optical system 10, the angle of incidence of the light beams (light beams LB1, LB2, etc.) incident on the second optical system 20 changes very little. Therefore, the second optical system 20 can maintain good imaging performance and form a high-resolution final image 35 on the image sensor 36.

[0136] (3) The maximum angle of view of the first optical system 10 may be 170° or more, in which case it is possible to capture an image of an imaging area with a wider field of view (such as wide-field imaging area IA0). (4) The image sensor 36 may further include an analysis unit 53 capable of analyzing image data of the subject generated by the image sensor 36 .

[0137] (5) The system may further include a field-of-view control unit 54 that drives the first driving unit 24 and / or changes the magnification of the final image 35 based on the results of the analysis by the analysis unit 53. This configuration allows a desired imaging area IA1 to be extracted at high speed and imaged at high resolution based on information about the subject imaged in a wide-field imaging area (such as wide-field imaging area IA0). Alternatively, an image of the subject can be formed at an appropriate size within the imaging area IA1.

[0138] (6) The camera may further include an imaging control unit 51 that controls the start and end of recording of image data generated by imaging by the imaging element 36 based on the results of the analysis by the analysis unit 53. With this configuration, the start and end of recording of image data can be controlled based on information about the subject, such as the facial expression, gestures, or the shape of an item held by the subject.

[0139] (Imaging device of second embodiment) 7 is a diagram showing an overview of an imaging device 1A according to the second embodiment. The configuration of the imaging device 1A according to the second embodiment is generally similar to that of the imaging device 1 according to the first embodiment described above, and therefore the same components are denoted by the same reference numerals and descriptions thereof will be omitted where appropriate.

[0140] The imaging device 1A of the second embodiment includes a distance measurement unit 40 enclosed by a dashed line, in addition to a first optical system 10, a second optical system 20, an image sensor 36, and a control unit 50. The control unit 50 also includes a virtual image generation unit 57.

[0141] The distance measurement unit 40 measures the distance from the imaging device 1A to at least a portion of the subject. The distance measurement unit 40 includes, for example, a light-emitting unit 41, a light-transmitting optical system 42, a light-receiving unit 44, a light-receiving optical system 45, and a measurement unit 47. In response to a command from the measurement unit 47, the light-emitting unit 41, such as a laser diode array, emits pulses of measurement light 43 toward the subject via the light-transmitting optical system 42, and the light-receiving unit 44 receives, via the light-receiving optical system 45, detection light 46, which is light that is the measurement light 43 reflected by at least a portion of the subject. Note that the light-receiving unit 44 can also be referred to as a detection element because it detects the detection light 46.

[0142] The measurement unit 47 calculates the distance to at least a part of the subject based on the time from when the light-emitting unit 41 emits the measurement light 43 to when the detection light 46 is received by the light-receiving unit 44 and the speed of light. The measurement unit 47 may be provided outside the image capture device 1A. For example, the measurement unit 47 may be provided in a control device arranged outside the image capture device 1A and configured to be able to communicate signals with each unit of the image capture device 1A (for example, the light-emitting unit 41, the light-receiving unit 44, and at least a part of the virtual image generation unit 57). The measurement unit 47 can also be called a distance calculation unit because it calculates the distance to at least a part of the subject.

[0143] As an example, the light receiving unit 44 may be an imaging element having two-dimensional resolution in which single photon avalanche diodes are arranged in a two-dimensional lattice. In this case, in addition to the distance to the subject, it is also possible to measure the azimuth angle of the subject relative to the imaging device 1A. Note that the measurement unit 47 may be included in the light receiving unit 44. In this case, the light receiving unit 44 itself may calculate the distance to at least a portion of the subject. For example, when an imaging element is used for the light receiving unit 44, the distance to at least a portion of the subject may be calculated by a processing circuit of a part of the imaging element.

[0144] The measurement unit 47 calculates the distance to the subject in response to a command in the form of a control signal S5 from the virtual image generation unit 57 of the control unit 50. Then, information relating to the distance to the subject is transmitted to the virtual image generation unit 57 as a distance signal S6.

[0145] The angle of view of the light-sending optical system 42 and the light-receiving optical system 45 may be the same as the angle of view of the first optical system 10 described above, or may be different from the angle of view of the first optical system 10. As an example, the angle of view (maximum angle of view) of the light-sending optical system 42 and the light-receiving optical system 45 may be 170° or more. The wavelengths of the measurement light 43 emitted by the light-emitting unit 41 and the detection light 46 received by the light-receiving unit 44 may be wavelengths included in the wavelength range of the light received by the image sensor 36, or may be wavelengths different from the wavelength range of the light received by the image sensor 36. For example, the light received by the image sensor 36 may be visible light, and the wavelength of the measurement light 43 emitted by the light-emitting unit 41 may be a wavelength in the infrared range.

[0146] The light-transmitting optical system 42 and the light-receiving optical system 45 do not have to be separate optical systems, but may be an optical system in which the optical path of the measurement light 43 emitted from the light-emitting unit 41 and the optical path of the detection light 46 received by the light-receiving unit 44 are at least partially overlapped. For example, a light-path branching element such as a half mirror may be combined with one optical system, and the light-emitting unit 41 may be disposed in one optical path branched by the light-path branching element, and the light-receiving unit 44 may be disposed in another optical path branched.

[0147] In FIG. 1, the optical path from the first optical system 10 through the second optical system 20 to the image sensor 36 is separated from the optical path of the measurement light 43 from the light-emitting unit 41 to the light-transmitting optical system 42, and the optical path of the detection light 46 from the light-receiving optical system 45 to the light-receiving unit 44, but these optical paths may overlap at least partially. It is also possible to use the first optical system 10 and the second optical system 20 as the light receiving optical system 45 and the image sensor 36 as the light receiving unit 44 .

[0148] The light emitting unit 41 is not limited to one that emits pulses of the measurement light 43, but may also emit measurement light 43 whose intensity is modulated over time. In this case, the distance measuring unit 40 may measure the distance to the subject based on the phase of the temporal change in the signal corresponding to the amount of detection light 46 received by the light receiving unit 44.

[0149] Instead of using an imaging element with two-dimensional resolution as the light receiving unit 44, the distance measurement unit 40 may scan and irradiate the subject with one or more laser beams that emit pulsed light or whose light intensity fluctuates, thereby measuring the distance to the subject. Alternatively, the distance to the subject may be measured using an imaging optical system equipped with an image sensor having a focus detection function based on a so-called image plane phase difference method. Note that the distance measurement unit 40 and the imaging device 1A may be separate assemblies. Note that the distance measurement unit 40 may have another existing component capable of measuring the distance to the subject.

[0150] The virtual image generation unit 57 generates image data (virtual image data) when the subject is imaged from a position different from the position where the imaging device 1A is located, based on the image data of the subject captured by the imaging element 36 and generated by the image generation unit 52 and the distance to the subject measured by the distance measurement unit 40.

[0151] The virtual image generation unit 57 communicates signals with the imaging control unit 51, image generation unit 52, analysis unit 53, field of view control unit 54, storage unit 55, and interface unit within the control unit 50 via wiring 58 such as a bus. The virtual image generation unit 57 may be independent, i.e., mechanically separated hardware.

[0152] Alternatively, any of the imaging control unit 51, image generation unit 52, analysis unit 53, field of view control unit 54, storage unit 55, and interface unit 56 may be integrated into one piece of hardware. The virtual image generation unit 57 may be composed of hardware and software that controls it. The virtual image generation unit 57 may be provided outside the imaging device 1A. For example, the virtual image generation unit 57 may be provided in a control device disposed outside the imaging device 1A, and configured to be able to communicate signals with each unit of the imaging device 1A.

[0153] The storage unit 55 may store image data generated by the virtual image generation unit 57 together with, or instead of, the image data of the subject captured by the imaging element 36 and generated by the image generation unit 52. The interface unit 56 may output image data of the subject captured by the imaging element 36 and generated by the image generation unit 52, or image data stored in the memory unit 55, or image data generated by the virtual image generation unit 57 to an external device via a network line NW or wirelessly.

[0154] Fig. 8 is a diagram showing an example of a usage state of the imaging device 1Aa of the second embodiment. In Fig. 8, the imaging device 1Aa is placed at a position that has a predetermined positional relationship with respect to a subject 60, which is, for example, an automobile. The imaging device 1Aa captures image data of the subject 60 using the imaging element 36. That is, the image generation unit 52 generates image data of the subject 60 based on an imaging signal S1 captured by the imaging element 36.

[0155] The distance measurement unit 40 of the imaging device 1Aa measures the distance and azimuth angle from the imaging device 1Aa to a plurality of points P1, . . . , Pj, . . . , Pn on the subject 60, roughly in synchronization with the imaging of the subject 60 described above. Here, n is the total number of points on the subject 60 to be measured, and j is an arbitrary natural number as a subscript.

[0156] Based on the measured distances and azimuth angles to the multiple points P1 to Pn, the virtual image generation unit 57 of the control unit 50 of the imaging device 1Aa calculates the three-dimensional positional relationship of each part of the subject 60 with respect to the imaging device 1Aa using a known method. As an example, the virtual image generation unit 57 calculates the XYZ coordinate values ​​of each part (points P1 to Pn) of the subject 60 when the position where the imaging device 1Aa is placed is set as the origin.

[0157] The virtual image generation unit 57 generates virtual image data, which is virtual image data obtained when the subject 60 is imaged from virtual imaging devices 1v1 and 1v2 located at positions different from those of the actual imaging device 1Aa, based on the image data of the subject 60 and the XYZ coordinate values ​​of each of the points P1 to Pn. The position of the virtual imaging device 1v1 or the virtual imaging device 1v2 may be input by the user to the imaging device 1a as a relative position with respect to the imaging device 1Aa, for example.

[0158] The imaging device 1Ab shown in FIG. 8 constitutes an imaging system 2A of the second embodiment together with the imaging device 1Aa, the details of which will be described later.

[0159] 9(a) shows image Im5, which is an example of an image based on image data of subject 60 captured by imaging device 1Aa of the second embodiment, and FIG. 9(b) shows virtual image Im6, which is an example of a virtual image based on virtual image data generated by virtual image generation unit 57. Image Im5 was captured when subject 60, which is, for example, an automobile, was close to imaging device 1Aa, so the perspective of subject 60 is exaggerated, i.e., the front of the automobile (subject 60) is depicted as being excessively large compared to the rear.

[0160] 9(b) is a virtual image based on virtual image data generated by the virtual image generation unit 57, and corresponds to an image obtained when the subject is imaged from a virtual imaging device 1v1 that is positioned farther from the subject 60 than the imaging device 1Aa. In virtual image Im6, the perspective of the subject has been corrected, resulting in a more preferable image than in image Im5.

[0161] Fig. 10(a) shows image Im7, which is an example of an image of subject 60 captured by imaging device 1Aa of the second embodiment, and Fig. 10(b) shows virtual image Im8, which is an example of a virtual image based on virtual image data generated by virtual image generation unit 57. Image Im7 was captured by imaging device 1a (see Fig. 8) placed at a position with a small difference in Z position from subject 60, which is, for example, an automobile, and therefore the structure of the top surface (the surface on the -Z side) of subject 60 is represented in a compressed form in image Im7.

[0162] 10(b) is a virtual image based on virtual image data generated by virtual image generation unit 57, and corresponds to an image obtained when capturing an image of subject 60 from a virtual imaging device 1v2 that is closer to subject 60 than imaging device 1Aa and on the -Z side of imaging device 1a. Compared to image Im7, virtual image Im8 is an image in which subject 60 is viewed from above (the -Z side), and the top surface of subject 60 is depicted in detail.

[0163] To summarize the above, the virtual image generation unit 57 generates image data (virtual image data) when the subject 60 is imaged from a position different from the position of the imaging device 1Aa, based on the image data of the subject imaged by the imaging element 36 and the distance to at least a part of the subject 60 measured by the distance measurement unit 40.

[0164] (Effects of the imaging device of the second embodiment) (7) In addition to the components of the imaging device 1 of the first embodiment described above, the imaging device 1A of the second embodiment includes a distance measurement unit 40 that measures the distance to at least a part of the subject. The imaging device 1A also includes a virtual image generation unit that generates image data (virtual image data) of the subject when it is imaged from a position different from the position of the imaging device 1, based on image data of the subject generated by imaging with the image sensor 36 and the distance to at least a part of the subject measured by the distance measurement unit 40. With this configuration, the imaging device 1A of the second embodiment can generate virtual image data (virtual images Im6 and Im8) in which a subject is imaged from a position different from the position where the imaging device 1A is actually placed.

[0165] The imaging device 1 of the first embodiment or the imaging device 1A of the second embodiment may not have one or more of the imaging control unit 51, image generation unit 52, analysis unit 53, field of view control unit 54, memory unit 55, or interface unit 56, which are included in the control unit 50 in the above description.

[0166] For example, one or more of the imaging control unit 51, image generation unit 52, analysis unit 53, field of view control unit 54, storage unit 55, and interface unit 56 may be provided outside the imaging device 1, 1A. For example, one or more of the imaging control unit 51, image generation unit 52, analysis unit 53, field of view control unit 54, storage unit 55, and interface unit 56 may be provided in a control device arranged outside the imaging device 1, 1A, and configured to be able to communicate signals with each unit of the imaging device 1, 1A.

[0167] In the imaging device 1 of the first embodiment or the imaging device 1A of the second embodiment, the first optical system 10 and the second optical system 20 do not necessarily have to be optical systems that are telecentric on the intermediate image 18 side. In addition, the second optical system 20 does not necessarily have to have the second drive unit 33. Alternatively, the second optical system 20 and the imaging element 36 do not necessarily have to be held by the holder 21.

[0168] In the imaging device 1A of the second embodiment, part of the optical system of the distance measurement unit 40 may be shared with the first optical system 10 and the second optical system 20. Fig. 11 is a diagram showing a distance measurement unit 40 in which part of the optical system is shared with the first optical system 10 and the second optical system 20. In Fig. 11, the same reference numerals are used to designate components that achieve the same functions as those in the above-described embodiments.

[0169] 11, measurement light 43 from light emitting unit 41 constituting distance measurement unit 40 is emitted along optical axis AX3 that intersects with optical axis AX2 of second optical system 20, and is reflected by half mirror 61 that is obliquely disposed with respect to optical axes AX2 and AX3. Measurement light 43 is then reflected by mirror 62 that is located on the opposite side of second optical system 20 with respect to optical axis AX2, passes through half mirror 61, and enters lens 28 of second optical system 20.

[0170] The measurement light 43 incident on the second optical system 20 is projected onto the object via lenses 27, 26, and 25 (not shown) of the second optical system 20 and the first optical system 10 (see FIG. 7 for all). The measurement light 43 is reflected by at least a part of the object, forming detection light 46, which reaches the half mirror 61 via the first optical system 10 and the second optical system 20 (see FIG. 7 for all). The detection light 46 reflected by the half mirror 61 is then reflected by the half mirror 63 and reaches the light receiving unit 44 that constitutes the distance measurement unit 40.

[0171] On the other hand, light LB from the subject for forming an image on the image sensor 36 passes through the first optical system 10 and the second optical system 20, is reflected by the half mirror 61, and then passes through the half mirror 63, before reaching the image sensor 36. In this manner, at least a part of the light sending optical system 42 and the light receiving optical system 45 that constitute the distance measurement unit 40 may be shared with the first optical system 10 and the second optical system 20.

[0172] (Imaging system of the first embodiment) As shown in Fig. 4, the imaging system 2 of the first embodiment includes a plurality of imaging devices 1 (1a to 1c) of the first embodiment. The plurality of imaging devices 1 (1a to 1c) constituting the imaging system 2 communicate with each other via a network line NW (see Fig. 1) or wirelessly. The number of imaging devices 1 included in the imaging system 2 is not limited to three as shown in the figure, and may be any number.

[0173] In the imaging system 2, when one of the multiple imaging devices 1a to 1c detects a region of interest IA, information about the position of that region of interest IA may be transmitted to the other imaging devices 1a to 1c in the imaging system 2. The information about the position of the region of interest IA shared by the multiple imaging devices 1Aa to 1Ac may be, for example, the azimuth angle of the subject included in the region of interest IA relative to one imaging device 1Aa, or may be the coordinates of the region of interest IA in an XYZ coordinate system virtually defined by the imaging system 2.

[0174] This allows the subject included in the area of ​​interest IA to be easily captured by the other imaging devices 1b and 1c in the imaging system 2. Furthermore, the imaging devices 1a to 1c can capture and record images of the same subject included in the area of ​​interest IA from different viewpoints.

[0175] (Imaging system of second embodiment) As shown in Fig. 8, the imaging system 2A of the second embodiment includes a plurality of imaging devices 1A (1Aa to 1Ac) of the second embodiment. The plurality of imaging devices 1Aa to 1Ac that make up the imaging system 2A communicate with each other via a network line NW (see Fig. 7) or wirelessly. The number of imaging devices 1A included in the imaging system 2A is not limited to three as shown in the figure, and may be any number.

[0176] In the imaging system 2A of the second embodiment, as in the imaging system 2 of the second embodiment described above, when one of the multiple imaging devices 1Aa to 1Ac detects an area of ​​interest IA, information regarding the position of that area of ​​interest IA may be transmitted to the other imaging devices 1Aa to 1Ac in the imaging system 2A.

[0177] In the imaging system 2A of the second embodiment, not only the azimuth angle of the subject included in the region of interest IA relative to one imaging device 1Aa can be shared, but also the distance of the subject, which allows the other imaging devices 1Ab and 1Ac in the imaging system 2A to more easily capture images of the subject included in the region of interest IA.

[0178] The above-described virtual image data may be generated using the imaging system 2A of the second embodiment. In this case, two or more of the multiple imaging devices 1Aa to 1ac capture an image of the subject 60 and measure the distance and azimuth angle to at least a part of the subject 60. This information is then shared among the multiple imaging devices 1Aa to 1ac. Then, based on this information, a virtual image generation unit 57 included in the imaging devices 1Aa to 1Ac may generate virtual image data corresponding to an image obtained when the subject 60 is captured by the virtual imaging device 1v1 or 1v2.

[0179] The imaging device 1 of the first embodiment, the imaging device 1A, imaging system 2, or imaging system 2A of the second embodiment described above may be installed in a predetermined location such as a nursery school, school, medical facility, nursing home, conference room, shop, station, park, vending machine, street light, telephone pole, prison, military facility, border, road, park, etc. When the imaging device 1, 1A, or the imaging system 2, 2A is installed in a nursery school, the imaging device 1, 1A, or the imaging system 2, 2A may, for example, set a part of a child's body as an area of ​​interest IA and change the position and size (magnification of the final image 35) of the imaging area IA1 (field of view of the imaging device 1) to follow changes in the position and size of the area of ​​interest IA due to the child's activities, etc.

[0180] In addition, when the imaging device 1, 1A or the imaging system 2, 2A is installed in a care facility, the imaging device 1, 1A or the imaging system 2, 2A may, for example, set the face or part of the body of the person requiring care as the area of ​​interest IA, and change the position and size of the imaging area IA1 to follow changes in the position and size of the area of ​​interest IA that occur due to changes in the posture of the person requiring care, etc.

[0181] When the imaging device 1, 1A or the imaging system 2, 2A is installed in a conference room, the position and size of the imaging area IA1 may be changed to follow changes in the position and size of the area of ​​interest IA due to changes in the posture of the conference attendees, for example, by using the face of a conference attendee as the area of ​​interest IA.

[0182] When the imaging device 1, 1A or the imaging system 2, 2A is installed near a ticket gate at a station, near a passageway or entrance / exit of a store, on a vending machine, a streetlight, or a telephone pole, the imaging device 1, 1A or the imaging system 2, 2A may set a passing person as the area of ​​interest IA and change the position and size of the imaging area IA1 to track changes in the position and size of the area of ​​interest IA as the person moves. The imaging device 1, 1A or the imaging system 2, 2A may detect the characteristics of the passing person based on the image data of the area of ​​interest IA by using the object detection processing described above or other existing image processing.

[0183] For example, the imaging device 1, 1A or the imaging system 2, 2A may detect at least one of the gender, age, body type (height, weight, etc.), race, hairstyle, clothing, movement speed, movement direction, etc. of a passing person based on image data of the area of ​​interest IA. In this case, the detected characteristics of the passing person can be used for market research, etc. Note that the imaging device 1, 1A or the imaging system 2, 2A may detect not only the characteristics of passing people, but also the characteristics of a moving object such as a passing vehicle.

[0184] For example, the imaging device 1, 1A or the imaging system 2, 2A may detect at least one of the vehicle type, color, moving speed, moving direction, etc. of a passing vehicle. Note that the imaging device 1, 1A or the imaging system 2, 2A may set a wide-field imaging area (such as wide-field imaging area IA0) until at least a part of a passing person or vehicle enters the field of view, and when at least a part of the person or vehicle enters the field of view of the imaging device 1, 1A or the imaging system 2, 2A, may detect a region of interest IA (i.e., an image of at least a part of the passing person or vehicle) using the above-mentioned object detection processing or other existing image processing, and adjust the position and size of the imaging area IA1 to match the detected region of interest IA.

[0185] The imaging device 1 of the first embodiment, the imaging device 1A of the second embodiment, the imaging system 2, or the imaging system 2A described above does not need to be fixedly installed indoors or outdoors, but may be installed on a moving body such as a vehicle, a ship, or an aircraft (e.g., a drone).

[0186] When the imaging device 1, 1A, or the imaging system 2, 2A is installed on an unmanned aerial vehicle such as a drone, the object being searched for (e.g., a person, animal, vehicle, ship, aircraft, etc.) may be used as the region of interest IA. For example, the imaging device 1, 1A, or the imaging system 2, 2A may fly the unmanned aerial vehicle and search for the object being searched for in a wide-field imaging area (e.g., wide-field imaging area IA0) until the object being searched for enters the field of view of the imaging device 1, 1A, or the imaging system 2, 2A. When at least a portion of the object being searched for enters the field of view of the imaging device 1, 1A, or the imaging system 2, 2A, the region of interest IA (i.e., an image of at least a portion of the object being searched for) may be detected using the object detection process described above or other existing image processing, and the position and size of the imaging area IA1 may be adjusted to match the detected region of interest IA. The imaging device 1, 1A, or the imaging system 2, 2A may change the position and size of the imaging area IA1 to track changes in the position and size of the region of interest IA due to relative movement between the object being searched for and the unmanned aerial vehicle.

[0187] When the imaging device 1, 1A or the imaging system 2, 2A is installed in an attack type unmanned aerial vehicle (for example, an attack drone), which is an example of an unmanned aerial vehicle, an object to be attacked (for example, a person, a vehicle, a ship, an aircraft, a building, etc.) may be set as the area of ​​interest IA. For example, the imaging device 1, 1A or the imaging system 2, 2A may search by flying the unmanned aerial vehicle until the object to be attacked comes into view in a wide-field imaging area (wide-field imaging area IA0, etc.).

[0188] When at least a portion of the object to be attacked enters the field of view of the imaging device 1, 1A or the imaging system 2, 2A, the area of ​​interest IA (i.e., an image of at least a portion of the object to be attacked) may be detected using the object detection process described above or other existing image processing, and the position and size of the imaging area IA1 may be adjusted to match the detected area of ​​interest IA. Based on the position and size of the area of ​​interest IA on the image, the attack unmanned aerial vehicle may approach the object to be attacked until it is within the attack range of the attack unmanned aerial vehicle and attack the object to be attacked. Note that the imaging device 1, 1A or the imaging system 2, 2A may change the position and size of the imaging area IA1 to track changes in the position and size of the area of ​​interest IA that accompany the relative movement of the object to be attacked and the unmanned aerial vehicle.

[0189] For example, it is conceivable to mount two cameras (a wide-angle camera and a telephoto camera with a gimbal) on an unmanned aerial vehicle such as a drone, but this is not preferable because objects mounted on unmanned aerial vehicles must be small and lightweight due to restrictions on payload, size, etc. On the other hand, imaging devices 1, 1A or imaging systems 2, 2A that can capture wide-angle and telephoto images in one unit are smaller and lighter than two cameras, and are therefore suitable for mounting on unmanned aerial vehicles, etc.

[0190] Furthermore, the subject of imaging by the imaging device 1, 1A or the imaging system 2, 2A is not limited to people or animals as described above, but may also be industrial machinery such as machine tools. In this case, the imaging device 1, 1A or the imaging system 2, 2A analyzes whether the industrial machinery is operating normally based on the captured image data. If an abnormality is found in the shape or operation of the industrial machinery, the imaging device 1, 1A or the imaging system 2, 2A takes appropriate action, such as transmitting image data of the portion related to the abnormality to an external device.

[0191] The imaging device 1, 1A or the imaging system 2, 2A may be installed inside the machining chamber of the machine tool (the space where the workpiece is machined with a machining tool). In this case, for example, the region of interest IA may be an area including the machining tool attached to the spindle, the workpiece placed on the stage, and at least a portion of the machining point of the workpiece by the machining tool.

[0192] The imaging device 1, 1A or the imaging system 2, 2A may change the position and size (magnification of the final image 35) of the imaging area IA1 (field of view of the imaging device 1) so as to follow changes in the position and size of the area of ​​interest IA that accompany relative movement between the spindle and the workpiece (stage). Note that the imaging device 1, 1A or the imaging system 2, 2A may detect the movement and change in size of the area of ​​interest IA and change the position and size of the imaging area IA1 (field of view of the imaging device 1) by appropriately combining and executing the processes described in the above-mentioned embodiments.

[0193] In addition, the imaging device 1, 1A or the imaging system 2, 2A may detect at least one of the distance between the processing tool and the workpiece, the length (amount of wear) of the processing tool, the shape of the processing tool, damage (breakage) of the processing tool, the protrusion amount of the processing tool, the shape of the workpiece, the positional deviation of the workpiece (relative to the reference position of the stage), the shape of the chips, the amount of chips, and the amount of cutting fluid applied to the processed part based on image data of the area of ​​interest IA generated by existing image processing.

[0194] As an example, the imaging device 1, 1A or the imaging system 2, 2A may be installed in at least one of the wall of the machining chamber, the ceiling of the machining chamber, the spindle head, and the stage. As an example, the imaging device 1 or 1A may be installed in the spindle.

[0195] The imaging device 1 or 1A may be configured to be detachable from the spindle via a shank. In this case, the imaging device 1 or 1A attached to the spindle may be replaced with any processing tool using a processing tool exchanger. Alternatively, the imaging device 1 or 1A may be stored in the processing tool exchanger, and the processing tool attached to the spindle may be replaced with the imaging device 1 or 1A.

[0196] When the imaging device 1 is attached to the spindle, the imaging device 1 attached to the spindle may be configured in such a way that the second optical system 20, the second housing 21, the first drive unit 24, the second drive unit 33, and the field of view control unit 54 are removed from the configuration shown in FIG. 1 , and the imaging device 36 is disposed on the image plane of the subject imaged by the first optical system 10 (i.e., the intermediate image formation area 19) (i.e., a wide-angle imaging device whose field of view cannot be moved or resized). In this configuration, the analysis unit 53 or the interface unit 56 may be omitted. The image generation unit 52 may be provided outside the imaging device 1. For example, the image generation unit 52 may be provided in a control device disposed outside the imaging device 1, and may be configured to be able to communicate signals with each unit of the imaging device 1 (e.g., the image sensor 36).

[0197] When the imaging device 1A is attached to the main shaft, the imaging device 1A attached to the main shaft may have the configuration shown in Fig. 7 or 11. In this case, the imaging device 1A attached to the main shaft may have a configuration in which the second optical system 20, the second housing 21, the first drive unit 24, the second drive unit 33, and the field of view control unit 54 are eliminated from the configuration shown in Fig. 7 or 11, and the imaging device 36 is disposed on the image plane of the subject on which the image is formed by the first optical system 10 (that is, the intermediate image formation area 19) (that is, a wide-angle imaging device capable of distance measurement whose field of view cannot be moved or changed in size).

[0198] In this configuration, the analysis unit 53, the interface unit 56, and the virtual image generation unit 57 may be omitted. The measurement unit 47 may be provided outside the imaging device 1A. For example, the measurement unit 47 may be provided in a control device arranged outside the imaging device 1A, and configured to be able to communicate signals with each unit of the imaging device 1A (for example, the light-emitting unit 41, the light-receiving unit 44, and at least a part of the virtual image generation unit 57).

[0199] The image generation unit 52 may be provided outside the imaging device 1A. For example, the image generation unit 52 may be provided in a control device arranged outside the imaging device 1A, and configured to be able to communicate signals with each unit (e.g., the image sensor 36) of the imaging device 1A.

[0200] Furthermore, the imaging device 1, 1A or imaging system 2, 2A may be installed in two or more locations among the wall surface of the machining chamber, the ceiling of the machining chamber, the spindle head, and the stage. For example, they may be installed in different locations on the wall surface of the machining chamber, or they may be installed in one or more locations on the wall surface of the machining chamber and one or more locations on the ceiling of the machining chamber.

[0201] When a plurality of imaging devices 1, 1A or imaging systems 2, 2A are installed in this manner, they may be installed so that the region of interest in the processing chamber can be imaged from different directions (so that they can be imaged from different viewpoints). Note that when a plurality of imaging devices 1, 1A or imaging systems 2, 2A are installed, they may be installed so that their maximum field of view areas partially overlap each other.

[0202] The imaging device 1, 1A or the imaging system 2, 2A may be installed in a location other than the machining chamber of the machine tool, for example, inside a machining tool exchanger that exchanges a machining tool attached to the spindle of the machining chamber with another type of machining tool. The processing tool changer may be, for example, an automatic tool changer (ATC).

[0203] Furthermore, the imaging device 1, 1A or the imaging system 2, 2A may be installed inside an optical processing device that processes a workpiece with processing light. In this case, the imaging device 1, 1A or the imaging system 2, 2A may detect the processing location on the workpiece (the location irradiated with processing light) based on image data of the region of interest IA generated by existing image processing. Note that if the optical processing device is a three-dimensional additive manufacturing device, the imaging device 1, 1A or the imaging system 2, 2A may detect the supply state of the modeling material based on image data of the region of interest IA generated by existing image processing.

[0204] (Effects of the imaging systems of the first and second embodiments) (9) The imaging system 2 of the first and second embodiments described above includes a plurality of imaging devices 1 (1a to 1c) of the first embodiment or imaging devices 1A (1Aa to 1Ac) of the second embodiment described above, and at least one of the plurality of imaging devices 1a to 1c, 1Aa to 1Ac images a subject using information from an imaging device different from the at least one imaging device. With this configuration, for example, information about the position of an area of ​​interest IA identified by one imaging device 1a or 1Aa from within a subject can be shared with the other imaging devices 1b, 1c, 1Ab, and 1Ac, allowing the imaging devices 1b, 1c, 1Ab, and 1Ac to easily capture images of the subject included in the area of ​​interest IA. Alternatively, the imaging devices 1a to 1c and 1Aa to 1Ac can each capture and record images of the same subject included in the area of ​​interest IA from different viewpoints. In the above example, there is one region of interest IA, but there may be multiple regions of interest IA.

[0205] Although various embodiments and modifications have been described above, the present invention is not limited to these. Furthermore, each embodiment and modification may be applied independently or in combination. Other aspects conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention.

[0206] (Addendum) It will be understood by those skilled in the art that the above-described embodiments or variations thereof are examples of the following aspects.

[0207] (Section 1) An imaging device comprising: a first optical system that forms an intermediate image of a subject; a second optical system that re-images at least a portion of the intermediate image to form a final image; an image sensor that captures the final image; and a drive unit that moves the first optical system and at least one of the second optical system and the image sensor in a direction intersecting the optical axis of the first optical system.

[0208] (Section 2) 2. The imaging device according to claim 1, wherein the first optical system is telecentric on the intermediate image side.

[0209] (Section 3) In the imaging device described in paragraph 2, the difference between the angle of the chief ray of light traveling from the first optical system to a first location in the intermediate image forming area relative to the optical axis of the first optical system and the angle of the chief ray of light traveling from the first optical system to a second location in the intermediate image forming area whose distance from the optical axis of the first optical system is different from the first location relative to the optical axis of the first optical system is within 1°.

[0210] (Section 4) 3. The imaging device according to claim 1, wherein the second optical system is telecentric on the side of the first optical system.

[0211] (Section 5) In the imaging device described in paragraph 4, the difference between the angle of the chief ray of light traveling from a first location in the intermediate image forming area to the second optical system relative to the optical axis of the second optical system and the angle of the chief ray of light traveling from a second location in the intermediate image forming area, which is at a different distance from the optical axis of the second optical system than the first location, relative to the optical axis of the second optical system, is within 1°.

[0212] (Section 6) In the imaging device described in any one of paragraphs 1 to 5, the difference between the angle of the chief ray of light from the first optical system relative to the optical axis of the first optical system and the angle of the chief ray of light incident on the second optical system relative to the optical axis of the second optical system is within 1°.

[0213] (Section 7) 7. The imaging device according to any one of paragraphs 1 to 6, wherein the maximum angle of view of the first optical system is 170° or more.

[0214] (Section 8) 8. The imaging device according to any one of paragraphs 1 to 7, wherein the drive unit moves the second optical system and the imaging element in a direction intersecting the optical axis of the first optical system.

[0215] (Section 9) 9. The imaging device according to claim 8, further comprising a holding unit that holds the second optical system and the imaging element, wherein the driving unit moves the holding unit in a direction that intersects with the optical axis of the first optical system.

[0216] (Section 10) 10. The imaging device according to claim 8 or 9, wherein the drive unit moves the second optical system and the imaging element in a direction parallel to a plane perpendicular to the optical axis of the first optical system.

[0217] (Section 11) 11. The imaging device according to any one of paragraphs 1 to 10, wherein the second optical system is capable of changing the magnification of the final image.

[0218] (Section 12) In the imaging device described in paragraph 11, the second optical system includes a plurality of optical elements, and when the driving unit is a first driving unit, the imaging device further includes a second driving unit that moves at least one of the plurality of optical elements along the optical axis of the second optical system, and the second optical system changes the magnification of the final image by driving the second driving unit.

[0219] (Section 13) 13. The imaging device according to claim 11 or 12, further comprising a field of view control unit that performs at least one of driving the drive unit and changing the magnification of the final image based on image data generated by imaging by the imaging element.

[0220] (Section 14) 14. The imaging device according to claim 13, further comprising an imaging control unit that controls the start and end of recording of image data generated by imaging with the imaging element based on the image data generated by imaging with the imaging element.

[0221] (Section 15) In the imaging device described in paragraph 13 or 14, the field of view control unit drives the drive unit so that the area of ​​interest of the subject does not move out of the field of view of the imaging device when at least one of the subject and the imaging device moves.

[0222] (Section 16) In the imaging device described in any one of paragraphs 13 to 15, the field of view control unit reduces the magnification of the final image when at least one of the subject and the imaging device moves and the subject's area of ​​interest moves out of the field of view of the imaging device.

[0223] (Section 17) 17. An imaging device according to any one of claims 13 to 16, wherein the field of view control unit changes the magnification of the final image depending on the size of the region of interest of the subject relative to the field of view of the imaging device.

[0224] (Section 18) 18. The imaging device according to claim 17, wherein the field of view control unit changes the magnification of the final image so that the size of the region of interest of the subject relative to the field of view of the imaging device remains constant.

[0225] (Section 19) An imaging device according to any one of paragraphs 13 to 19, further comprising an analysis unit that analyzes image data generated by imaging of the imaging element, and when the analysis unit cannot recognize an image of at least a portion of the subject, the field of view control unit performs at least one of driving the drive unit and changing the magnification of the final image.

[0226] (Section 20) 20. The imaging device according to claim 19, wherein the field of view control unit reduces the magnification of the final image.

[0227] (Section 21) 20. The imaging device according to claim 19, wherein the field of view control unit increases the magnification of the final image.

[0228] (Section 22) 14. The imaging device according to claim 13, wherein the field of view control unit drives the drive unit and changes the magnification of the final image based on image data generated by imaging with the imaging element.

[0229] (Section 23) 11. The imaging device according to any one of claims 1 to 10, further comprising a field of view control unit that drives the drive unit based on image data generated by imaging by the imaging element.

[0230] (Section 24) 24. An imaging device according to claim 23, wherein the field of view control unit controls the start and end of recording of image data generated by imaging by the imaging element based on the image data generated by imaging by the imaging element.

[0231] (Section 25) In the imaging device described in paragraph 23 or 24, the field of view control unit drives the drive unit so that the area of ​​interest of the subject does not move out of the field of view of the imaging device when at least one of the subject and the imaging device moves.

[0232] (Section 26) 26. The imaging device according to any one of paragraphs 1 to 25, further comprising an analysis unit capable of analyzing image data of the subject generated by imaging with the imaging element.

[0233] (Section 27) An imaging device comprising: an imaging element; a first optical system that forms an intermediate image of a subject; a second optical system that re-images at least a portion of the intermediate image to form a final image on the imaging element; and a portion of the intermediate image that is re-imaged on the imaging element by the second optical system, the center position of which is variable.

[0234] (Section 28) 28. The imaging device according to claim 27, wherein the second optical system is capable of changing the magnification of the final image, and the size of the portion of the intermediate image that is re-imaged on the imaging element by the second optical system is variable.

[0235] (Section 29) An imaging device comprising: a light source that emits measurement light for measuring the distance to the subject; an optical system that has a maximum angle of view of 170° or more and irradiates the subject with the measurement light from the light source; a detection element that detects, via the optical system, detection light from the subject that is generated when the measurement light is irradiated onto the subject; and an imaging element that captures an image of the subject formed by the optical system.

[0236] (Section 30) 29. An imaging device according to claim 29, wherein at least a portion of the optical path of the detection light from the optical system detected by the detection element and the optical path of the light from the optical system received by the imaging element overlap.

[0237] (Section 31) 31. The imaging device according to claim 29 or 30, wherein the wavelength of the detection light detected by the detection element is different from the wavelength of the light received by the imaging element.

[0238] (Section 32) An imaging system comprising: an imaging device according to any one of paragraphs 29 to 31; a distance calculation unit that calculates a distance to at least a part of the subject based on a detection result of a detection element of the imaging device; and a virtual image generation unit that generates image data of the subject when it is imaged from a position different from the position of the imaging device, based on image data of the subject generated by imaging of the imaging element of the imaging device and the calculation result of the distance to at least a part of the subject by the distance calculation unit.

[0239] (Section 33) an optical system having a maximum angle of view of 170° or more and irradiating the measurement light from the light source onto the subject; a detection element that detects, via the optical system, detection light from the subject generated when the measurement light is irradiated onto the subject; a distance calculation unit that calculates a distance to at least a part of the subject based on the detection light detected by the detection element; and an image sensor that captures an image of the subject formed by the optical system.

[0240] (Section 34) In the imaging system described in paragraph 33, at least a portion of the optical path of the detection light from the optical system detected by the detection element and the optical path of the light from the optical system received by the imaging element overlap.

[0241] (Section 35) 35. An imaging system according to claim 33 or 34, wherein the wavelength of the detection light detected by the detection element is different from the wavelength of the light received by the imaging element.

[0242] (Section 36) The imaging system according to any one of items 33 to 35, further comprising a virtual image generation unit that generates image data of the subject when the subject is imaged from a position different from the position of the imaging element, based on image data of the subject generated by imaging with the imaging element and a result of distance calculation to at least a part of the subject by the distance calculation unit.

[0243] (Section 37) An imaging device comprising: a light source that emits measurement light; an optical system that has a maximum angle of view of 170° or more and that irradiates the measurement light from the light source onto the subject; a detection element that detects, via the optical system, detection light from the subject that is generated when the measurement light is irradiated onto the subject; and an imaging element that captures an image of the subject formed by the optical system, and that calculates a distance to at least a part of the subject based on the detection light.

[0244] (Section 38) An imaging device according to claim 37, wherein at least a portion of the optical path of the detection light from the optical system detected by the detection element and the optical path of the light from the optical system received by the imaging element are overlapped.

[0245] (Section 39) 39. The imaging device according to claim 37 or 38, wherein the wavelength of the detection light detected by the detection element is different from the wavelength of the light received by the imaging element.

[0246] (Section 40) An imaging system comprising: an imaging device according to any one of items 37 to 39; and a virtual image generation unit that generates image data of the subject when the subject is imaged from a position different from the position of the imaging device, based on image data of the subject generated by imaging with the imaging element of the imaging device and a result of calculation of the distance to at least a part of the subject by the imaging device.

[0247] (Section 41) An imaging system comprising: an imaging element that captures an image of a subject; a distance measurement unit that measures the distance to at least a portion of the subject; and a virtual image generation unit that generates image data of the subject when the subject is imaged from a position different from the position of the imaging device, based on image data of the subject generated by imaging with the imaging element and the calculation result of the distance to at least a portion of the subject by the distance measurement unit. [Explanation of symbols]

[0248] 1, 1a to 1c: imaging device, 2: imaging system, 10: first optical system, 12 to 15: lenses, 18: intermediate image, 19: intermediate image forming area, 20: second optical system, 24: first driving unit, 25 to 28: lenses, 35: final image, 36: imaging element, 38: housing, 40: distance measurement unit, 41: light emitting unit, 42: light receiving unit, 43: light transmitting lens, 44: light receiving lens, 47: measurement unit, 50: control unit, 51: imaging control unit, 52: image generation unit, 53: analysis unit, 54: field of view control unit, 55: memory unit, 56: interface unit, 57: virtual image generation unit, IA0: wide field of view imaging area, IA1: imaging area

Claims

1. a first optical system that forms an intermediate image of a subject; a second optical system that re-images at least a part of the intermediate image to form a final image, the second optical system being capable of changing the magnification of the final image; an imaging element for capturing the final image; a drive unit that moves the second optical system and the image sensor in a direction intersecting an optical axis of the first optical system; An imaging device comprising:

2. 2. The imaging device according to claim 1, The imaging device, wherein the first optical system is telecentric on the intermediate image side.

3. 3. The imaging device according to claim 2, An imaging device in which the difference between the angle of a chief ray of light traveling from the first optical system to a first location in an intermediate image forming area relative to the optical axis of the first optical system and the angle of a chief ray of light traveling from the first optical system to a second location in the intermediate image forming area whose distance from the optical axis of the first optical system is different from the first location relative to the optical axis of the first optical system is within 1°.

4. 4. The imaging device according to claim 1, The second optical system is telecentric on the first optical system side.

5. 5. The imaging device according to claim 4, An imaging device in which the difference between the angle of a chief ray of light traveling from a first location in an intermediate image forming area to the second optical system relative to the optical axis of the second optical system and the angle of a chief ray of light traveling from a second location in the intermediate image forming area, which is at a different distance from the optical axis of the second optical system than the first location, relative to the optical axis of the second optical system, is within 1°.

6. 4. The imaging device according to claim 1, An imaging device, wherein the difference between the angle of the chief ray of light from the first optical system relative to the optical axis of the first optical system and the angle of the chief ray of light incident on the second optical system relative to the optical axis of the second optical system is within 1°.

7. 4. The imaging device according to claim 1, An imaging device, wherein the maximum angle of view of the first optical system is 170° or more.

8. 4. The imaging device according to claim 1, When the drive unit is a first drive unit, the second optical system includes a plurality of optical members, and further includes a second driving unit that moves at least one of the plurality of optical members along an optical axis of the second optical system; The second optical system is an imaging device that changes the magnification of the final image by driving the second driving unit.

9. 4. The imaging device according to claim 1, a holder for holding the second optical system and the image sensor; The driving unit moves the holding unit in a direction intersecting with the optical axis of the first optical system.

10. 4. The imaging device according to claim 1, The driving unit moves the second optical system and the imaging element in a direction parallel to a plane perpendicular to the optical axis of the first optical system.

11. 4. The imaging device according to claim 1, The imaging device further includes an analysis unit capable of analyzing image data of the subject generated by imaging with the imaging element.

12. 12. The imaging device according to claim 11, The imaging device further includes a field of view control unit that performs at least one of driving the drive unit and changing the magnification of the final image based on the result of the analysis by the analysis unit.

13. 12. The imaging device according to claim 11, The imaging device further includes an imaging control unit that controls the start and end of recording of image data captured by the imaging element based on the analysis result of the analysis unit.

14. 4. The imaging device according to claim 1, further comprising a distance measurement unit that measures a distance to at least a part of the subject.

15. 15. The imaging device according to claim 14, The distance measurement unit a light emitting unit that irradiates the subject with light; a light receiving unit that receives light emitted from the light emitting unit and reflected by at least a part of the subject; a measuring unit that measures a distance to at least a part of the subject in accordance with a time period from when light emitted from the light emitting unit is reflected by at least a part of the subject until the light is received by the light receiving unit, Imaging device.

16. 15. The imaging device according to claim 14, Image data of the subject generated by capturing an image of the image sensor; Based on the distance to at least a part of the subject measured by the distance measurement unit, The imaging device further comprises a virtual image generation unit that generates image data when the subject is imaged from a position different from the position of the imaging device.

17. a plurality of the imaging devices according to any one of claims 1 to 3; An imaging system in which at least one of the plurality of imaging devices images the subject using information from an imaging device other than the at least one imaging device.

18. a first optical system that forms an intermediate image of a subject; a second optical system that forms a final image by magnifying and re-imaging at least a portion of the intermediate image; an imaging element for capturing the final image; a drive unit that moves the second optical system and the image sensor in a direction intersecting the optical axis of the first optical system.

19. 19. The imaging device according to claim 18, The imaging device, wherein the first optical system is telecentric on the intermediate image side.

20. 20. The imaging device according to claim 19, an angle of a chief ray of light traveling from the first optical system toward a first location in an intermediate image forming area with respect to the optical axis of the first optical system; An imaging device in which the difference between the angle of the chief ray of light traveling from the first optical system to a second location in the intermediate image forming area whose distance from the optical axis of the first optical system is different from the first location and the angle of the chief ray of light traveling from the first optical system to the optical axis of the first optical system is within 1°.

21. 20. The imaging device of claim 18, The second optical system is telecentric on the first optical system side.

22. 22. The imaging device according to claim 21, an angle of a chief ray of light traveling from a first location of the intermediate image forming area to the second optical system with respect to the optical axis of the second optical system; An imaging device in which the difference in angle between the chief ray of light traveling from a second location in the intermediate image forming area, which has a different distance from the optical axis of the second optical system from the first location, to the second optical system and the optical axis of the second optical system is within 1°.

23. 20. The imaging device of claim 18, an angle of a chief ray of light from the first optical system with respect to the optical axis of the first optical system; An imaging device, wherein the difference between the angle of a chief ray of light incident on the second optical system and the angle of the principal ray of light relative to the optical axis of the second optical system is within 1°.

24. 20. The imaging device of claim 18, An imaging device, wherein the maximum angle of view of the first optical system is 170° or more.

25. 20. The imaging device of claim 18, The imaging device further includes an analysis unit capable of analyzing image data of the subject generated by imaging with the imaging element.

26. 26. The imaging device according to claim 25, The imaging device further includes a field of view control unit that performs at least one of driving the drive unit and changing the magnification of the final image based on the result of the analysis by the analysis unit.

27. 26. The imaging device according to claim 25, The imaging device further includes an imaging control unit that controls the start and end of recording of image data captured by the imaging element based on the analysis result of the analysis unit.

28. 20. The imaging device of claim 18, The imaging device further includes a distance measurement unit that measures the distance to at least a part of the subject.

29. 29. The imaging device according to claim 28, The distance measurement unit includes a light emitting unit that irradiates light onto the subject, and a light receiving unit that receives light that is emitted from the light emitting unit and reflected by at least a part of the subject. a measuring unit that measures a distance to at least a part of the subject in accordance with a time period from when light emitted from the light emitting unit is reflected by at least a part of the subject until the light is received by the light receiving unit, Imaging device.

30. a plurality of the imaging devices according to claim 18; An imaging system in which at least one of the plurality of imaging devices images the subject using information from an imaging device other than the at least one imaging device.

Images