Imaging device and light receiving device

Abstract

The present invention provides an imaging device that can rapidly move the imaging range being captured, and can also rapidly move the imaging field of view on the object being captured (target side). [Solution] The imaging device 1 comprises an imaging optical system 10, an optical splitting member 13 that amplitude-splits light that has passed through at least a part of the imaging optical system, a first imaging unit that images a first image 14a formed by one of the optical beams split by the optical splitting member, a second imaging unit that images a part of the second image 14b formed by the other optical beam split by the optical splitting member, and a drive unit 16 that moves the second imaging unit in a direction that crosses the optical path of the other optical beam.

Description

[Technical Field]

[0001] The present invention relates to an image pickup apparatus and a light receiving apparatus. [Background Art]

[0002] A camera is attached to a rotary head device capable of panning and tilting to point the camera at an object (Patent Document 1). [Prior Art Literature] [Patent Literature]

[0003] [Patent Document 1] U.S. Pat. No. 4,855,838, Specification [Summary of the Invention]

[0004] According to a first aspect, an image pickup apparatus includes: an imaging optical system; a light splitting member that amplitude-splits light having passed through at least a part of the imaging optical system; a first image pickup unit that picks up a first image formed by one light beam split by the light splitting member; a second image pickup unit that picks up a part of a second image formed by the other light beam split by the light splitting member; and a driving unit that moves the second image pickup unit in a direction crossing an optical path of the other light beam.

[0005] According to a second aspect, a light receiving apparatus includes: an imaging optical system; a light splitting member that amplitude-splits light having passed through at least a part of the imaging optical system; an image pickup unit that picks up an image formed by one light beam split by the light splitting member; a light receiving unit having a light incident portion on which a part of the other light beam split by the light splitting member is incident; and a driving unit that moves the light incident portion in a direction crossing an optical path of the other light beam. [Brief Description of the Drawings]

[0006] [Figure 1] Figure 1 is a diagram outlining the image pickup apparatus according to the first embodiment. [Figure 2] Figure for explaining an outline of a first image pickup device and a second image pickup device. [Figure 3] This diagram shows the positional relationship between the first image and the first imaging range of the first image sensor, and the positional relationship between the second image and the second imaging range of the second image sensor. [Figure 4] A diagram showing an example of the usage state of the imaging device according to the first embodiment. [Figure 5] A diagram showing an example of an image captured by the imaging device. [Figure 6] A diagram showing another example of an image captured by the imaging device. [Figure 7] A diagram showing an overview of the imaging device according to the second embodiment. [Figure 8] A diagram showing a portion of the imaging optical system and re-imaging optical system of the imaging device according to the second embodiment. [Modes for carrying out the invention]

[0007] (Imaging device of the first embodiment) Figure 1 shows an overview of the imaging device 1 of the first embodiment. The X, Y, and Z directions indicated by arrows in Figure 1 are defined as positive directions. The X, Y, and Z directions are mutually orthogonal directions. In the following, the position in the X direction will be referred to as the X position, the position in the Y direction as the Y position, and the position in the Z direction as the Z position. Note that the X, Y, and Z directions in each figure referred to later are the same directions as the X, Y, and Z directions shown in Figure 1.

[0008] The imaging device 1 of the first embodiment includes an imaging optical system 10, an optical segmentation member 13, a first image sensor 15a, a second image sensor 15b, a drive unit 16 for moving the second image sensor 15b, and a control unit 30, etc. The imaging optical system 10 includes, as an example, a plurality of lenses 11a to 11d arranged along the optical axis AX0. The plurality of lenses 11a to 11d are held by a holding frame 12, which is held by the housing 19 of the imaging device 1.

[0009] At least some of the lenses 11a to 11d constituting the imaging optical system 10 may be held so as to be movable in the ±Z direction relative to the holding frame 12 for focusing. In Figure 1, lenses 11a to 11d are all shown as single lenses, but they may also be lens groups consisting of multiple lenses, and may include optical components such as mirrors and diffractive optical elements. Furthermore, the number of lens groups constituting the imaging optical system 10 is not limited to the four shown (lenses 11a to 11d), but may be any other number. Note that the imaging optical system 10 can be said to be a form of a light-gathering optical system because it focuses incident light. Therefore, the imaging optical system 10 may also be called a light-gathering optical system.

[0010] As shown in Figure 1, the light splitting member 13 is positioned on the side of the imaging optical system 10 where light from the object side (outside the imaging device 1) passes through the imaging optical system 10 and is emitted (the light emission side of the imaging optical system 10). The light splitting member 13 amplitude-splits the light that has passed through the imaging optical system 10. The light splitting member 13 is, for example, a cube-shaped beam splitter and has a split reflective surface 13a that amplitude-splits the light from the imaging optical system 10, that is, reflects a portion of it and transmits a portion of it. The amount of light reflected by the split reflective surface 13a and the amount of light transmitted may be equal or different. The light splitting member 13 may also be a plate type in which a split reflective surface is formed on a transparent flat plate.

[0011] Furthermore, the light division performed by the light division member 13 is not limited to the amplitude division described above. For example, the light splitting member 13 may split the polarization component of the incident light. In this case, the light splitting member 13 may be an existing optical component that splits the polarization component of light, such as a polarizing beam splitter. The light splitting member 13 may also split the wavelength component of the incident light. In this case, the light splitting member 13 may be an existing optical component that splits the wavelength component of light, such as a dichroic mirror.

[0012] The light-splitting member 13 may also split the incident light by reflecting a portion of the incident light beam (i.e., transmitting the other portion of the incident light beam). In this case, the light-splitting member 13 may be an existing reflective member such as a mirror placed in the optical path of a portion of the incident light beam. Furthermore, if the light-splitting member 13 reflects a portion of the incident light beam, it can be said that the light-splitting member 13 splits the field of view of the imaging optical system 10. In other words, the light-splitting member 13 only needs to be a member that splits the incident light beam.

[0013] Furthermore, in order to split the light that has passed through the imaging optical system 10, the light splitting member 13 does not have to be positioned on the side of the imaging optical system 10 where light from the object side (outside the imaging device 1) passes through the imaging optical system 10 and is emitted (the light emission side of the imaging optical system 10). For example, the light splitting member 13 may split light that has passed through a part of the imaging optical system 10. For example, the light splitting member 13 may be positioned between any of the lenses (lens groups) 11a to 11d that constitute the imaging optical system 10.

[0014] Furthermore, an optical component separate from the imaging optical system 10 may be provided on the optical path of the light beam reflected by the light-dividing member 13 so that an image (the first image 14a described later) is formed near the imaging surface of the first image sensor 15a by the light beam reflected by the light-dividing member 13. Also, an optical component separate from the imaging optical system 10 may be provided on the optical path of the light beam that has passed through the light-dividing member 13 so that an image (the second image 14b described later) is formed near the imaging surface of the second image sensor 15b by the light beam that has passed through the light-dividing member 13.

[0015] In other words, the light-splitting member 13 only needs to split the light that has passed through at least a part of the imaging optical system 10. It can also be said that the light-splitting member 13 splits the light that has passed through at least one optical component of the imaging optical system 10. Furthermore, it can also be said that the light-splitting member 13 splits the light that has passed through at least one lens of the imaging optical system 10. Finally, it can also be said that the light-splitting member 13 splits the light that has passed through at least one lens group of the imaging optical system 10.

[0016] Among the light split in amplitude by the light splitting member 13, the light beam that has transmitted through the split reflection surface 13a travels generally along the optical axis AX0 of the imaging optical system 10, and exits from the light splitting member 13. The light beam reflected by the split reflection surface 13a travels generally along the optical axis AX1, which is a geometrically reflected axis of the optical axis AX0 of the imaging optical system 10 by the split reflection surface 13a, and exits from the light splitting member 13. Since the optical axis AX1 is an optically equivalent axis to the optical axis AX0, the optical axis AX1 is also referred to as the optical axis of the imaging optical system 10 hereinafter.

[0017] Hereinafter, among the light split in amplitude by the light splitting member 13, one of the light beams forms a first image 14a after exiting the light splitting member 13, and the other light beam forms a second image 14b after exiting the light splitting member 13. Here, among the light split in amplitude by the light splitting member 13, it can be stated that one light beam is used for forming the first image 14a, and the other light beam is used for forming the second image 14b.

[0018] It should be noted that, since light incident through at least a part of the imaging optical system 10 is split into respective light beams (each of the one light beam and the other light beam) by the light splitting member 13 (for example, amplitude splitting), and the first image 14a and the second image 14b are formed by the respective split light beams, it can also be stated that the light splitting member 13 contributes to forming the first image 14a and the second image 14b. Therefore, the light splitting member 13 may be a part of the imaging optical system 10.

[0019] An image of an object within any predetermined angular range on the object side of the imaging optical system 10 is formed as the first image 14a. Then, an image of an object within the same predetermined angular range on the object side of the imaging optical system 10 is formed as the second image 14b. It should be noted that any predetermined angular range on the object side of the imaging optical system 10 may also be referred to as the field of view of the imaging optical system 10.

[0020] In the vicinity of the first image 14a, a first image sensor 15a is arranged such that its imaging surface substantially coincides with the first image 14a. The first image sensor 15a may also be referred to as a first imaging unit. The first imaging unit may include at least a part of a control unit 30 (for example, an image generation unit 32) that will be described in detail later. Since the optical axis AX1 is the optical axis directed toward the first imaging unit, it may also be referred to as the optical axis of the image-forming optical system 10 on the first imaging unit side. The combination of optical axis AX0 and optical axis AX1 may also be referred to as the optical axis of the image-forming optical system 10 on the first imaging unit side. The first imaging unit may include other components such as a fixing unit 18 described later, in addition to the first image sensor 15a.

[0021] In the vicinity of the second image 14b, a second image sensor 15b is arranged such that its imaging surface substantially coincides with the second image 14b in the direction of the optical axis AX0. The second image sensor 15b may also be referred to as a second imaging unit. The first imaging unit may include at least a part of a control unit 30 (for example, an image generation unit 32) that will be described in detail later. Since the optical axis AX0 is the optical axis directed toward the second imaging unit, it may also be distinguished from the optical axis AX1 and referred to as the optical axis of the image-forming optical system 10 on the second imaging unit side. As a part of the optical axis AX0, the optical axis from the light splitting member 13 to the second imaging unit may also be referred to as the optical axis of the image-forming optical system 10 on the second imaging unit side. Since the second imaging unit receives the other light flux from the light splitting member 13, it may also be referred to as a light receiving unit.

[0022] It should be noted that light from the object side of the image-forming optical system 10 (for example, light from a target or the like) enters the light splitting member 13 via the image-forming optical system 10, and respective images are formed by the respective light fluxes split by the light splitting member 13 (for example, amplitude splitting). Here, each formed image can also be said to be an image in an arbitrary range on the object side of the image-forming optical system 10. This arbitrary range can also be said to be an image-pickupable range of the image pickup apparatus 1 (or an image pickup apparatus 1A described later). In this case, it can be said that the first imaging unit (the first image sensor 15a) picks up an image in this arbitrary range (that is, the first image 14a). On the other hand, it can be said that the second imaging unit (the second image sensor 15b) picks up a part of the image in this arbitrary range (that is, the second image 14b).

[0023] From the above, it can be said that the imaging device 1 (or the imaging device 1A described later) is capable of imaging any range and comprises an imaging optical system 10, a light splitting member 13 that splits light that has passed through at least a part of the imaging optical system 10, a first imaging unit that images a first image 14a of any range formed by one of the light beams split by the light splitting member 13, a second imaging unit that images a part of a second image 14b of any range formed by the other light beam split by the light splitting member 13, and a drive unit 16 (or the first drive unit 16a described later) that moves the second imaging unit. Note that the second imaging unit may include not only the second image sensor 15b but also other components. For example, the second imaging unit may include, in addition to the second image sensor 15b, at least one of the holding unit 17 and the drive unit 16 described later.

[0024] The first image sensor 15a is held in place by a fixing part 18 attached to the housing 19. The second image sensor 15b is held by a holding unit 17, and the second image sensor 15b and the holding unit 17 are held by a drive unit 16 so as to be movable, for example, in the X and Y directions. When the second image sensor 15b is moved, for example, in the X and Y directions, the imaging range captured by the second image sensor 15b in the second image 14b is changed. As a result, the imaging field of view on the object side (target side) captured by the second image sensor 15b is also changed. The second image sensor 15b and the holding unit 17 may be held by the drive unit 16 so as to be movable in either the X or Y direction. The second image sensor 15b and the holding unit 17 may be held by the drive unit 16 so as to be movable in any one direction in the XY plane defined by the X and Y directions.

[0025] In other words, the drive unit 16 moves the second image sensor 15b in two directions (for example, the X and Y directions) that intersect the optical axis AX0 parallel to the Z direction of the imaging optical system 10 on the second image sensor 15b side. Another way to express this is that the drive unit 16 moves the second image sensor 15b in a direction that crosses the optical path of the other light beam.

[0026] Alternatively, the drive unit 16 can be described as moving the second image sensor 15b in a direction that intersects with the direction in which the other light beam, which travels roughly along the optical axis AX0, enters the second image sensor 15b. Furthermore, the drive unit 16 can be described as moving the second image sensor 15b in the in-plane direction of the image plane of the second image 14b. Note that the drive unit 16 may be an existing drive mechanism such as a linear motor or a stepping motor.

[0027] The configuration of the first image sensor 15a and the second image sensor 15b will be described below with reference to Figure 2. Figure 2(a) shows the first image sensor 15a as viewed from the imaging plane side (-X side), and Figure 2(b) shows the second image sensor 15b as viewed from the imaging plane side (+Z side).

[0028] In the first effective region EA1 of the first image sensor 15a, multiple first pixels PEa are arranged two-dimensionally with an array pitch PZa in the Z direction and an array pitch PYa in the Y direction. The array pitch PZa in the Z direction and the array pitch PYa in the Y direction may be the same or different. The width of the first pixels PEa in the Z direction and the width of the first pixels PEa in the Y direction may be the same or different. For example, there may be 1,000 or more first pixels PEa arranged in both the Z and Y directions. The size of the first effective region EA1 is such that the width in the Z direction is WZa and the width in the Y direction is WYa.

[0029] Similarly, in the second effective region EA2 of the second image sensor 15b, multiple second pixels PEb are arranged two-dimensionally with an array pitch PXb in the X direction and an array pitch PYb in the Y direction. The array pitch PXb in the X direction and the array pitch PYb in the Y direction may be the same or different. The width of the second pixels PEb in the Z direction and the width of the second pixels PEb in the Y direction may be the same or different. For example, there may be 1,000 or more second pixels PEb arranged in both the X and Y directions. The size of the second effective region EA2 is WXb in the X direction and WYb in the Y direction.

[0030] The array pitches PXb and PYb of the second pixel PEb of the second image sensor 15b are smaller than the array pitches PZa and PYa of the first pixel PEa of the first image sensor 15a. Therefore, the number of second pixels PEb per unit area of ​​the second effective region EA2 (i.e., density) of the second image sensor 15b is higher than the number of second pixels PEb in the first effective region EA1 of the first image sensor 15a (i.e., density), making it a higher-resolution image sensor than the first image sensor 15a. It can also be said that the width of the second pixel PEb in the Z direction and the Y direction of the second image sensor 15b are smaller than the width of the first pixel PEa in the Z direction and the Y direction of the first image sensor 15a.

[0031] The width WXb in the X direction of the second effective region EA2 of the second image sensor 15b is shorter than the width WZa in the Z direction of the first effective region EA1 of the first image sensor 15a. Also, the width WYb in the Y direction of the second effective region EA2 of the second image sensor 15b is shorter than the width WYa in the X direction of the first effective region EA1 of the first image sensor 15a. Therefore, the area of ​​the second effective region EA2 is smaller than the area of ​​the first effective region EA1.

[0032] Figure 3(a) shows the positional relationship between the first imaging range CA1 of the first image sensor 15a and the first image 14a, and Figure 3(b) shows the positional relationship between the second imaging range CA2 of the second image sensor 15b and the second image 14b. The first imaging range CA1 corresponds to the first effective area EA1 of the first image sensor 15a shown in Figure 2(a). The second imaging range CA2 corresponds to the second effective area EA2 of the second image sensor 15b shown in Figure 2(b).

[0033] Furthermore, the imaging range can be rephrased as the imaging area. In other words, the first imaging range CA1 can be rephrased as the first imaging area, and the second imaging range CA2 can be rephrased as the second imaging area.

[0034] Figure 3(b) shows the state in which the center CC of the second imaging range CA2 by the second image sensor 15b has been moved by DX in the X direction and DY in the Y direction with respect to the optical axis AX0 by the drive unit 16 described above. In Figures 3(a) and (b) of this embodiment, the ratio of the area of ​​the first effective region EA1 to the area of ​​the second effective region EA2 and the ratio of the area of ​​the first imaging range CA1 to the area of ​​the second imaging range CA2 are substantially the same, but for the sake of explanation, the ratios of their areas have been changed.

[0035] As described above, the imaging optical system 10 and the light-splitting member 13 are substantially identical in size to the first image 14a and the second image 14b. However, the second effective region EA2 of the second image sensor 15b, located at the position of the second image 14b, is smaller than the first effective region EA1 of the first image sensor 15a, located at the position of the first image 14a.

[0036] Therefore, the proportion of the second imaging range CA2 captured by the second image sensor 15b in the second image 14b is lower than the proportion of the first imaging range CA1 captured by the first image sensor 15a in the first image 14a.

[0037] The second image sensor 15b captures a portion of the second image 14b. As mentioned above, the second image sensor 15b has a higher resolution than the first image sensor 15a, so it can also be said that the second image sensor 15b captures a portion of the second image 14b with higher resolution than the first image sensor 15a. By moving the second image sensor 15b in the XY plane direction by the drive unit 16, a large portion of the second image 14b can be captured with high resolution. Furthermore, by moving the second image sensor 15b in the XY plane direction by the drive unit 16, a desired portion of the second image 14b can be captured with high resolution.

[0038] Furthermore, when the second imaging unit (second image sensor 15b) images a portion of the second image 14b, it can be said that a portion of the other light beam from the light splitting member 13 is incident on the second imaging unit. In this case, it can be said that the second imaging unit has a light incident section 29 into which a portion of the other light beam is incident. Furthermore, it can be said that the light incident section 29 is at least a part of the second imaging unit. For example, the light incident section 29 may be the light incident surface of a plurality of first pixels PEa arranged in the first effective region EA1.

[0039] The second imaging unit can also be referred to as the light-receiving unit. The light-receiving unit can be said to receive a portion of the other light beam from the light-dividing member 13. In this case, the light-receiving unit can also be said to have a light-incident unit 29 into which a portion of the other light beam is incident. The light-incident unit 29 can also be said to be at least a part of the light-receiving unit.

[0040] Furthermore, when the second imaging unit images a portion of the second image 14b, it can be said that a portion of the other light beam from the light-dividing member 13 is incident on the second imaging unit. The second imaging range CA2 can also be called the light-receiving region, since a portion of the other light beam from the light-dividing member 13 is incident on it. In this case, the light-receiving region can also be said to be a part of the region where the second image 14b is formed (i.e., the region through which the other light beam from the light-dividing member 13 passes). Furthermore, the second imaging unit can also be called the light-receiving unit. The light-receiving unit can also be said to receive light that has passed through the light-receiving region, which is a part of the region through which the other light beam divided by the light-dividing member 13 passes.

[0041] On the other hand, the first image sensor 15a can simultaneously image a large portion of the first image 14a. In other words, the first image sensor 15a can image a wide area of ​​the first image 14a at once. The diameter of the first image 14a, shown as a circle in Figure 3(a) as an example, is, for example, 170° or more when converted to an angle of view on the object side opposite to the light-dividing member 13 of the imaging optical system 10 (opposite to the side of the first image 14a formed by the imaging optical system 10). Therefore, it can be said that the maximum angle of view on the object side of the imaging optical system 10 is 170° or more. Note that the maximum angle of view on the object side of the imaging optical system 10 may be less than 170°, for example, 60°. The same applies to the diameter of the second image 14b, shown as a circle in Figure 3(b) as an example, and its converted value to an angle of view on the object side.

[0042] In Figure 3(a), a portion of the first image 14a is shown to be included in the first imaging range CA1, but the entire first image 14a may be included in the first imaging range CA1, or conversely, the entire first imaging range CA1 may be included in the first image 14a. Also, the center of the first imaging range CA1 may or may not coincide with the optical axis AX1. Furthermore, the first image sensor 15a does not have to have multiple first pixels PEa arranged in it, and may have only one first pixel PEa. Also, a single photodiode may be used instead of the first image sensor 15a. Similarly, the second image sensor 15b does not have to have multiple second pixels PEb arranged in it, and may have only one second pixel PEb. Also, a single photodiode may be used instead of the second image sensor 15b.

[0043] Referring again to Figure 1, the control unit 30 of the imaging device 1 will be described. The control unit 30 controls various operations of the imaging device 1. The control unit 30 comprises an imaging control unit 31, an image generation unit 32, an analysis unit 33, an imaging range control unit 34, a storage unit 35, and an interface unit 36. The imaging control unit 31, image generation unit 32, analysis unit 33, imaging range control unit 34, storage unit 35, and interface unit 36 ​​communicate signals with each other via wiring 37 such as a bus.

[0044] The imaging control unit 31 controls the first image sensor 15a by sending a control signal S2a to the first image sensor 15a. Specifically, the imaging control unit 31 controls the first image sensor 15a by starting and ending the output of low-resolution video (through image), taking still images, setting the exposure time (light reception time), and setting the gain of photoelectric conversion. The imaging control unit 31 also sends a control signal S2b to the second image sensor 15b and controls the second image sensor 15b in the same way as the first image sensor 15a.

[0045] The image generation unit 32 receives the imaging signal S1a from the first image sensor 15a and generates image data of the first image 14a captured by the first image sensor 15a. The image generation unit 32 receives the imaging signal S1b from the second image sensor 15b and generates image data of the second image 14b captured by the second image sensor 15b.

[0046] The image generation unit 32 may have a function to correct distortion of the first image 14a or the second image 14b caused by distortion of the imaging optical system 10 when generating image data. Furthermore, the image generation unit 32 may have a function to correct deterioration of the first image 14a or the second image 14b caused by aberrations other than distortion of the imaging optical system 10.

[0047] The analysis unit 33 analyzes target information that may be included in the image data based on the image data of a portion of the first image 14a or the second image 14b generated by the image generation unit 32. If the image data of the first image 14a or the image data of a portion of the second image 14b contains image data of at least a portion of the target, the analysis unit 33 can detect at least a portion of the image data of the target through analysis. In other words, the analysis unit 33 can detect at least a portion of the target image included in the image corresponding to the image data of a portion of the first image 14a or the second image 14b generated by the image generation unit 32.

[0048] Furthermore, the analysis unit 33 can also be called a detection unit, since it detects at least a portion of the target (at least a portion of the target image). It can also be said that the analysis unit 33 can detect at least a portion of the target image data by analyzing the image data of the first image 14a or a portion of the image data of the second image 14b generated by the image generation unit 32. Furthermore, the analysis unit 33 can also be said to recognize at least a portion of the target (at least a portion of the target image). Therefore, the analysis unit 33 can also be called a recognition unit. The analysis performed by the analysis unit 33 will be described later.

[0049] The imaging range control unit 34 sends a drive signal S3 to the drive unit 16 based on the results of the analysis unit 33's analysis of target information based on image data of a portion of the first image 14a or the second image 14b (i.e., the results of detecting at least a portion of the target image), which drives a drive member such as a linear motor included in the drive unit 16, moving the holding unit 17 and the second image sensor 15b, for example, in the XY plane direction. That is, as shown in Figure 3(b), the imaging range control unit 34 moves the position of the center CC of the second imaging range CA2 of the second image sensor 15b to a predetermined position relative to the optical axis AX0. Alternatively, the imaging range control unit 34 can be said to move the position of the center CC of the second imaging range CA2 of the second image sensor 15b to a predetermined position within the second image 14b.

[0050] This changes the second imaging range CA2 captured by the second image sensor 15b within the second image 14b. Consequently, the imaging field of view on the object side (target side) captured by the second image sensor 15b is also changed. The imaging range control unit 34 may, regardless of the target information detected by the analysis unit 33, send a drive signal S3 to the drive unit 16 based on a signal input from an operation unit (not shown) via the interface unit 36, and move the second image sensor 15b.

[0051] The storage unit 35 includes storage materials such as memory elements and magnetic disks, and stores image data generated by the image generation unit 32 based on images captured by at least one of the first image sensor 15a and the second image sensor 15b, as needed.

[0052] The interface unit 36 ​​outputs image data generated by the image generation unit 32 based on the image captured by at least one of the first image sensor 15a and the second image sensor 15b, or image data stored in the storage unit 35, to an external device via a network line NW. The interface unit 36 ​​may also receive commands from an external device to the imaging device 1. The interface unit 36 ​​may be equipped with a wireless transmission and reception mechanism, and the output of image data and input of commands may be performed wirelessly.

[0053] Furthermore, the imaging control unit 31, image generation unit 32, analysis unit 33, imaging range control unit 34, storage unit 35, and interface unit 36 ​​may each be independent, that is, mechanically separated hardware. Alternatively, some of the imaging control unit 31, image generation unit 32, analysis unit 33, imaging range control unit 34, storage unit 35, and interface unit 36 ​​may be integrated as one or more hardware units.

[0054] Furthermore, at least one of the imaging control unit 31, image generation unit 32, analysis unit 33, imaging range control unit 34, storage unit 35, and interface unit 36 ​​may be composed of hardware and software that controls it.

[0055] Figure 4 shows an example of the usage state of the imaging device 1 of the first embodiment. In the example shown in Figure 4, as an example, multiple imaging devices 1 of the first embodiment are installed on the wall of the living room LM of a home where one or more users live. Note that the number of imaging devices 1 installed in a room such as a living room LM is not limited to multiple devices, and may be just one.

[0056] In this example, the imaging device 1 generally continuously images a target, which is at least a part of a person or an animal such as a pet, within the living room LM, and records, for example, the daily life of a family, capturing suitable photo opportunities. The target is not limited to a person or an animal, but may be any other object.

[0057] The first effective area EA1 of the first image sensor 15a captures a wide area of ​​the first image 14a, which is the first imaging range CA1, and transmits the imaging signal S1a to the image generation unit 32. Based on the imaging signal S1a, the image generation unit 32 generates wide-area image data corresponding to the wide area of ​​the first image 14a, which is the first imaging range CA1. The image corresponding to this wide-area image data will be referred to as the wide-area image below.

[0058] The second effective region EA2 of the second image sensor 15b captures a second imaging range CA2, which is a part of the second image 14b, and transmits the imaging signal S1b to the image generation unit 32. Based on the imaging signal S1b, the image generation unit 32 generates narrow-area image data corresponding to the second imaging range CA2, which is a part of the second image 14b. The image corresponding to this narrow-area image data will be referred to as the narrow-area image below.

[0059] As described above, the pixel array pitch (PXb, PYb) of the second image sensor 15b is smaller than the pixel array pitch (PZa, PYa) of the first image sensor 15a. Therefore, the resolution of the narrow-area image captured by the second image sensor 15b is higher than the resolution of the wide-area image captured by the first image sensor 15a. For this reason, the narrow-area image captured by the second image sensor 15b can also be called a high-resolution image.

[0060] Figure 5(a) shows a wide-area image Im1 based on wide-area image data of the living room LM, as an example of an image captured by the first image sensor 15a of the imaging device 1. As described above, the living room LM is imaged over a wide area. The analysis unit 33 analyzes target information that may be included in the wide-area image Im1 based on the wide-area image data captured by the first image sensor 15a.

[0061] For example, the analysis unit 33 analyzes wide-area image data generated by the image generation unit 32, which may contain target information, to detect the presence or absence of a region of interest IA in the wide-area image Im1 that includes at least a portion of the image of the target to be captured or recorded. If the analysis unit 33 detects a region of interest IA, it calculates the location of the region of interest IA in the wide-area image Im1.

[0062] The region of interest (IA) is, for example, a region that includes at least a part of the image of a target (e.g., at least a part of a person or animal) contained within an image (e.g., data from at least one of a wide-area image and a narrow-area image). Alternatively, the region of interest (IA) can be rephrased as a region that includes at least a part of the target contained within image data (e.g., data from at least one of a wide-area image and a narrow-area image).

[0063] Furthermore, detecting the presence or absence of a region of interest IA by analyzing the image data using the analysis unit 33 can be rephrased as detecting the presence or absence of at least a portion of the target image (at least a portion of the target) by analyzing the image data using the analysis unit 33. Furthermore, detecting the position of the region of interest IA on the image using the analysis unit 33 can be rephrased as calculating the position of at least a portion of the target image on the image using the analysis unit 33.

[0064] Furthermore, the analysis unit 33 does not need to detect the presence or absence of the region of interest IA, and may instead analyze the image data generated by the image generation unit 32 (for example, data from at least one of the wide-area image data and the narrow-area image data) to detect the presence or absence of at least a part of the target image contained in the image (for example, at least one of the wide-area image and the narrow-area image), and calculate the position of at least a part of the target image in the image.

[0065] Furthermore, the target is not limited to at least a part of a person or animal, but may be any object of interest (e.g., an object to be imaged or recorded). For example, the target may be at least a part of an object worn or carried by a person (clothing, shoes, bag, cane, etc.), dangerous goods such as guns or explosives, moving objects such as vehicles, ships, aircraft (drones, etc.), or buildings. For example, the target may be at least a part of the machining chamber of a machine tool (the space where a workpiece is processed with a machining tool), or the inside of a tool changing device that exchanges a machining tool attached to the spindle of the machining chamber with another type of machining tool.

[0066] For example, the analysis unit 33 may, as part of the analysis of wide-area image data (i.e., image data of wide-area image Im1), perform object detection processing using machine learning to detect the presence or absence of a region of interest IA from wide-area image Im1, and (if a region of interest IA is detected) calculate the position of the region of interest IA in wide-area image Im1. Note that calculating the position of the region of interest IA can also be said to be detecting the position of the region of interest IA. As an example, the analysis unit 33 may, as part of the analysis of wide-area image data, perform object detection processing using deep learning to detect the presence or absence of a region of interest IA. Note that, depending on the type of object detection processing to be performed, the analysis unit 33 may not need to detect the presence or absence of a region of interest IA, and may instead analyze the wide-area image data generated by the image generation unit 32 to detect the presence or absence of at least a part of the target image in wide-area image Im1, and (if a region of interest IA is detected) calculate the position of the region of interest IA in wide-area image Im1.

[0067] More specifically, the analysis unit 33 may use a Convolutional Neural Network (CNN) object detection algorithm to detect the presence or absence of a region of interest (IA) in the wide-area image Im1 and calculate the location of the region of interest (IA) in the wide-area image Im1. Alternatively, the analysis unit 33 may use other object detection processes based on deep learning to detect the presence or absence of a region of interest (IA) in the wide-area image Im1 and calculate the location of the region of interest (IA) in the wide-area image Im1.

[0068] For example, the analysis unit 33 may use object detection algorithms such as R-CNN (Region with CNN features), Faster R-CNN (Faster Region with CNN features), or Mask R-CNN (Mask Region with CNN features) as region-proposal type algorithms. Furthermore, the analysis unit 33 is not limited to region-proposal type algorithms and may use other deep learning-based object detection algorithms such as YOLO (You Only Look Once) or SSD (Single Shot Multibox Detector).

[0069] Furthermore, the analysis unit 33 may detect the presence or absence of a region of interest (IA) using algorithms other than deep learning, such as linear regression, decision trees, or SVM (Support Vector Machine). Furthermore, the analysis unit 33 may detect the presence or absence of a region of interest (IA) and calculate its location using existing template matching processes, rather than being limited to machine learning. Furthermore, the analysis unit 33 may detect the presence or absence of a region of interest (IA) and calculate its location using existing segmentation processes.

[0070] For example, the wide-area image Im1 shown in Figure 5(a) contains images of multiple people as target images, so the analysis unit 33 detects the region containing the multiple people, indicated by the dashed line, as the region of interest IA. Note that the detection of a region of interest IA is not limited to one; if the wide-area image Im1 contains images of multiple targets, the analysis unit 33 may detect multiple regions of interest IA containing images of each target from the wide-area image Im1.

[0071] When the analysis unit 33 detects a region of interest IA, it calculates the position of that region of interest IA in the wide-area image Im1. The analysis unit 33 then transmits the position of the region of interest IA in the wide-area image Im1 to the imaging range control unit 34.

[0072] The detection of the region of interest IA by the analysis unit 33 can be rephrased as the analysis unit 33 recognizing the region of interest IA. Similarly, the calculation of the position of the region of interest IA in the wide-area image Im1 by the analysis unit 33 can be rephrased as the analysis unit 33 recognizing the position of the region of interest IA.

[0073] The imaging range control unit 34 sends a drive signal S3 to the drive unit 16 in response to an information signal from the analysis unit 33 (i.e., information regarding the position of the region of interest IA in the wide-area image Im1 as a detection result), causing the second image sensor 15b to move in a direction intersecting the optical axis AX0 (for example, in the XY plane direction), so that the center CC of the second imaging range CA2 (see Figure 3(b)) approximately coincides with the center position of the region of interest IA. In other words, based on the wide-area image data, the imaging range control unit 34 performs the movement of the second image sensor 15b by the drive unit 16 so that at least a portion of the target image included in the second image 14b is included in the second imaging range CA2 by the second image sensor 15b (second imaging unit).

[0074] Furthermore, the imaging range control unit 34 does not necessarily have to move the second image sensor 15b so that the center CC of the second imaging range CA2 substantially coincides with the center position of the region of interest IA. It is sufficient that the region of interest IA is included in the second imaging range CA2. For example, the imaging range control unit 34 may move the second image sensor 15b so that the center CC of the second imaging range CA2 is positioned at a predetermined distance from the center position of the region of interest IA.

[0075] Furthermore, if the analysis unit 33 detects multiple regions of interest IA from the wide-area image Im1, it may calculate the position of each of the detected regions of interest IA in the wide-area image Im1. In this case, the imaging range control unit 34 may move the second image sensor 15b in a direction intersecting the optical axis AX0 (for example, in the XY plane direction) so that at least one of the multiple regions of interest IA is included in the second imaging range CA2.

[0076] As a result, an image corresponding to the region of interest IA is formed on the second image sensor 15b, and the second image sensor 15b captures a narrow-area image Im2, shown in Figure 5(b), which corresponds to the portion of the region of interest IA in the wide-area image Im1. If the size of the region of interest IA is larger than the second imaging range CA2 of the second image sensor 15b, the second image sensor 15b captures a narrow-area image Im2 corresponding to a portion of the region of interest IA in the wide-area image Im1. If the size of the region of interest IA is smaller than the second imaging range CA2 of the second image sensor 15b, the second image sensor 15b captures a narrow-area image Im2 that includes the portion corresponding to the region of interest IA in the wide-area image Im1.

[0077] In the imaging device 1, the position of the center CC of the second imaging range CA2 (see Figure 3(b)) can be changed by moving the second image sensor 15b in the XY plane direction only, without rotating the entire imaging device 1 or the imaging optical system 10. This allows the imaging device 1 to change the position of the second imaging range CA2 within the second image 14b at high speed. Consequently, the imaging field of view on the object side (target side) imaged by the second image sensor 15b can be changed at high speed.

[0078] Furthermore, the analysis unit 33 may not be limited to wide-area image data, but may also analyze narrow-area image data (i.e., image data of the narrow-area image Im2) to detect the presence or absence of a region of interest IA in the narrow-area image Im2. If the analysis unit 33 detects a region of interest IA on the narrow-area image Im2, it may calculate the position of the region of interest IA in the narrow-area image Im2.

[0079] The imaging range control unit 34 may send a drive signal S3 to the drive unit 16 in response to an information signal from the analysis unit 33 (i.e., information regarding the position of region of interest IA in the narrow-area image Im2 as a detection result), and move the second image sensor 15b so that the region of interest IA is included in the second imaging range CA2. In other words, the imaging range control unit 34 may perform the movement of the second image sensor 15b by the drive unit 16 based on the narrow-area image data.

[0080] Furthermore, the analysis unit 33 may also analyze the target in the narrow-area image Im2, and for example, detect whether the target person (i.e., a person included in the region of interest) is showing a specific facial expression such as a smile, or whether it is performing a specific gesture or hand movement.

[0081] For example, the analysis unit 33 may detect a specific facial expression, such as a smile, or a specific gesture or hand movement of a target person (a person included in the region of interest) by performing the object detection process described above or existing tracking processes (described later). The analysis unit 33 may also detect a specific facial expression, such as a smile, or a specific gesture or hand movement of a target person (a person included in the region of interest) by other image processing methods, not limited to the object detection process described above or existing tracking processes.

[0082] Furthermore, the analysis unit 33 may, not limited to narrow-area image data, but based on wide-area image data, detect whether a person or other object included in the region of interest IA displays a specific facial expression such as a smile, or whether it performs a specific gesture or hand movement, and may perform one or more of the following actions A through F. Furthermore, if the analysis unit 33 detects the above-mentioned facial expressions, gestures, or hand movements of the target, it may perform one or more of the following actions A through F.

[0083] A: A command is issued to the imaging control unit 31 to capture a still image with the second image sensor 15b. B: A command is issued to the imaging control unit 31 to start or stop the output of low-resolution video data to the second image sensor 15b. C: A command is issued to the imaging control unit 31 to start or stop the output of high-resolution video data to the second image sensor 15b.

[0084] D: A command is issued to the memory unit 35 to start or stop storing the narrow-area image data captured by the second image sensor 15b and generated by the image generation unit 32. E: A command is issued to the memory unit 35 to start or stop adding a predetermined identification signal (flag) to the narrow-area image data stored in it. F: The interface unit 36 ​​is given a command to start or stop the transmission of the narrow-area image data captured by the second image sensor 15b and generated by the image generation unit 32 to an external device (for example, a display device that displays the narrow-area image Im2).

[0085] Furthermore, if the analysis unit 33 detects the above-mentioned facial expressions, gestures, or hand movements of the target and initiates any of the actions A through F described above, the analysis unit 33 may terminate those actions after a predetermined time has elapsed.

[0086] Furthermore, if the positional relationship between the imaging device 1 and the target changes, the analysis unit 33 may detect whether or not the region of interest IA in the wide-area image Im1 has moved (i.e., whether or not at least a portion of the target image has moved). If the region of interest IA has moved, the analysis unit 33 may calculate the direction and amount of movement of the region of interest IA in the wide-area image Im1.

[0087] The analysis unit 33 transmits the direction and amount of movement of the region of interest IA in the wide-area image Im1 to the imaging range control unit 34. The direction and amount of movement of the region of interest IA in the wide-area image Im1 calculated by the analysis unit 33 can be rephrased as the position of the region of interest IA in the wide-area image Im1. Furthermore, the fact that the analysis unit 33 calculates the direction and amount of movement of the region of interest IA can be rephrased as the analysis unit 33 detecting the direction and amount of movement of the region of interest IA.

[0088] The imaging range control unit 34 may send a drive signal S3 to the drive unit 16 in response to an information signal from the analysis unit 33 (i.e., information regarding the direction and amount of movement of the region of interest IA in the wide-area image Im1 as a detection result), and move the second image sensor 15b in a direction intersecting the optical axis AX0 (for example, in the XY plane direction) so that the region of interest IA does not move out of the second imaging range CA2. In other words, if the positional relationship between the target and the imaging device 1 changes, the imaging range control unit 34 may, based on the image data of the wide-area image Im1, execute the movement of the second image sensor 15b (second imaging unit) by the drive unit 16 so that the region of interest IA does not move out of the second imaging range CA2.

[0089] Therefore, even if the region of interest IA (i.e., an image of at least a portion of the target) in the wide-area image Im1 moves due to a change in the positional relationship between the imaging device 1 and the target, the imaging device 1 can continue to generate a narrow-area image Im2 of the region of interest IA (i.e., a high-resolution image of at least a portion of the target) without the region of interest IA moving out of the second imaging range CA2 of the imaging device 1.

[0090] For example, the analysis unit 33 may detect the movement of the region of interest IA between multiple wide-area images Im1 captured by the first image sensor 15a at different times by performing an existing tracking process (that is, it may detect whether or not the region of interest IA has moved and calculate the direction and amount of movement of the region of interest IA).

[0091] As an example, the analysis unit 33 may use the image data of the region of interest IA detected from the wide-area image Im1 by the object detection process described above as a template, and use an existing template matching process to detect the region of interest IA from each wide-area image Im1 captured at different times, and calculate the direction and amount of movement of the region of interest IA on the wide-area image Im1 (in other words, the position of the region of interest IA on the wide-area image Im1).

[0092] The analysis unit 33 may also detect regions of interest IA from each wide-area image Im1 captured at different times, and detect whether or not the region of interest IA has moved based on whether or not there is a displacement of the region of interest IA on the wide-area image Im1.

[0093] The analysis unit 33 may then transmit the calculated movement direction and amount of the region of interest IA in the wide-area image Im1 to the imaging range control unit 34. The imaging range control unit 34 may receive an information signal from the analysis unit 33 (i.e., information regarding the movement direction and amount of the region of interest IA in the wide-area image Im1 as a detection result) and calculate the movement direction and amount of the second imaging range CA2 based on that information signal. The imaging range control unit 34 may send a drive signal S3 to the drive unit 16 corresponding to the calculated movement direction and amount of the second imaging range CA2, and move the second image sensor 15b in a direction intersecting the optical axis AX0 so that the region of interest IA does not move out of the second imaging range CA2.

[0094] Furthermore, even if at least a portion of the region of interest IA moves outside the second imaging range CA2 due to a change in the positional relationship between the imaging device 1 and the target, if the region of interest IA is included in the wide-area image Im1, the analysis unit 33 can detect the position (direction of movement and amount of movement) of the region of interest IA in the wide-area image Im1. Therefore, the imaging range control unit 34 can move the position of the second image sensor 15b so that the region of interest IA is included in the second imaging range CA2.

[0095] Furthermore, if the analysis unit 33 detects the movement of multiple regions of interest IA in the wide-area image Im1 through the tracking process described above, it may calculate the direction and amount of movement of each of the detected regions of interest IA in the wide-area image Im1. In this case, the imaging range control unit 34 may move the second image sensor 15b in a direction intersecting the optical axis AX0 (for example, in the XY plane direction) so that at least one of the multiple regions of interest IA does not move outside the second imaging range CA2.

[0096] Furthermore, the analysis unit 33 may perform detection of whether or not the region of interest IA in the image (at least one of the wide-area image Im1 and the narrow-area image Im2) has moved, and calculate the direction and amount of movement in the image, not limited to template matching processing. Furthermore, the analysis unit 33 may perform detection of whether or not the region of interest IA in the image has moved, and calculate the direction and amount of movement in the image, not limited to existing tracking processing, and calculate the direction and amount of movement in the image, not limited to existing tracking processing.

[0097] Furthermore, the analysis unit 33 may use existing deep learning to predict the direction and amount of movement of the region of interest IA per predetermined time period, based on the direction and amount of movement of the region of interest IA in the wide-area image Im1 calculated by the tracking process described above, based on wide-area image data captured at different times.

[0098] In this case, the analysis unit 33 may transmit the predicted direction and amount of movement of the region of interest IA in the wide-area image Im1 per predetermined time to the imaging range control unit 34. Note that the analysis unit 33 predicting the direction and amount of movement can be rephrased as the analysis unit 33 detecting the future direction and amount of movement. Furthermore, the direction and amount of movement of the region of interest IA in the wide-area image Im1 predicted by the analysis unit 33 can be rephrased as the future position of the region of interest IA in the wide-area image Im1.

[0099] The imaging range control unit 34 may receive an information signal from the analysis unit 33 (i.e., predicted information regarding the direction and amount of movement of the region of interest IA in the wide-area image Im1 as a detection result) and calculate (predict) the direction and amount of movement of the second imaging range CA2. The imaging range control unit 34 may send a drive signal S3 to the drive unit 16 according to the calculated direction and amount of movement of the second imaging range CA2 and move the second image sensor 15b in a direction intersecting the optical axis AX0 so that the region of interest IA does not move out of the second imaging range CA2.

[0100] In other words, if the positional relationship between the target and the imaging device 1 changes, the imaging range control unit 34 may, based on the image data of the wide-area image Im1, cause the drive unit 16 to move the second image sensor 15b (second imaging unit) so that the region of interest IA does not move outside the second imaging range CA2.

[0101] Furthermore, the imaging range control unit 34 may receive an information signal from the analysis unit 33 (i.e., predicted information regarding the direction and amount of movement of the region of interest IA in the wide-area image Im1 as a detection result) and, based on that information signal, move the second image sensor 15b so that the position of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1. In other words, the imaging range control unit 34 may move the position of the second image sensor 15b based on the image data of the wide-area image Im1 so that the position of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1.

[0102] Furthermore, the analysis unit 33 is not limited to the wide-area image Im1, but may also perform the tracking process described above based on the narrow-area image data to detect whether or not the region of interest IA has moved in the narrow-area image Im2, and to calculate the direction and amount of movement in the narrow-area image Im2. For example, the analysis unit 33 may use the image data of the region of interest IA detected by the object detection process described above from the narrow-area image Im2 captured at a first time as a template, detect the region of interest IA from the narrow-area image Im2 captured at a second time after a predetermined time using an existing template matching process, and perform the detection of whether or not the region of interest IA has moved between the first and second time points, and to calculate the direction and amount of movement of the region of interest IA.

[0103] The analysis unit 33 may then transmit the direction and amount of movement of the region of interest IA in the narrow-area image Im2 to the imaging range control unit 34. The direction and amount of movement of the region of interest IA in the narrow-area image Im2 calculated by the analysis unit 33 can be rephrased as the position of the region of interest IA in the narrow-area image Im2.

[0104] The imaging range control unit 34 receives an information signal from the analysis unit 33 (i.e., information regarding the direction and amount of movement of the region of interest IA in the narrow-area image Im2 as a detection result), and sends a drive signal S3 corresponding to the information signal to the drive unit 16, which may move the second image sensor 15b in a direction intersecting the optical axis AX0 so that the region of interest IA does not move out of the second imaging range CA2.

[0105] In other words, if the positional relationship between the target and the imaging device 1 changes, the imaging range control unit 34 may, based on the image data of the narrow-area image Im2, cause the drive unit 16 to move the second image sensor 15b (second imaging unit) so that the region of interest IA does not move out of the second imaging range CA2. Therefore, even if the positional relationship between the target and the imaging device 1 changes, the imaging device 1 can continue to generate a narrow-area image Im2 of the region of interest IA without the region of interest IA moving out of the second imaging range CA2 of the imaging device 1.

[0106] Furthermore, the analysis unit 33 may predict the direction and amount of movement of the region of interest IA in the narrow-area image Im2 per predetermined time interval using existing deep learning, based on the direction and amount of movement of the region of interest IA on the narrow-area image Im2 between different time intervals calculated by the tracking process described above, based on the narrow-area image data captured at different time intervals. In this case, the analysis unit 33 may transmit the predicted direction and amount of movement of the region of interest IA per predetermined time interval to the imaging range control unit 34. The direction and amount of movement of the region of interest IA in the narrow-area image Im2 predicted by the analysis unit 33 can be rephrased as the predicted position of the region of interest IA in the narrow-area image Im2.

[0107] The imaging range control unit 34 receives an information signal from the analysis unit 33 (i.e., predicted information regarding the direction and amount of movement of the region of interest IA in the subsequent narrow-area image Im2 as a detection result), and sends a drive signal S3 corresponding to the information signal to the drive unit 16, which may move the second image sensor 15b in a direction intersecting the optical axis AX0 so that the region of interest IA does not move outside the second imaging range CA2.

[0108] In other words, if the positional relationship between the target and the imaging device 1 changes, the imaging range control unit 34 may, based on the image data of the narrow-area image Im2, cause the drive unit 16 to move the second image sensor 15b (second imaging unit) so that the region of interest IA does not move outside the second imaging range CA2.

[0109] Furthermore, the imaging range control unit 34 may receive an information signal from the analysis unit 33 (i.e., predicted information regarding the direction and amount of movement of the region of interest IA in the narrow-area image Im2 as a detection result) and, based on that information signal, move the second image sensor 15b so that the position of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1.

[0110] In other words, the imaging range control unit 34 may move the position of the second image sensor 15b based on the image data of the narrow-area image Im2 so that the position of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1.

[0111] Furthermore, if the analysis unit 33 cannot detect the region of interest IA on the narrow-area image Im2 by the object detection process described above due to changes in the positional relationship between the target and the imaging device 1, it may determine that the region of interest IA has moved outside the second imaging range CA2. In this case, the analysis unit 33 detects the presence or absence of a region of interest IA in the wide-area image Im1 by analyzing the wide-area image data at that time. If the analysis unit 33 detects a region of interest IA, it may then calculate the position of the region of interest IA in the wide-area image Im1.

[0112] The imaging range control unit 34 may receive an information signal from the analysis unit 33 (i.e., information regarding the position of the region of interest IA in the wide-area image Im1) and move the second image sensor 15b in accordance with that information signal so that the region of interest IA is included in the second imaging range CA2. In other words, if the positional relationship between the target and the imaging device 1 changes and at least a portion of the target's image falls outside the second imaging range CA2, the imaging range control unit 34 may, based on the image data of the wide-area image Im1, cause the drive unit 16 to move the second image sensor 15b (second imaging unit) so that the region of interest IA is included in the second imaging range CA2.

[0113] Furthermore, if the analysis unit 33 cannot detect the region of interest IA on the narrow-area image Im2, the analysis unit 33 does not need to analyze the wide-area image data at that time. In this case, the imaging range control unit 34 may search for the region of interest IA by moving the second image sensor 15b in a predetermined path in a direction intersecting the optical axis AX0 (for example, in the XY plane direction). The second image sensor 15b may then be stopped at a position where the region of interest IA is included in the second imaging range CA2. Furthermore, the analysis unit 33 may detect that the region of interest IA is included in the second imaging range CA2 by the object detection process described above.

[0114] Furthermore, the imaging device 1 can continue to acquire a narrow-area image Im2 of the region of interest IA, not only when only the target to be imaged or recorded within the region of interest IA moves relative to the imaging device 1, but also when the imaging device 1 and the target move relative to each other.

[0115] In the example shown in Figure 4, the imaging device 1 is assumed to be installed in the living room LM, but this is not limited to that. For example, one or more imaging devices 1 may be installed in predetermined locations inside a building to image and record any object (the target to be imaged or recorded).

[0116] For example, the imaging device may be installed in a designated indoor location such as a nursery school, school, medical facility, nursing home, conference room, shop, or train station, and any object, such as a person, may be used as a target for imaging or recording. Alternatively, one or more imaging devices may be installed in a designated location inside a mobile body to image and record any object (target).

[0117] For example, the imaging device may be installed in a designated location on a moving object such as a vehicle, ship, or aircraft, and any object from that moving object may be imaged or recorded as a target. Alternatively, one or more imaging devices 1 may be installed in a designated location outdoors to image and record any object (target) such as a person or animal. For example, one or more imaging devices 1 may be installed in a designated location on a building such as a vending machine, streetlamp, telephone pole, bridge, shop entrance or station entrance, and image and record any object (target).

[0118] Alternatively, the imaging device 1 may be installed inside the machining chamber of a machine tool. The target may be a workpiece attached to the spindle of the machining chamber, a workpiece placed on the stage of the machining chamber, the machining point of the workpiece by the workpiece, chips generated by machining the workpiece by the workpiece, and cutting fluid applied to the machining area by the workpiece. For example, the target may be a workpiece stored inside a tool exchange device that exchanges a workpiece attached to the spindle of the machining chamber for another type of workpiece.

[0119] Figure 6 shows an example of an image based on image data generated by imaging by the imaging device 1 in another example of the usage state of the imaging device 1 of the first embodiment. In this example, the imaging device 1 is installed outdoors as an example and used as a surveillance camera to monitor the surroundings of the imaging device 1. However, even when used as a surveillance camera, one or more imaging devices 1 may be installed indoors.

[0120] Furthermore, one or more imaging devices 1 may be installed as surveillance cameras in designated locations where surveillance is necessary, such as shops, train stations, airports, medical facilities, nursing homes, prisons, military facilities, borders, roads, and parks, and may image and record any object as a target to be imaged or recorded. Furthermore, one or more imaging devices 1 may be installed on a moving object such as a vehicle, ship, or aircraft (drone, etc.), and may image and record any object (target) that requires surveillance from the moving object.

[0121] Figure 6(a) shows a wide-area image Im3 based on wide-area image data of the surrounding scenery of the imaging device 1 installed outdoors, as an example of an image captured by the first imaging range CA1 of the first image sensor 15a of the imaging device 1. The analysis unit 33 analyzes target information that may be included in the wide-area image Im3 based on the wide-area image data, and in particular detects the presence or absence of a region of interest IA that includes at least a part of the target to be imaged or recorded. The analysis unit 33 may also detect the region of interest IA by performing the object detection process described above in the same manner as the usage state of the imaging device 1 shown in Figures 4 and 5 above. The image data of the wide-area image Im3 may also be referred to as wide-area image data.

[0122] The wide-area image Im3 shown in Figure 6(a) contains an image of a person as the target. For example, the analysis unit 33 detects the region containing at least a portion of the image of the person, indicated by the dashed line, as the region of interest IA. The analysis unit 33 calculates the position of the region of interest IA in the wide-area image Im3 and transmits this information to the imaging range control unit 34.

[0123] The imaging range control unit 34, in the same manner as the operating state of the imaging device 1 shown in Figures 4 and 5 above, sends a drive signal S3 to the drive unit 16 in response to the information signal from the analysis unit 33 (i.e., information regarding the position of the region of interest IA in the wide-area image Im3 as a detection result), and moves the second image sensor 15b in the XY plane direction, for example, to make the center CC of the second imaging range CA2 (see Figure 3(b)) approximately coincide with the center position of the region of interest IA.

[0124] As a result, an image of the target corresponding to the region of interest IA is formed on the second image sensor 15b, and the imaging device 1 captures a narrow-area image Im4, shown in Figure 6(b), which corresponds to a narrow-area image of the portion of the region of interest IA in the wide-area image Im3.

[0125] Furthermore, since it is sufficient that the region of interest IA is included in the second imaging range CA2, the imaging range control unit 34 may move the second image sensor 15b so that the center CC of the second imaging range CA2 is positioned at a predetermined distance from the center of the region of interest IA. The image data of the narrow-area image Im4 may also be referred to as narrow-area image data.

[0126] The analysis unit 33 may detect whether or not the region of interest IA in the wide-area image Im3 has moved, similar to the usage status of the imaging device 1 shown in Figures 4 and 5 above. If the target to be imaged or recorded (person, animal, etc.) and the imaging device 1 move relative to each other (i.e., if the positional relationship between the target and the imaging device 1 changes), the analysis unit 33 may calculate the direction and amount of movement of the region of interest IA in the wide-area image Im3. The analysis unit 33 transmits the calculated direction and amount of movement of the region of interest IA in the wide-area image Im3 to the imaging range control unit 34.

[0127] The imaging range control unit 34 may, in the same manner as the usage state of the imaging device 1 shown in Figures 4 and 5 above, send a drive signal S3 to the drive unit 16 in response to the information signal from the analysis unit 33 (i.e., information regarding the direction and amount of movement of the region of interest IA in the wide-area image Im3 as a detection result), and move the second image sensor 15b in a direction intersecting the optical axis AX0 so that the region of interest IA does not move outside the second imaging range CA2.

[0128] Therefore, even if the region of interest IA in the wide-area image Im3 moves due to a change in the positional relationship between the imaging device 1 and the target, the imaging device 1 can continue to generate a narrow-area image Im4 of the region of interest IA (i.e., a high-resolution image of at least a portion of the target) without the region of interest IA moving out of the second imaging range CA2 of the imaging device 1.

[0129] Furthermore, the analysis unit 33 may detect the movement of the region of interest IA between multiple wide-area images Im3 captured by the second image sensor 15b at different times by performing the tracking process described above in the same manner as the usage state of the imaging device 1 shown in Figures 4 and 5 above (that is, it may detect whether or not the region of interest IA has moved and calculate the direction and amount of movement of the region of interest IA). Furthermore, the analysis unit 33 may detect the movement of the region of interest IA not only by the existing tracking process but also by other existing image processing methods.

[0130] Furthermore, even if at least a portion of the region of interest IA moves outside the second imaging range CA2 due to a change in the positional relationship between the imaging device 1 and the target, if the region of interest IA is included in the wide-area image Im3, the analysis unit 33 can detect the position (direction of movement and amount of movement) of the region of interest IA in the wide-area image Im3. Therefore, the imaging range control unit 34 can move the position of the second image sensor 15b so that the region of interest IA is included in the second imaging range CA2.

[0131] Furthermore, the analysis unit 33 may, in the same manner as the usage state of the imaging device 1 shown in Figures 4 and 5 above, predict the direction and amount of movement of the region of interest IA per predetermined time using existing deep learning based on the direction and amount of movement of the region of interest IA in the wide-area image Im3 calculated by the tracking process described above, based on wide-area image data captured at different times.

[0132] In this case, the analysis unit 33 may transmit the predicted direction and amount of movement of the region of interest IA in the wide-area image Im3 per predetermined time to the imaging range control unit 34. The direction and amount of movement of the region of interest IA in the wide-area image Im3 predicted by the analysis unit 33 can be rephrased as the position of the region of interest IA in the wide-area image Im3.

[0133] The imaging range control unit 34 may, in the same manner as the usage state of the imaging device 1 shown in Figures 4 and 5 above, send a drive signal S3 to the drive unit 16 in response to the information signal from the analysis unit 33 (i.e., predicted information regarding the direction and amount of movement of the region of interest IA in the wide-area image Im3 as a detection result), and move the second image sensor 15b in a direction intersecting the optical axis AX0 so that the region of interest IA does not move outside the second imaging range CA2.

[0134] Furthermore, the imaging range control unit 34 may, in the same manner as the usage state of the imaging device 1 shown in Figures 4 and 5 above, receive an information signal from the analysis unit 33 (i.e., predicted information regarding the direction and amount of movement of the region of interest IA in the wide-area image Im3 as a detection result), and based on that information signal, move the second image sensor 15b so that the position of the region of interest IA in the second imaging range CA2 does not change substantially before and after the change in the positional relationship between the target and the imaging device 1.

[0135] Furthermore, the analysis unit 33 is not limited to wide-area image data, but may also perform the tracking process described above based on narrow-area image data, similar to the usage state of the imaging device 1 shown in Figures 4 and 5 above, to detect whether or not the region of interest IA in the narrow-area image Im4 has moved, and to calculate the direction and amount of movement in the narrow-area image Im4.

[0136] The analysis unit 33 may also transmit the direction and amount of movement of the region of interest IA in the narrow-area image Im4 to the imaging range control unit 34. The direction and amount of movement of the region of interest IA in the narrow-area image Im4 calculated by the analysis unit 33 can be rephrased as the position of the region of interest IA in the narrow-area image Im4.

[0137] The imaging range control unit 34 may, in the same manner as the operating state of the imaging device 1 shown in Figures 4 and 5 above, receive an information signal from the analysis unit 33 (i.e., information regarding the direction and amount of movement of the region of interest IA in the narrow-area image Im4 as a detection result), send a drive signal S3 corresponding to the information signal to the drive unit 16, and move the second image sensor 15b in a direction intersecting the optical axis AX0 so that the region of interest IA does not move outside the second imaging range CA2.

[0138] Furthermore, the analysis unit 33 may predict the direction and amount of movement of the region of interest IA in the narrow-area image Im4 per predetermined time period using existing deep learning, based on the direction and amount of movement of the region of interest IA in the narrow-area image Im4 calculated by the tracking process described above, based on the narrow-area image data captured at different times, similar to the usage state of the imaging device 1 shown in Figures 4 and 5 above.

[0139] In this case, the analysis unit 33 may transmit the predicted direction and amount of movement of the region of interest IA per predetermined time interval to the imaging range control unit 34. The direction and amount of movement of the region of interest IA in the narrow-area image Im4 predicted by the analysis unit 33 can be rephrased as the predicted position of the region of interest IA in the narrow-area image Im4.

[0140] The imaging range control unit 34 may, in the same manner as the operating state of the imaging device 1 shown in Figures 4 and 5 above, receive an information signal from the analysis unit 33 (i.e., predicted information regarding the direction and amount of movement of the region of interest IA in the subsequent narrow-area image Im4 as a detection result), send a drive signal S3 corresponding to the information signal to the drive unit 16, and move the second image sensor 15b in a direction intersecting the optical axis AX0 so that the region of interest IA does not move outside the second imaging range CA2.

[0141] Furthermore, the imaging range control unit 34 may receive an information signal from the analysis unit 33 (i.e., predicted information regarding the direction and amount of movement of the region of interest IA in the narrow-area image Im4 as a detection result) and, based on that information signal, move the second image sensor 15b so that the position of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1.

[0142] Furthermore, the analysis unit 33 may determine that the region of interest IA has moved outside the second imaging range CA2 if, similar to the usage conditions of the imaging device 1 shown in Figures 4 and 5 above, the region of interest IA cannot be detected on the narrow-area image Im4 by the object detection process described above due to changes in the positional relationship between the target and the imaging device 1.

[0143] In this case, the analysis unit 33 detects the presence or absence of a region of interest IA in the wide-area image Im3 by analyzing the wide-area image data at that time. If the analysis unit 33 detects a region of interest IA, it may then calculate the position of the region of interest IA in the wide-area image Im3.

[0144] The imaging range control unit 34 may, in the same manner as the usage state of the imaging device 1 shown in Figures 4 and 5 above, receive an information signal from the analysis unit 33 (i.e., information regarding the position of the region of interest IA in the wide-area image Im3) and move the second image sensor 15b in accordance with that information signal so that the region of interest IA is included in the second imaging range CA2.

[0145] Furthermore, if the region of interest IA cannot be detected on the narrow-area image Im4, the analysis unit 33 does not need to analyze the wide-area image data at that time, similar to the usage state of the imaging device 1 shown in Figures 4 and 5 above. In this case, the imaging range control unit 34 may search for the region of interest IA by moving the second image sensor 15b in a predetermined path in a direction intersecting the optical axis AX0. The second image sensor 15b may then be stopped at a position where the region of interest IA is included in the second imaging range CA2.

[0146] In the example shown in Figure 6, the imaging device 1 may perform at least one of the operations A through F described above, in the same manner as in the example shown in Figure 5. The analysis unit 33 may also analyze whether a person in the region of interest IA possesses a gun, bomb, or other dangerous object, and if the person possesses a predetermined dangerous object, it may perform at least one of the operations A through F described above.

[0147] Furthermore, the positional relationship between the first image 14a and the first image sensor 15a and the second image 14b and the second image sensor 15b may be reversed. That is, the first image 14a may be an image formed by a light beam that passes through the divided reflective surface 13a of the light-dividing member 13 and travels roughly along the optical axis AX0. The first image sensor 14a may be positioned at a location that roughly coincides with this first image 14a. In this case, the light beam that passes through the divided reflective surface 13a of the light-dividing member 13 and travels roughly along the optical axis AX0 can be called one of the light beams.

[0148] Furthermore, the second image 14b may be an image formed by a light beam reflected by the divided reflective surface 13a of the light-dividing member 13 and traveling approximately along the optical axis AX1. The second image sensor 15b may be positioned at a location that approximately coincides with this second image 14b. In this case, the second image sensor 15b is held by the holding unit 17 and is held together with the holding unit 17 by the drive unit 16 so as to be movable in the Y and Z directions, which are directions intersecting the optical axis AX1. In this case, the light beam reflected by the divided reflective surface 13a of the light-dividing member 13 and traveling approximately along the optical axis AX1 can be called the other light beam.

[0149] In the above description, it was assumed that the array pitch (PXb, PYb) of the second pixel PEb of the second image sensor 15b is smaller than the array pitch (PZa, PYa) of the first pixel PEa of the first image sensor 15a, but this is not necessarily the only case. The array pitch (PXb, PYb) of the second pixel PEb may be the same as the array pitch (PZa, PYa) of the first pixel PEa of the first image sensor 15a, or it may be larger than the array pitch (PZa, PYa) of the first pixel PEa of the first image sensor 15a.

[0150] Furthermore, although the above assumes that the area of ​​the second effective region EA2 of the second image sensor 15b is smaller than the area of ​​the first effective region EA1 of the first image sensor 15a, this is not necessarily the only limitation. The area of ​​the second effective region EA2 of the second image sensor 15b may be the same as the area of ​​the first effective region EA1 of the first image sensor 15a, or it may be smaller than the area of ​​the first effective region EA1 of the first image sensor 15a.

[0151] The control unit 30 does not necessarily have to be integrated with the imaging device 1 (or the imaging device 1A described later), and may be located separately from the imaging device 1 (or the imaging device 1A described later). In other words, the imaging device 1 (or the imaging device 1A described later) does not necessarily have to be equipped with a control unit 30. In this case, the imaging signals S1a, S1b, the control signals S2a, S2b, and the drive signal S3 may be transmitted between the imaging device 1 (or the imaging device 1A described later) and the control unit 30, for example, via a network line or wirelessly.

[0152] The imaging device 1 (or imaging device 1A described later) includes the imaging control unit 31, image generation unit 32, analysis unit 33, imaging range control unit 34, storage unit 35, and part of the interface unit 36 ​​of the control unit 30, while the remaining parts may be arranged separately from the imaging device 1 (or imaging device 1A described later).

[0153] For example, when the imaging device 1 (or the imaging device 1A described later) is installed on an object, the control device that controls the object may include the control unit 30 (or a part of the imaging control unit 31, image generation unit 32, analysis unit 33, imaging range control unit 34, storage unit 35, and interface unit 36). In other words, the control device that controls the object may be configured to perform the functions of the control unit 30 (or a part of the functions of the imaging control unit 31, image generation unit 32, analysis unit 33, imaging range control unit 34, storage unit 35, and interface unit 36).

[0154] For example, when the imaging device 1 (or the imaging device 1A described later) is installed on a mobile device, machine tool, optical processing machine, etc., the control device that controls the mobile device, machine tool, or optical processing machine may include the control unit 30 (or a part of the imaging control unit 31, image generation unit 32, analysis unit 33, imaging range control unit 34, storage unit 35, and interface unit 36). In other words, the control device that controls the mobile device or machine tool may be configured to perform the functions of the control unit 30 (or a part of the functions of the imaging control unit 31, image generation unit 32, analysis unit 33, imaging range control unit 34, storage unit 35, and interface unit 36).

[0155] Furthermore, the imaging device 1 does not necessarily need to have a holding unit 17 for holding the second image sensor 15b; the drive unit 16 may directly hold the second image sensor 15b without the holding unit 17.

[0156] The imaging device 1 may also include a re-imaging optical system (not shown) for re-imaging the second image 14b. For example, a re-imaging optical system may be placed between the light-splitting member 13 and the second image sensor 15b, and the second image 14b formed by the light beam split by the light-splitting member 13 (the other light beam) may be re-imaged. Then, a portion of the re-imaged second image 14b may be captured by the second image sensor 15b. In this case, the second image sensor 15b may be placed near the position where the second image 14b is re-imaged by the re-imaging optical system. In this case, the re-imaging optical system and the second image sensor 15b may also be referred to as the second imaging unit.

[0157] The re-imaging optical system may be fixed to the housing 19 via a retaining part (not shown). The re-imaging optical system may include at least one optical element. The optical element included in the re-imaging optical system may include a lens. The re-imaging optical system may include multiple lenses. In this case, each of the multiple lenses may be a single lens, or it may be a group of multiple lenses. The re-imaging optical system may include four lenses. The re-imaging optical system may include four lens groups.

[0158] The imaging magnification of the re-imaging optical system may be an enlargement magnification, a 1:1 magnification, or a reduction magnification. Furthermore, the imaging magnification may be changed by moving at least some of the lenses of the re-imaging optical system along the optical axis (for example, in the optical axis AX0 direction) using a drive unit (not shown). In this case, the imaging magnification may be increased or decreased while maintaining the enlargement magnification, increased or decreased while maintaining the reduction magnification, or increased or decreased within a range from the enlargement magnification to the reduction magnification.

[0159] Furthermore, when the imaging magnification of this re-imaging optical system is set to the magnification magnification, it is possible to capture an image of a target that is magnified more than when there is no re-imaging optical system (i.e., compared to 1x magnification) (to obtain a narrow-area image of the magnified target). In this case, it can also be said that at least a portion of the second image 14b, which is magnified more than when there is no re-imaging optical system (i.e., compared to 1x magnification), can be captured.

[0160] Furthermore, if the imaging magnification of the re-imaging optical system is set to an enlarged magnification, the positioning resolution of the second image sensor 15b relative to the second image 14b can be increased compared to the case without the re-imaging optical system (i.e., compared to 1x magnification). Conversely, if the positioning resolution of the second image sensor 15b relative to the second image 14b can be the same as in the case without the re-imaging optical system (i.e., 1x magnification), the drive resolution of the drive unit 16 (the positioning resolution of the second image sensor 15b by the drive unit 16) can be reduced, and an existing inexpensive drive mechanism can be used for the drive unit 16.

[0161] Furthermore, if the imaging magnification of this re-imaging optical system is set to an enlarged magnification, it can be said that the positioning resolution of the range on the object side (target side) imaged by the second image sensor 15b via the imaging optical system 20 can be increased compared to the case without the re-imaging optical system (i.e., compared to 1x magnification).

[0162] Furthermore, when the imaging magnification of this re-imaging optical system is set to a reduction magnification, it is possible to capture an image of the target over a wider area (i.e., compared to 1x magnification) compared to when there is no re-imaging optical system (i.e., compared to 1x magnification). In this case, it can also be said that a wider portion of the second image 14b can be captured compared to when there is no re-imaging optical system (i.e., compared to 1x magnification).

[0163] Furthermore, if the imaging magnification of this re-imaging optical system is set to a reduction magnification, the position of the second image sensor 15b relative to the second image 14b can be changed at a faster speed compared to the case without the re-imaging optical system (i.e., compared to 1x magnification). Conversely, if the displacement speed of the second image sensor 15b relative to the second image 14b can be the same as in the case without the re-imaging optical system (i.e., 1x magnification), the driving speed of the drive unit 16 (the displacement speed of the second image sensor 15b by the drive unit 16) can be reduced, and an existing inexpensive drive mechanism can be used for the drive unit 16.

[0164] Furthermore, if the imaging magnification of the re-imaging optical system is set to a reduction magnification, it can be said that the range of the object (target) imaged by the second image sensor 15b via the imaging optical system 20 can be changed more rapidly compared to the case without a re-imaging optical system (i.e., compared to 1x magnification).

[0165] The imaging device 1 may also have a distance measuring unit that measures the distance from the imaging device 1 to the target, etc. The distance measuring unit may be, for example, a so-called LiDAR that measures the distance by shining light onto the target, etc. and measuring the time it takes for the light to return to the distance measuring unit. Alternatively, it may be a system that estimates the distance to the target based on the focus state of the target, etc., similar to the autofocus detection mechanism in a camera.

[0166] By having a distance measuring unit such as LiDAR, the imaging device 1 can determine the distance from the imaging device 1 to the target, etc. Furthermore, it can grasp the general three-dimensional structure of the target, etc.

[0167] (Effects of the imaging device of the first embodiment) (1) The imaging device 1 of the first embodiment includes an imaging optical system 10, an optical splitting member 13 that amplitude-splits light that has passed through at least a part of the imaging optical system 10, a first imaging unit (first image sensor 15a) that images a first image 14a formed by one of the optical beams split by the optical splitting member 13, and a second imaging unit (second image sensor 15b) that images a part of the second image 14b formed by the other optical beam split by the optical splitting member 13. The device also includes a drive unit 16 that moves the second imaging unit in a direction that crosses the optical path of the other optical beam. In this configuration, the imaging device 1 forms two images, a first image 14a and a second image 14b, using a single imaging optical system 10. The first imaging unit (15a) captures wide-area images Im1 and Im3, which may include the target, from the first image 14a, and the second imaging unit (15b) captures narrow-area images Im2 and Im4, which may include the target, from the second image 14b. The drive unit 16 moves the second imaging unit (15b), thereby rapidly moving the imaging range captured by the second image sensor 15b within the second image 14b. This also allows the imaging field of view on the object side (target side) captured by the second image sensor 15b to move rapidly.

[0168] (2) The first imaging unit includes a first image sensor 15a that captures a first image 14a, and the second imaging unit includes a second image sensor 15b that captures an image of a part of the second image 14b, wherein the array pitch (PXb, PYb) of the pixels (second pixels PEb) of the second image sensor 15b is smaller than the array pitch (PZa, PYa) of the pixels (first pixels PEa) of the first image sensor 15a, and the drive unit 16 may move the second image sensor 15b in a direction that crosses the optical path of the other light beam. In this configuration, the resolution of the narrow-area images Im2 and Im4 captured by the second image sensor 15b can be made higher than the resolution of the wide-area images Im1 and Im3 captured by the first image sensor 15a.

[0169] (3) The maximum field of view of the imaging optical system 10, that is, the maximum field of view on the object side of the wide-area images Im1 and Im3 captured by the first image sensor 15a, may be 170° or more. In this case, the wide-area images Im1 and Im3 can simultaneously capture a wider range on the object side.

[0170] (Imaging device of the second embodiment) Figure 7 shows an overview of the imaging device 1A of the second embodiment. The imaging device 1A of the second embodiment shares many components with the imaging device 1 of the first embodiment described above. Therefore, the following explanation will mainly focus on the differences from the imaging device 1 of the first embodiment, and the same reference numerals will be used for common components, with explanations omitted as appropriate.

[0171] The imaging device 1A of the second embodiment differs from the imaging device 1 of the first embodiment in that it has a re-imaging optical system 20 that forms a third image 14c by re-imaging at least a portion of the second image 14b formed by a light beam that passes through the light-dividing member 13 and travels generally along the optical axis AX0. Furthermore, the second image sensor 15b is positioned near the third image 14c, not near the second image 14b. In this case, the light beam that reflects off the light-dividing member 13 and travels generally along the optical axis AX1 can be called one type of light beam. The light beam that passes through the light-dividing member 13 and travels generally along the optical axis AX0 can be called the other type of light beam.

[0172] The re-imaging optical system 20 and the second image sensor 15b can also be referred to as the second imaging unit. The second image 14b is formed by a light beam (the other light beam) traveling roughly along the optical axis AX0, and the second image sensor 15b captures at least a portion of the third image 14c, which is formed when a portion of the second image 14b is re-imaged by the re-imaging optical system 20. In other words, the second imaging unit captures a portion of the second image formed by the other light beam divided by the light-dividing member. Furthermore, since at least a portion of the second image 14b is re-imaged as the third image 14c by the re-imaging optical system 20, the second image 14b can also be called an intermediate image.

[0173] Furthermore, when the second imaging unit (re-imaging optical system 20 and second image sensor 15b) images a portion of the second image 14b, it can be said that a portion of the other light beam from the light splitting member 13 is incident on the second imaging unit. In this case, it can be said that the second imaging unit has a light incident section 29 into which a portion of the other light beam is incident. Furthermore, it can be said that the light incident section 29 is at least a part of the second imaging unit. For example, the light incident section 29 may be the light incident side (light splitting member 13 side) of the lens 21a in the re-imaging optical system 20.

[0174] The second imaging unit can also be referred to as the light-receiving unit. The light-receiving unit can be said to receive a portion of the other light beam from the light-dividing member 13. In this case, the light-receiving unit can also be said to have a light-incident unit 29 into which a portion of the other light beam is incident. The light-incident unit 29 can also be said to be at least a part of the light-receiving unit.

[0175] The second imaging unit may include not only the re-imaging optical system 20 and the second image sensor 15b, but also other components. For example, in addition to the re-imaging optical system 20 and the second image sensor 15b, the second imaging unit may include at least one of the following: the second drive unit 28, the movable support units 24 and 25, the first drive unit 16a, the second holding frame 17a, and the second holding unit 17b.

[0176] The optical axis of the re-imaging optical system 20 is the optical axis AX2 shown in Figure 7. The re-imaging optical system 20 includes, as an example, a plurality of lenses 21a to 21d arranged along the optical axis AX2. The plurality of lenses 21a to 21d are held by a second holding frame 17a. The second image sensor 15b is held by a second holding part 17b attached to the second holding frame 17a, and is positioned so that its imaging surface generally coincides with the third image 14c. In the imaging device 1A of the second embodiment, the center of the second effective area EA2 of the second image sensor 15b generally coincides with the optical axis AX2 of the re-imaging optical system 20.

[0177] The second retaining frame 17a is held in the housing 19 via the first drive unit 16a. The first drive unit 16a includes, for example, a stator 23 fixed to the housing 19 and a mover 22 fixed to the second holding frame 17a and movable in the XY plane relative to the stator 23. The first drive unit 16a holds the second holding frame 17a and moves it in the XY plane relative to the housing 19. In other words, the first drive unit 16a moves the re-imaging optical system 20 and the second image sensor 15b (i.e., the second imaging unit), which are supported by the second holding frame 17a, in the XY plane relative to the imaging optical system 10 fixed to the housing 19.

[0178] When the first drive unit 16a moves the re-imaging optical system 20 and the second image sensor 15b, the region in the second image 14b that is re-imaging as the third image 14c and imaged on the second image sensor 15b, i.e., the second imaging range CA2 (see Figure 3(b)) described above, changes. In the second embodiment, the size of the second imaging range CA2 in the second image 14b is not necessarily the same as the size of the second effective region EA2 of the second image sensor 15b, and may be enlarged or reduced according to the imaging magnification of the re-imaging optical system 20.

[0179] The first drive unit 16a may, for example, be a linear motor having a stator 23 and a mover 22, or it may be a configuration having a stepping motor and a lead screw. The first drive unit 16a may also be a configuration having a stepping motor and a guide bar. That is, the second holding frame 17a is moved in the XY plane by the first drive unit 16a while supporting the re-imaging optical system 20 and the second image sensor 15b (i.e., the second imaging unit) so that the optical positional relationship between the re-imaging optical system 20 and the second image sensor 15b (for example, the relative positional relationship between the re-imaging optical system 20 and the second image sensor 15b) does not change. The drive unit 16a may be able to move the second holding frame 17a (re-imaging optical system 20 and second image sensor 15b) in the X direction or the Y direction. The drive unit 16a may be able to move the second holding frame 17a in any one direction in the XY plane.

[0180] Furthermore, the direction of movement of the second holding frame 17a by the first drive unit 16a is not necessarily limited to the XY plane direction, but can be any direction intersecting the Z direction, that is, a direction intersecting the optical axis AX0 of the imaging optical system 10. In other words, the first drive unit 16a may move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0 of the imaging optical system 10.

[0181] Furthermore, the direction of movement of the second retaining frame 17a may also be a rotational direction, for example, at least one of the following rotational directions: rotation around the X axis (θx direction), rotation around the Y axis (θy direction), and rotation around the Z axis (θz direction).

[0182] In other words, the first drive unit 16a moves the second holding frame 17a, that is, the re-imaging optical system 20 and the second image sensor 15b (second imaging unit), in a direction that intersects with the direction in which the other light beam, which travels roughly along the optical axis AX0, is incident on the second image sensor 15b. Alternatively, the first drive unit 16a can be said to move the second holding frame 17a in the in-plane direction of the image plane of the second image 14b.

[0183] The first drive unit 16a can also be said to have a configuration corresponding to the drive unit 16 in the imaging device 1 of the first embodiment described above. Furthermore, the second holding frame 17a and the second holding portion 17b can be said to have a configuration corresponding to the holding portion 17 in the imaging device 1 of the first embodiment described above.

[0184] Furthermore, if the second holding frame 17a directly holds the second image sensor 15b, the second holding portion 17b does not need to be provided. The second holding frame 17a may integrally hold the re-imaging optical system 20 and the second image sensor 15b. Furthermore, the second holding frame 17a may be a single, integrally formed structure, or it may be a structure in which multiple components are joined together.

[0185] Of the lenses 21a to 21d that constitute the re-imaging optical system 20, lenses 21a and 21d are fixed to the second retaining frame 17a. In contrast, lens 21b is supported by a movable support part 24 that is supported so as to be movable relative to the second retaining frame 17a in the direction of the optical axis AX2 (for example, in the ±Z direction) by a drive mechanism 26 such as a linear motor. Lens 21c is supported by a movable support part 25 that is supported so as to be movable relative to the second retaining frame 17a in the direction of the optical axis AX2 (for example, in the ±Z direction) by a drive mechanism 27 such as a linear motor.

[0186] In other words, the drive mechanisms 26 and 27 allow the lenses 21b and 21c to move in the direction of the optical axis AX2 relative to the lenses 21a and 21d. It is sufficient that the drive mechanisms 26 and 27 allow the lenses 21b and 21c to move approximately in the direction of the optical axis AX2 relative to the lenses 21a and 21d, but they may also move in a direction inclined from the direction of the optical axis AX2 relative to the lenses 21a and 21d. Furthermore, the drive mechanisms 26 and 27 are not limited to the linear motor mechanism described above, but may also be a moving mechanism having a stepping motor and a lead screw.

[0187] Lens 21b may be configured to be movable in either the +Z direction or the -Z direction by the drive mechanism 26, or it may be configured to be movable in only one of the +Z or -Z directions. Similarly, lens 21c may be configured to be movable in either the +Z direction or the -Z direction by the drive mechanism 27, or it may be configured to be movable in only one of the +Z or -Z directions.

[0188] Hereinafter, the drive mechanisms 26 and 27, together or individually, will also be referred to as the second drive unit 28. The second drive unit 28 drives lenses 21b and 21c, causing lenses 21b and 21c to move in the Z direction, and changing the imaging magnification of the re-imaging optical system 20 (the magnification of the third image 14c relative to the second image 14b, i.e., the third image 14c).

[0189] In Figure 7, lenses 21a to 21d are shown as single lenses, but they may also be lens groups consisting of multiple lenses, or they may include mirrors, diffractive optical elements, etc. Furthermore, the number of lens groups constituting the re-imaging optical system 20 is not limited to the four shown, but may be any number of one or more. Moreover, the number of lens groups (or lenses) among the lens groups constituting the re-imaging optical system 20 that are moved in the Z direction by the second drive unit 28 is not limited to the two groups (lenses 21b and 21c) mentioned above, but may be any other number.

[0190] The imaging position of the third image 14c may be aligned with the imaging plane of the second image sensor 15b by moving some of the lenses 21b, 21c, etc., in the direction of the optical axis AX2 by the second drive unit 28. Alternatively, there may be a drive mechanism that moves other lenses 21a, 21d, etc., which are not moved in the optical axis direction by the second drive unit 28, in the direction of the optical axis AX2, and this drive mechanism may be used to align the imaging position of the third image 14c with the imaging plane of the second image sensor 15b. Alternatively, there may be a drive mechanism that moves the second image sensor 15b in the direction of the optical axis AX2, and this drive mechanism may be used to align the imaging plane of the second image sensor 15b with the imaging position of the third image 14c.

[0191] The first drive unit 16a drives the center CC of the second imaging range CA2 in the second image 14b, which changes the position. The second drive unit 28 drives the size of the second imaging range CA2, which changes the size. In other words, the first drive unit 16a or the second drive unit 28 drives the imaging field of view on the object side (target side) captured by the second image sensor 15b, which changes.

[0192] The imaging magnification of the re-imaging optical system 20 may be an enlargement magnification that enlarges at least a part of the second image 14b to form the third image 14c, or it may be 1x magnification or a reduction magnification. The imaging magnification of the re-imaging optical system 20 may be increased or decreased while maintaining the enlargement magnification, increased or decreased while maintaining the reduction magnification, or increased or decreased within a range from the enlargement magnification to the reduction magnification by moving some of the lenses 21b, 21c, etc., in the optical axis AX2 direction by the second drive unit 28.

[0193] If the imaging magnification of the re-imaging optical system 20 is the magnification magnification, the third image 14c will be an image in which at least a part of the second image 14b is magnified, and the resolution of the narrow-field image captured by the second image sensor 15b will be further improved.

[0194] Furthermore, the re-imaging optical system 20 and the second image sensor 15b do not necessarily have to be held together by the second holding frame 17a. For example, the re-imaging optical system 20 may be held by the second holding frame 17a, and the second image sensor 15b may be held by a holding mechanism other than the second holding frame 17a. The first drive unit 16a may move the second holding frame 17a and the other holding mechanism in a direction intersecting the optical axis AX0 (Z direction) of the imaging optical system 10 so as not to change the optical positional relationship between the re-imaging optical system 20 and the second image sensor 15b (the relative positional relationship between the re-imaging optical system 20 and the second image sensor 15b). In this case, the second holding frame 17a and the other holding mechanism will move synchronously.

[0195] Furthermore, the imaging magnification of the re-imaging optical system 20 (i.e., the magnification of the third image 14c) may be changed by means other than moving the lenses 21b and 21c in the Z direction as described above. For example, a so-called liquid lens may be provided within the re-imaging optical system 20, and the imaging magnification of the re-imaging optical system 20 may be changed by changing the voltage applied to the liquid lens. In this case, the second drive unit 28 becomes a voltage control unit that controls the voltage applied to the liquid lens.

[0196] The imaging magnification of the re-imaging optical system 20 may also be changed by inserting or removing a predetermined optical system from the re-imaging optical system 20. In this case, the second drive unit 28 becomes an insertion / removal unit for inserting or removing the predetermined optical system. The optical system arranged in the optical path of light from the imaging optical system 10 (i.e., at least a part of the re-imaging optical system 20) may be configured to be interchangeable between a first magnification optical system and a second magnification optical system, which have different imaging magnifications.

[0197] In this case, the magnification of the third image 14c changes depending on whether the optical system placed on the optical path of the light from the imaging optical system 10 is a first-magnification optical system or a second-magnification optical system. In this case, the second drive unit 28 acts as an exchange unit that swaps the optical system placed on the optical path of the light from the imaging optical system 10 between a first-magnification optical system and a second-magnification optical system. Alternatively, the imaging magnification of the optical system (re-imaging optical system 20) may be changed by reversing at least a part of the optical system placed on the optical path of the light from the imaging optical system 10.

[0198] Figure 8 is an enlarged view showing a portion of the imaging optical system 10 and the re-imaging optical system 20 of the imaging device 1A of the second embodiment. The two light beams LB1 and LB2 shown in Figure 8 are light beams that enter the imaging optical system 10 from different parts of an object (not shown) (or from different objects), form a portion of the second image 14b, and enter the re-imaging optical system 20. In the following, the region in which the second image 14b is formed will also be referred to as the image-forming region. Note that objects not shown may be targets to be imaged or recorded.

[0199] The light beam LB1 is directed from the imaging optical system 10 to the first location Q1 in the image-forming region where the second image 14b is formed. The light beam LB2 is directed from the imaging optical system 10 to the second location Q2 in the image-forming region. The distance D1 from the optical axis AX0 of the imaging optical system 10 to the first location Q1 is different from the distance D2 from the optical axis AX0 of the imaging optical system 10 to the second location Q2. The principal ray PR1 of luminous flux LB1 is the ray that passes through the center of luminous flux LB1, and the principal ray PR2 of luminous flux LB2 is the ray that passes through the center of luminous flux LB2.

[0200] As described above, in the imaging device 1A of the second embodiment, the re-imaging optical system 20 re-images at least a portion of the second image 14b formed by the imaging optical system 10 to form a third image 14c on the second image sensor 15b. Generally, in such a configuration, if the optical axis AX2 of the re-imaging optical system 20 is misaligned in a direction that intersects the optical axis AX0 of the imaging optical system 10, the incident angle of the light beam incident on the re-imaging optical system 20 changes, which deteriorates the imaging performance of the re-imaging optical system 20.

[0201] In contrast, in the imaging device 1A of the second embodiment, the principal rays PR1, PR2, etc. of each light beam (light beams LB1, LB2, etc.) formed by the imaging optical system 10 are incident on the image-forming region approximately parallel to the optical axis AX0. In other words, the imaging optical system 10 is so-called telecentric on the image-forming region (second image 14b) side.

[0202] The angle between the optical axis AX0 and the direction of propagation of the principal rays (principal rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident on the image-forming region may, for example, be within 1°. In this case, the difference between the angle between the optical axis AX0 and the principal ray PR1 of the light beam LB1 traveling from the imaging optical system 10 to the first location Q1 in the image-forming region, and the angle between the optical axis AX0 and the optical axis AX0 of the principal ray PR2 of the light beam LB2 traveling from the imaging optical system 10 to the second location Q2 in the image-forming region, can be said to be within 1°.

[0203] Furthermore, the angle between the optical axis AX0 and the direction of propagation of the principal rays (principal rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident on the image-forming region may be within 0.5° or within 0.3°. In this case, the difference between the angle between the optical axis AX0 and the principal ray PR1 of the light beam LB1 traveling from the imaging optical system 10 to the first location Q1 in the image-forming region, and the angle between the optical axis AX0 and the optical axis AX0 of the principal ray PR2 of the light beam LB2 traveling from the imaging optical system 10 to the second location Q2 in the image-forming region, may be within 0.5° or within 0.3°.

[0204] Furthermore, the angle between the direction of propagation of the principal rays of all light beams incident on the image-forming region and the optical axis AX0 may be 0.5° or 0.3° or less. Also, the angle between the direction of propagation of two principal rays incident on the image-forming region at different positions and the optical axis AX0 may be 0.5° or 0.3° or less. These conditions may be satisfied for at least one light beam incident on the image-forming region.

[0205] Therefore, the principal rays (principal rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) that enter the re-imaging optical system 20 via the image-forming region (second image 14b) also travel approximately parallel to the optical axis AX2 of the re-imaging optical system 20 and enter the re-imaging optical system 20. For this reason, even if the re-imaging optical system 20 is moved by the first drive unit 16a in a direction perpendicular to the optical axis AX0 of the imaging optical system 10 (in the XY plane), the incident angle of the light beams (light beams LB1, LB2, etc.) entering the re-imaging optical system 20 hardly changes. Therefore, even if the re-imaging optical system 20 is moved in the XY plane relative to the imaging optical system 10, the re-imaging optical system 20 maintains good imaging performance, and a high-resolution third image 14c can be captured by the second image sensor 15b.

[0206] To make the imaging optical system 10 telecentric on the second image 14b side (image formation region side), for example, an aperture diaphragm 10S may be provided at a predetermined position. Alternatively, the telecentricity on the second image 14b side (image formation region side) may be achieved by limiting the effective diameter of a predetermined lens, such as lens 11c, to a predetermined size. Note that the position where the aperture diaphragm 10S is placed is not limited to the position shown in Figure 8. Note that the aperture diaphragm 10S does not necessarily have to be provided in the imaging optical system 10.

[0207] Furthermore, by making the re-imaging optical system 20 telecentric on the second image 14b side, i.e., the imaging optical system 10 side, it is possible to prevent changes in the incident angle of the light beams (light beams LB1, LB2, etc.) incident on the re-imaging optical system 20 due to movement of the re-imaging optical system 20 in the XY plane. To achieve this, for example, an aperture diaphragm (not shown) can be provided inside the re-imaging optical system 20, and this aperture diaphragm can be set so that the principal rays PR1, PR2, etc. of the light beams (light beams LB1, LB2, etc.) passing through the re-imaging optical system 20 travel approximately parallel to the optical axis AX2 of the re-imaging optical system 20. Instead of an aperture diaphragm, the effective diameter of the lenses 21a to 21d, etc. that constitute the re-imaging optical system 20 can be limited to a predetermined system to make the re-imaging optical system 20 telecentric as described above.

[0208] The angle between the optical axis AX2 and the direction of propagation of the principal rays (principal rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident from the image-forming region (second image 14b) to the re-imaging optical system 20 may, for example, be within 1°. In this case, the difference between the angle between the optical axis AX2 and the principal ray PR1 of the light beam LB1 traveling from the first location Q1 in the image-forming region to the re-imaging optical system 20, and the angle between the optical axis AX2 and the principal ray PR2 of the light beam LB2 traveling from the second location Q2 in the image-forming region to the re-imaging optical system 20, can be said to be within 1°.

[0209] Furthermore, the angle between the direction of propagation of the principal rays (principal rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident from the image-forming region to the re-imaging optical system 20 and the optical axis AX2 may be within 0.5° or within 0.3°. In this case, the difference between the angle between the principal ray PR1 of the light beam LB1 traveling from the first location Q1 in the image-forming region to the re-imaging optical system 20 and the optical axis AX2, and the angle between the principal ray PR2 of the light beam LB2 traveling from the second location Q2 in the image-forming region to the re-imaging optical system 20 and the optical axis AX2, may be within 0.5° or within 0.3°.

[0210] Furthermore, the angle between the direction of propagation of the principal rays of all light beams incident from the image-forming region to the re-imaging optical system 20 and the optical axis AX2 may be within 0.5° or within 0.3°. Also, the angle between the direction of propagation of two principal rays from different positions in the light beams incident from the image-forming region to the re-imaging optical system 20 and the optical axis AX2 may be within 0.5° or within 0.3°. These conditions may be satisfied for at least one light beam incident from the image-forming region to the re-imaging optical system 20.

[0211] Furthermore, both the imaging optical system 10 and the re-imaging optical system 20 may be telecentric on the second image 14b side (image formation region side). In this case, the difference between the angle of the direction of propagation of the principal rays (principal rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident from the imaging optical system 10 to the image formation region (second image 14b) with respect to the optical axis AX0, and the angle of the direction of propagation of the principal rays (principal rays PR1, PR2, etc.) of the light beams (light beams LB1, LB2, etc.) incident from the image formation region (second image 14b) to the re-imaging optical system 20 with respect to the optical axis AX2 may, for example, be within 1°.

[0212] Furthermore, the optical axis AX0 of the imaging optical system 10 and the optical axis AX2 of the re-imaging optical system 20 in the image-forming region may be approximately parallel. Also, not limited to the image-forming region, the optical axis AX0 of the entire imaging optical system 10 and the optical axis AX2 of the entire re-imaging optical system 20 may be approximately parallel.

[0213] Furthermore, the difference between the angle of the direction of propagation of the principal rays (principal rays PR1, PR2, etc.) of light beams (light beams LB1, LB2, etc.) incident from the imaging optical system 10 to the image-forming region (second image 14b) with respect to the optical axis AX0, and the angle of the direction of propagation of the principal rays (principal rays PR1, PR2, etc.) of light beams (light beams LB1, LB2, etc.) incident from the image-forming region to the re-imaging optical system 20 with respect to the optical axis AX2, may be within 0.5° or within 0.3°. These conditions may be satisfied for at least one of the light beams passing through the image-forming region.

[0214] Furthermore, it is not necessary for both the imaging optical system 10 and the re-imaging optical system 20 to be telecentric on the second image 14b side. If either the imaging optical system 10 or the re-imaging optical system 20 is telecentric on the second image 14b side, the other only needs to be an optical system with a large numerical aperture (or a bright optical system) that allows light beams (light beams LB1, LB2, etc.) passing through the other optical system and the second imaging range CA2 of the second image 14b to pass without obstruction.

[0215] The imaging device 1A of the second embodiment may also be used for the same purposes as the imaging device 1 of the first embodiment described above. The configuration and function of the control unit 30 of the imaging device 1A in the second embodiment are substantially the same as those of the control unit 30 of the imaging device 1 in the first embodiment described above. However, the imaging range control unit 34 may control not only the position of the second imaging unit (second image sensor 15b and re-imaging optical system 20, etc.) in the direction intersecting the optical axis AX0, but also the imaging magnification of the re-imaging optical system 20.

[0216] The analysis unit 33 of the control unit 30 analyzes, similar to the first embodiment described above, the target information that may be included in the wide-area image data corresponding to the wide-area images Im1 and Im3 captured by the first image sensor 15a, or the target information that may be included in the narrow-area image data corresponding to the narrow-area images Im2 and Im4 captured by the second image sensor 15b. The analysis unit 33 may also analyze the target information that may be included in the wide-area image data corresponding to the wide-area images Im1 and Im3 captured by the first image sensor 15a and the target information that may be included in the narrow-area image data corresponding to the narrow-area images Im2 and Im4 captured by the second image sensor 15b.

[0217] In the imaging device 1A of the second embodiment, the analysis unit 33 may perform at least one of the following processes: detection of the presence or absence of a region of interest IA in the above-described images (i.e., at least one of the wide-area images Im1, Im3, and narrow-area images Im2, Im4); calculation of the position of the region of interest IA in the image; calculation of the direction and amount of movement of the region of interest IA in the image in accordance with changes in the positional relationship between the imaging device 1A and the target (i.e., calculation of the position of the region of interest IA in the image after the positional relationship between the imaging device 1A and the target has changed); and detection of facial expressions or gestures or hand movements, in addition to / or calculation of the size of the region of interest IA in the image and calculation of the amount of change in the size of the region of interest IA in the image in accordance with changes in the positional relationship between the imaging device 1A and the target (i.e., calculation of the size of the region of interest IA in the image after the positional relationship between the imaging device 1A and the target has changed). In other words, the analysis unit 33 calculates the size of the region of interest IA, which can be described as the analysis unit 33 detecting the size of the region of interest IA.

[0218] For example, the analysis unit 33 may perform the object detection process described above on the wide-area image data to detect the region of interest IA and calculate the size of the region of interest IA in the wide-area images Im1 and Im3. The analysis unit 33 may then transmit the size of the region of interest IA in the wide-area images Im1 and Im3 to the imaging range control unit 34. Note that the size of the region of interest IA in the images (wide-area images Im1 and Im3 and narrow-area images Im2 and Im4) can be calculated using known methods, so a description of that calculation method will be omitted.

[0219] Furthermore, the detection of the region of interest IA by the analysis unit 33 can be rephrased as the analysis unit 33 recognizing the region of interest IA. Also, the calculation of the size of the region of interest IA in the wide-area images Im1 and Im3 by the analysis unit 33 can be rephrased as the analysis unit 33 recognizing the size of the region of interest IA.

[0220] The imaging range control unit 34 may send a drive signal S4 to the second drive unit 28 in response to an information signal from the analysis unit 33 (i.e., information regarding the size of the region of interest IA in the wide-area images Im1 and Im3 as a detection result), thereby changing the magnification of the re-imaging optical system 20, i.e., the magnification of the third image 14c (the size of the region of interest IA in the narrow-area images Im2 and Im4). In other words, the imaging range control unit 34 may move the lenses 21b and 21c in the Z direction by driving the drive mechanisms 26 and 27 of the second drive unit 28, thereby changing the magnification of the re-imaging optical system 20. Note that changing the magnification of the re-imaging optical system 20 (i.e., changing the magnification of the third image 14c) can be rephrased as changing the size of the region of interest IA in the narrow-area images Im2 and Im4.

[0221] Furthermore, the analysis unit 33 may, in addition to calculating the size of the region of interest IA in the wide-area images Im1 and Im3, and / or, similar to the first embodiment, calculate the position of the region of interest IA in the wide-area images Im1 and Im3. Then, the imaging range control unit 34 may, in addition to changing the magnification of the re-imaging optical system 20, and / or, based on the above-mentioned detection results by the analysis unit 33 (information regarding the position of the region of interest IA in the wide-area images Im1 and Im3), send a drive signal S3 to the first drive unit 16a, similar to the first embodiment described above, and move the re-imaging optical system 20 and the second image sensor 15b (i.e., the second imaging unit) in a direction intersecting the optical axis AX0.

[0222] In other words, the imaging range control unit 34 may, based on wide-area image data, perform at least one of the following: moving the re-imaging optical system 20 and the second image sensor 15b (second imaging unit) by the first drive unit 16a, and changing the magnification of the re-imaging optical system 20 by the second drive unit 28. When the imaging range control unit 34 performs both changing the magnification of the re-imaging optical system 20 and moving the re-imaging optical system 20 and the second image sensor 15b, it may send a drive signal S3 to the first drive unit 16a and a drive signal S4 to the second drive unit 28, moving the positions of the re-imaging optical system 20 and the second image sensor 15b while simultaneously moving the positions of the lenses 21b and 21c.

[0223] In this case, at least a portion of the period during which the re-imaging optical system 20 and the second image sensor 15b move will overlap with at least a portion of the period during which the lenses 21b and 21c move. However, the entire period during which the re-imaging optical system 20 and the second image sensor 15b move does not necessarily have to overlap completely with the entire period during which the lenses 21b and 21c move.

[0224] Furthermore, the imaging range control unit 34 may change the magnification of the re-imaging optical system 20 so that the second imaging range CA2 of the second image sensor 15b includes the region of interest IA, in accordance with the information signal from the analysis unit 33 (i.e., information regarding the size of the region of interest IA in the wide-area images Im1 and Im3 as detection results).

[0225] Furthermore, the imaging range control unit 34 may, in addition to changing the magnification of the re-imaging optical system 20, and / or based on the above-mentioned detection results by the analysis unit 33 (information regarding the position of region of interest IA in wide-area images Im1 and Im3), move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0 so that the region of interest IA is included in the second imaging range CA2.

[0226] In other words, based on wide-area image data, the imaging range control unit 34 may perform at least one of the following actions: moving the re-imaging optical system 20 and the second image sensor 15b (second imaging unit) by the first drive unit 16a and changing the magnification of the re-imaging optical system 20 by the second drive unit 28, so that the region of interest IA is included in the second imaging range CA2.

[0227] As an example, if the size of the second imaging range CA2 and the size of the region of interest IA differ relatively significantly, and the region of interest IA is far from the second imaging range CA2, the imaging range control unit 34 may, as described above, change the magnification of the re-imaging optical system 20 and move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0 so that the region of interest IA is included in the second imaging range CA2.

[0228] Furthermore, if the positional relationship between the imaging device 1A and the target changes, the analysis unit 33 may detect whether or not there is a change in the size of the region of interest IA in the wide-area images Im1 and Im3 (i.e., a change in the size of at least a portion of the image of the target). For example, the analysis unit 33 may perform the tracking process described above to detect changes in the size of the region of interest IA between multiple wide-area images Im1 and Im3 captured by the first image sensor 15a at different times, and calculate the amount of change in the size of the region of interest IA.

[0229] Furthermore, if the size of the region of interest IA changes, the analysis unit 33 may calculate the amount of change in the size of the region of interest IA in the wide-area images Im1 and Im3 (i.e., the size of the region of interest IA after the positional relationship between the imaging device 1A and the target has changed). The analysis unit 33 transmits the amount of change in the size of the region of interest IA in the wide-area images Im1 and Im3 to the imaging range control unit 34. Note that the calculation of the amount of change in the size of the region of interest IA by the analysis unit 33 can be rephrased as the detection of the amount of change in the size of the region of interest IA by the analysis unit 33.

[0230] Furthermore, the change in the size of the region of interest IA in the wide-area images Im1 and Im3 calculated by the analysis unit 33 can be rephrased as the size of the region of interest IA in the wide-area images Im1 and Im3 after the positional relationship between the imaging device 1A and the target has changed.

[0231] The imaging range control unit 34 may send a drive signal S4 to the drive unit 28 in response to an information signal from the analysis unit 33 (i.e., information regarding the change in the size of the region of interest IA in the wide-area images Im1 and Im3 as a detection result), and change the magnification of the re-imaging optical system 20 so that the region of interest IA does not move outside the second imaging range CA2.

[0232] Furthermore, the imaging range control unit 34 may change the magnification of the re-imaging optical system 20 in accordance with the information signal from the analysis unit 33 (i.e., information regarding the amount of change in the size of the region of interest IA in the wide-area images Im1 and Im3 as detection results) to offset the change in the size of the region of interest IA. For example, if the region of interest IA becomes larger, the imaging range control unit 34 may decrease the magnification of the re-imaging optical system 20. On the other hand, if the region of interest IA becomes smaller, the imaging range control unit 34 may increase the imaging magnification of the re-imaging optical system 20. In this case, even if the distance between the target and the imaging device 1A changes, the imaging device 1A can continue to image the region of interest IA at a roughly constant size relative to the second imaging range CA2.

[0233] Furthermore, the analysis unit 33 may, in addition to calculating the change in the size of the region of interest IA in the wide-area images Im1 and Im3, and / or, similar to the first embodiment, calculate the direction and amount of movement of the region of interest IA in the wide-area images Im1 and Im3. The imaging range control unit 34 may, in addition to changing the magnification of the re-imaging optical system 20, and / or move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0, so that the region of interest IA does not move outside the second imaging range CA2.

[0234] In other words, based on wide-area image data, the imaging range control unit 34 may perform at least one of the following actions: moving the re-imaging optical system 20 and the second image sensor 15b (second imaging unit) by the first drive unit 16a and changing the magnification of the re-imaging optical system 20 by the second drive unit 28, so that the region of interest IA does not move outside the second imaging range CA2.

[0235] Furthermore, the analysis unit 33 may use existing deep learning to predict (calculate) the change in the size of the region of interest IA per predetermined time period in the wide-area images Im1 and Im3, based on the change in the size of the region of interest IA calculated by the tracking process described above, based on wide-area image data captured at different times. The change in the size of the region of interest IA predicted by the analysis unit 33 can be rephrased as the analysis unit 33 detecting the future change in the size of the region of interest IA. The change in the size of the region of interest IA in the wide-area images Im1 and Im3 predicted by the analysis unit 33 can be rephrased as the size of the region of interest IA in the wide-area images Im1 and Im3 in the future.

[0236] The imaging range control unit 34 may change the magnification of the re-imaging optical system 20 so that the region of interest IA does not move outside the second imaging range CA2, in accordance with the information signal from the analysis unit 33 (i.e., predicted information regarding the amount of change in the size of the region of interest IA in the wide-area images Im1 and Im3 as detection results).

[0237] Furthermore, the analysis unit 33 may, in addition to predicting the change in the size of the region of interest IA in the wide-area images Im1 and Im3, and / or predict the direction and amount of movement of the region of interest IA in the wide-area images Im1 and Im3, similar to the first embodiment.The imaging range control unit 34 may, in addition to changing the magnification of the re-imaging optical system 20, and / or move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0, so that the region of interest IA does not move outside the second imaging range CA2.

[0238] Furthermore, the imaging range control unit 34 may change the magnification of the re-imaging optical system 20 in accordance with the information signal from the analysis unit 33 (i.e., predicted information regarding the amount of change in the size of the region of interest IA in the wide-area images Im1 and Im3 as detection results) so that the size of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1A. In other words, the imaging range control unit 34 may change the magnification of the re-imaging optical system 20 based on the wide-area image data so that the size of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1A.

[0239] The analysis unit 33 may predict the direction and amount of movement of the region of interest IA in the wide-area images Im1 and Im3, similar to the first embodiment. The imaging range control unit 34 may move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0 so that the position of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1A, in accordance with the information signal from the analysis unit 33 (i.e., predicted information regarding the direction and amount of movement of the region of interest IA in the wide-area images Im1 and Im3 as a detection result).

[0240] In other words, the imaging range control unit 34 may, based on wide-area image data, move the re-imaging optical system 20 and the second image sensor 15b so that the position of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1A.

[0241] Furthermore, the analysis unit 33 is not limited to wide-area image data, but may also calculate the size of the region of interest IA in the narrow-area images Im2 and Im4 by analyzing narrow-area image data (i.e., image data of narrow-area images Im2 and Im4) using the object detection process described above. The imaging range control unit 34 may change the magnification of the re-imaging optical system 20 according to the information signal from the analysis unit 33 (i.e., information regarding the position of the region of interest IA in the narrow-area images Im2 and Im4 as a detection result).

[0242] Furthermore, the analysis unit 33 may, in addition to calculating the size of the region of interest IA in the narrow-field images Im2 and Im4, and / or, similar to the first embodiment, calculate the position of the region of interest IA in the narrow-field images Im2 and Im4. The imaging range control unit 34 may, in addition to changing the magnification of the re-imaging optical system 20, and / or move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0.

[0243] In other words, the imaging range control unit 34 may, based on the narrow-area image data, perform at least one of the following: moving the re-imaging optical system 20 and the second image sensor 15b by the first drive unit 16a, and changing the magnification of the re-imaging optical system 20 by the second drive unit 28.

[0244] Furthermore, if the positional relationship between the imaging device 1A and the target changes, the analysis unit 33 may use the tracking process described above to calculate the change in the size of the region of interest IA in the narrow-area images Im2 and Im4. The change in the size of the region of interest IA in the wide-area images Im2 and Im4 calculated by the analysis unit 33 can be rephrased as the size of the region of interest IA in the wide-area images Im2 and Im4 after the positional relationship between the imaging device 1A and the target has changed.

[0245] The imaging range control unit 34 may change the magnification of the re-imaging optical system 20 so that the region of interest IA does not move outside the second imaging range CA2, in accordance with the information signal from the analysis unit 33 (i.e., information regarding the amount of change in the size of the region of interest IA in the narrow-area images Im2 and Im4 as detection results).

[0246] Furthermore, the imaging range control unit 34 may change the magnification of the re-imaging optical system 20 in accordance with the information signal from the analysis unit 33 (i.e., information regarding the amount of change in the size of the region of interest IA in the narrow-area images Im2 and Im4 as detection results) to cancel out the change in the size of the region of interest IA. For example, if the region of interest IA becomes larger, the imaging range control unit 34 may decrease the magnification of the re-imaging optical system 20. On the other hand, if the region of interest IA becomes smaller, the imaging range control unit 34 may increase the imaging magnification of the re-imaging optical system 20.

[0247] Furthermore, the analysis unit 33 may, in addition to calculating the change in the size of the region of interest IA in the narrow-field images Im2 and Im4, and / or, similar to the first embodiment, calculate the direction and amount of movement of the region of interest IA in the narrow-field images Im2 and Im4.The imaging range control unit 34 may, in addition to changing the magnification of the re-imaging optical system 20, and / or move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0, so that the region of interest IA does not move outside the second imaging range CA2.

[0248] In other words, based on the narrow-area image data, the imaging range control unit 34 may perform at least one of the following: moving the re-imaging optical system 20 and the second image sensor 15b by the first drive unit 16a and changing the magnification of the re-imaging optical system 20 by the second drive unit 28, so that the region of interest IA does not move outside the second imaging range CA2.

[0249] Furthermore, the analysis unit 33 may use existing deep learning to predict the future change in the size of the region of interest IA per predetermined time period based on the change in the size of the region of interest IA in the narrow-area images Im2 and Im4, calculated by the tracking process described above based on narrow-area image data captured at different times. The change in the size of the region of interest IA in the narrow-area images Im2 and Im4 predicted by the analysis unit 33 can be rephrased as the future size of the region of interest IA in the narrow-area images Im2 and Im4.

[0250] The imaging range control unit 34 may change the magnification of the re-imaging optical system 20 so that the region of interest IA does not move outside the second imaging range CA2, in accordance with the information signal from the analysis unit 33 (i.e., predicted information regarding the amount of change in the size of the region of interest IA in the narrow-area images Im2 and Im4 as detection results).

[0251] Furthermore, the analysis unit 33 may, in addition to predicting the change in the size of the region of interest IA in the narrow-field images Im2 and Im4, and / or predict the direction and amount of movement of the region of interest IA in the narrow-field images Im2 and Im4, similar to the first embodiment.The imaging range control unit 34 may, in addition to changing the magnification of the re-imaging optical system 20, and / or move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0, so that the region of interest IA does not move outside the second imaging range CA2.

[0252] Furthermore, the imaging range control unit 34 may change the magnification of the re-imaging optical system 20 in accordance with the information signal from the analysis unit 33 (i.e., predicted information regarding the amount of change in the size of the region of interest IA in the narrow-area images Im2 and Im4 as detection results) so that the size of the region of interest IA in the second imaging range CA2 does not change substantially before and after the change in the positional relationship between the target and the imaging device 1A.

[0253] In other words, the imaging range control unit 34 may change the magnification of the re-imaging optical system 20 based on the narrow-area image data so that the size of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1A.

[0254] The analysis unit 33 may predict the direction and amount of movement of the region of interest IA in the narrow-area images Im2 and Im4, similar to the first embodiment. The imaging range control unit 34 may move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0 so that the position of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1A, in accordance with the information signal from the analysis unit 33 (i.e., predicted information regarding the direction and amount of movement of the region of interest IA in the wide-area images Im2 and Im4 as a detection result).

[0255] In other words, the imaging range control unit 34 may move the re-imaging optical system 20 and the second image sensor 15b based on the narrow-area image data so that the position of the region of interest IA in the second imaging range CA2 does not change substantially before and after a change in the positional relationship between the target and the imaging device 1A.

[0256] Furthermore, similar to the first embodiment described above, if the region of interest IA cannot be detected on the narrow-area images Im2 and Im4 by the object detection process described above due to changes in the positional relationship between the target and the imaging device 1A, the analysis unit 33 may determine that the region of interest IA has moved outside the second imaging range CA2.

[0257] In this case, the analysis unit 33 may analyze the wide-area image data at that time to re-detect the region of interest IA (i.e., the target image) and calculate at least one of the size and position of the region of interest IA in the wide-area images Im1 and Im3. The imaging range control unit 34 may, in response to the information signal from the analysis unit 33 (i.e., information regarding at least one of the size and position of the region of interest IA in the wide-area images Im1 and Im3), perform at least one of the following: change the magnification of the re-imaging optical system 20 so that the region of interest IA is included in the second imaging range CA2, and move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX0.

[0258] In other words, if the region of interest IA falls outside the second imaging range CA2, the imaging range control unit 34 may, based on the wide-area image data, perform at least one of the following: move the re-imaging optical system 20 and the second image sensor 15b, or change the magnification of the re-imaging optical system 20, so that the region of interest IA is included in the second imaging range CA2.

[0259] Furthermore, if the region of interest IA cannot be detected on the narrow-area images Im2 and Im4, the analysis unit 33 does not need to analyze the wide-area image data at that time. In this case, the imaging range control unit 34 may reduce the magnification of the re-imaging optical system 20. By reducing the magnification of the re-imaging optical system 20, the field of view on the object side corresponding to the second imaging range CA2 is widened, thereby expanding the target search range (i.e., the size of the region of interest IA can be reduced).

[0260] Furthermore, the analysis unit 33 may detect, through the object detection process described above, that the region of interest IA is included in the second imaging range CA2. In other words, if the region of interest IA is outside the second imaging range CA2, the imaging range control unit 34 may reduce the magnification of the re-imaging optical system 20 so that the region of interest IA is included in the second imaging range CA2.

[0261] Furthermore, similar to the first embodiment, if the region of interest IA cannot be detected on the narrow-area images Im2 and Im4, the analysis unit 33 does not need to analyze the wide-area image data at that time. In this case, the imaging range control unit 34 may search for the region of interest IA by moving the re-imaging optical system 20 and the second image sensor 15b in a predetermined path in a direction intersecting the optical axis AX0. The second image sensor 15b may then be stopped at a position where the region of interest IA is included in the second imaging range CA2.

[0262] In other words, if the region of interest IA falls outside the second imaging range CA2, the imaging range control unit 34 may perform at least one of the following: move the re-imaging optical system 20 and the second image sensor 15b by the first drive unit 16a, or change the magnification of the re-imaging optical system 20 by the second drive unit 28, so that the region of interest IA is included in the second imaging range CA2.

[0263] Furthermore, if the analysis unit 33 determines that the target has moved out of the second imaging range CA2, the imaging range control unit 34 may move the re-imaging optical system 20 and the second image sensor 15b while lowering the imaging magnification of the re-imaging optical system 20 in order to re-detect the region of interest IA (i.e., the image of the target).

[0264] Furthermore, when the analysis unit 33 detects the aforementioned predetermined facial expressions or actions of a person or other target included in the wide-area images Im1, Im3 or the narrow-area images Im2, Im4 based on the wide-area image data or narrow-area image data, it may perform one or more of the actions G below, in addition to the actions A to F described in the first embodiment.

[0265] G: A command is issued to the imaging range control unit 34 to decrease the imaging magnification of the re-imaging optical system 20, or to increase the imaging magnification of the re-imaging optical system 20. Furthermore, if any of operations A through G is initiated, the analysis unit 33 may terminate those operations after a predetermined time has elapsed.

[0266] Furthermore, the imaging range control unit 34 may, regardless of the target information analyzed by the analysis unit 33, send a drive signal S4 to the second drive unit 28 based on a signal input from an operation unit (not shown) via the interface unit 36, thereby moving the lenses 21b and 21c. Furthermore, the imaging range control unit 34 may, based on a signal input from an operation unit (not shown) via the interface unit 36, send a drive signal S3 to the first drive unit 16a, thereby moving the imaging optical system 20 and the second image sensor 15b.

[0267] Furthermore, the imaging range control unit 34 may send a drive signal S3 to the first drive unit 16a and a drive signal S4 to the second drive unit 28 based on signals related to the change of the center position of the second imaging range CA2 and the change of the magnification of the imaging optical system 20, which are input from an operation unit (not shown) via the interface unit 36.

[0268] Furthermore, the operator may input at least one of the signals related to changing the center position of the field of view and the signal related to changing the magnification of the imaging magnification 20 to the imaging range control unit 34 via the interface unit 36 ​​by operating an operation unit (not shown).

[0269] In addition, in the imaging device 1A of the second embodiment, the positional relationship between the first image 14a and the first image sensor 15a and the second image 14b, the re-imaging optical system 20 and the second image sensor 15b may be reversed.

[0270] In other words, the first image 14a may be an image formed by a light beam that passes through the divided reflective surface 13a of the light dividing member 13 and propagates roughly along the optical axis AX0. The first image sensor 14a may be positioned at a location that roughly coincides with this first image 14a. In this case, the light beam that passes through the divided reflective surface 13a and propagates roughly along the optical axis AX0 can be called one of the light beams.

[0271] The second image 14b is an image formed by a light beam reflected by the divided reflective surface 13a of the light-dividing member 13 and traveling along the optical axis AX1. This second image 14b may be re-imaged by a re-imaging optical system 20 having an optical axis AX2 that is roughly parallel to the optical axis AX1. The second image sensor 15b may be positioned at a location that roughly coincides with the third image 14c re-imaged by the re-imaging optical system 20. In this case, the light beam reflected by the divided reflective surface 13a of the light-dividing member 13 and traveling along the optical axis AX1 can be called the other light beam.

[0272] In this case, the second imaging unit, including the re-imaging optical system 20 and the second image sensor 15b, may be held in the second holding frame 17a and held together with the second holding frame 17a by the first drive unit 16a so as to be movable in the Y and Z directions, which are directions intersecting the optical axis AX1. Note that the first drive unit 16a is not limited to the Y and Z directions, but may also move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX1.

[0273] The first drive unit 16a may move the re-imaging optical system 20 and the second image sensor 15b in a direction intersecting the optical axis AX1 which is generally parallel to the X direction of the imaging optical system 10 on the second image sensor 15b side (for example, at least one of the Y direction and the Z direction). The first drive unit 16a may also move the re-imaging optical system 20 and the second image sensor 15b in a direction that crosses the optical path of the other light beam. Furthermore, the second drive unit 28 may move the lenses 21b and 21c along the optical axis AX1 which is generally parallel to the X direction.

[0274] Furthermore, the re-imaging optical system 20 is not limited to the variable imaging magnification optical system described above, but may also be an optical system with a fixed imaging magnification. The imaging magnification may be an enlargement magnification, 1:1 magnification, or a reduction magnification. However, as described above, if the imaging magnification of the re-imaging optical system 20 is an enlargement magnification, the third image 14c will be an image in which at least a part of the second image 14b is enlarged, and the resolution of the narrow-field image captured by the second image sensor 15b will be further improved.

[0275] If the imaging magnification of the re-imaging optical system 20 is a reduction magnification, the magnification of the first image 15a formed on the first image sensor 15a will be greater than the magnification of the third image 14c formed on the second image sensor 15b. In this case, a wide-area image may be captured by the re-imaging optical system 20 and the second image sensor 15b, and a narrow-area image may be captured by the first image sensor 15a, or the first image sensor 15a may be moved in a direction intersecting the optical axis AX0. Furthermore, at least one of the imaging optical system 10 or the re-imaging optical system 20 does not need to be telecentric on the side of the second image 14b.

[0276] (Effects of the imaging device in the second embodiment) (4) The image pickup apparatus 1A according to the second embodiment is the image pickup apparatus 1 according to the first embodiment, wherein the second image pickup unit includes a re-imaging optical system 20 that re-images a part of the second image 14b at a predetermined magnification, and an image pickup device (second image pickup device 15b) that picks up an image of a part of the second image (third image 14c) re-formed by the re-imaging optical system 20. The driving unit (first driving unit 16a) moves the re-imaging optical system 20 and the image pickup device (second image pickup device 15b) in a direction crossing the optical path of the other light beam (the light beam traveling along the optical axis AX0). With this configuration, in addition to the effects of the image pickup apparatus 1 according to the first embodiment, the image pickup apparatus 1A can make the narrow-area images (Im2, Im4) picked up by the second image pickup device 15b into higher-resolution images by means of the re-imaging optical system 20.

[0277] (5) The re-imaging optical system 20 may be configured to be capable of changing the magnification. In this case, the object-side field of view of the narrow-area images (Im2, Im4) picked up by the second image pickup device 15b can be optimized.

[0278] (6) The re-imaging optical system 20 may be telecentric on the second image 14b side. In this case, even when the re-imaging optical system 20 moves in a direction intersecting the optical axis AX0 of the imaging optical system 10, the incident angles of the light beams (light beams LB1, LB2, etc.) incident on the re-imaging optical system 20 hardly change. Accordingly, the re-imaging optical system 20 can maintain favorable imaging performance and form a high-resolution third image 14c on the second image pickup device 15b.

[0279] (7) The arrangement pitches PXb and PYb of pixels PEb of the second image pickup device 15b included in the second image pickup unit may be smaller than the arrangement pitches PXa and PYa of pixels PEa of the first image pickup device 15a included in the first image pickup unit. In this case, the narrow-area images (Im2, Im4) picked up by the second image pickup device 15b can be made into even higher-resolution images.

[0280] In (8), the area of the region (first effective area EA1) where pixels PEa of the first image sensor 15a included in the first imaging unit are arranged may be larger than the area of the region (second effective area EA2) where pixels PEb of the second image sensor 15b included in the second imaging unit are arranged. In this case, the object-side field of view of the wide-area images (Im1, Im3) captured by the first image sensor 15a can be made wider.

[0281] It should be noted that the imaging device 1 according to the first embodiment and the imaging device 1A according to the second embodiment described above may be installed at predetermined locations such as nurseries, schools, medical facilities, nursing care facilities, conference rooms, stores, train stations, parks, vending machines, street lamps, telephone poles, prisons, military facilities, borderlines, roads and parks for applications such as monitoring, watching over and searching, for example. When the imaging devices 1, 1A are installed in a nursery, the imaging devices 1, 1A may target, for example, a part of a child's body. When referring to at least one of one or more imaging devices 1 and one or more imaging devices 1A, they are referred to as imaging devices 1, 1A.

[0282] It should be noted that when the imaging devices 1, 1A are installed in a nursing care facility, the imaging devices 1, 1A may target, for example, the face or a part of the body of a person requiring nursing care. It should be noted that when the imaging devices 1, 1A are installed in a conference room, for example, the faces of conference attendees may be targeted. In this case, the present invention can also be used as a video conference system with a remote location.

[0283] It should be noted that when the imaging devices 1, 1A are installed near ticket gates of train stations, near aisles or entrances / exits of stores, at vending machines, street lamps or telephone poles, the imaging devices 1, 1A may target passing persons. The imaging devices 1, 1A may detect features of passing persons based on at least one of narrow-area image data and wide-area image data through the above-described object detection processing or other existing image processing.

[0284] For example, imaging devices 1 and 1A may detect at least one of the following characteristics of a passing person based on at least one of the image data from the narrow-area image data and the wide-area image data: gender, age, body type (height, weight, etc.), race, hairstyle, clothing, speed of movement, direction of movement, etc. In this case, the detected characteristics of the passing person can be used for market research, etc.

[0285] Furthermore, the imaging devices 1 and 1A may detect characteristics of moving objects other than people passing by, for example, vehicles passing by. For example, the imaging devices 1 and 1A may detect at least one of the following characteristics of a passing vehicle: vehicle type, color, speed, and direction of movement. Furthermore, the imaging devices 1 and 1A do not need to be fixedly installed indoors or outdoors, and may be installed on mobile devices such as vehicles, ships, or aircraft (e.g., drones).

[0286] Furthermore, when the imaging devices 1 and 1A are installed on unmanned aerial vehicles such as drones, the target may be an object being searched for (for example, a person, an animal, a vehicle, a ship, an aircraft, etc.). Furthermore, when the imaging devices 1 and 1A are installed on an attack drone (for example), which is an example of an unmanned aerial vehicle, the target may be an object to be attacked (for example, a person, a vehicle, a ship, an aircraft, a building, etc.).

[0287] For example, it is conceivable to mount two cameras (a wide-angle camera and a telephoto camera with a gimbal) on an unmanned aerial vehicle such as a drone, but this is undesirable because, due to limitations on payload weight and size, objects mounted on unmanned aerial vehicles must be small and lightweight. On the other hand, imaging devices 1, 1A, or imaging systems 2, 2A, which can capture both wide-angle and telephoto images with a single unit, are smaller and lighter than two cameras, making them suitable for mounting on unmanned aerial vehicles.

[0288] Furthermore, the objects to be imaged by imaging devices 1 and 1A may be industrial machinery such as machine tools. For example, imaging devices 1 and 1A analyze whether the industrial machinery is operating normally based on at least one of the captured narrow-area image data and wide-area image data. If an abnormality is found in the shape or operation of the industrial machinery, imaging devices 1 and 1A take action such as transmitting the image data of the abnormal part to an external device.

[0289] Furthermore, the imaging devices 1 and 1A may be installed inside the machining chamber of the machine tool (the space where the workpiece is machined with the machining tool). In this case, as an example, the target area may include the machining tool to which the spindle is attached, the workpiece placed on the stage, and at least a portion of the machining points of the workpiece by the machining tool.

[0290] Furthermore, the imaging devices 1 and 1A may detect at least one of the following based on target image data (at least one of narrow-area image data and wide-area image data) generated by existing image processing: the distance between the workpiece and the workpiece, the length of the workpiece (amount of wear), the shape of the workpiece, the breakage (fracture) of the workpiece, the amount of protrusion of the workpiece, the shape of the workpiece, the positional displacement of the workpiece (relative to the reference position of the stage), the shape of the chips, the amount of chips, and the amount of cutting fluid applied to the machined area.

[0291] For example, imaging devices 1 and 1A may be installed in at least one of the following locations: the wall of the processing room, the ceiling of the processing room, the spindle head, and the stage. For example, imaging devices 1 and 1A may be installed on the spindle.

[0292] The imaging devices 1 and 1A may be configured to be detachably attached to the spindle via a shank. In this case, the imaging device 1 or 1A attached to the spindle may be replaced with any workpiece by a tool changing device. Alternatively, the imaging devices 1 and 1A may be stored in the tool changing device, and the workpiece attached to the spindle may be replaced with the imaging devices 1 and 1A.

[0293] Furthermore, imaging devices 1 and 1A may be installed in two or more locations among the walls of the processing room, the ceiling of the processing room, the spindle head, and the stage. For example, they may be installed in different locations on the walls of the processing room, or they may be installed in one or more locations on the walls of the processing room and one or more locations on the ceiling of the processing room. When multiple imaging devices 1 and 1A are installed in this manner, they may be installed so that targets in the processing room can be imaged from different directions (from different viewpoints). In addition, when multiple imaging devices 1 and 1A are installed, they may be installed so that their respective maximum field of view areas partially overlap.

[0294] Furthermore, the imaging devices 1 and 1A may be installed in a part of the machine tool other than the machining chamber. For example, they may be installed inside a tool changing device that exchanges a machining tool attached to the spindle in the machining chamber for a different type of machining tool. The tool changing device may, for example, be an automatic tool changer (ATC).

[0295] Furthermore, the imaging devices 1 and 1A may be installed inside an optical processing apparatus that processes a workpiece with processing light. In this case, the imaging devices 1 and 1A may detect the processing area on the workpiece (the area irradiated with processing light) as a target. If the optical processing apparatus is a three-dimensional additive manufacturing apparatus, the imaging devices 1 and 1A may detect the molding material as a target based on at least one of the image data of the narrow-area image data and the wide-area image data, and detect the supply status of the molding material using existing image processing.

[0296] (Light receiving device of the third embodiment) Each of the imaging devices 1 and 1A in the above-described embodiments and modifications can also be described as a light-receiving device comprising an imaging unit (image sensor 15a) that captures an image formed by one of the light beams divided by the light-dividing member 13, and a light-receiving unit (second imaging unit including a second image sensor 15b) having a light-incident unit into which a portion of the other light beam divided by the light-dividing member 13 is incident.

[0297] (Effects of the light receiving device according to the third embodiment) (9) The light receiving devices 1, 1A according to the third embodiment comprise: an imaging optical system 10; a light splitting member 13 that amplitude-splits light that has passed through at least a part of the imaging optical system 10; an imaging unit 15a that captures an image 14a formed by one light beam split by the light splitting member 13; a light receiving unit (a second imaging unit including a second image pickup device 15b) having a light incident portion 29 on which a part of the other light beam split by the light splitting member 13 is incident; and a driving unit 16 that moves the light incident portion 29 in a direction crossing the optical path of the other light beam. With this configuration, in the light receiving devices 1, 1A, similar to the image pickup apparatuses 1, 1A according to each of the embodiments and modifications described above, the driving unit 16 moves the light incident portion 29, whereby the range of the second image 14b received by the light receiving unit can be moved at high speed. Accordingly, the field of view on the object side (target side) received by the light receiving unit can also be moved at high speed.

[0298] Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Furthermore, each embodiment and modification may be applied independently or may be used in combination. Other aspects conceivable within the scope of the technical idea of the present invention are also included within the scope of the present invention.

[0299] (Supplementary Note) It is understood by those skilled in the art that the plurality of embodiments or modifications described above are specific examples of the following aspects.

[0300] (Clause 1) An image pickup apparatus comprising: an imaging optical system; a light splitting member that splits light having passed through at least a part of the imaging optical system; a first imaging unit that captures a first image formed by one light beam split by the light splitting member; a second imaging unit that captures a part of a second image formed by the other light beam split by the light splitting member; and a driving unit that moves the second imaging unit in a direction crossing an optical path of the other light beam.

[0301] (Clause 2) The imaging device described in paragraph 1, wherein the first imaging unit includes a first image sensor for capturing the first image, the second imaging unit includes a second image sensor for capturing an image of a part of the second image, the pixel array pitch of the second image sensor is smaller than the pixel array pitch of the first image sensor, and the drive unit moves the second image sensor in a direction that crosses the optical path of the other light beam.

[0302] (Section 3) An imaging device according to paragraph 2, wherein the area of ​​the region in which the pixels of the first image sensor are arranged is larger than the area of ​​the region in which the pixels of the second image sensor are arranged.

[0303] (Section 4) The imaging device described in paragraph 1, wherein the second imaging unit includes a re-imaging optical system that re-images a portion of the second image at a predetermined magnification, and an image sensor that captures an image of the portion of the second image re-imaged by the re-imaging optical system, and the drive unit moves the re-imaging optical system and the image sensor in a direction that crosses the optical path of the other light beam.

[0304] (Section 5) In the imaging apparatus described in paragraph 4, the re-imaging optical system is an imaging apparatus capable of changing the magnification ratio. (Section 6) An imaging apparatus according to paragraph 4 or 5, wherein the imaging optical system is telecentric on the side of the second image.

[0305] (Section 7) The imaging apparatus described in paragraph 6, wherein the re-imaging optical system re-imaging a portion of the second image formed in the image-forming region by the imaging optical system, and the difference between the angle between the principal ray of the other light beam directed from the light-dividing member to a first location in the image-forming region and the optical axis of the imaging optical system on the second imaging unit side, and the angle between the principal ray of the other light beam directed from the light-dividing member to a second location in the image-forming region at a different distance from the optical axis than the first location and the optical axis, is within 1°.

[0306] (Section 8) An imaging apparatus according to any one of paragraphs 4 to 7, wherein the re-imaging optical system is telecentric on the imaging optical system side.

[0307] (Section 9) The imaging apparatus described in paragraph 8, wherein the re-imaging optical system re-imagings a portion of the second image formed in the image-forming region by the imaging optical system, and the difference between the angle between the principal ray of a light beam directed from a first location in the image-forming region toward the re-imaging optical system and the optical axis of the re-imaging optical system, and the angle between the principal ray of a light beam directed toward the re-imaging optical system from a second location in the image-forming region, which is at a different distance from the optical axis of the re-imaging optical system than the first location, and the optical axis of the re-imaging optical system, is within 1°.

[0308] (Section 10) An imaging apparatus according to any one of paragraphs 4 to 9, wherein the difference between the angle between the principal ray of the other light beam from the light splitting member and the optical axis of the imaging optical system on the second imaging unit side, and the angle between the principal ray of the light beam incident on the re-imaging optical system and the optical axis of the re-imaging optical system is within 1°.

[0309] (Section 11) An imaging device according to any one of paragraphs 4 to 10, wherein the re-imaging optical system includes a plurality of optical members, and when the drive unit is a first drive unit, it further includes a second drive unit that moves at least one of the plurality of optical members along the optical axis of the re-imaging optical system, and the re-imaging optical system changes the magnification by moving at least one of the plurality of optical members by the second drive unit.

[0310] (Section 12) An imaging apparatus according to any one of paragraphs 4 to 11, comprising a holding unit for holding the re-imaging optical system and the image sensor, wherein the drive unit moves the holding unit in a direction that crosses the optical path of the other light beam.

[0311] (Section 13) An imaging device according to any one of paragraphs 4 to 12, wherein the first imaging unit includes a first image sensor for capturing a first image, and the image sensor included in the second imaging unit is a second image sensor, wherein the pixel array pitch of the second image sensor is smaller than the pixel array pitch of the first image sensor.

[0312] (Section 14) An imaging device according to any one of paragraphs 4 to 13, wherein the first imaging unit includes a first image sensor for capturing a first image, and the image sensor included in the second imaging unit is a second image sensor, wherein the area of ​​the region in which the pixels of the first image sensor are arranged is larger than the area of ​​the region in which the pixels of the second image sensor are arranged.

[0313] (Section 15) An imaging device according to paragraph 2 or 3, comprising a holding portion for holding the second image sensor, wherein the drive portion moves the holding portion in a direction that crosses the optical path of the other light beam. (Section 16) An imaging device according to any one of paragraphs 1 to 15, wherein the maximum field of view of the imaging optical system is 170° or more.

[0314] (Section 17) An imaging apparatus according to any one of paragraphs 1 to 16, wherein the light splitting member is an imaging apparatus that amplitude-splits light that has passed through at least a part of the imaging optical system. (Section 18) In the imaging apparatus described in paragraph 17, the light splitting member is an imaging apparatus that amplitude-splits light that has passed through the imaging optical system.

[0315] (Section 19) An imaging device according to any one of paragraphs 1 to 16, wherein the light splitting member is an imaging device that splits light that has passed through the imaging optical system.

[0316] (Section 20) An imaging apparatus according to paragraph 5, any one of paragraphs 6 to 14 dependent on paragraph 5, or any one of paragraphs 16 to 19 dependent on paragraph 5, further comprising an imaging range control unit that performs at least one of the movement of the re-imaging optical system and the image sensor by the drive unit and the change of the magnification ratio based on image data of the first image generated by the first imaging unit.

[0317] (Section 21) The imaging apparatus described in paragraph 20, wherein at least light from a target is incident on the imaging optical system, and the imaging range control unit performs at least one of moving the re-imaging optical system and the image sensor and changing the magnification ratio by the drive unit, based on the image data of the first image, such that at least a portion of the image of the target included in the second image is included in the imaging range of the second image by the second imaging unit.

[0318] (Section 22) In the imaging apparatus described in paragraph 21, when the positional relationship between the target and the imaging apparatus changes, the imaging range control unit performs at least one of the following actions based on the image data of the first image: moving the re-imaging optical system and the image sensor by the drive unit and changing the magnification ratio, so that at least a portion of the image of the target falls out of the imaging range of the second image taken by the second imaging unit.

[0319] (Section 23) In the imaging apparatus described in paragraph 22, when the positional relationship between the target and the imaging apparatus changes, the imaging range control unit performs movement of the re-imaging optical system and the image sensor by the drive unit based on the image data of the first image, so that the position of at least a portion of the image of the target in the imaging range of the second image taken by the second imaging unit does not change before and after the change in positional relationship.

[0320] (Section 24) In the imaging apparatus described in paragraph 22 or 23, if the positional relationship between the target and the imaging apparatus changes, the imaging range control unit performs a change in the magnification ratio based on the image data of the first image such that the size of at least a portion of the image of the target captured by the second imaging unit remains unchanged before and after the change in positional relationship.

[0321] (Section 25) An imaging apparatus according to any one of paragraphs 21 to 24, further comprising an analysis unit that detects at least a portion of the image of the target included in the image of the first image based on the image data of the first image, wherein the imaging range control unit performs at least one of moving the re-imaging optical system and the image sensor and changing the magnification based on the detection result of the at least portion by the analysis unit.

[0322] (Section 26) The imaging apparatus described in paragraph 25, wherein the analysis unit detects the position of at least a portion of the image of the target on the image of the first image based on the image data of the first image, and the imaging range control unit performs movement of the re-imaging optical system and the image sensor by the drive unit based on the position of at least a portion as the result of the analysis unit's detection.

[0323] (Section 27) An imaging device according to paragraph 25 or 26, wherein the analysis unit detects the size of at least a portion of the image of the target on the image of the first image based on the image data of the first image, and the imaging range control unit performs a change in the magnification ratio based on the size of the at least portion as the detection result by the analysis unit.

[0324] (Section 28) An imaging apparatus according to any one of paragraphs 20 to 27, wherein the imaging range control unit performs the movement of the re-imaging optical system and the image sensor and the change of the magnification ratio by the drive unit based on the image data of the first image.

[0325] (Section 29) An imaging apparatus according to any one of paragraphs 20 to 28, wherein the imaging range control unit performs at least one of the following actions based on image data of at least a portion of the second image generated by the second imaging unit: moving the re-imaging optical system and the image sensor by the drive unit and changing the magnification ratio.

[0326] (Section 30) In the imaging apparatus described in paragraph 29, at least light from a target is incident on the imaging optical system, and when the positional relationship between the target and the imaging apparatus changes, the imaging range control unit performs at least one of the following actions: moving the re-imaging optical system and the image sensor by the drive unit and changing the magnification ratio, based on image data of at least a portion of the second image, so that at least a portion of the image of the target included in the second image falls out of the imaging range of the second image by the second imaging unit.

[0327] (Section 31) In the imaging apparatus described in paragraph 29 or 30, if the positional relationship between the target and the imaging apparatus changes, the imaging range control unit performs movement of the re-imaging optical system and the image sensor by the drive unit based on image data of at least a portion of the second image, so that the position of at least a portion of the image of the target in the imaging range of the second image by the second imaging unit does not change before and after the change in positional relationship.

[0328] (Section 32) In an imaging device according to any one of paragraphs 29 to 31, if the positional relationship between the target and the imaging device changes, the imaging range control unit performs a change in the magnification ratio based on image data of at least a portion of the second image, such that the size of at least a portion of the image of the target captured by the second imaging unit remains unchanged before and after the change in positional relationship.

[0329] (Section 33) In the imaging apparatus described in paragraph 29, if the positional relationship between the target and the imaging apparatus changes and at least a portion of the image of the target falls outside the imaging range of the second image taken by the second imaging unit, the imaging range control unit performs at least one of moving the re-imaging optical system and the image sensor by the drive unit and changing the magnification ratio so that at least a portion is included in the imaging range.

[0330] (Section 34) In the imaging apparatus described in paragraph 29 or 33, if the positional relationship between the target and the imaging apparatus changes and at least a portion of the image of the target falls outside the imaging range of the second image taken by the second imaging unit, the imaging range control unit reduces the magnification ratio so that at least a portion is included in the imaging range.

[0331] (Section 35) In the imaging apparatus described in paragraph 29, if the positional relationship between the target and the imaging apparatus changes and at least a portion of the image of the target falls outside the imaging range of the second image taken by the second imaging unit, the imaging range control unit performs at least one of the following actions based on the image data of the first image: moving the re-imaging optical system and the image sensor by the drive unit and changing the magnification ratio, so that at least a portion of the target is included in the imaging range.

[0332] (Section 36) An imaging apparatus according to any one of paragraphs 30 to 35, further comprising an analysis unit that detects at least a portion of the image of the target included in at least a portion of the image of the second image based on image data of at least a portion of the second image.

[0333] (Section 37) An imaging apparatus according to paragraph 36, which is dependent on any one of paragraphs 30 to 32, wherein the imaging range control unit performs at least one of moving the re-imaging optical system and the image sensor by the drive unit and changing the magnification ratio, based on the detection result of at least a portion of the image of the target by the analysis unit, so that at least a portion of the image of the target falls out of the imaging range of the second image by the second imaging unit.

[0334] (Section 38) The imaging apparatus described in paragraph 37, wherein the analysis unit detects the position of at least a portion of the target image on the image of at least a portion of the second image based on image data of at least a portion of the second image, and the imaging range control unit moves the re-imaging optical system and the image sensor by the drive unit so that at least a portion of the target image does not fall outside the imaging range of the second image by the second imaging unit, based on the position of at least a portion of the target image on the image of at least a portion of the second image as the detection result by the analysis unit.

[0335] (Section 39) An imaging device according to paragraph 37 or 38, wherein the analysis unit detects the size of at least a portion of the image of the target on the image of at least a portion of the second image based on image data of at least a portion of the second image, and the imaging range control unit changes the magnification based on the size of at least a portion of the image of the target on the image of at least a portion of the second image as the detection result by the analysis unit, so that at least a portion of the image of the target falls outside the imaging range of the second image by the second imaging unit.

[0336] (Section 40) An imaging apparatus according to any one of paragraphs 1 to 19, further comprising an imaging range control unit that performs movement of the second imaging unit by the drive unit based on image data of the first image generated by the first imaging unit.

[0337] (Section 41) The imaging apparatus described in paragraph 40, wherein at least light from a target is incident on the imaging optical system, and the imaging range control unit moves the second imaging unit by the drive unit based on the image data of the first image, such that at least a portion of the image of the target included in the second image is included in the imaging range of the second image by the second imaging unit.

[0338] (Section 42) In the imaging apparatus described in paragraph 40, at least light from a target is incident on the imaging optical system, and when the positional relationship between the target and the imaging apparatus changes, the imaging range control unit performs movement of the second imaging unit by the drive unit based on the image data of the first image so that at least a portion of the image of the target falls out of the imaging range of the second image taken by the second imaging unit.

[0339] (Section 43) In the imaging apparatus described in paragraph 41 or 42, if the positional relationship between the target and the imaging apparatus changes, the imaging range control unit performs movement of the second imaging unit by the drive unit based on the image data of the first image so that the position of at least a portion of the image of the target in the imaging range remains unchanged before and after the change in positional relationship.

[0340] (Section 44) An imaging device according to any one of paragraphs 41 to 43, further comprising an analysis unit that detects at least a portion of the image of the target included in the image of the first image based on the image data of the first image, wherein the imaging range control unit executes the movement of the second imaging unit by the drive unit based on the detection result of at least a portion of the image of the target by the analysis unit.

[0341] (Section 45) The imaging device described in paragraph 44, wherein the analysis unit detects the position of at least a portion of the image of the target on the image of the first image based on the image data of the first image, and the imaging range control unit executes the movement of the second imaging unit by the drive unit based on the position of at least a portion as the result of the analysis unit's detection.

[0342] (Section 46) An imaging device according to any one of paragraphs 40 to 45, wherein the imaging range control unit performs movement of the second imaging unit by the drive unit based on image data of at least a portion of the second image generated by the second imaging unit.

[0343] (Section 47) In the imaging apparatus described in paragraph 46, at least light from a target is incident on the imaging optical system, and when the positional relationship between the target and the imaging apparatus changes, the imaging range control unit performs movement of the second imaging unit by the drive unit based on image data of at least a portion of the second image, so that at least a portion of the image of the target included in the second image falls out of the imaging range of the second image by the second imaging unit.

[0344] (Section 48) In the imaging apparatus described in paragraph 46 or 47, at least light from a target is incident on the imaging optical system, and when the positional relationship between the target and the imaging apparatus changes, the imaging range control unit performs movement of the second imaging unit by the drive unit based on image data of at least a portion of the second image, such that the position of at least a portion of the image of the target in the imaging range of the second image by the second imaging unit remains unchanged before and after the change in positional relationship.

[0345] (Section 49) In the imaging apparatus described in paragraph 46, at least light from a target is incident on the imaging optical system, and when the positional relationship between the target and the imaging apparatus changes and at least a portion of the image of the target falls outside the imaging range of the second image taken by the second imaging unit, the imaging range control unit performs movement of the second imaging unit by the drive unit so that at least a portion is included in the imaging range.

[0346] (Section 50) In the imaging apparatus described in paragraph 46, at least light from a target is incident on the imaging optical system, and when the positional relationship between the target and the imaging apparatus changes and at least a portion of the image of the target falls outside the imaging range of the second image taken by the second imaging unit, the imaging range control unit performs movement of the second imaging unit by the drive unit based on the image data of the first image so that at least a portion is included in the imaging range.

[0347] (Section 51) An imaging apparatus according to any one of paragraphs 46 to 50, wherein at least light from a target is incident on the imaging optical system, and the imaging apparatus further comprises an analysis unit that detects at least a portion of the image of the target included in the image of at least a portion of the second image based on image data of at least a portion of the second image.

[0348] (Section 52) An imaging apparatus according to paragraph 51, which is dependent on paragraph 47 or 48, wherein the imaging range control unit performs movement of the second imaging unit by the drive unit so that at least a portion of the image of the target falls outside the imaging range, based on the detection result of at least a portion of the image of the target by the analysis unit.

[0349] (Section 53) The imaging apparatus described in paragraph 52, wherein the analysis unit detects the position of at least a portion of the image of the target on the image of at least a portion of the second image based on image data of at least a portion of the second image, and the imaging range control unit performs movement by the drive unit so that at least a portion of the image of the target does not fall outside the imaging range, based on the position of at least a portion as a result of the analysis unit's detection.

[0350] (Section 54) An imaging apparatus according to any one of paragraphs 1 to 53, further comprising an imaging recording control unit that controls the start and end of recording of the image data of the first image and the image data of at least one of the image data of at least a portion of the second image, based on the image data of the first image generated by the first imaging unit and the image data of at least a portion of the second image generated by the second imaging unit.

[0351] (Section 55) The imaging apparatus described in paragraph 54, wherein at least light from a target is incident on the imaging optical system, and the imaging apparatus further comprises an analysis unit that detects at least one of at least a portion of the image of the target included in the image of the first image and at least one of at least a portion of the image of the target included in the image of at least a portion of the second image, and the imaging recording control unit controls the start and end of recording of the image data of at least one of the image data of the first image and at least a portion of the image data of at least a portion of the second image based on the detection result of the analysis unit.

[0352] (Section 56) An imaging device comprising: an imaging optical system having an optical splitting member for amplitude splitting of incident light; a first imaging unit for capturing a first image formed by one of the light beams split by the optical splitting member; a second imaging unit for capturing a portion of a second image formed by the other light beam split by the optical splitting member; and a drive unit for moving the second imaging unit in a direction that crosses the optical path of the other light beam.

[0353] (Section 57) The imaging apparatus described in paragraph 56, wherein the first imaging unit includes a first image sensor for capturing the first image, the second imaging unit includes a second image sensor for capturing an image of a part of the second image, the pixel array pitch of the second image sensor is smaller than the pixel array pitch of the first image sensor, and the drive unit moves the second image sensor in a direction that crosses the optical path of the other light beam.

[0354] (Section 58) An imaging device as described in paragraph 57, wherein the area of ​​the region in which the pixels of the first image sensor are arranged is larger than the area of ​​the region in which the pixels of the second image sensor are arranged.

[0355] (Section 59) The imaging device described in paragraph 56, wherein the second imaging unit includes a re-imaging optical system that re-images a portion of the second image at a predetermined magnification, and an image sensor that captures an image of the portion of the second image re-imaged by the re-imaging optical system, and the drive unit moves the re-imaging optical system and the image sensor in a direction that crosses the optical path of the other light beam.

[0356] (Section 60) In the imaging apparatus described in paragraph 59, the re-imaging optical system is an imaging apparatus capable of changing the magnification ratio. (Section 61) An imaging apparatus according to paragraph 59 or 60, wherein the imaging optical system is telecentric on the side of the second image.

[0357] (Section 62) The imaging apparatus described in paragraph 61, wherein the re-imaging optical system re-imaging a portion of the second image formed in the image-forming region by the imaging optical system, and the difference between the angle between the principal ray of the other light beam directed from the light-dividing member to a first location in the image-forming region and the optical axis of the imaging optical system on the second imaging unit side, and the angle between the principal ray of the other light beam directed from the light-dividing member to a second location in the image-forming region at a different distance from the optical axis than the first location and the optical axis, is within 1°.

[0358] (Section 63) An imaging apparatus according to any one of paragraphs 59 to 62, wherein the re-imaging optical system is telecentric on the imaging optical system side.

[0359] (Section 64) The imaging apparatus described in paragraph 63, wherein the re-imaging optical system re-imagings a portion of the second image formed in the image-forming region by the imaging optical system, and the difference between the angle between the principal ray of a light beam directed from a first location in the image-forming region toward the re-imaging optical system and the optical axis of the re-imaging optical system, and the angle between the principal ray of a light beam directed toward the re-imaging optical system from a second location in the image-forming region, which is at a different distance from the optical axis of the re-imaging optical system than the first location, and the optical axis of the re-imaging optical system, is within 1°.

[0360] (Section 65) An imaging apparatus according to any one of paragraphs 59 to 64, wherein the difference between the angle between the principal ray of the other light beam from the light splitting member and the optical axis of the imaging optical system on the second imaging unit side, and the angle between the principal ray of the light beam incident on the re-imaging optical system and the optical axis of the re-imaging optical system is within 1°.

[0361] (Section 66) An imaging device according to any one of paragraphs 59 to 65, wherein the re-imaging optical system includes a plurality of optical members, and when the drive unit is a first drive unit, it further includes a second drive unit that moves at least one of the plurality of optical members along the optical axis of the re-imaging optical system, and the re-imaging optical system changes the magnification by moving at least one of the plurality of optical members by the second drive unit.

[0362] (Section 67) An imaging apparatus according to any one of paragraphs 59 to 66, comprising a holding unit for holding the re-imaging optical system and the image sensor, wherein the drive unit moves the holding unit in a direction that crosses the optical path of the other light beam.

[0363] (Section 68) An imaging device according to any one of paragraphs 59 to 67, wherein the first imaging unit includes a first image sensor for capturing a first image, and the image sensor included in the second imaging unit is a second image sensor, wherein the pixel array pitch of the second image sensor is smaller than the pixel array pitch of the first image sensor.

[0364] (Section 69) An imaging device according to any one of paragraphs 59 to 68, wherein the first imaging unit includes a first image sensor for capturing a first image, and the image sensor included in the second imaging unit is a second image sensor, wherein the area of ​​the region in which the pixels of the first image sensor are arranged is larger than the area of ​​the region in which the pixels of the second image sensor are arranged.

[0365] (Section 70) An imaging device according to paragraph 57 or 58, comprising a holding portion for holding the second image sensor, wherein the drive portion moves the holding portion in a direction that crosses the optical path of the other light beam.

[0366] (Section 71) An imaging device according to any one of paragraphs 56 to 70, wherein the maximum field of view of the imaging optical system is 170° or more. (Section 72) An imaging device according to any one of paragraphs 56 to 71, wherein the light splitting member is an imaging device that amplitude-splits incident light.

[0367] (Section 73) The imaging apparatus according to paragraph 72, wherein the imaging optical system further comprises at least one lens member, and the light splitting member amplitude splits the light that has passed through the at least one lens member.

[0368] (Section 74) An imaging apparatus according to any one of paragraphs 56 to 71, wherein the imaging optical system further comprises at least one lens member, and the light-splitting member splits the light that has passed through the at least one lens member.

[0369] (Section 75) An imaging apparatus according to paragraph 60, any one of paragraphs 61 to 69 dependent on paragraph 60, or any one of paragraphs 71 to 74 dependent on paragraph 5, further comprising an imaging range control unit that performs at least one of the following actions: moving the re-imaging optical system and the image sensor by the drive unit and changing the magnification ratio, based on image data of the first image generated by the first imaging unit.

[0370] (Section 76) The imaging apparatus described in paragraph 75 further comprises an analysis unit that detects at least a portion of the image of the target included in the image of the first image based on the image data of the first image, and the imaging range control unit performs at least one of moving the re-imaging optical system and the image sensor and changing the magnification based on the detection result of the at least portion by the analysis unit.

[0371] (Section 77) The imaging apparatus described in paragraph 76, wherein the analysis unit detects the position of at least a portion of the image of the target on the image of the first image based on the image data of the first image, and the imaging range control unit performs movement of the re-imaging optical system and the image sensor by the drive unit based on the position of at least a portion as the result of the analysis unit's detection.

[0372] (Section 78) An imaging apparatus according to paragraph 76 or 77, wherein the analysis unit detects the size of at least a portion of the image of the target on the image of the first image based on the image data of the first image, and the imaging range control unit performs a change in the magnification ratio based on the size of the at least portion as the detection result by the analysis unit.

[0373] (Section 79) An imaging apparatus according to any one of paragraphs 75 to 78, wherein the imaging range control unit performs the movement of the re-imaging optical system and the image sensor and the change of the magnification ratio by the drive unit based on the image data of the first image.

[0374] (Section 80) An imaging apparatus according to any one of paragraphs 56 to 74, further comprising an imaging range control unit that performs movement of the second imaging unit by the drive unit based on image data of the first image generated by the first imaging unit.

[0375] (Section 81) The imaging apparatus described in paragraph 80 further comprises an analysis unit that detects at least a portion of the image of the target included in the image of the first image based on the image data of the first image, and the imaging range control unit executes the movement of the second imaging unit by the drive unit based on the detection result of at least a portion of the image of the target by the analysis unit.

[0376] (Section 82) The imaging device described in paragraph 81, wherein the analysis unit detects the position of at least a portion of the image of the target on the image of the first image based on the image data of the first image, and the imaging range control unit performs movement of the second imaging unit by the drive unit based on the position of at least a portion as the result of the analysis unit's detection.

[0377] (Section 83) An imaging apparatus according to any one of paragraphs 56 to 82, further comprising an imaging recording control unit that controls the start and end of recording of the image data of the first image and the image data of at least one of the image data of at least a portion of the second image, based on the image data of the first image generated by the first imaging unit and the image data of at least a portion of the second image generated by the second imaging unit.

[0378] (Section 84) An imaging device capable of imaging an arbitrary range, comprising: an imaging optical system; an optical splitting member for splitting light that has passed through at least a part of the imaging optical system; a first imaging unit for imaging a first image of the arbitrary range formed by one of the optical beams split by the optical splitting member; a second imaging unit for imaging a portion of a second image of the arbitrary range formed by the other optical beam split by the optical splitting member; and a drive unit for moving the second imaging unit in a direction that crosses the optical path of the other optical beam.

[0379] (Section 85) The imaging apparatus described in paragraph 84, wherein the first imaging unit includes a first image sensor for capturing a first image, the second imaging unit includes a second image sensor for capturing an image of a part of the second image, the pixel array pitch of the second image sensor is smaller than the pixel array pitch of the first image sensor, and the drive unit moves the second image sensor in a direction that crosses the optical path of the other light beam.

[0380] (Section 86) An imaging device as described in paragraph 85, wherein the area of ​​the region in which the pixels of the first image sensor are arranged is larger than the area of ​​the region in which the pixels of the second image sensor are arranged.

[0381] (Section 87) The imaging device described in paragraph 84, wherein the second imaging unit includes a re-imaging optical system that re-images a portion of the second image at a predetermined magnification, and an image sensor that captures an image of the portion of the second image re-imaged by the re-imaging optical system, and the drive unit moves the re-imaging optical system and the image sensor in a direction that crosses the optical path of the other light beam.

[0382] (Section 88) In the imaging apparatus described in paragraph 87, the re-imaging optical system is an imaging apparatus capable of changing the magnification ratio. (Section 89) An imaging apparatus according to paragraph 87 or 88, wherein the imaging optical system is telecentric on the side of the second image.

[0383] (Section 90) The imaging apparatus described in paragraph 89, wherein the re-imaging optical system re-imaging a portion of the second image formed in the image-forming region by the imaging optical system, and the difference between the angle between the principal ray of the other light beam directed from the light-dividing member to a first location in the image-forming region and the optical axis of the imaging optical system on the second imaging unit side, and the angle between the principal ray of the other light beam directed from the light-dividing member to a second location in the image-forming region at a different distance from the optical axis than the first location and the optical axis, is within 1°.

[0384] (Section 91) An imaging apparatus according to any one of paragraphs 87 to 90, wherein the re-imaging optical system is telecentric on the imaging optical system side.

[0385] (Section 92) The imaging apparatus described in paragraph 91, wherein the re-imaging optical system re-imagings a portion of the second image formed in the image-forming region by the imaging optical system, and the difference between the angle between the principal ray of a light beam directed from a first location in the image-forming region toward the re-imaging optical system and the optical axis of the re-imaging optical system, and the angle between the principal ray of a light beam directed toward the re-imaging optical system from a second location in the image-forming region, which is at a different distance from the optical axis of the re-imaging optical system than the first location, and the optical axis of the re-imaging optical system, is within 1°.

[0386] (Section 93) An imaging apparatus according to any one of paragraphs 87 to 92, wherein the difference between the angle between the principal ray of the other light beam from the light splitting member and the optical axis of the imaging optical system on the second imaging unit side, and the angle between the principal ray of the light beam incident on the re-imaging optical system and the optical axis of the re-imaging optical system is within 1°.

[0387] (Section 94) An imaging device according to any one of paragraphs 87 to 93, wherein the re-imaging optical system includes a plurality of optical members, and when the drive unit is a first drive unit, it further includes a second drive unit that moves at least one of the plurality of optical members along the optical axis of the re-imaging optical system, and the re-imaging optical system changes the magnification by moving at least one of the plurality of optical members by the second drive unit.

[0388] (Section 95) An imaging apparatus according to any one of paragraphs 87 to 94, comprising a holding unit for holding the re-imaging optical system and the image sensor, wherein the drive unit moves the holding unit in a direction that crosses the optical path of the other light beam.

[0389] (Section 96) An imaging apparatus according to any one of paragraphs 87 to 95, wherein the first imaging unit includes a first image sensor for capturing a first image, and the image sensor included in the second imaging unit is a second image sensor, wherein the pixel array pitch of the second image sensor is smaller than the pixel array pitch of the first image sensor.

[0390] (Section 97) An imaging apparatus according to any one of paragraphs 87 to 96, wherein the first imaging unit includes a first image sensor for capturing a first image, and the image sensor included in the second imaging unit is a second image sensor, the area of ​​the region in which the pixels of the first image sensor are arranged is larger than the area of ​​the region in which the pixels of the second image sensor are arranged.

[0391] (Section 98) An imaging device according to paragraph 85 or 86, comprising a holding portion for holding the second image sensor, wherein the drive portion moves the holding portion in a direction that crosses the optical path of the other light beam.

[0392] (Section 99) An imaging device according to any one of paragraphs 84 to 98, wherein the maximum field of view of the imaging optical system is 170° or more. (Section 100) An imaging apparatus according to any one of paragraphs 84 to 99, wherein the light splitting member is an imaging apparatus that amplitude-splits light that has passed through at least a part of the imaging optical system.

[0393] (Section 101) In the imaging apparatus described in paragraph 100, the light splitting member is an imaging apparatus that amplitude-splits light that has passed through the imaging optical system. (Section 102) An imaging apparatus according to any one of paragraphs 87 to 99, wherein the light splitting member is an imaging apparatus that splits light that has passed through the imaging optical system.

[0394] (Section 103) An imaging apparatus according to paragraph 88, any one of paragraphs 89 to 97 dependent on paragraph 88, or any one of paragraphs 99 to 102 dependent on paragraph 88, further comprising an imaging range control unit that performs at least one of the movement of the re-imaging optical system and the image sensor by the drive unit and the change of the magnification ratio based on image data of the first image generated by the first imaging unit.

[0395] (Section 104) The imaging apparatus described in paragraph 103 further comprises an analysis unit that detects at least a portion of the image of the target included in the image of the first image based on the image data of the first image, and the imaging range control unit performs at least one of moving the re-imaging optical system and the image sensor by the drive unit and changing the magnification ratio based on the detection result of the at least portion by the analysis unit.

[0396] (Section 105) The imaging apparatus described in paragraph 104, wherein the analysis unit detects the position of at least a portion of the image of the target on the image of the first image based on the image data of the first image, and the imaging range control unit performs movement of the re-imaging optical system and the image sensor by the drive unit based on the position of at least a portion as the result of the analysis unit's detection.

[0397] (Section 106) An imaging apparatus according to paragraph 104 or 105, wherein the analysis unit detects the size of at least a portion of the image of the target on the image of the first image based on the image data of the first image, and the imaging range control unit performs a change in the magnification ratio based on the size of the at least portion as the result of the analysis unit's detection.

[0398] (Section 107) An imaging apparatus according to any one of paragraphs 103 to 106, wherein the imaging range control unit performs the movement of the re-imaging optical system and the image sensor and the change of the magnification ratio by the drive unit based on the image data of the first image.

[0399] (Section 108) An imaging apparatus according to any one of paragraphs 85 to 102, further comprising an imaging range control unit that performs movement of the second imaging unit by the drive unit based on image data of the first image generated by the first imaging unit.

[0400] (Section 109) The imaging apparatus described in paragraph 108 further comprises an analysis unit that detects at least a portion of the image of the target included in the image of the first image based on the image data of the first image, and the imaging range control unit executes the movement of the second imaging unit by the drive unit based on the detection result of at least a portion of the image of the target by the analysis unit.

[0401] (Section 110) The imaging device described in paragraph 109, wherein the analysis unit detects the position of at least a portion of the image of the target on the image of the first image based on the image data of the first image, and the imaging range control unit performs movement of the second imaging unit by the drive unit based on the position of at least a portion as the result of the analysis unit's detection.

[0402] (Section 111) An imaging apparatus according to any one of paragraphs 84 to 110, further comprising an imaging recording control unit that controls the start and end of recording of the image data of the first image and the image data of at least one of the image data of at least a portion of the second image, based on the image data of the first image generated by the first imaging unit and the image data of at least a portion of the second image generated by the second imaging unit.

[0403] (Section 112) A light receiving device comprising: a light-gathering optical system; a light-splitting member for splitting light that has passed through at least a part of the light-gathering optical system; an imaging unit disposed in the optical path of one of the light beams split by the light-splitting member; and a light-receiving unit for receiving light that reaches a light-receiving region which is a part of the region occupied by the other light beam on a plane that crosses the optical path of the other light beam split by the light-splitting member, wherein the light-receiving device displaces the light-receiving region within the region.

[0404] (Section 113) A light-receiving device according to paragraph 112, wherein the light-collecting optical system includes an imaging optical system that forms an image on the imaging surface of the imaging unit, and the imaging optical system forms an image in the region. (Section 114) A light-receiving device according to paragraph 112 or 113, wherein the light-receiving unit is a light-receiving device having an image sensor having a plurality of pixels arranged in the portion of the region.

[0405] (Section 115) A light-receiving device comprising: an imaging optical system; an optical splitting member for splitting light that has passed through at least a part of the imaging optical system; an imaging unit for capturing an image formed by one of the optical beams split by the optical splitting member; a light-receiving unit for receiving light that has passed through a light-receiving region which is a part of the region through which the other optical beam split by the optical splitting member passes; and a drive unit for displacing the light-receiving region within the region.

[0406] (Section 116) A light-receiving device as described in paragraph 115, wherein an image is formed in the region by the other light beam, and when the imaging unit is a first imaging unit, the light-receiving unit includes a second imaging unit, the imaging region by the second imaging unit is the light-receiving region, and the drive unit displaces the imaging region by moving the second imaging unit.

[0407] (Section 117) In the light-receiving device described in paragraph 116, when the image formed by one of the light beams is defined as a first image, the image formed in the region by the other light beam is defined as a second image, the imaging region by the second imaging unit includes a portion of the second image, and the second imaging unit includes a re-imaging optical system for re-imaging a portion of the second image, and an image sensor for imaging the portion of the second image re-imaging by the re-imaging optical system. [Explanation of symbols]

[0408] 1,1A: Imaging device, 10: Imaging optical system, 11a~11d: Lens, 13: Light splitting member, 14a: First image, 14b: Second image, 14c: Third image, 15a: First image sensor, 15b: Second image sensor, 16: Drive unit, 16a: First drive unit, 17: Holding unit, 17a: Second holding frame, 17b: Second holding unit, 19: Housing, 20: Re-imaging optical system, 21a~21d: Lens, 28: Second drive unit, 30: Control unit, 31: Imaging control unit, 32: Image generation unit, 33: Analysis unit, 34: Imaging range control unit, 35: Storage unit, 36: Interface unit, EA1: First effective area, EA2: Second effective area, CA1: First imaging range, CA2: Second imaging range, IA: Region of interest

Claims

[Claim 1] Imaging optical system, A light splitting member that amplitude-splits light that has passed through at least a part of the imaging optical system, A first imaging unit captures a first image formed by one of the light beams divided by the light splitting member, A second imaging unit captures a portion of the second image formed by the other light beam divided by the aforementioned light splitting member, A drive unit that moves the second imaging unit in a direction that crosses the optical path of the other light beam, An imaging device equipped with the following features.

Images