Imaging device

Abstract

To provide an imaging apparatus capable of obtaining an image having the resolution of a long-focus optical system despite a wide angle.SOLUTION: An imaging apparatus (10) includes an optical system (11) for forming the image of incident first light on the predetermined light-receiving area of an imaging element (12), an optical element (13) for guiding second light different from the first light in an angle formed by the optical axis of the optical system and a main light beam made incident on the optical system to the predetermined light-receiving area and allowing the imaging element to image by superimposing on the image by the first light, and a controller (14) for separating the image by the first light and the image by the second light from the image signal of a superimposition image obtained by superimposing the image by the first light and the image by the second light outputted by the imaging element. The controller separates the image by the first light and the image by the second light on the basis of that the movement direction of an arbitrary subject image in the superimposition image is different between the image by the first light and the image by the second light.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an imaging apparatus.

Background Art

[0002] An imaging optical system that forms an image of an observation target has various physical properties such as focal length and angle of view. When the focal length increases, an image in which the observation target is magnified is formed, so detailed optical information of a distant observation target, in other words, magnified optical information can be obtained. The wider the angle of view, the more optical information of observation targets located in a wider range can be obtained. However, there is a trade-off relationship between focal length and angle of view. When the focal length increases, the angle of view narrows, and when the focal length decreases, the angle of view widens.

[0003] Therefore, the focal length is adjusted so that desired optical information can be obtained depending on the situation. For example, the focal length is adjusted by displacing a zoom lens included in the imaging optical system. Further, the focal length is adjusted by switching between a plurality of single-focus lenses (see Patent Documents 1 and 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problem to be Solved by the Invention

[0005] The technologies of Patent Documents 1 and 2 can obtain an image having the resolution of a long-focus optical system (a distant captured image) or a wide-angle image which is a captured image with a wide angle of view by switching the focal length. However, the technologies of Patent Documents 1 and 2 cannot simultaneously obtain a distant captured image and a wide-angle image with high resolution.

[0006] In view of these circumstances, the purpose of this disclosure is to provide an imaging device capable of obtaining images that have the resolution of a long-focus optical system while being wide-angle. [Means for solving the problem]

[0007] (1) An imaging device according to one embodiment of the present disclosure, An optical system that images the incoming first light onto a predetermined light-receiving area of ​​an image sensor, An optical element that guides a second light, whose angle between the optical axis of the optical system and the principal ray incident on the optical system is different from that of the first light, into the predetermined light-receiving region, and superimposes it on the image produced by the first light so that it is imaged by the image sensor, The system includes a controller that separates the first light image and the second light image from the superimposed image signal obtained by superimposing the first light image and the second light image output by the image sensor, The controller separates the image produced by the first light and the image produced by the second light based on the fact that the direction of movement of any subject image in the superimposed image is different for the image produced by the first light and the image produced by the second light.

[0008] (2) As one embodiment of the present disclosure, in (1), The controller uses optical flow to detect the direction of movement of the arbitrary subject image.

[0009] (3) In one embodiment of the present disclosure, in (1) or (2), A stereo camera comprising a plurality of optical systems and a plurality of optical elements, The controller uses a plurality of superimposed images captured from different positions at the same time to detect the direction of movement of the arbitrary subject image.

[0010] (4) In one embodiment of the present disclosure, in (1) or (2), The controller uses a plurality of superimposed images captured at different times to detect the direction of movement of the arbitrary subject image.

[0011] (5) In one embodiment of the present disclosure, in any of (1) to (4), The controller separates the image from the first light and the image from the second light using an image separation model generated by machine learning. The aforementioned image separation model is constructed by creating a training superimposed image by overlapping multiple images in advance, and then using the training superimposed image to perform machine learning on the multiple images as the correct answers.

[0012] (6) An imaging device according to one embodiment of the present disclosure An optical system that images the incoming first light onto a predetermined light-receiving area of ​​an image sensor, The system includes a controller that separates the image from the first light source and the image from the second light source from the image signal of a superimposed image obtained by superimposing an image from a second light source, where the angle between the optical axis of the optical system and the principal light ray incident on the optical system is different from that of the first light source. The direction of movement of any subject image in the superimposed image is different for the image produced by the first light and the image produced by the second light. [Effects of the Invention]

[0013] According to this disclosure, it is possible to provide an imaging device that can obtain images with a wide angle while having the resolution of a long-focus optical system. [Brief explanation of the drawing]

[0014] [Figure 1] Figure 1 is a schematic diagram showing the main configuration of an imaging device according to one embodiment of the present disclosure. [Figure 2] Figure 2 shows an imaging device configured as a stereo camera. [Figure 3] Figure 3 is a diagram illustrating the positions of the image sensor and optical system shown in Figure 1. [Figure 4] Figure 4 is a conceptual diagram illustrating the image that reaches the light-receiving region in Figure 1. [Figure 5] Figure 5 is a conceptual diagram illustrating the process by which the superimposed image reaching the light-receiving region in Figure 1 is formed. [Figure 6] Figure 6 is a conceptual diagram for explaining a process of generating a restored image from a superimposed image. [Figure 7] Figure 7 is a diagram for explaining a difference in movement direction between an image formed by first light and an image formed by second light. [Figure 8] Figure 8 is a flowchart for explaining distance measurement processing executed by a controller. MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, an imaging apparatus 10 (see FIG. 1) according to an embodiment of the present disclosure will be described with reference to the drawings. In the components shown in the following drawings, the same components are denoted by the same reference numerals. The drawings used for describing the embodiment are schematic. Dimensional ratios and the like in the drawings do not necessarily match actual values.

[0016] FIG. 1 is a schematic diagram showing a main configuration of the imaging apparatus 10 according to the present embodiment. FIG. 2 is a diagram showing the imaging apparatus 10 configured as a stereo camera. The following description is based on the assumption that the imaging apparatus 10 according to the embodiment of the present disclosure is configured as a stereo camera. However, since a plurality of identical components are arranged when configured as a stereo camera, description may be given using a diagram showing the main component configuration of the imaging apparatus 10 as shown in FIG. 1 to avoid duplication and make the description easy to understand.

[0017] As shown in FIG. 1, the imaging apparatus 10 includes an optical device 21 and a controller 14. The imaging apparatus 10 may further include an imaging element 12. The optical device 21 includes an imaging optical system (optical system 11) and an optical element 13.

[0018] The optical system 11 forms an image of the incident subject light beam. The optical system 11 forms an image of the incident first light on a predetermined light-receiving area pa of the image sensor 12. The predetermined light-receiving area pa is part or all of the light-receiving area ra of the image sensor 12. The first light may be light emitted from an object point located within the field of view of the optical system 11 alone. The predetermined light-receiving area pa may be, for example, a virtual plane or curved surface in three-dimensional space whose center intersects the optical axis ox of the optical system 11. Hereafter, the field of view of the optical system 11 alone, in other words, the field of view of the optical system 11 in a configuration that does not include the optical elements 13, may be referred to as the direct field of view in the explanation. The optical system 11 is composed of optical components that, individually, in other words, without the optical elements 13, form images of light beams emitted from object points at different positions on different image points. The optical components that make up the optical system 11 are, for example, lenses, mirrors, diaphragms, etc.

[0019] The optical system 11 does not have to be image-side telecentric. In other words, the angle of the principal ray of any light beam passing through the optical system 11 with respect to the optical axis ox may be greater than 0°. Alternatively, the optical system 11 may be image-side telecentric.

[0020] The optical element 13 guides the second light incident on the optical system 11 to a predetermined light-receiving region pa. The angle between the optical axis ox of the optical system 11 and the principal ray incident on the optical system 11 of the second light is different from that of the first light. The second light may be light emitted from an object point located outside the field of view of the optical system 11, in other words, directly outside the field of view. Therefore, the angle between the principal ray of the second light and the optical axis ox may be larger than the angle between the principal ray of the first light and the optical axis ox. The principal ray may be a ray passing through the center of the aperture diaphragm of the optical system 11, a ray passing through the center of the entrance pupil of the optical system 11, or a ray at the center of a light beam emitted from any one object point and incident on the optical system 11. Furthermore, the optical element 13 may image the second light that has passed through the optical system 11 into the predetermined light-receiving region pa.

[0021] The optical element 13 may be a mirror that reflects the second light and guides it to a predetermined light-receiving region pa. In this embodiment, the optical element 13 guides the second light to the predetermined light-receiving region pa as an inverted image of the image obtained by the first light. The optical element 13 superimposes the inverted image obtained by the second light onto the image obtained by the first light and causes the image sensor 12 to capture it. The optical element 13 may be, for example, a planar mirror, a curved mirror, a DMD, or a Fresnel mirror.

[0022] The reflective surface of the mirror may be parallel to the optical axis ox of the optical system 11. Alternatively, the reflective surface of the mirror may not be parallel to the optical axis ox. For example, the reflective surface of the mirror may be tilted with respect to the optical axis ox so that it is tilted outward, facing the optical system 11. With the outward tilt, the field of view of the entire optical device 21 can be widened compared to a configuration in which the reflective surface of the mirror is parallel to the optical axis ox. For example, the reflective surface of the mirror may be tilted with respect to the optical axis ox so that it is tilted inward, facing the image-forming plane of the optical system 11. With the inward tilt, the entire optical device 21 can be miniaturized compared to a configuration in which the reflective surface of the mirror is parallel to the optical axis ox.

[0023] Here, the optical device 21 may have a lens for adjusting the optical path length between the optical system 11 and the optical element 13, which is a mirror. By arranging the lens for adjusting the optical path length, the shift in the focus position due to the longer optical path length compared to a configuration in which the reflective surface of the mirror is parallel to the optical axis ox can be reduced. The lens for adjusting the optical path length may be, for example, a cylindrical lens. The optical device 21 may also be configured to include a prism. The second light may be reflected by the optical element 13, which is a mirror, and then reflected again by the prism, and guided to a predetermined light-receiving region pa.

[0024] The mirror may be positioned such that its reflective surface is located outside the exit pupil of the optical system 11 when viewed from the optical axis ox of the optical system 11. Alternatively, the mirror may be located inside the exit pupil when viewed from the optical axis ox of the optical system 11. In particular, in configurations where the light-receiving area ra is smaller than the pupil diameter, the mirror may be located inside the exit pupil. The mirror may consist of multiple planar mirrors. As shown in Figure 3, the distance H between each of two planar mirrors with parallel reflective surfaces and the optical axis ox may be equal. The two parallel planar mirrors, the optical system 11, and the image sensor 12 satisfy CRA ≤ tan -1 The optical system may be designed and positioned to satisfy (H / B). CRA is the angle of the principal ray by the optical system 11 with respect to the optical axis ox of the light beam emitted from the object point (pp) at an angle twice the direct field of view. B is the back focus of the optical system 11.

[0025] As shown in Figure 4, the first image component im1, corresponding to an object point directly within the field of view, in other words, an object point emitting the first light, reaches the light-receiving region ra of the image sensor 12 without passing through the optical element 13. More specifically, the first image component im1, corresponding to an object point directly within the field of view, corresponds to the image of the subject located directly within the field of view. In addition, the second image component im2, corresponding to an object point directly outside the field of view, in other words, an object point emitting the second light, reaches the light-receiving region ra inverted via the optical element 13. More specifically, the second image component im2, corresponding to an object point directly outside the field of view, corresponds to the image of the subject located directly outside the field of view.

[0026] The image sensor 12 captures an image that is formed within the light-receiving region ra. Here, the direct field of view may be a field of view corresponding to the range of object points that are imaged within the light-receiving region ra without passing through the optical element 13. At least a portion of the light beam, which is the first light incident on the optical system 11 from within the direct field of view of the optical system 11, may be imaged in the light-receiving region ra. In addition, at least a portion of the light beam, which is the second light incident on the optical system 11 from outside the direct field of view of the optical system 11 and passing through the optical element 13 may be imaged in the light-receiving region ra.

[0027] The image sensor 12 may be capable of capturing images using invisible light such as visible light, infrared light, or ultraviolet light. The image sensor 12 may be, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor. The image sensor 12 may be a color image sensor. The image sensor 12 generates an image signal corresponding to the image received by imaging.

[0028] In the image sensor 12, the outer edge of the light-receiving region ra on the side where the optical element 13 is provided may be located outside the outer edge of the exit pupil of the optical system 11, as viewed from the optical axis ox of the optical system 11. Outside the outer edge of the exit pupil means outside with respect to the optical axis ox of the optical system 11. When viewed in the direction along the optical axis ox of the optical system 11, the light-receiving region ra may be rectangular.

[0029] The imaging device 10 may include a plurality of optical systems 11 and a plurality of optical elements 13. As described above, the imaging device 10 according to this embodiment is configured as a stereo camera comprising a plurality of cameras having parallax with respect to each other. As shown in Figure 2, one set of optical systems 11 and optical elements 13 constitutes an optical device 21R having the configuration of Figure 1, and another set of optical systems 11 and optical elements 13 constitutes an optical device 21L having the configuration of Figure 1. The imaging device 10 also includes a plurality of image sensors 12. Each of the optical devices 21R and 21L forms an image on a predetermined light-receiving area pa of its respective image sensor 12 such that the inverted image from the second light is superimposed on the image from the first light.

[0030] As shown in Figure 5, for one camera of the imaging device 10, a first image component im1 and a second image component im2, which is inverted by the optical element 13 acting as a mirror, are superimposed in the light-receiving region ra. Therefore, the image sensor 12 of one camera captures a superimposed image olim of the first image component im1 and the second image component im2, which is inverted by the optical element 13 acting as a mirror. Here, since the imaging device 10 according to this embodiment is configured as a stereo camera, it is possible to acquire multiple superimposed images olim captured from different positions at the same time. The multiple superimposed images olim obtained are captured by multiple cameras having parallax with respect to each other, so they are images of substantially the same object being captured from different viewpoints. Here, as another example, the multiple superimposed images olim may be captured at different times. In this case, the imaging device 10 may be equipped with multiple cameras, and the camera being used for capturing may be switched according to time. Alternatively, the imaging device 10 may be equipped with one camera capable of changing viewpoints, and may be configured to capture while changing viewpoints according to time. Furthermore, the imaging device 10 may be configured to take images at different times without changing the viewpoint.

[0031] The controller 14 comprises at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. The dedicated circuit may be, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The controller 14 may perform image processing on the image signal acquired from the image sensor 12.

[0032] The controller 14 separates the image from the superimposed image olim, which is formed by superimposing the image from the first light and the image from the second light output by the image sensor 12, into the image from the first light (first image component im1) and the image from the second light (second image component im2). As shown in Figure 6, the controller 14 performs image processing to separate the superimposed image olim, which corresponds to the image signal, into the first image component im1 and the second image component im2. The controller 14 may separate the superimposed image olim by applying image processing methods such as independent component analysis, wavelet method, or image separation model. The image separation model is a model constructed by, for example, creating a training superimposed image by superimposing multiple images in advance, and then using machine learning on the training superimposed image with multiple images as the ground truth. In this embodiment, the controller 14 separates the image from the first light and the image from the second light using an image separation model (trained model) generated by machine learning. Here, the image separation model only needs to be generated before the controller 14 performs the process of separating the superimposed image olim, and stored in a memory accessible to the controller 14. The image separation model may be generated by the controller 14, for example, or by an information processing device (computer) different from the controller 14. Also, the machine learning method is, for example, deep learning, but is not limited to any particular method.

[0033] In this embodiment, the controller 14 separates the image obtained by the first light and the image obtained by the second light based on the fact that the direction of movement of an arbitrary subject image in the superimposed image olim is different for the image obtained by the first light and the image obtained by the second light. The controller 14 uses multiple superimposed images olim to detect the direction of movement of an arbitrary subject image. In this embodiment, the controller 14 uses the superimposed image olim_R of the right eye (right camera) and the superimposed image olim_L of the left eye (left camera) captured at the same time. However, as described above, the controller 14 can also use multiple superimposed images captured at different times. For example, when the imaging device 10 is mounted on a vehicle and used to measure the distance to objects such as vehicles and pedestrians ahead, it can be difficult to distinguish whether the difference in the position of an object is due to the movement of the object itself or a change in the camera's viewpoint. Therefore, in cases where the imaging device 10 is mounted on a vehicle, it is preferable that the imaging device 10 is configured as a stereo camera and the controller 14 uses multiple superimposed images olim captured at the same time in order to improve detection accuracy. However, if the object being measured for distance using the imaging device 10 is a building or other object that does not move on its own, the controller 14 may use multiple superimposed images olim taken at different times instead of multiple superimposed images olim taken at the same time.

[0034] Figure 7 illustrates the difference in the direction of movement between the image produced by the first light source and the image produced by the second light source. The three subjects ob1 to ob3 included in the superimposed image olim_R of the right eye are also included in the superimposed image olim_L of the left eye. The three subjects ob1 to ob3 in the superimposed image olim_R of the right eye are shown by solid lines. The three subjects ob1 to ob3 in the superimposed image olim_L of the left eye are shown by dashed lines. Subjects ob1 and ob2 are images produced by the first light source, and their positions in the superimposed image olim_L of the left eye are to the right of those in the superimposed image olim_R of the right eye. That is, if the change from the position in the superimposed image olim_R of the right eye to the position in the superimposed image olim_L of the left eye is considered the direction of movement, then the direction of movement is to the right. In contrast, subject ob3 is an image produced by the second light source, and it is an inverted image of the image produced by the first light source. Therefore, the position of subject ob3 in the left eye superimposed image olim_L is to the left of the position of the right eye superimposed image olim_R, and the direction of movement is to the left. In this way, since the image produced by the second light is an inverted image with respect to the image produced by the first light, the direction of movement of any subject image in the superimposed image olim is opposite. Therefore, the controller 14 can determine whether any subject image in the superimposed image olim is an image produced by the first light or the second light, according to the direction of movement identified by comparing the positions in multiple superimposed images olim. Here, the subject image whose direction of movement is detected may be, for example, a feature point of an object, or it may be a block unit or pixel unit of a predetermined size that divides the superimposed image olim.

[0035] For example, controller 14 may use optical flow to detect the direction of movement of an arbitrary subject image. While optical flow generally uses images of consecutive frames, controller 14 may treat the superimposed image olim_R for the right eye and the superimposed image olim_L for the left eye as images of consecutive frames and perform processing. Controller 14 calculates the optical flow at each pixel of the superimposed image olim using any method, such as the Lucas-Kanade method. Based on the calculated optical flow for each pixel, controller 14 can determine that an image is caused by a first light source if the motion vector for each pixel is in a first direction, and that an image is caused by a second light source if it is in a second direction. Here, the second direction is the opposite direction to the first direction. Furthermore, controller 14 may determine that pixels that are close to each other and have the same motion vector belong to the same subject, based on the optical flow at each pixel of the superimposed image olim. Furthermore, by incorporating optical flow information into machine learning during the generation of an image separation model, for example, it becomes possible to construct an image separation model that accurately separates the image produced by the first light source from the image produced by the second light source according to the direction of movement of the subject image.

[0036] The controller 14 may generate a reconstructed image rcim by combining the separated first image component im1 and the second image component im2 (see Figure 6). The controller 14 may use the reconstructed image rcim to measure the distance of a subject captured around the imaging device 10. The controller 14 may, for example, measure the distance using the reconstructed image rcim based on DFD (Depth From Defocus). The controller 14 may also measure the distance using the reconstructed image rcim based on methods such as motion disparity (SLAM: Simultaneous Localization and Mapping, Motion Stereo) or ground-level distance measurement. Ground-level distance measurement is a method that calculates three-dimensional coordinates based on image coordinates, assuming that the lower end of the subject image is located on the ground.

[0037] The controller 14 may generate a distance image based on the calculated distance. The distance image is an image in which the pixel value of each pixel corresponds to the distance. The controller 14 may output the distance image to an external device.

[0038] Figure 8 is a flowchart of the distance measurement process performed by the controller 14 in this embodiment. The distance measurement process is started each time an image signal is acquired from the image sensor 12.

[0039] In step S100, the controller 14 separates the second image component im2 from the superimposed image olim, which corresponds to the acquired image signal. After separation, the process proceeds to step S101.

[0040] In step S101, the controller 14 generates the first image component im1 by subtracting the second image component im2, which was separated in step S100, from the superimposed image olim. After generating the first image component im1, the process proceeds to step S102.

[0041] In step S102, the controller 14 generates a reconstructed image rcim by combining the second image component im2, which was separated in step S101, with the first image component im1, which was generated in step S101. After the generation of the reconstructed image rcim, the process proceeds to step S103.

[0042] In step S103, the controller 14 uses the reconstructed image rcim generated in step S102 to measure the distance to each subject in the reconstructed image rcim. After measuring the distance, the process proceeds to step S104.

[0043] In step S104, the controller 14 generates a distance image based on the distance calculated in step S103 and the position of the reconstructed image rcim corresponding to that distance. The controller 14 also outputs the distance image to an external device. After the distance image is generated, the distance measurement process ends.

[0044] As described above, the imaging device 10 according to this embodiment, with the above configuration, can obtain images that are wide-angle yet have the resolution of a long-focus optical system.

[0045] Furthermore, while distance measurement processing using the imaging device 10 has been described, embodiments of this disclosure may also include not only methods or programs for implementing the device, but also embodiments of a storage medium on which a program is recorded. Examples of storage media include optical discs, magneto-optical discs, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, hard disks, or memory cards.

[0046] Furthermore, the implementation form of the program is not limited to application programs such as object code compiled by a compiler or program code executed by an interpreter. The program may take the form of a program module embedded in an operating system. Moreover, the program may or may not be configured so that all processing is performed only on the CPU on the control board. The program may be configured so that some or all of its processing is performed by another processing unit implemented on an expansion board or expansion unit attached to the board, as needed.

[0047] While embodiments relating to this disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications or alterations based on this disclosure. Therefore, it should be noted that these modifications or alterations are within the scope of this disclosure. For example, the functions included in each component can be rearranged in a logically consistent manner, and multiple components can be combined into one or separated.

[0048] All of the constituent elements described in this disclosure, and / or all of the disclosed methods or steps of processing, can be combined in any combination except for any combination in which these features are mutually exclusive. Furthermore, each of the features described in this disclosure can be replaced by an alternative feature that works for the same, equivalent, or similar purpose, unless expressly disregarded. Thus, unless expressly disregarded, each of the disclosed features is merely an example of a comprehensive set of identical or equivalent features.

[0049] Furthermore, the embodiments relating to this disclosure are not limited to any specific configuration of the embodiments described above. The embodiments relating to this disclosure can be extended to all novel features or combinations thereof described herein, or all novel methods or processing steps or combinations thereof described herein.

[0050] In this disclosure, designations such as "First" and "Second" are identifiers used to distinguish between configurations. Configurations distinguished by designations such as "First" and "Second" in this disclosure may have their numbers swapped. The swapping of identifiers occurs simultaneously. The configurations remain distinguishable even after the swapping of identifiers. Identifiers may be deleted. Configurations from which identifiers have been deleted are distinguished by codes. The designations such as "First" and "Second" in this disclosure should not be used alone to interpret the order of configurations or to justify the existence of smaller numbered identifiers. [Explanation of Symbols]

[0051] 10 Imaging device 11 Optical system 12 Image sensor 13 Optical elements 14 Controllers 21, 21L, 21R optical equipment im1 First image component im2: Second image component olim, olim_L, olim_R superimposed images ox optical axis pa predetermined light receiving area ra light receiving area rcim restored image

Claims

1. An optical system that images the incoming first light onto a predetermined light-receiving area of ​​an image sensor, An optical element that guides a second light, whose angle between the optical axis of the optical system and the principal ray incident on the optical system is different from that of the first light, into the predetermined light-receiving region, and superimposes it on the image produced by the first light so that it is imaged by the image sensor, The system includes a controller that separates the first image and the second image from the superimposed image signal obtained by superimposing the first image and the second image output by the image sensor. The controller separates the image produced by the first light and the image produced by the second light based on the fact that the direction of movement of an arbitrary subject image in the superimposed image is different for the image produced by the first light and the image produced by the second light.

2. The imaging apparatus according to claim 1, wherein the controller uses optical flow to detect the direction of movement of the arbitrary subject image.

3. A stereo camera comprising a plurality of optical systems and a plurality of optical elements, The imaging apparatus according to claim 1 or 2, wherein the controller uses a plurality of superimposed images captured from different positions at the same time to detect the direction of movement of the arbitrary subject image.

4. The imaging apparatus according to claim 1 or 2, wherein the controller uses a plurality of superimposed images captured at different times to detect the direction of movement of the arbitrary subject image.

5. The controller separates the image from the first light and the image from the second light using an image separation model generated by machine learning. The imaging apparatus according to claim 1 or 2, wherein the image separation model is constructed by creating a training superimposed image by overlapping multiple images in advance, and then using machine learning on the training superimposed image with the multiple images as the correct answers.

6. An optical system that images the incoming first light onto a predetermined light-receiving area of ​​an image sensor, The system includes a controller that separates the image from the first light source and the image from the second light source from the image signal of a superimposed image obtained by superimposing an image from a second light source, where the angle between the optical axis of the optical system and the principal light ray incident on the optical system is different from that of the first light source. The controller separates the image produced by the first light and the image produced by the second light based on the fact that the direction of movement of an arbitrary subject image in the superimposed image is different for the image produced by the first light and the image produced by the second light.

Images