Imaging device

Abstract

The system tracks subjects that can be focused on even with a narrow field of view, among the subjects recognized in the wide-angle image. [Solution] The field of view of the second camera 200 is narrower than the field of view α of the first camera 100, which is a field of view β. The pan-tilt device 300 holds the first camera 100 and the second camera 200 together so that they can rotate in the pan and tilt directions. The first camera control unit 15 determines, based on the distance measurement results of the subject recognized from the image acquired by the first camera 100, that the subject within the distance range that the second camera 200 can focus on is the subject to be tracked, and controls the pan-tilt device 300 to track the subject to be tracked.

Description

Technical Field

[0001] The present invention relates to an imaging device.

Background Art

[0002] There is known an imaging device having a tracking function for framing a subject using a pan-tilt mechanism. For example, in an imaging device used for shooting sports competitions, wild animals, etc., the shooting direction is determined by remote control and shooting is carried out. Patent Document 1 discloses a technique for controlling the shooting range of a telephoto camera using a mirror block and a pan motor based on the movement of a subject detected by a wide-angle camera. Further, Patent Document 2 discloses a technique for correctly detecting the eyes of a driver driving an automobile, detecting the position of the driver's eyes from a wide-angle image, panning and tilting a telephoto camera based on the result, and imaging the driver's eyes after magnifying them.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, although the focus adjustment of the telephoto camera is mentioned, it is premised that the assumed subject can be photographed by the telephoto camera. Further, in Patent Document 2, when detecting the eyes of the driver, since the position of the driver and the position of the camera are known to some extent, there is no need to consider whether it is possible to focus on the eyes with the telephoto camera.

[0005] However, when attempting to photograph sports competitions using a telephoto camera like the one described in Patent Document 1, the following problems arise. Specifically, even if a subject can be focused on in the wide-angle image obtained by pan-tilt control, the telephoto camera intended for this shooting may not be able to focus on the subject due to reasons such as the subject being too close, and a focused image may not be obtained.

[0006] The present invention aims to track subjects that can be focused on even with a narrow field of view, among subjects recognized in a wide-angle image. [Means for solving the problem]

[0007] To achieve the above objective, the imaging device of the present invention is characterized by comprising: a first imaging unit for acquiring a wide-angle image; a second imaging unit for acquiring an image with a field of view narrower than that of the first imaging unit but included within the field of view of the first imaging unit; a pan-tilt device for holding the first imaging unit and the second imaging unit so as to be rotatable in the pan and tilt directions; and a control means for determining, based on the distance measurement result of a subject recognized from the image acquired by the first imaging unit, a subject that is within a distance range that can be focused by the second imaging unit from among the recognized subjects as a tracking target, and controlling the pan-tilt device to track the subject that is the tracking target. [Effects of the Invention]

[0008] According to the present invention, it is possible to track subjects that can be focused on even with a narrow field of view, among subjects recognized in a wide-angle image. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic cross-sectional view of the imaging device. [Figure 2] This is a schematic block diagram of the imaging device. [Figure 3] This is a flowchart for pan-tilt processing. [Figure 4] This is a schematic block diagram of a modified imaging device. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings.

[0011] Figure 1 is a schematic cross-sectional view of an imaging device according to one embodiment of the present invention. Figure 2 is a schematic block diagram of the imaging device 1000. The imaging device 1000 consists of a first camera 100 (first imaging unit), a second camera 200 (second imaging unit), and a pan-tilt device 300.

[0012] The field of view of the first camera 100 is field of view α. The field of view of the second camera 200 is field of view β, which is narrower than field of view α. For example, the first camera 100 may be called a wide-angle camera, and the second camera 200 may be called a telephoto camera. The optical axis 12 of the first camera 100 and the optical axis 22 of the second camera 200 are parallel to each other, and the first camera 100 and the second camera 200 always point in the same direction. Field of view β is encompassed by field of view α. In other words, the area that can be imaged by the second camera 200 can also be imaged by the first camera 100.

[0013] The pan-tilt device 300 holds the first camera 100 and the second camera 200 together so that they can rotate in the pan and tilt directions. The pan-tilt device 300 includes a pan-tilt control unit 31 and a pan-tilt mechanism 32 (Figure 2). The pan-tilt device 300 uses a pan motor 322 and a tilt motor 321 included in the pan-tilt mechanism 32 to pan-tilt the first camera 100 and the second camera 200 as shown by the arrows.

[0014] While not particularly limited, the rotation center of the tilt motor 321 is set on the optical axis 22 of the second camera 200, and the rotation center of the pan motor 322 is set near the front principal point of the second camera 200. This makes it possible for the pan-tilt operation to mainly impart only an angle change to the pan-tilt operation of the second camera 200, without introducing any unnecessary parallel movement components.

[0015] As shown in FIG. 2, the first camera 100 includes a wide-angle optical system 11, an image pickup device 13, an image processing unit 14, a first camera control unit 15, and a storage unit 16. The wide-angle optical system 11 includes a focus mechanism 17. The second camera 200 includes an optical system 21, an image pickup device 23, an image processing unit 24, a second camera control unit 25, and a storage unit 26. The optical system 21 includes a focus mechanism 27.

[0016] The first camera 100 is a so-called digital camera that obtains a wide-angle image in which a light beam transmitted through the wide-angle optical system 11 and imaged on the image pickup device 13 is converted into an image signal by photoelectric conversion and various image processes are performed in the image processing unit 14.

[0017] Based on the image signal obtained by the image pickup device 13, the first camera control unit 15 performs distance measurement of the recognized subject and a focusing drive operation of the focus mechanism 17. Further, the first camera control unit 15 has a function of calculating a predicted distance measurement of the subject from the history of the obtained distance measurement results of the subject. The first camera 100 can also store (record) the image signal processed in the image processing unit 14 in the storage unit 16. The case of storing an image in the first camera 100 will be described later.

[0018] The second camera 200 is a so-called digital camera that converts a light beam imaged on the image pickup device 23 into an image signal by photoelectric conversion through an optical system 21 having a narrower angle of view than the wide-angle optical system 11, and various image processes are performed in the image processing unit 24.

[0019] Based on the image signal obtained by the image pickup device 23, the second camera control unit 25 performs distance measurement of the recognized subject and a focusing drive operation of the focus mechanism 27.

[0020] Note that the image pickup device 23 may have a higher number of pixels than the image pickup device 13, and the optical system 21 may have a higher resolution than the wide-angle optical system 11. In this case, the second camera 200 can obtain a clearer and higher-resolution image than the first camera 100.

[0021] In the pan-tilt device 300, based on the recognition information and distance measurement information of the subject obtained by the first camera control unit 15, the pan-tilt control unit 31 drives the pan-tilt mechanism 32 to track the main subject.

[0022] Since the first camera 100 has a wide-angle field of view, in a normal shooting scene, the influence of the translation component caused by the deviation of the pan-tilt rotation center from the wide-angle optical system 11 is small and can be ignored. By the pan-tilt operation, it is possible to change the direction while maintaining the relationship between the optical axis 12 of the first camera 100 and the optical axis 22 of the second camera 200 that are parallel to each other.

[0023] In the imaging device 1000, the first camera 100 and the second camera 200 always face the same direction, but it is not limited to such a configuration. Even if a pan-tilt operation is performed, it is only necessary that the angle of view α of the wide-angle optical system 11 includes the angle of view β of the optical system 21. The imaging device 1000 is an imaging device mainly for saving the image captured by the second camera 200. The main role of the first camera 100 is to provide the recognition result and distance measurement result of the main subject for the pan-tilt device 300 to track and drive the main subject.

[0024] The second camera 200 can save the image signal processed in the image processing unit 24 in the storage unit 26. The second camera control unit 25 can refer to the subject recognition result and distance measurement result obtained from the first camera control unit 15. Also, the first camera control unit 15 can refer to the subject recognition result and distance measurement result obtained from the second camera control unit 25. That is, the two camera control units 15 and 25 can refer to each other's information.

[0025] Note that the focal length of the optical system 21 of the second camera 200 is longer than that of the wide-angle optical system 11 of the first camera 100, but it is not limited to this, and any optical system that can cover the required angle of view range of the photographer may be used.

[0026] The first camera control unit 15 is capable of storing a history of distance measurement results, and by performing extrapolation calculations from this history of distance measurement results, it is possible to predict the distance measurement result at a predetermined frame ahead and acquire it as distance measurement prediction information. Pan-tilt operation using this distance measurement prediction information will be described later.

[0027] Figure 3 is a flowchart of the pan-tilt process performed by the imaging device 1000. This process starts when the main power supply of the imaging device 1000 is turned on. Specifically, the pan-tilt process starts when cameras 100 and 200 and the pan-tilt device 300 are started up, and the first camera control unit 15 and the second camera control unit 25 are able to communicate with each other.

[0028] Pan-tilt processing is performed by control means. The first camera control unit 15, the second camera control unit 25, and the pan-tilt control unit 31 cooperate to function as control means. Each control unit 15, 25, and 31 is realized by its own CPU loading the corresponding program into the corresponding RAM and executing it. The corresponding programs are stored in ROMs (not shown) in the cameras 100, 200, and the pan-tilt device 300, respectively.

[0029] In step S101, the first camera control unit 15 captures one frame using the first camera 100 and then recognizes a subject from the output of the image processing unit 14. If one subject is recognized, the control unit 15 determines it to be the "main subject". If multiple subjects are recognized, the control unit 15 determines one to be the main subject from among them, then prioritizes the subjects other than the main subject (secondary subjects), and then determines which subject could become the next main subject. For example, prioritization may be based on whether a face has been detected, whether a moving subject has been detected, etc.

[0030] In step S102, the first camera control unit 15 determines whether or not there is a subject that was recognized in step S101. If no subject is recognized in the current frame, the first camera control unit 15 proceeds to step S111; if a subject is recognized, it proceeds to step S103.

[0031] In step S103, the camera control unit 15 of the first camera performs distance measurement on the subject recognized in step S101. This allows the first camera control unit 15 to obtain the subject distances of the main subject and any sub-subjects. Furthermore, if the captured frame is the second or later frame, the first camera control unit 15 predicts the distance measurement results for subsequent subjects based on the history of distance measurement results obtained up to the previous frame and the current distance measurement result, and stores this as distance measurement prediction information. The distance measurement prediction information is stored in the storage unit 16 or a memory (not shown).

[0032] In step S104, the first camera control unit 15 determines whether the main subject can be focused by the focus mechanism 27 of the second camera 200, based on the distance measurement information and distance measurement prediction information acquired in step S103. Specifically, if the main subject can be focused in the current frame and is predicted to be able to be focused in the next frame, it is determined that it can be focused. However, if the main subject cannot be focused in the current frame, or is predicted to become impossible to focus in the next frame, it is determined that it cannot be focused. If it is determined that it can be focused, the first camera control unit 15 proceeds to step S105; if it is determined that it cannot be focused, it proceeds to step S108.

[0033] In step S105, the first camera control unit 15 determines whether the main subject recognized by the first camera 100 is within the field of view of the second camera 200. This is to determine whether the main subject is in a state where it can be recognized by the second camera 200 as a result of the pan-tilt operation described later.

[0034] The first camera control unit 15 proceeds to step S106 if the main subject is within the field of view of the second camera 200, and to step S107 if the main subject is not within the field of view of the second camera 200. Therefore, until the second camera 200 can recognize the main subject through pan-tilt operation, step S106 is skipped and the process proceeds to step S107.

[0035] In step S106, the first camera control unit 15 determines the reliability of the distance measurement results (distance measurement information, distance measurement prediction information) acquired in step S103. Specifically, the first camera control unit 15 instructs the second camera control unit 25 to perform distance measurement or distance measurement prediction on the same subject that was the target of distance measurement or distance measurement prediction in step S103, compares the results of both, and determines whether the difference exceeds a predetermined value.

[0036] In other words, the first camera control unit 15 obtains the difference between the distance measurement result (including distance measurement prediction result) of the subject recognized from the image acquired by the first camera 100 and the distance measurement result (including distance measurement prediction result) of the subject recognized from the image acquired by the second camera 200. If at least one of the differences between the distance measurement results or the differences between the distance measurement prediction results exceeds a predetermined value, the first camera control unit 15 can determine that the distance measurement results obtained by the first camera 100 are unreliable and proceed to step S108. In this case, even if the subject is within the distance range that the second camera 200 can focus on, it will not be determined to be a tracking target, thus improving the accuracy of determining the tracking target. On the other hand, if none of the above differences exceed a predetermined value, the first camera control unit 15 proceeds to step S107.

[0037] Furthermore, the determination may be made by weighting the distance measurement results and distance prediction results from the second camera 200. Also, even if the difference exceeds a predetermined value, the following processing may be performed.

[0038] In other words, the first camera control unit 15 instructs the second camera control unit 25 to move the focus mechanism 27 of the optical system 21 of the second camera 200 to the focus position calculated by the first camera control unit 15, after which the second camera control unit 25 performs distance measurement. The focus position is the position indicated by the distance measurement result of the subject recognized from the image acquired by the first camera 100. The first camera control unit 15 then repeats the acquisition of the difference. That is, the first camera control unit 15 compares the newly acquired distance measurement result with the distance measurement result from the first camera control unit 15 again, and performs reliability determination by comparing the aforementioned difference with a predetermined value. This improves the accuracy of the determination.

[0039] In step S107, the first camera control unit 15 determines the main subject to be tracked and starts tracking the main subject. To this end, the first camera control unit 15 notifies the pan-tilt control unit 31 of the main subject's position information for pan-tilt control. The pan-tilt control unit 31 drives and controls the pan-tilt mechanism 32 so that the most recent main subject is captured in the center of the second camera 200's screen.

[0040] Therefore, among the subjects recognized from the image acquired by the first camera 100, subjects that are within the distance range that can be focused by the second camera 200 are determined to be the subjects to be tracked, and the pan-tilt device 300 is controlled to track the subjects to be tracked.

[0041] Furthermore, the drive control may be performed in a way that allows the user to position the main subject in a compositional balance desired by the user, rather than being limited to capturing the main subject in the center of the screen.

[0042] In step S108, the first camera control unit 15 stops tracking the main subject if there is a main subject being tracked (tracking is in progress). More precisely, the first camera control unit 15 instructs the pan-tilt control unit 31, and the pan-tilt control unit 31 controls the pan-tilt mechanism 32 to stop the drive for tracking the main subject. For example, if the pan-tilt control unit 31 was performing pan-tilt control in the previous frame, the pan-tilt control for tracking the main subject that was being tracked in the previous frame is temporarily stopped. In this case, the pan-tilt mechanism 32 does not return to its initial position, but stops at the drive position at that time.

[0043] For example, if the first camera control unit 15 predicts that the main subject being tracked will move out of the distance range in which the second camera 200 can focus, it transitions from step S104 to step S108 and stops tracking the subject. This suppresses unnecessary pan-tilt movements.

[0044] The pan-tilt mechanism 32 is stopped because continuing pan-tilt control for tracking the main subject, which has been determined to be impossible to focus on, will not result in an in-focus image. The stopped pan-tilt mechanism 32 remains in place (stopped) until a new subject to be tracked is determined in the subsequent step S107, or until an instruction for initial position return drive (S112), described later, is given.

[0045] Furthermore, if the system transitions from step S106 to step S108, even if a subject is within the distance range that the second camera 200 can focus on, it will not be determined to be a subject to be tracked.

[0046] In step S108, the first camera control unit 15 further instructs the first camera control unit 15 to start saving images captured by the first camera 100 to the storage unit 16 if saving of images captured by the first camera 100 has not yet started. Saving images captured by the first camera 100 is useful for saving images of decisive moments in focus, even though they are not as high-resolution and have a wider angle than the second camera 200, considering that good images in focus cannot be obtained with the second camera 200.

[0047] Here, it is stated that if it is predicted in step S104 that the main subject will become impossible to focus on, the images captured by the first camera 100 are saved immediately, but this is not limited to this. For example, it may be performed only if the main subject has been successfully captured over a predetermined number of frames. Furthermore, the saving of images captured by the first camera 100 may continue after it has started until the subject being tracked moves out of the field of view of the second camera 200.

[0048] In step S109, the first camera control unit 15 determines whether there is a next candidate for the main subject, that is, whether there is a subject other than the subject that was determined to be impossible to focus on in step S104. If there is no next candidate, the first camera control unit 15 proceeds to step S111, and if there is a next candidate, it proceeds to step S110.

[0049] In step S110, the first camera control unit 15 newly sets the next candidate determined in step S109 as the main subject. If there are multiple next candidates, the first camera control unit 15 newly sets the secondary subject with the highest priority among them as the main subject. Once the setting is complete, the first camera control unit 15 returns to step S104.

[0050] If the system proceeds to step S107 after steps S110 and S114, a new target for tracking will be determined. Therefore, after the first camera control unit 15 stops tracking the target subject, if there is another subject among the recognized subjects that is within the distance range that the second camera 200 can focus on, the first camera control unit 15 will determine that other subject to be newly tracked.

[0051] In step S111, the first camera control unit 15 determines whether the state in which the subject to be tracked cannot be determined continues for a predetermined number of frames. This occurs when the second camera 200 fails to recognize a subject that can be focused on and is highly reliable, etc. If the state in which the subject to be tracked cannot be determined does not continue for a predetermined number of frames, the first camera control unit 15 proceeds to step S113; if the state in which the subject to be tracked cannot be determined continues for a predetermined number of frames, it proceeds to step S112.

[0052] In step S112, the first camera control unit 15 instructs the pan-tilt control unit 31 to start control (initial position return drive) to return the pan-tilt mechanism 32 to its initial position. At the point of transitioning to step S112, the pan-tilt mechanism 32 has already stopped because the state in which the subject to be tracked cannot be determined has continued. Therefore, the pan-tilt mechanism 32 is driven to the initial position, which is a predetermined pan-tilt position. The initial position is, for example, the position where the pan angle and tilt angle are zero (the position shown in Figure 1).

[0053] After steps S112 and S107, the first camera control unit 15 proceeds to step S113. In step S113, the first camera control unit 15 determines whether or not there has been an instruction to end shooting. If there is no instruction to end shooting, the first camera control unit 15 returns to step S101; if there is an instruction to end shooting, it terminates the process shown in Figure 3.

[0054] If the system returns to step S101, the sequence of operations from the shooting of the next frame onward is repeated. Then, when a new subject to be tracked is determined in step S107, the pan-tilt mechanism 32 for tracking is started to be driven. Therefore, if the first camera control unit 15 fails to determine a subject to be tracked for a predetermined number of frames, it returns the pan-tilt mechanism 32 to its initial position and keeps it waiting in the initial position until a subject to be tracked is determined.

[0055] In step S112, the pan-tilt mechanism 32 is returned to its initial position, but it may also be driven to a predetermined position where the next main subject is easily recognizable.

[0056] According to this embodiment, based on the distance measurement results of subjects recognized from images acquired by the first camera 100, only subjects that can be focused on by the second camera 200, or are predicted to be focused on, are determined to be tracked and tracked. Therefore, subjects that can be focused on even with a narrow field of view among subjects recognized in a wide-angle image can be tracked.

[0057] Furthermore, if it is predicted that the subject being tracked will move out of the distance range in which the second camera 200 can focus, image saving will begin with the first camera 100. Therefore, even in situations where it is difficult to successfully capture an image with the second camera 200, the decisive moment will not be missed.

[0058] Furthermore, in situations where a focused image of the main subject cannot be obtained or is expected not to be obtained, the tracking target can be changed to a new main subject from among the secondary subjects. This ensures that the second camera 200 can always capture a subject that is in focus.

[0059] The first camera 100 has one set of wide-angle optical system 11 and image sensor 13, but is not limited to this. For example, as shown in the modified example in Figure 4, the first camera 100 may be a stereo camera consisting of, for example, two wide-angle optical systems and two image sensors.

[0060] Figure 4 is a schematic block diagram of a modified imaging device 1000. The first camera 100 in the imaging device 1000 shown in Figure 4 includes an image sensor 13-2 and a wide-angle optical system 11-2 in addition to the configuration shown in Figure 2. The wide-angle optical system 11-2 includes a focusing mechanism 17-2.

[0061] If the first camera 100 is a stereo camera, the distance measurement result of the subject may be obtained by utilizing the parallax information of the two wide-angle images obtained by the image sensor 13 and the image sensor 13-2.

[0062] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention.

[0063] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above embodiments to a system or device via a network or a non-transient storage medium, and by having one or more processors in the computer of that system or device read and execute the program. The above program and the storage medium that stores the above program constitute the present invention. Furthermore, the present invention can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0064] This embodiment includes the following configuration. (Composition 1) A first imaging unit for acquiring wide-angle images, A second imaging unit that acquires an image with a field of view narrower than that of the first imaging unit but included within the field of view of the first imaging unit, A pan-tilt device that holds the first imaging unit and the second imaging unit so as to be rotatable in the pan direction and the tilt direction, An imaging device characterized by comprising: a control means that, based on the distance measurement result of a subject recognized from an image acquired by the first imaging unit, determines a subject that is within a distance range that can be focused by the second imaging unit from among the recognized subjects as a tracking target, and controls the pan-tilt device to track the subject that is the tracking target. (Configuration 2) The imaging device according to configuration 1, characterized in that the control means discontinues tracking the subject to be tracked if it is predicted, based on the history of the distance measurement results, that the subject to be tracked will move out of the distance range in which the second imaging unit can focus. (Composition 3) The imaging apparatus according to configuration 2, characterized in that, after discontinuing tracking of the subject to be tracked, if there is another subject among the recognized subjects within the distance range that can be focused by the second imaging unit, the control means determines that other subject to be tracked as a new subject and controls the pan-tilt device to track the subject to be tracked. (Composition 4) The imaging apparatus according to any one of configurations 1 to 3, characterized in that the control means starts saving the image acquired by the first imaging unit when it is predicted, based on the history of the distance measurement results, that the subject to be tracked will move out of the distance range that can be focused by the second imaging unit. (Composition 5) The imaging apparatus according to configuration 4, characterized in that the storage of the image acquired by the first imaging unit is continued until the subject being tracked moves out of the field of view of the second imaging unit. (Composition 6) The imaging device according to any one of configurations 1 to 5, wherein the control means obtains the difference between the distance measurement result of a subject recognized from an image acquired by the first imaging unit and the distance measurement result of a subject recognized from an image acquired by the second imaging unit, and if the difference exceeds a predetermined value, the imaging device does not determine that the subject is a tracking target even if it is within the distance range that the second imaging unit can focus on. (Composition 7) The imaging apparatus according to configuration 6, characterized in that, if the difference exceeds the predetermined value, the control means moves the focus position of the second imaging unit to the position indicated by the distance measurement result of the subject recognized from the image acquired by the first imaging unit, and then repeats the acquisition of the difference. (Composition 8) The imaging apparatus according to any one of configurations 1 to 7, characterized in that, if the control means stops tracking the subject to be tracked, it stops the pan-tilt device at that point and continues to stop the pan-tilt device until a new subject to be tracked is determined. (Composition 9) The imaging apparatus according to any one of configurations 1 to 8, characterized in that, if the control means fails to position the pan-tilt device at a predetermined location and the subject to be tracked cannot be determined for a predetermined number of frames, it returns the pan-tilt device to the predetermined location and keeps the pan-tilt device waiting at the predetermined location until the subject to be tracked is determined. (Composition 10) The first imaging unit is a stereo camera, The imaging apparatus according to any one of configurations 1 to 9, characterized in that the distance measurement result is obtained based on the disparity information of two images acquired by the first imaging unit. [Explanation of symbols]

[0065] 15 First camera control unit 25 Second camera control unit 31 Pan-Tilt Control Unit 100 First Camera 200 Second Camera 300 Pan-Tilt Device

Claims

1. A first imaging unit for acquiring wide-angle images, A second imaging unit that acquires an image with a field of view narrower than that of the first imaging unit but included within the field of view of the first imaging unit, A pan-tilt device that holds the first imaging unit and the second imaging unit so as to be rotatable in the pan direction and the tilt direction, An imaging device characterized by comprising: control means that, based on the distance measurement result of a subject recognized from an image acquired by the first imaging unit, determines a subject that is within a distance range that can be focused by the second imaging unit from among the recognized subjects as a tracking target, and controls the pan-tilt device to track the subject that is the tracking target.

2. The imaging apparatus according to claim 1, characterized in that the control means discontinues tracking the subject to be tracked if it is predicted, based on the history of the distance measurement results, that the subject to be tracked will move out of the distance range in which the second imaging unit can focus.

3. The imaging apparatus according to claim 2, characterized in that, after discontinuing tracking of the subject to be tracked, if there is another subject among the recognized subjects within the distance range that can be focused by the second imaging unit, the control means determines the other subject to be tracked as a new subject and controls the pan-tilt device to track the subject to be tracked.

4. The imaging apparatus according to claim 1, characterized in that the control means, based on the history of the distance measurement results, predicts that the subject to be tracked will move out of the distance range in which the second imaging unit can focus, and initiates saving of the image acquired by the first imaging unit.

5. The imaging apparatus according to claim 4, characterized in that the storage of the image acquired by the first imaging unit is continued until the subject being tracked moves out of the field of view of the second imaging unit.

6. The imaging device according to claim 1, wherein the control means obtains the difference between the distance measurement result of a subject recognized from an image acquired by the first imaging unit and the distance measurement result of a subject recognized from an image acquired by the second imaging unit, and if the difference exceeds a predetermined value, the imaging device does not determine that the subject is a tracking target even if it is within the distance range that the second imaging unit can focus on.

7. The imaging apparatus according to claim 6, characterized in that, if the difference exceeds the predetermined value, the control means moves the focus position of the second imaging unit to the position indicated by the distance measurement result of the subject recognized from the image acquired by the first imaging unit, and then repeats the acquisition of the difference.

8. The imaging apparatus according to claim 1, characterized in that, if the control means stops tracking the subject to be tracked, it stops the pan-tilt device at that point and continues to stop the pan-tilt device until a new subject to be tracked is determined.

9. The imaging apparatus according to claim 1, characterized in that if the control means returns the pan-tilt device to the predetermined position and the subject to be tracked cannot be determined for a predetermined number of frames, the pan-tilt device is returned to the predetermined position and the pan-tilt device is kept waiting at the predetermined position until the subject to be tracked is determined.

10. The first imaging unit is a stereo camera, The imaging apparatus according to claim 1, characterized in that the distance measurement result is obtained based on the disparity information of two images acquired by the first imaging unit.

Images