Control unit, control method for the same, program, and imaging apparatus

Abstract

To provide a control unit that can appropriately photograph a subject to be tracked according to a scene.SOLUTION: A control unit according to the present embodiment has: detection means that detects an area of a subject in a picked-up image; determination means that determines the target position of the subject on the image when tracking the detected subject; and control means that changes the angle of view at which the subject is photographed so as to set the position of the subject on the image to the target position. When the entirety of the subject is included in the picked-up image, the determination means determines the target position through first processing, and when the entirety of the subject is not included in the picked-up image, determines the target position through second processing different from the first processing.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present invention relates to a control device, a control method and a program thereof, and an imaging device.

Background Art

[0002] In recent years, in imaging devices such as digital still cameras, a technique is known in which an object in an image is recognized using object recognition technology, and the object is tracked so that the recognized object is at a predetermined position within the angle of view, thereby stabilizing framing. In Patent Document 1, a technique is disclosed in which a tracking correction amount of a correction lens for tracking an object is calculated so that the center-of-gravity position of a recognized object region becomes the center of the image, and the tracking correction amount is changed based on the likelihood of the recognized object. With the technique disclosed in Patent Document 1, it is possible to prevent malfunction of tracking and perform highly responsive tracking.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when photographing a moving object while panning, for example, it can be photographed in various compositions according to the photographer's shooting intention. In the above-described conventional technology, appropriately tracking an object in a shooting scene with different intentions has not been considered.

[0005] The present invention has been made in view of the above problems, and an object thereof is to realize a technology capable of appropriately photographing a subject to be tracked according to a scene.

Means for Solving the Problems

[0006] To solve this problem, for example, the control device of the present invention has the following configuration. That is, it has a detection means for detecting the region of the subject in the captured image, a determination means for determining the target position of the subject on the image when tracking the detected subject, and a control means for changing the shooting angle so that the position of the subject on the image becomes the target position. The determination means determines the target position by a first process when the entire subject is included in the captured image, and determines the target position by a second process different from the first process when the entire subject is not included in the captured image.

Advantages of the Invention

[0007] According to the present invention, it becomes possible to appropriately photograph the subject to be tracked according to the scene.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] In the following embodiments, a case where a lens interchangeable digital camera as an example of an imaging device is used will be described as an example. However, the present invention can be implemented with any electronic device having an imaging function. Such electronic devices include video cameras, computer devices (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game machines, robots, drones, and drive recorders. These are examples, and the present invention can be implemented with other electronic devices. Further, the control device according to the present embodiment is configured to include a part of the components of the imaging device. The control device may be configured by a part of the configuration of the main body of the imaging device, or may be configured by a part of the configuration of the interchangeable lens.

[0011] <Configuration of Digital Camera> Referring to FIG. 1, a configuration example of the digital camera 10 according to the present embodiment will be described. The digital camera 10 according to the present embodiment includes a main body 100 and a lens unit 200 detachably attached to the main body 100. The main body 100 and the lens unit 200 are connected through a mount that is detachably and mechanically engaged. The mount provided on the lens unit 200 and the mount provided on the main body 100 are provided with terminals configured to come into contact when the lens unit 200 is attached to the main body 100. The terminals include terminals for supplying power from the main body 100 to the lens unit 200, terminals for the camera-side communication unit 135 and the lens-side communication unit 128 to communicate, and the like.

[0012] The lens unit 200 has an imaging optical system (or imaging lens) 150 that generates an optical image of a subject on the imaging surface of the imaging element 106 included in the main body 100. The imaging optical system 150 includes a plurality of optical lenses including movable lenses and a diaphragm 104. For the sake of convenience, FIG. 1 shows only the zoom lens 101, the image stabilization lens 102, and the focus lens 103, which are movable lenses among the optical lenses. It should be noted that each of these movable lenses can actually be composed of a plurality of lenses.

[0013] The zoom lens 101 is driven in the optical axis direction by the zoom lens driving unit 124 to change the focal length (angle of view) of the imaging optical system 150. The image stabilization lens 102 is movable in a direction perpendicular to the optical axis by the correction lens driving unit 122 to optically correct (suppress) image blur caused by the movement of the digital camera 10. The focus lens 103 is driven in the optical axis direction by the focus lens driving unit 121 to change the distance at which the imaging optical system 150 is in focus. The aperture 104 has its aperture amount controlled by the aperture driving unit 120 to adjust the amount of light incident from the imaging optical system 150 to the main body 100.

[0014] The lens control unit 160 includes, for example, one or more processors capable of executing a program. The one or more processors may be at least any one of, for example, a CPU, an MPU, a microprocessor, etc. The lens control unit 160 controls the operation of the lens unit 200 by reading the program stored in the ROM 141 into the RAM 142 and executing it. The lens control unit 160 also controls the operation of the lens unit 200 or outputs information of the lens unit 200 to the camera control unit 115 in response to an instruction or request from the camera control unit 115.

[0015] In FIG. 1, the functional blocks described inside the lens control unit 160 schematically show various functions realized when the lens control unit 160 executes a program. Therefore, the operations of the functional blocks in the lens control unit 160 of the present embodiment are actually performed by the lens control unit 160. The lens-side motion detection unit 125, the lens-side anti-shake control unit 126, and the lens information management unit 129 in the lens control unit 160 will be described later.

[0016] The ROM 141 is a rewritable non-volatile memory and stores programs executed by the lens control unit 160, information and setting values of the lens unit 200, etc. The RAM 142 is used to read programs executed by the lens control unit 160 and temporarily store information of the lens unit 200. Note that at least one of the ROM 141 and the RAM 142 may be integrated with the lens control unit 160.

[0017] The shutter 105 is, for example, a mechanical focal plane shutter. Note that an electronic shutter may be used instead of or in addition to the shutter 105. The electronic shutter is a shutter operation realized by controlling the operation of the imaging device 106.

[0018] The imaging device 106 may be a known CCD or CMOS color image sensor having a color filter in a primary color Bayer array, for example. The imaging device 106 includes a pixel array in which a plurality of pixels are two-dimensionally arranged, and peripheral circuits for reading signals from each pixel. Each pixel accumulates charges corresponding to the amount of incident light by photoelectric conversion. By reading out a signal having a voltage corresponding to the amount of charges accumulated during the exposure period from each pixel, a group of pixel signals (analog image signals) representing the subject image formed on the imaging surface is obtained.

[0019] In addition, the imaging device 106 in the present embodiment is configured to be movable in a direction perpendicular to the optical axis of the imaging optical system 150. The position of the imaging device 106 is controlled by the sensor driving unit 130. Thus, the digital camera 10 according to the present embodiment has an optical image blur correction function in each of the lens unit 200 and the main body 100.

[0020] The AD converter 107 applies noise removal processing, gain adjustment processing, and AD conversion processing to the analog image signal read from the imaging device 106 to generate a digital image signal (image data). The AD converter 107 outputs the digital image data to the image processing circuit 109. Note that the AD converter 107 may be provided in the imaging device 106.

[0021] The timing generator 108 supplies signals for controlling the operation timing to the imaging device 106 and the AD converter 107 in accordance with instructions from the camera control unit 115.

[0022] The image processing circuit 109 applies predetermined image processing to the image data output by the AD converter 107 to generate signals and image data according to the application, or to acquire and generate various types of information. The image processing circuit 109 may be a dedicated hardware circuit such as an ASIC (Application Specific Integrated Circuit) designed to implement a specific function. Alternatively, the image processing circuit 109 may be configured such that a processor such as a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit) executes software to implement a specific function. The image processing circuit 109 outputs the acquired or generated information and data to the camera control unit 115, the RAM 110, etc. according to the application.

[0023] The image processing applied by the image processing circuit 109 may include, for example, preprocessing, color interpolation processing, correction processing, detection processing, data processing, evaluation value calculation processing, special effect processing, etc. Preprocessing may include signal amplification, reference level adjustment, defective pixel correction, etc. Color interpolation processing is performed when a color filter is provided in the imaging device, and is a process of interpolating the values of color components not included in the individual pixel data constituting the image data. Color interpolation processing is also called demosaicing processing. Correction processing may include processes such as white balance adjustment, tone correction, correction of image degradation caused by optical aberration of the imaging optical system 150 (image restoration), correction of the influence of peripheral light reduction of the imaging optical system 150, color correction, etc. Detection processing may include detection of a feature region (for example, a face region or a human body region) and its movement, person recognition processing, etc. Data processing may include processes such as region extraction (trimming), synthesis, scaling, encoding and decoding, header information generation (data file generation), etc. Generation of display image data (including image data for live view display) and recording image data is also included in data processing. Evaluation value calculation processing may include processes such as generation of signals and evaluation values used for autofocus detection (AF), generation of evaluation values used for automatic exposure control (AE), etc. Special effect processing may include processes such as addition of a blur effect, change of color tone, relighting, etc.

[0024] Note that these are examples of processes applicable to the image processing circuit 109 and do not limit the processes applied by the image processing circuit 109. Also, a part of the image processing enumerated here may be executed by a signal processing circuit incorporated in the imaging device 106, or may be executed by the camera control unit 115.

[0025] The camera control unit 115 includes, for example, one or more processors capable of executing a program. The one or more processors may be at least any one of, for example, a CPU, an MPU, a microprocessor, etc. The camera control unit 115 controls the operations of the main body 100 and the lens unit 200 and realizes the functions of the digital camera 10 by reading the program stored in the ROM 113 into the RAM 110 and executing it. The camera control unit 115 controls the operation of the lens unit 200 through communication with the lens control unit 160. In FIG. 1, the functional blocks described inside the camera control unit 115 schematically show various functions realized when the camera control unit 115 executes a program. Therefore, the operations of the functional blocks in the camera control unit 115 in this specification are actually performed by the camera control unit 115.

[0026] The ROM 113 is a rewritable non-volatile memory and stores programs executed by the camera control unit 115, various setting values of the digital camera 10, GUI data, etc. The RAM 110 is used to read the programs executed by the camera control unit 115, temporarily store data to be processed by the image processing circuit 109, intermediate data, processing result data, etc. Also, the RAM 110 is used as a buffer memory for image data or as a video memory for the display unit 111.

[0027] The display unit 111 is provided, for example, on the surface of the housing of the digital camera 10 and displays a live view image, information of the digital camera, information of the imaging scene, a menu screen, etc. The display unit 111 may be a touch display.

[0028] The operation unit 114 is a general term for input devices (buttons, switches, dials, etc.) provided for a photographer to input various instructions to the digital camera 10. The input devices constituting the operation unit 114 have names corresponding to the assigned functions. For example, the operation unit 114 includes a release switch, a video recording switch, a shooting mode selection dial for selecting a shooting mode, a menu button, arrow keys, a determination key, and the like. The release switch is a switch for still image recording, and the camera control unit 115 recognizes the half-pressed state of the release switch as an instruction for shooting preparation and the fully-pressed state as an instruction for starting shooting. Also, when the video recording switch is pressed in the shooting standby state, the camera control unit 115 recognizes it as an instruction for starting video recording, and when it is pressed during video recording, it recognizes it as an instruction for stopping recording. Note that the functions assigned to the same input device may be variable.

[0029] When the camera control unit 115 detects an operation on the operation unit 114, it executes an operation corresponding to the detected operation. For example, when the camera control unit 115 detects an operation for changing the angle of view of the lens unit 200, it periodically transmits an angle-of-view change instruction to the lens control unit 160 while the operation is being detected. Each time the zoom lens drive unit 124 of the lens control unit 160 receives the angle-of-view change instruction, it drives the zoom lens 101 by a certain amount.

[0030] The exposure control unit 138 determines exposure conditions (aperture value, shutter speed (or exposure time), imaging sensitivity) based on, for example, the evaluation value for AE generated by the image processing circuit 109 and the settings of the digital camera (e.g., imaging mode). When determining the exposure conditions for still image imaging, the exposure control unit 138 outputs the shutter speed to the shutter drive unit 136 and the aperture value to the lens control unit 160, respectively. Also, when the exposure control unit 138 determines the exposure conditions for video shooting, the exposure control unit 138 outputs the exposure time to the timing generator 108. Further, when changing the aperture value, the exposure control unit 138 outputs the aperture value to the lens control unit 160. The exposure control unit 138 outputs the imaging sensitivity to the imaging device 106 or the image processing circuit 109. The aperture drive unit 120 of the lens control unit 160 drives the aperture 104 according to the aperture value received from the camera control unit 115.

[0031] The focus lens control unit 139 determines the drive amount and drive direction for moving the focus lens 103 from the current position of the focus lens to the target position of the focus lens based on, for example, the evaluation value for AF generated by the image processing circuit 109. If the evaluation value for AF is the contrast evaluation value, the focus lens control unit 139 determines the drive amount and drive direction with the focus lens position where the evaluation value is maximized as the target position. Also, if the evaluation value for AF is the defocus amount, the focus lens control unit 139 converts the defocus amount into the drive amount and drive direction. The focus lens control unit 139 transmits the determined drive amount and drive direction to the lens control unit 160. The focus lens drive unit 121 of the lens control unit 160 drives the focus lens 103 according to the drive amount and drive direction received from the camera control unit 115.

[0032] Hereinafter, the functional blocks related to image blur correction will be described. The digital camera 10 performs both correction of image blur (camera shake) due to the movement of the main body 100 and the lens unit 200 and correction of image blur (subject blur) due to the movement of the subject during the exposure period.

[0033] The camera-side motion detection unit 134 detects the motion of the main body 100 and outputs a signal corresponding to the detected motion to the camera-side anti-shake control unit 133. The lens-side motion detection unit 125 detects the motion of the lens unit 200 and outputs a signal corresponding to the detected motion to the lens-side anti-shake control unit 126. The camera-side motion detection unit 134 and the lens-side motion detection unit 125 may be, for example, a combination of a gyro sensor that outputs a signal corresponding to the angular velocity around each axis of a rectangular coordinate system, and an acceleration sensor that outputs a signal corresponding to the acceleration in each axis direction of the rectangular coordinate system.

[0034] In this embodiment, the main body 100 and the lens unit 200 each have a separate motion detection unit, but a motion detection unit may be provided on either the main body 100 or the lens unit 200. For example, when there is no motion detection unit in the lens unit 200, the output signal of the motion detection unit of the main body 100 may be transmitted to both the camera-side anti-shake drive unit 133 and the lens-side anti-shake control unit 126.

[0035] The sensor position detection unit 132 detects the current position of the imaging element 106. The current position may be, for example, a relative displacement with respect to a reference position. The position where the center of the pixel array of the imaging element 106 is orthogonal to the optical axis can be set as the reference position. Note that the reference position may be set based on other known methods. Also, the displacement may be a displacement in two directions parallel to the short side and the long side of the imaging element 106.

[0036] The camera-side anti-shake control unit 133 determines a correction amount for moving the imaging element 106 so as to cancel out the motion detected by the camera-side motion detection unit 134. Note that the camera-side anti-shake control unit 133 determines the correction amount within a range that does not exceed the maximum movement amount in each direction with respect to the reference position. Then, the camera-side anti-shake control unit 133 corrects image blur by moving the imaging element 106 (changing the shooting angle of view) through the sensor drive unit 130 according to the determined correction amount.

[0037] The correction lens position detection unit 123 detects the current position of the image blur correction lens 102. The current position may be, for example, a relative displacement with respect to a reference position. The reference position can be the position when the optical axis is orthogonal to the center of the image blur correction lens 102. Note that the reference position may be set based on other known methods. Also, the displacement may be displacements in two directions parallel to the short side and the long side of the imaging device.

[0038] The lens-side anti-shake control unit 126 determines a correction amount for moving the image blur correction lens 102 so as to cancel out the movement detected by the lens-side movement detection unit 125. Note that the lens-side anti-shake control unit 126 determines the correction amount within a range not exceeding the maximum movement amount in each direction with respect to the reference position. Then, the lens-side anti-shake control unit 126 moves the image blur correction lens 102 (changes the shooting angle of view) through the correction lens driving unit 122 according to the determined correction amount, thereby correcting the image blur.

[0039] The image blur correction described here may be executed on either one of the main body 100 and the lens unit 200, or on both. When the image blur correction is executed on both the main body 100 and the lens unit 200, for example, the movement amount exceeding the maximum movement amount of one is compensated by the other. Alternatively, the required movement amount may be allocated to the main body 100 and the lens unit 200 at a predetermined ratio. Or, the blur to be canceled by the main body 100 and the lens unit 200 may be shared according to the components of the blur.

[0040] Instead of, or in addition to, moving the anti-shake member (here, one or both of the image blur correction lens 102 and the imaging device 106), electronic image blur correction can be executed. The electronic image blur correction can be realized by moving the position where the effective region is cut out from the captured image so as to cancel out the movement of the digital camera 10. For example, the movement amount of the imaging device 106 determined by the camera-side anti-shake control unit 133 can be converted into the movement amount on the pixel array, and the cut-out position can be changed (the shooting angle of view can be changed). In the present embodiment, the image composition processing unit 131 executes the conversion of the movement amount and the cut-out.

[0041] The motion vector detection unit 143 detects the motion between frames as a motion vector by, for example, the template matching method. The motion vector detection unit 143 can also be used to detect the motion of the digital camera 10, but in this embodiment, it is mainly used to detect subject blur. The method for detecting subject blur will be described later. When detecting the motion vector of the entire frame, electronic image blur correction can be executed by changing the cutout position based on the amount of movement represented by the motion vector.

[0042] The camera information management unit 137 is used to store information about the main body 100, such as the setting information of the main body 100, the current position of the imaging element 106, the driving limit (maximum movement amount for each direction), the subject blur correction amount, and information acquired by the main body 100.

[0043] In addition, the lens information management unit 129 is used to store information about the lens unit 200, such as the optical characteristics, current position, driving limit (maximum movement amount) of the image blur correction lens 102, and information acquired by the lens unit 200.

[0044] <Configuration related to image blur correction> With reference to FIG. 2, the configuration related to image blur correction in the lens unit 200 and the main body 100 will be further described. In FIG. 2, the functional blocks described inside the lens-side anti-vibration control unit 126 schematically show various functions realized by the lens control unit 160 executing a program. Therefore, the operations of the functional blocks in the lens-side anti-vibration control unit 126 described below are actually performed by the lens control unit 160. Similarly, the operations of the functional blocks described inside the camera-side anti-vibration drive unit 133 are actually performed by the camera control unit 115. The camera control unit 115 and the lens control unit 160 function as control means for performing image blur correction control in cooperation.

[0045] First, the configuration regarding image blur correction in the lens unit 200 will be described. The lens-side integrating unit 151 integrates the angular velocity signal output by the lens-side motion detection unit 125 and converts it into an angular signal.

[0046] The lens-side blur correction amount determination unit 152 determines a correction amount corresponding to the movement amount of the image blur correction lens 102 necessary to correct image blur based on the angular signal output by the lens-side integrating unit 151.

[0047] The adder 153 adds (reflects) the correction amount determined by the lens-side blur correction amount determination unit 152, the subject blur correction amount determined by the main body 100 and acquired by the subject blur correction amount acquisition unit 157, and the subject tracking amount determined by the main body 100 and acquired by the subject tracking amount acquisition unit 158. The methods for determining the subject blur correction amount and the subject tracking amount will be described later.

[0048] The lens-side ratio determination unit 154 determines the ratio of the movement amount borne by the lens unit 200 when the total movement amount required for correcting hand shake, subject blur, and subject tracking is set to 100%. When only one of the lens unit 200 and the main body 100 is used, the ratio determined by the lens-side ratio determination unit 154 is 100% or 0%.

[0049] Whether to perform image blur correction with both the lens unit 200 and the main body 100 or only with one of them can be determined according to arbitrary conditions. For example, it may be according to user settings, or it may be determined to perform with both the main body 100 and the lens unit 200 when the total movement amount exceeds the maximum movement amount of the image blur correction lens 102 or the imaging element 106. The lens-side ratio determination unit 154 can be determined based on the information stored in the camera information management unit 137 and the lens information management unit 129.

[0050] The lens-side drive range limiting unit 155 restricts the correction amount so that the image blur correction lens 102 moves within a range not exceeding the maximum movement amount. The correction lens controller unit 156 controls the position of the image blur correction lens 102 through the correction lens drive unit 122. The correction lens controller unit 156 performs feedback control based on the current position of the image blur correction lens 102 detected by the correction lens position detection unit 123 so that the image blur correction lens 102 moves to a target position corresponding to the movement amount.

[0051] Subsequently, the configuration related to image blur correction in the main body 100 will be described. The camera-side integration unit 161 integrates the angular velocity signal output by the camera-side motion detection unit 134 and converts it into an angle signal.

[0052] The camera-side blur correction amount determination unit 162 determines a correction amount corresponding to the movement amount of the imaging element 106 necessary for correcting image blur based on the angle signal output by the camera-side integration unit 161. The subject vector detection unit 163 detects a subject vector representing subject blur based on the motion vector between frames detected by the motion vector detection unit 143. Details of the subject vector detection operation will be described later. The subject blur correction amount determination unit 164 determines a correction amount for correcting subject blur based on the subject vector.

[0053] The subject reference position determination unit 169 determines whether the entire body of the subject is within the shooting angle of view based on the information of the object recognition unit 140 and determines the tracking detection position of the subject. Details of the operation of the subject reference position determination unit 169 will be described later. The subject target position determination unit 170 determines a position for tracking the subject at a predetermined position within the shooting angle of view based on the information of the subject reference position determination unit 169. Details of the operation of the subject target position determination unit will be described later.

[0054] The adder 171 adds the subject blur correction amount for subject blur correction determined by the subject blur correction amount determination unit 164 and the subject tracking amount for tracking the subject at an arbitrary predetermined position within the imaging angle of view determined by the subject target position determination unit 170. The adder 165 adds the correction amount for hand shake correction determined by the camera side blur correction amount determination unit 162, the subject blur correction amount added by the adder 171, and the subject tracking amount.

[0055] The camera side ratio determination unit 166 determines the ratio of the amount of movement borne by the main body 100 when the total amount of movement required for correction of hand shake, subject blur, and subject tracking is set to 100%. When only one of the lens unit 200 and the main body 100 is used, the ratio determined by the camera side ratio determination unit 166 is 100% or 0%. The camera side ratio determination unit 166 can be determined based on the information stored in the camera information management unit 137 and the lens information management unit 129.

[0056] The camera side drive range limit unit 167 limits the correction amount so that the imaging element 106 moves within a range not exceeding the maximum movement amount. The camera side controller unit 168 controls the position of the imaging element 106 through the sensor drive unit 130. The camera side controller unit 168 performs feedback control based on the current position of the imaging element 106 detected by the sensor position detection unit 132 so that the imaging element 106 moves to the target position according to the movement amount.

[0057] <Detection of Subject Vector> Here, with reference to FIG. 3, the detection of the subject vector in the subject vector detection unit 163 will be described. Note that the subject is an object existing in front of the background, and generally is a person or the like. Here, an area that moves between frames is detected as a subject area, and a vector representing the movement of the subject area is detected as a subject vector.

[0058] First, the motion vector detection unit 143 detects the motion vector between the target frame and the reference frame for each region set in the target frame. FIG. 3(A) shows an example of setting the region for detecting the motion vector in the target frame. Here, in order to reduce the processing load, the detection range 301 of the motion vector can be set in a part of the target frame. The motion vector detection unit 143 detects the motion vector for each region obtained by dividing the detection range 301. The detection of the motion vector can be executed, for example, by template matching that uses the image of each region as a template and detects the region with the highest correlation in the reference frame as the destination region.

[0059] FIG. 3(B) schematically shows the histogram of the magnitude of the motion vectors detected for each region within the detection range 301 shown in FIG. 3(A). The subject vector detection unit 163 detects the subject vector based on this histogram. The subject vector detection unit 163 converts the angle signal output by the camera-side integration unit 161 or the lens-side integration unit 151 into the amount of movement on the imaging surface. This conversion can be performed, for example, using the focal length of the imaging optical system 150, the imaging time difference or frame rate between the target frame and the reference frame, and the pixel pitch of the imaging device 106.

[0060] In the camera-side motion detection unit 134 or the lens-side motion detection unit 125, the origin of the output fluctuates due to disturbances such as temperature drift (offset phenomenon). Therefore, the subject vector detection unit 163 sets a predetermined range centered on the movement amount 302 as the background determination region 303. Then, the subject vector detection unit 163 determines that the motion vectors within the magnitude included in the background determination region 303 are background vectors 304. Also, the subject vector detection unit 163 determines that the motion vectors outside the magnitude included in the background determination region 303 are subject vectors 305. The subject vector 305 can be used for correcting subject blur, and the background vector 304 can be used for correcting camera shake.

[0061] In the above example, a method of separating the motion vector into the subject vector 305 and the background vector 304 based on the angular velocity signal (or the angle signal) has been described. However, other methods may also be used. For example, when the digital camera 10 or the subject has little movement, it becomes difficult to separate the subject vector and the background vector by the method based on the angular velocity or the angle.

[0062] For example, by using the subject distance information, it is possible to separate the motion vector into the subject vector and the background vector even when the digital camera 10 or the subject has little movement. The subject distance information may be, for example, a depth map indicating the subject distance for each region or each pixel obtained by dividing the imaging range. Here, as a simpler method, an example of separating the subject region and the background region based on the AF frame with the highest focus degree will be described. In FIG. 3(C), 306 indicates a plurality of AF frames (focus detection regions) that can be set by the digital camera. Further, 307 indicates the AF frame with the highest focus degree among the plurality of AF frames 306. In this case, the subject vector detection unit 163 sets the subject region 308 based on the subject distance of the AF frame 307, and sets the motion vector detected within the subject region 308 as the subject vector and the motion vector detected in other regions as the background vector. The subject vector detection unit 163 can set, for example, an AF frame whose difference in focus degree from the AF frame 307 is within a predetermined threshold range, or an AF frame within a predetermined threshold range (for example, ±2 depths) based on the depth of field of the AF frame 307 as the subject region 308.

[0063] Note that the above method using the angular velocity signal (or the angle signal) and the method using the subject distance information may be used in combination. In this case, the representative value (average value, mode value, etc.) of the same type of vectors determined by each method may be used. Further, the subject region may be detected based on machine learning or a known feature region detection technique, and the motion vector of the subject region may be detected as subject blur. FIG. 3(D) schematically shows a state in which a predetermined subject region 309 (here, a human face region) has been detected. The detection of subject blur can be performed using an evaluation image captured for live view display.

[0064] <Calculation of Movement Amount on Imaging Plane> FIG. 4 schematically shows a case where a subject moving in a direction parallel to the imaging plane is detected by comparing distance information for two consecutive times. For example, assuming that distance information is acquired at a cycle of 1 / 30 second, the time t required for the subject to move from position A to B is 33.3 ms.

[0065] In FIG. 4, assuming that the principal point (one) of the imaging optical system 150 is C, the triangle ABC with the distance between positions A and B (subject movement amount Δ) as one side is similar to the triangle abc with the movement amount δ of the subject on the imaging plane as one side. Also, the ratio between the triangle ABC and the triangle abc is L - f:f, where L is the subject distance and f is the focal length of the imaging optical system 150. Therefore, the movement amount δ of the subject on the imaging plane = the movement amount Δ of the subject × f / (L - f), and it can be converted to the movement amount on the imaging plane. This relationship holds even when the speed of the subject is different at points A and B.

[0066] The movement amount on the imaging plane can be converted to an angular velocity by, for example, treating it as a horizontal motion vector and performing the reverse procedure of converting the movement of the digital camera 10 to the movement amount on the imaging plane. Note that the method described here is merely an example, and the movement of the subject may be detected by other methods without using the imaging element 106.

[0067] <Example of Subject Tracking> Next, with reference to FIG. 5, an example of subject tracking according to this embodiment will be described. FIG. 5(A) schematically shows a shooting scene in which a moving subject 501 is being chased while panning. Since the photographer is not stably chasing the subject, it shows a state where local components (for example, the cockpit) within the subject 501 are about to frame out of the shooting angle of view. When the tracking detection position 502 within the subject 501 is made to follow the tracking target position 503 without considering the entire body of the subject with respect to the shooting angle of view and corrected so that the tracking detection position 502 becomes the tracking target position 503, the composition schematically shown in FIG. 5(B) is obtained. In this case, local components (for example, the cockpit) of the subject 501 that were about to frame out in FIG. 5(A) can be included within the shooting angle of view. However, another local component (for example, the tail fin) of the subject 501 that was within the shooting angle of view in FIG. 5(A) frames out of the shooting angle of view due to the correction in FIG. 5(B). In the shooting angle of view of FIG. 5(A), since the entire body of the subject 501 is shot under shooting conditions where it fits, the composition after the tracking control expected by the photographer is as shown in FIG. 5(C).

[0068] Therefore, in this embodiment, the subject target position determination unit 170 determines the tracking target position based not only on the subject 501 or local components that are part of the subject 501, but also taking into account the entire body of the subject.

[0069] Next, with reference to FIG. 6, an example of determining whether the entire body of the subject is within the shooting angle of view (included) will be described. Note that the determination of whether the entire body of the subject 601 is within the shooting angle of view is performed by the subject reference positioning unit 169. The determination can be made in various ways. In the example shown in FIG. 6(A), the entire body of the subject 601 is detected based on machine learning such as deep learning in the object recognition unit 140 or a known feature area detection technique. For example, by detecting the area of the subject 601 or local components 602 (e.g., cockpit, tail fin) that make up the subject 601 using a trained model trained by supervised learning, it can be determined whether the entire body of the subject 601 is within the shooting angle of view. FIG. 6(B) shows an example when using motion vectors. The subject reference positioning unit 169 determines whether the subject vector frame exists as one entity from the distribution of the vector frame 603 determined as the subject vector and the vector frame 604 determined as the background vector by the subject vector detection unit 163. And if the subject vector frame exists as one solid, it can be determined that the entire body of the subject 601 is within the shooting angle of view. Also, as described above, instead of the camera motion detection unit 134, a depth map, a LiDAR sensor, etc. may be utilized to detect the subject vector based on the subject depth and the subject distance to determine whether the entire body of the subject is within the shooting angle of view. Also, the camera motion detection unit 134 and the subject distance may be used in combination. As shown in FIG. 6(C), the outline (edge) 605 of the subject may be extracted to determine whether the subject 601 is within the shooting angle of view. For the detection of the outline of the subject, an image filter that performs differentiation such as a Laplacian filter may be used for the evaluation image. Also, the methods from FIG. 6(A) to FIG. 6(C) may be used in combination to determine whether the subject 601 is within the shooting angle of view.

[0070] Furthermore, with reference to FIG. 7, a method for determining the target position on the image for tracking the subject will be described. Specifically, when it is determined by the subject reference positioning unit 169 that the entire body of the subject is within the shooting angle of view, a method for the subject target position determination unit 170 to determine the target position for tracking the subject will be described.

[0071] FIG. 7(A) schematically shows a composition when the subject target positioning unit 170 controls tracking so that the entire body of the subject is at an equal position within the shooting angle of view. For example, regarding the horizontal direction of the evaluation image in FIG. 7(A), the subject target positioning unit 170 determines the X coordinate of the tracking target position at a position where the right margin part 703 and the left margin part 704 are equal. Subsequently, regarding the vertical direction of the evaluation image, the subject target positioning unit 170 determines the Y coordinate of the tracking target position at a position where the upper margin part 701 and the lower margin part 702 are equal. That is, FIG. 7(A) shows a composition in which tracking is controlled so that the entire body of the subject is at an equal position in the evaluation image.

[0072] FIG. 7(B) schematically shows a composition when the subject target positioning unit 170 determines the X coordinate of the tracking target position so that the traveling direction of the subject is evenly arranged within the shooting angle of view. The difference from FIG. 7(A) is, for example, in a scene of photographing an airplane taking off at a night airport, in order to create a composition that incorporates an element other than the subject, such as the approach lights on the runway, into a part of the example, a large margin is provided in either the upper or lower direction of the shooting angle of view. In this case, for the X coordinate of the tracking target position, the subject target positioning unit 170 determines the X coordinate of the tracking target position so that the right margin part 703 and the left margin part 704, which are the traveling direction of the subject, are equal, similar to FIG. 7(A). On the other hand, different from FIG. 7(A), for the Y coordinate of the tracking target position, the subject target positioning unit 170 determines it at a Y coordinate position shifted toward the lower margin part 702 of the subject. Also, the subject target positioning unit 170 may determine it at a Y coordinate position shifted toward the upper margin part 701 of the subject.

[0073] In the example shown in FIG. 7(C), the type of the subject is different from that in FIGS. 7(A) and 7(B), and a person is the target to be tracked. When photographing a person, generally, there is a wide blank area in the horizontal direction with respect to the photographing angle of view. In such a case, the subject target position determination unit 170 determines the Y coordinate of the tracking target position so that the positions in the vertical direction of the evaluation image are equal, and determines the X coordinate near the right blank area 703 or near the left blank area 704. In the case of the airplane in FIG. 7(B) and the person in FIG. 7(C), in the point where the coordinates for making the tracking target positions equal are different, the horizontal and vertical blank areas may be made equal in the direction where they are small according to the type of the subject.

[0074] Furthermore, with reference to FIG. 8, another method for the subject target position determination unit 170 to determine the target position for tracking the subject will be described. In the method shown in FIG. 8, when it is determined that the entire body of the subject does not fit within the photographing angle of view by the method described above with reference to FIG. 6, the target position for tracking the subject is determined.

[0075] FIG. 8(A) schematically shows a composition in which the photographer can stably track the subject at a telephoto focal length. Since the photographer is photographing at a photographing angle of view (focal length) where the entire body of the subject does not fit, there are an upper blank area 901 and a lower blank area 902, but there is no right blank area 903. On the other hand, the left blank area 904 has a margin of a predetermined value or more. When this state can be stably maintained in a predetermined number of or more frames, the tracking control is not performed. In this composition, the positions of the local components (for example, the cockpit) of the subject are stably maintained.

[0076] FIG. 8(B) schematically shows a composition in which the photographer cannot stably track the subject at a telephoto focal length. In this example, since the photographer lags the camera with respect to the subject, the subject gradually frames out in the direction of travel of the subject (the left side direction of the shooting angle of view). In the case of such a composition, the subject target positioning unit 170 performs tracking control in the direction opposite to the direction of travel of the subject so that a local component of the subject (for example, the cockpit) fits within the shooting angle of view. FIG. 8(C) is a composition in which the subject is driven to track a predetermined amount in the direction opposite to the subject travel direction (the left side direction of the shooting angle of view, i.e., the right side of the shooting angle of view) from FIG. 8(B). Although a local component of the subject (for example, the cockpit) fits within the shooting angle of view, since there is no margin in the left margin portion 904 and the subject is cut off, the subject tracking control is continuously performed. Note that the local components of the subject to be tracked may be determined in advance. For example, the cockpit of an airplane may be determined in advance as a local component to be tracked. FIG. 8(D) shows continuing to perform tracking control from FIG. 8(C) and continuing to track the subject while maintaining the point where the left margin portion 904 becomes a predetermined value (for example, 100 pixels).

[0077] Next, with reference to FIG. 9, a series of operations in image blur correction in the present embodiment will be described. The series of operations described here are realized by the camera control unit 115 expanding and executing a program stored in the ROM 113 or the like in the RAM 110, and each part in the camera control unit 115 (including each part of the camera-side anti-shake control unit 133) functioning. However, it may be realized by the lens control unit 160 executing a program stored in the ROM 141 or the like, or may be realized by the camera control unit 115 and the lens control unit 160 cooperating. Further, this process can be started, for example, when an image (evaluation image) captured by the image sensor 106 is read into the camera control unit 115. At this time, the image to be read is an image in which the photographer is shooting while panning at least a part of the subject which is a moving object.

[0078] In S901, the camera-side motion detection unit 134 (and / or the lens-side motion detection unit 125) detects the motion (camera shake) of the digital camera 10 as an angular velocity signal. In S902, the camera-side blur correction amount determination unit 162 (and / or the lens-side blur correction amount determination unit 152) determines a correction amount (camera shake correction amount) corresponding to the movement amounts of the imaging device 106 and the image blur correction lens 102 necessary to cancel out the motion detected in S901.

[0079] In S903, the subject vector detection unit 163 detects a subject vector representing subject blur in the evaluation image by any of the methods described above with reference to FIG. 3. For example, the subject vector detection unit 163 detects the subject vector based on the histogram of the magnitudes of the motion vectors detected for each region. In S904, the subject blur correction amount determination unit 164 converts the subject vector detected in S903 into the angular velocity of the subject. This conversion may be performed by reversing the procedure for converting the motion of the digital camera into the movement amount on the imaging surface, which was described with reference to FIG. 4. Note that the angular velocity of the subject is used to predict the motion of the subject within the frame. Therefore, the subject blur correction amount determination unit 164 stores, for example, the angular velocity of the subject obtained in a predetermined number of most recent frames in the RAM 110.

[0080] In S905, the camera control unit 115 determines whether the tracking control function is valid. The setting of the tracking control function may be, for example, a manual setting in which the photographer gives an instruction via the operation unit 114, or the camera control unit 115 may detect the tracking state and perform automatic setting. If the camera control unit 115 determines that the tracking control function is valid, the process proceeds to S906; otherwise, the process proceeds to S912.

[0081] In S906, the subject reference position determination unit 169 detects the entire body of the subject in the evaluation image by the method described above with reference to FIG. 6. The subject reference position determination unit 169 detects a part or the entire body of the subject based on, for example, machine learning such as deep learning in the object recognition unit 140 or known feature region detection techniques.

[0082] In S907, based on the result of S906, the subject reference positioning unit 169 determines whether the entire body of the subject is within the shooting angle of view (or included). If the subject reference positioning unit 169 determines that the entire body of the subject is within the shooting angle of view, the process proceeds to S908; otherwise, the process proceeds to S909.

[0083] In S908, when the entire body of the subject is within the shooting angle of view, the subject target positioning unit 170 determines the position of the subject (the coordinates of the target position for tracking the subject) such that the blank space within the frame (i.e., the distance from the end of the subject to the end of the angle of view) is equal (substantially equal). Note that, as described above with reference to FIGS. 7(B) and 7(C), the subject target positioning unit 170 may deliberately assume a composition including elements other than the subject and determine the coordinates where the distance of the blank space is equal for only one of the horizontal or vertical directions of the screen. For example, the subject target positioning unit 170 can determine the coordinates of the target position such that the distance of the blank space (the distance from the end of the subject to the end of the angle of view) is equal for the direction in which the subject has a large increase in height among the horizontal and vertical directions. In this embodiment, in order to have a composition where the subject is located at the center of the shooting angle of view, the distances of the blank spaces are made substantially equal. However, within the range where it can be regarded as a composition where the subject is located at the center of the shooting angle of view, there may be a difference in the distances of the blank spaces at both ends of the subject. For example, if it is within a distance corresponding to 5% of the effective area in the horizontal direction, a difference in the distances of the blank spaces at both ends of the subject may be regarded as substantially equal.

[0084] Also, the subject target positioning unit 170 can determine the coordinates of the target position such that the subject area approaches one of the angle-of-view ends for the direction in which the distance from the end of the subject to the end of the angle of view is large among the horizontal and vertical directions. Alternatively, the subject target positioning unit 170 can determine the coordinates of the target position such that the subject area approaches one of the angle-of-view ends for only the direction in which the subject has a small increase in height among the horizontal and vertical directions.

[0085] For example, when shooting with the composition shown in Fig. 7(B), the subject being photographed has the long side direction of the subject area substantially aligned with the horizontal direction of the angle of view. The distance from the end of the subject in the short side direction of the subject area to the end of the angle of view is greater than the distance from the end of the subject in the long side direction of the subject area to the end of the angle of view. The subject target position determination unit 170 makes the distances of the blank portions in the long side direction with respect to the subject area (for example, from the left end of the subject to the left end of the angle of view and from the right end of the subject to the right end of the angle of view) substantially equal. Then, the subject target position determination unit 170 makes one of the distances of the blank portions in the short side direction (for example, the vertical direction of the angle of view) with respect to the subject area (for example, from the upper end of the subject to the upper end of the angle of view and from the lower end of the subject to the lower end of the angle of view) smaller than the other. Of course, not limited to this method, the subject target position determination unit 170 can determine the coordinates of the target position so that the subject area is not biased in the horizontal direction of the angle of view and the subject area is biased in either direction in the vertical direction of the screen.

[0086] Also, when shooting with the composition shown in Fig. 7(C), the subject being photographed has the long side direction of the subject area substantially aligned with the vertical direction of the angle of view. The distance from the end of the subject in the short side direction of the subject area to the end of the angle of view is greater than the distance from the end of the subject in the long side direction of the subject area to the end of the angle of view. The subject target position determination unit 170 makes the distances of the blank portions in the long side direction with respect to the subject area (for example, from the upper end of the subject to the upper end of the angle of view and from the lower end of the subject to the lower end of the angle of view) substantially equal. Also, the subject target position determination unit 170 makes one of the distances of the blank portions in the short side direction (for example, the horizontal direction of the angle of view) with respect to the subject area (for example, from the left end of the subject to the left end of the angle of view and from the right end of the subject to the right end of the angle of view) smaller than the other. Also in this case, not limited to this method, the subject target position determination unit 170 can determine the coordinates of the target position so that the subject area is not biased in the vertical direction of the screen and the subject area is biased in either direction in the horizontal direction of the screen.

[0087] In S909, the subject reference position determination unit 169 determines whether local components of the subject are within the shooting angle of view. When the subject reference position determination unit 169 determines that local components of the subject are within the shooting angle of view, the process proceeds to S910; otherwise, the process proceeds to S911. This process is, for example, to consider a state where local components of the subject are shot to fill the shooting angle of view at a telephoto focal length. In such a state, it is assumed that the photographer can track the component and that the photographer cannot track the component (because the subject is likely to be out of the frame due to the telephoto focal length), and thus corresponding processing is performed.

[0088] For example, when a specific local component (e.g., the cockpit) is detected within the shooting angle of view by the object recognition unit 140, the subject reference position determination unit 169 can determine that local components of the subject are within the shooting angle of view. Also, when a specific local component is not detected within the shooting angle of view (even if a local component that is not the tracking target of the subject is detected within the shooting angle of view), the subject reference position determination unit 169 can determine that local components of the subject are not within the shooting angle of view.

[0089] In S910, the camera-side anti-shake control unit 133 does not perform tracking correction. This is because it is estimated that even in the shooting angle of view with a telephoto focal length, the photographer can stably track local components of the subject within the shooting angle of view, and thus assistance by tracking control is unnecessary. The camera-side anti-shake control unit 133 may be configured not to perform tracking correction when a YES determination is made in S909 for a predetermined number of frames.

[0090] In addition, when it is determined that a local component of the subject (e.g., the cockpit) is within the shooting angle of view, the subject target position determination unit 170 may further perform processing considering the blank space. For example, the subject target position determination unit 170 determines whether the distance of the blank space in the traveling direction of the subject (e.g., from the left end of the subject to the left end of the angle of view) is greater than a predetermined value (e.g., 0). This is to determine whether the subject is cut off even when a local component is included within the shooting angle of view. When the distance of the blank space (e.g., from the left end of the subject to the left end of the angle of view) is less than or equal to the predetermined value, the subject target position determination unit 170 may perform the same processing as in S911. On the other hand, when the distance of the blank space is greater than the predetermined value, tracking correction may not be performed.

[0091] In S911, the subject target position determination unit 170 performs tracking control until a local component of the subject frames into the shooting angle of view. This is because in the shooting angle of view with a telephoto focal length, it is presumed that the photographer cannot stably track a local component of the subject within the shooting angle of view, and by the processing of this step, a local component of the subject can be appropriately tracked. For example, the subject target position determination unit 170 changes the position of the subject in the direction opposite to the traveling direction of the subject so that the distance of the blank space in the traveling direction of the subject (e.g., from the left end of the subject to the left end of the angle of view) becomes equal to or greater than a predetermined threshold value.

[0092] In S912, the camera-side anti-shake control unit 133 determines an image blur correction amount based on the hand shake correction amount, the subject blur correction amount, and the coordinates of the target position for tracking the subject determined from S901 to S911. In S913, the camera-side anti-shake control unit 133 and / or the lens-side anti-shake control unit 126 drive the image sensor 106 and / or the image blur correction lens 102 as described above based on this correction amount and the correction amount for hand shake correction. When the camera control unit 115 finishes the processing of S913, it finishes a series of operations of this processing.

[0093] As described above, according to the present embodiment, it is determined whether the entire subject is included in the captured image. When the entire subject is included in the image, the target position for tracking is determined by the first process so that the distances of the margin portions of the subject are equalized within the target position in the image. Further, when the entire subject is not included in the image, the target position is determined by the second process so that a specific component of the subject is included in the image. That is, the target position for tracking the subject is determined so as to obtain an appropriate composition according to whether the entire subject is included in the image or not. Thereby, it becomes possible to perform tracking control that results in an appropriate composition expected by the photographer, and a high-quality captured image can be provided. That is, it becomes possible to appropriately photograph the subject to be tracked according to the scene. When it is determined in S907 that the entire body of the subject does not fit within the shooting angle of view, the shooting angle may be expanded (made wider) so that the entire body of the subject fits within the shooting angle of view, and then the process may proceed to S908. The method of expanding the shooting angle may be a method of driving the zoom lens 101 to change the focal length of the imaging optical system 150 or a method of changing the cutout size of the effective area. When such a process is performed, the entire body of the subject will fit within the shooting angle of view, but the size of the subject image within the effective area will automatically change. Therefore, when it is determined that the entire body of the subject does not fit within the shooting angle of view, the photographer may be able to select whether or not to expand the shooting angle using the operation unit 114. Further, when the photographer selects that the shooting angle may be expanded, the photographer may also be able to select how much the shooting angle may be expanded. When it is necessary to expand beyond the maximum value by which the shooting angle may be expanded to fit the entire body of the subject within the shooting angle of view, the process may proceed to S909.

[0094] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0095] (Disclosure of this specification) The disclosure of this specification includes the following control device, its control method, program, and imaging device. (Item 1) Detection means for detecting a subject area in a captured image, Determination means for determining a target position of the subject on the image when tracking the detected subject, Control means for changing an imaging angle so that the position of the subject on the image becomes the target position, and has, When the entire subject is included in the captured image, the determination means determines the target position by a first process, and when the entire subject is not included in the captured image, the determination means determines the target position by a second process different from the first process. A control device characterized by that. (Item 2) In the first process, the determination means determines the target position at which the distance from the first end of the imaging angle to the first end of the subject and the distance from the second end of the imaging angle to the second end of the subject are substantially equal for a first direction among the horizontal and vertical directions of the imaging angle. The control device according to Item 1, wherein (Item 3) The first direction is a direction in which the height increase of the subject is large among the horizontal and vertical directions of the imaging angle. The control device according to Item 2, wherein (Item 4) In the first process, the determination means determines the target position at which the subject approaches an end of any imaging angle for a second direction different from the first direction among the horizontal and vertical directions of the imaging angle. The control device according to Item 2 or 3, wherein (Item 5) In the second process, the determination means determines the target position so that at least a specific component of the subject is included in the captured image. The control device according to any one of Items 1 to 4, wherein (Item 6) In the second process, when the specific component of the subject is not included in the captured image, the determination means includes determining a target position for moving the subject in a direction opposite to the moving direction of the subject, the control device according to item 5, characterized in that. (Item 7) In the second process, when the specific component of the subject is included in the captured image and the distance from the end of the angle of view corresponding to the moving direction of the subject to the subject is equal to or less than a predetermined value, the determination means determines the target position so that the distance becomes greater than the predetermined value, the control device according to item 5 or 6, characterized in that. (Item 8) When the entire subject is not included in the captured image and a specific component of the subject is included in the captured image, the control means does not change the angle of view for tracking the subject, the control device according to any one of items 5 to 7, characterized in that. (Item 9) An imaging means for imaging an image; An imaging device, characterized by comprising the control device according to any one of items 1 to 8. (Item 10) A detection step of detecting a subject area in the captured image; A determination step of determining a target position of the subject on the image when tracking the detected subject; A control step of changing an imaging angle so that the position of the subject on the image becomes the target position, and having, In the determination step, when the entire subject is included in the captured image, the target position is determined by a first process, and when the entire subject is not included in the captured image, the target position is determined by a second process different from the first process, a control method of a control device, characterized in that. (Item 11) A program for causing a computer to function as each means of the control device according to any one of items 1 to 8.

[0096] The invention is not limited to the above embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, the claims are appended to disclose the scope of the invention.

Explanation of Signs

[0097] 100... Main body, 115... Camera control unit, 126... Lens side anti-vibration control unit, 133... Camera side anti-vibration control unit, 160... Lens control unit, 200... Lens unit

Claims

1. detection means for detecting a subject area in a captured image; determination means for determining a target position of the subject on the image when tracking the detected subject; control means for changing an imaging angle so that the position of the subject on the image becomes the target position, and having: The determination means determines the target position by a first process when the entire subject is included in the captured image, and determines the target position by a second process different from the first process when the entire subject is not included in the captured image. A control device characterized by that.

2. In the first process, the determination means determines, for a first direction among the horizontal direction and the vertical direction of the imaging angle, the distance from the first end of the imaging angle to the first end of the subject and the imaging angle. The control device according to claim 1, wherein the target position where the distance to the second end of the subject from the second end is substantially equal is determined.

3. The control device according to claim 2, wherein the first direction is a direction in which the height increase of the subject is large among the horizontal direction and the vertical direction of the imaging angle.

4. In the first process, the determination means determines the target position where the subject approaches an end of any imaging angle for a second direction different from the first direction among the horizontal direction and the vertical direction of the imaging angle. The control device according to claim 2, wherein

5. In the second process, the determination means determines the target position so that at least a specific component of the subject is included in the captured image. The control device according to claim 1, wherein

6. In the second process, the determination means includes determining the target position for moving the subject in a direction opposite to the traveling direction of the subject when a specific component of the subject is not included in the captured image. The control device according to claim 5, wherein

7. In the second process, the determination means includes the case where the specific component of the subject is included in the captured image and the distance from the end of the imaging angle corresponding to the traveling direction of the subject to the subject is a predetermined value or less. The control device according to claim 5, wherein the target position is determined so that the distance becomes larger than the predetermined value.

8. When the entire subject is not included in the captured image and a specific component of the subject is included in the captured image, the control means does not change the angle of view for tracking the subject. The control device according to claim 5, characterized in that.

9. An imaging means for imaging an image, An imaging device, comprising: the control device according to any one of claims 1 to 8.

10. A detection step of detecting a subject area in the captured image, A determination step of determining a target position of the subject on the image when tracking the detected subject, A control step of changing the angle of view for shooting so that the position of the subject on the image becomes the target position, and In the determination step, when the entire subject is included in the captured image, the target position is determined by a first process, and when the entire subject is not included in the captured image, the target position is determined by a second process different from the first process. A control method for a control device, characterized in that.

11. A program for causing a computer to function as each means of the control device according to any one of claims 1 to 8.

Images