Control device, imaging apparatus, and control method for imaging apparatus

Abstract

To provide an imaging apparatus capable of recording moving images with good visibility by setting a subject tracking range according to camera movements of a photographer.SOLUTION: The control device has subject information acquiring means for acquiring information on a subject detected in a captured image, tracking amount calculating means for calculating a tracking amount on the basis of a position of the subject in a captured image and a target position, tracking means for controlling subject tracking to bring the position of the subject in the image closer to the target position on the basis of the tracking amount, and region setting means for setting a first region and a second region on the basis of a shooting state of an imaging apparatus for capturing an image. The first region has a lower degree of subject tracking than the second region.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to stabilization of a subject image using an image blur correction means. [Background technology]

[0002] There is a function to stabilize camera shake in moving images by using an imaging device equipped with an image stabilization means. This function is performed by driving the image stabilization unit to cancel camera shake according to the camera shake signal detected by the detection means, or by changing the position of the area to be cut out from the imaging area by image processing. The former is called optical camera shake correction, and the latter is called electronic camera shake correction.

[0003] However, when recording a moving image using an imaging device equipped with the above-mentioned image stabilization means, the subject may go out of frame even if camera shake is corrected. This is because the movement of the subject itself cannot be corrected even if camera shake, which is the movement of the imaging device, is corrected. Therefore, in order to prevent a moving subject from going out of frame, the photographer needs to frame the shot while paying attention to the movement of the subject.

[0004] To address the above-mentioned problem, the image stabilization device described in Patent Document 1 proposes determining whether to give priority to subject tracking or image stabilization depending on the shooting conditions.

[0005] With the above-described configuration, it becomes possible to move the image blur correction means in accordance with the movement of the subject, so that it is possible to achieve both subject tracking and camera shake correction. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2017-215350 Summary of the Invention [Problem to be solved by the invention]

[0007] However, through investigations by the inventors of the present invention, it has become clear that, depending on the shooting conditions, it may not be possible to properly track a subject.

[0008] SUMMARY OF THE PRESENT EMBODIMENTS Accordingly, an object of the present invention is to provide a control device and an imaging device that are capable of tracking a subject more appropriately than ever before. [Means for solving the problem]

[0009] A control device according to one aspect of the present invention includes a subject information acquisition means for acquiring information of a subject detected from a captured image, a tracking amount calculation means for calculating a tracking amount based on a position of the subject in the captured image and a target position, a tracking control means for controlling subject tracking to bring the position of the subject in the image closer to the target position based on the tracking amount, and an area setting means for setting a first area and a second area based on a shooting state of an imaging device that captures the captured image, and the first area is characterized in that the degree of subject tracking is lower than that of the second area. Other aspects of the present invention will be described in the description of the embodiment of the invention. Effect of the Invention

[0010] According to the present invention, it is possible to provide a control device and an imaging device that are capable of tracking a subject more appropriately than ever before. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing an example of the arrangement of an imaging apparatus according to a first embodiment; [Diagram 2] FIG. 1 is a block diagram showing an example of a configuration related to image blur correction control and subject tracking control in a first embodiment. [Diagram 3] Flowchart relating to subject tracking according to the first embodiment [Figure 4] Graph of angular velocity signal while the photographer is walking [Diagram 5] FIG. 1 is a diagram showing an example of a tracking area and a tracking amount; [Figure 6] Table relating to setting of tracking area according to the second and third embodiments [Figure 7] FIG. 13 is a diagram showing an example of setting a tracking area according to the third embodiment; [Figure 8] FIG. 13 is a diagram showing an example of setting a tracking area according to the fourth embodiment; [Figure 9] Time series graph of subject detection position and tracking amount DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. However, the following embodiments do not limit the scope of the invention as claimed.

[0013] Although a plurality of features are described in the embodiment, not all of these features are essential to the invention, and the plurality of features may be combined in any desired manner.

[0014] Furthermore, in the accompanying drawings, the same reference numbers are used for the same or similar components, and duplicate explanations are omitted.

[0015] [First embodiment] In this embodiment, an image capture device will be described that uses a shake detection means to determine the state in which the image capture device is being held by a photographer (camera work determination), and sets a subject tracking area based on the result.

[0016] First, referring to FIG. 9, a description will be given of a situation in which subject tracking may result in a video that looks unnatural, depending on the shooting conditions.

[0017] Fig. 9 shows a time series graph of the subject detection position and subject tracking amount (hereinafter referred to as tracking amount) when neither the subject nor the background are stationary. In Fig. 9, the vertical axis shows angle, and the horizontal axis shows time. L901 shown by a dotted line shows the subject detection position, and the further away from the axis it is, the further away it is from the target position. On the other hand, L902 shown by a solid line shows the subject tracking amount calculated from the subject detection position L901, and the further away from the axis it is, the larger the tracking amount is, in other words, the larger the amount by which the imaging range is changed by subject tracking control.

[0018] As shown in FIG. 9, a delay time Td often occurs during the calculation of the object tracking amount. This delay time Td occurs, for example, because the output of the object detection unit varies, and filter processing is required. If a delay time occurs during the calculation of the tracking amount, the object tracking amount does not become 0 even when the object detection position coincides with the target position, or conversely, the object tracking amount becomes 0 even when object tracking is required. This may result in a video in which the object and the background do not stop properly.

[0019] In particular, in a situation where the photographer is holding the camera firmly, the above problem is easily visible because the fluctuation of the imaging range due to camera shake is small, and the photographer and viewers of the video may feel unnatural. In addition, when the photographer is holding the camera firmly, the scene is one in which the photographer can frame the scene, and the probability of the subject being out of the frame is considered to be low. Therefore, in such a scene, the unnaturalness is more likely to stand out than the effect of subject tracking. Therefore, in this embodiment, a tracking area, which is a range in which the subject is tracked, is set based on the holding state of the imaging device. In this specification, the imaging range refers to the range of the image that is captured and recorded. In other words, when electronic camera shake correction or crop shooting is performed, it refers to the range of the image after clipping.

[0020] 1 is a block diagram showing the configuration of an image capture device according to this embodiment. The image capture device according to this embodiment is a lens-interchangeable image capture device, and is made up of an image capture device body (hereinafter, camera body) 1 and a lens device 2 that can be attached to the camera body.

[0021] The lens device 2 includes a photographing optical system 200. The photographing optical system 200 includes a zoom lens 101, an image blur correction lens 102, a focus lens 103, and an aperture 104.

[0022] The zoom lens 101 moves in the direction of the optical axis to optically change the focal length of the photographing optical system (photographing lens) 200 that forms a subject image, thereby changing the photographing angle of view. The image blur correction lens 102 moves in a direction perpendicular to the optical axis to optically correct image blur caused by shaking of the imaging device. The focus lens 103 moves in the direction of the optical axis to optically adjust the focus position. The aperture 104 and shutter 105 can adjust the amount of light by opening and closing them, and are used for exposure control.

[0023] The aperture drive unit 120 and the shutter drive unit 135 drive the aperture 104 and the shutter 105. The zoom lens drive unit 124 drives the zoom lens 101 to change the angle of view. The zoom lens control unit 127 controls the position of the zoom lens 101 in accordance with a zoom operation instruction from the operation unit 114. The zoom lens 101 may be moved by operating a zoom ring provided around the lens 2. The focus lens drive unit 121 drives the focus lens 103. Light that has passed through the photographing optical system 200 is received by an image sensor 106 that uses a CCD (charge-coupled device) or a CMOS sensor (complementary metal-oxide semiconductor) or the like, and the optical signal is converted into an electrical signal.

[0024] The AD converter 107 performs noise removal processing, gain adjustment processing, and AD conversion processing on the image signal read out from the image sensor 106 .

[0025] The timing generator 108 controls the drive timing of the image sensor 106 and the output timing of the AD converter 107 according to instructions from the camera control unit 115. The image processing circuit 109 performs pixel interpolation processing, color conversion processing, and the like on the output from the AD converter 107, and then sends the processed image data to an internal memory 110. The image processing circuit 109 includes a position alignment circuit for a plurality of continuously captured images, a geometric transformation circuit for performing cylindrical coordinate transformation and lens group distortion correction, a synthesis circuit for performing trimming and synthesis processing, and the like. Furthermore, electronic image stabilization is performed using a projective transformation circuit provided in the image processing circuit 109. The operation of each circuit is well known, so a detailed description will be omitted.

[0026] The display unit 111 displays image data stored in the internal memory 110 as well as shooting information and the like.

[0027] A compression / decompression processing unit 112 performs compression or decompression processing on the data stored in the internal memory 110 in accordance with the image format.

[0028] The storage memory 113 stores various data such as parameters.

[0029] The operation unit 114 is a user interface for the user to perform various imaging operations, menu operations, and mode switching operations.

[0030] The camera control unit 115 is composed of a calculation device such as a CPU (Central Processing Unit), and executes various control programs stored in the internal memory 110 in response to a user's operation via the operation unit 114. The control programs are programs for performing, for example, zoom control, image blur correction control, automatic exposure control, automatic focus adjustment control, and processing for detecting the face of a subject.

[0031] In the case of a lens-interchangeable camera, information is transmitted between the camera and the lens via a camera side communication unit 140 and a lens side communication unit 128 .

[0032] A luminance signal detection unit 137 detects a signal that is read out from the image sensor 106 in a shooting preparation state (a so-called live view state) and passes through an AD converter 107 as the luminance of the subject and the scene.

[0033] The exposure control unit 136 calculates the exposure value (aperture value and shutter speed) based on the luminance information obtained by the luminance signal detection unit 137, and notifies the calculation result to the aperture driving unit 120 and the shutter driving unit 135 via the camera side communication unit 140 and the lens side communication unit 128. The exposure control unit 136 also simultaneously controls the amplification of the imaging signal read out from the imaging element 106. This performs automatic exposure control (AE control).

[0034] The evaluation value calculation unit 138 extracts a specific frequency component from the luminance information obtained by the luminance signal detection unit 137, and then calculates a contrast evaluation value based on the extracted frequency component.

[0035] Focus lens control unit 139 issues a command to focus lens driving unit 121 via camera side communication unit 140 and lens side communication unit 128 to drive focus lens 103 by a predetermined drive amount over a predetermined range. At the same time, it acquires an evaluation value that is a calculation result of evaluation value calculation unit 138 at each focus lens position. As a result, it calculates a focus position in the contrast AF method from the focus lens position where the change curve of the contrast evaluation value reaches its peak, and transmits it to focus lens driving unit 121. Having received the focus position, focus lens driving unit 121 drives the focus lens, thereby performing automatic focusing control (AF control) in which a light beam is focused on the surface of image sensor 106.

[0036] Although the contrast AF method has been described here, the AF method is not particularly limited, and may be, for example, a phase difference AF method. The details of the phase difference AF method are well known, so a description thereof will be omitted.

[0037] The lens side shake detector 125 and the camera side shake detector 134 detect shake and vibration applied to the imaging device. In this embodiment, the shake detectors are arranged on both the camera side and the lens side.

[0038] The lens-side image stabilization control unit 126 calculates an image stabilization amount for suppressing shake using an image stabilization lens, based on a shake detection signal detected by the lens-side shake detection unit 125, the camera-side shake detection unit 134, or both. Then, based on the calculated image stabilization amount and the position of the image stabilization lens 102 detected by the image stabilization lens position detection unit 123, a drive signal for the image stabilization lens is sent to an image stabilization lens 122 drive unit, thereby controlling image stabilization using the image stabilization lens.

[0039] The image stabilization lens driving unit 122 is an actuator constituted by a voice coil motor or the like, and drives (displaces) the image stabilization lens 102 in a direction perpendicular to the optical axis based on a drive signal for the image stabilization lens received from a lens-side image stabilization control unit 126. A method of controlling image stabilization using the image stabilization lens 102 will be described in detail later.

[0040] The camera-side image stabilization control unit 133 can communicate with the lens anti-shake control unit 126 via the camera communication unit 140 and the lens communication unit 128 in the interchangeable lens 200. The camera-side image stabilization control unit 133 calculates an image stabilization amount for suppressing shake using the image sensor, based on a shake detection signal detected by the camera-side shake detection unit 134, the lens-side shake detection unit 125, or both. Then, based on the calculated image stabilization amount and the position of the image sensor 106 detected by the image sensor position detection unit 132, a drive signal for the image sensor is transmitted to the image sensor drive unit 130, thereby controlling image stabilization by the image sensor.

[0041] The image sensor driver 130 is an actuator formed of a voice coil motor, an ultrasonic motor, or the like, and drives (displaces) the image sensor 106 in a direction perpendicular to the optical axis based on an image sensor drive signal received from a camera-side image stabilization controller 133. A method of controlling image stabilization using the image sensor 106 will be described in detail later.

[0042] The motion vector detection unit 131 uses a block matching method to calculate the correlation value between the current frame and the previous frame for each block that the frame is divided into. It then searches for the block in the previous frame that minimizes the calculation result, and detects the deviation of other blocks relative to that block as a motion vector.

[0043] The subject detection unit 141 generates subject detection information by detecting an image area of ​​a subject included in a captured image based on a captured image signal output from the image sensor 106. The subject detection information includes information on the position of the subject, and may also include information on the type of subject (e.g., person, animal, vehicle), part (e.g., pupil, face, body), size, etc.

[0044] The subject setting unit 143 sets a specific subject in the captured image. The photographer can set any subject from among multiple subjects as the tracking target subject via the operation unit 114 by touching or operating a button. The tracking target subject may be determined by an automatic subject setting program of the camera without the photographer operating any member. In addition, when there is only one subject in the captured image, that subject is set as the tracking target subject.

[0045] The subject tracking calculation unit 142 calculates the amount of tracking of the subject, which will be described in detail later with reference to FIG.

[0046] 2 is a block diagram showing an example of the configuration of a mechanism related to image stabilization control and subject tracking control in this embodiment. In this embodiment, image stabilization control is performed by the camera-side image stabilization control unit 133 and the lens-side image stabilization control unit 126 controlling the positions of the image stabilization lens 102 and the image sensor 106. Then, subject tracking control is performed by the tracking calculation unit 142 controlling the image processing circuit 109.

[0047] First, a description will be given of the configuration of the camera-side camera shake correction control unit 133. As described above, the camera shake correction control unit 133 can drive the image sensor 106 to perform camera shake correction (image blur correction) using the image sensor.

[0048] When the camera-side image stabilization control unit 133 acquires a shake angular velocity signal detected by the camera shake detection unit 134 from the camera shake detection unit 134, the camera-side image stabilization control unit 133 converts the shake angular velocity signal into a shake angle signal by performing integration processing using a camera integration / LPF unit 1331. Here, an integral low-pass filter is used for the camera integration / LPF unit 1331 (hereinafter referred to as an integral LPF).

[0049] The shake correction amount calculation unit 1332 calculates the correction amount to cancel the shake angle, taking into account the frequency band of the shake angle and the drivable range of the camera side. Specifically, the shake correction amount is calculated by integrating the gain related to the zoom magnification and the subject distance with respect to the shake angle signal.

[0050] A correction ratio calculation unit 1333 calculates the correction ratio to be performed by the camera side when the sum of the shake correction amounts on the camera side and the lens side is 100%. In this embodiment, the correction ratio is determined based on the respective movable ranges of the image sensor 106 and the image stabilization lens 102. In addition to the movable range of the correction members described above, the correction ratio may also be determined taking into consideration the movable range for correction by clipping in image processing (electronic image stabilization).

[0051] A correction ratio accumulating unit 1334 multiplies the calculation result in the correction ratio calculating unit 1333 by the shake correction amount, and calculates the image shake correction amount on the camera side based on the correction ratio.

[0052] The position control unit 1335 is a control unit for performing PID control (ratio control, integral control, and fine control) on the deviation between the target position of the image sensor 106 based on the camera-side shake correction amount calculated by the correction ratio accumulating unit 1334 and the current position of the image sensor 106. The deviation between the target position and the current position is converted into an image sensor drive signal and input to the image sensor drive unit 130. The current position is the output result of the image sensor position detection unit 132. Since PID control is a common technique, a detailed description will be omitted. The image sensor drive unit 130 drives the image sensor 106 in response to the image shake correction drive signal.

[0053] Next, the lens-side image stabilization control unit 126 will be described. As described above, the lens-side image stabilization control unit 126 can perform image stabilization (image blur correction) using the image stabilization lens by driving the image stabilization lens 102. When the lens-side image stabilization control unit 133 acquires from the lens shake detection unit 125 a shake angular velocity signal detected by the lens shake detection unit 125, it converts the signal into a shake angle signal by performing integration processing using the lens integration LPF 1261. Here, an integration LPF is used for the lens integration unit 1261.

[0054] The shake correction amount calculation unit 1262 calculates the correction amount to cancel the shake angle, taking into account the frequency band of the shake angle and the drivable range on the camera side. Specifically, the shake correction amount on the lens side is calculated by integrating the gain related to the zoom magnification and the subject distance with respect to the shake angle.

[0055] The correction ratio accumulator 1263 obtains a correction amount based on the correction ratio by multiplying the correction ratio provided by the lens side when the sum of the shake correction amounts on the camera side and the lens side is taken as 100%. In this embodiment, the correction ratio provided by the lens side is obtained from the calculation result by the division ratio calculator 1333 on the camera side. The correction ratio provided by the camera side or the lens side is notified via the communication units of the camera side communication unit 140 and the lens side communication unit 128.

[0056] The position control unit 1264 is a control unit for performing PID control (ratio control, integral control, fine control) on the deviation between the target position of the image blur correction lens 102 based on the shake correction amount on the lens side calculated by the correction ratio integrating unit 1263 and the current position of the image blur correction lens 102. The deviation between the target position and the current position is converted into an image blur correction lens drive signal and input to the image blur correction lens drive unit 122. The current position is the output result of the image blur correction lens position detection unit 123. Since PID control is a common technique, a detailed description will be omitted. The image blur correction lens drive unit 122 drives the image blur correction lens 102 in accordance with the image blur correction drive signal.

[0057] By driving the image blur correction lens 102 and the image sensor 106 in the above manner, image blur caused by camera shake can be reduced.

[0058] Furthermore, in this embodiment, subject tracking calculation unit 142 can change the image cut-out position like electronic camera shake correction, based on subject detection information acquired from subject detection unit 141. As described above, subject setting unit 143 can set any subject in the captured image as a subject to be tracked (hereinafter, sometimes referred to as a main subject). Subject detection unit 141 obtains information such as position information, size, and subject type of the main subject set by subject setting unit 143.

[0059] The determination unit 1422 determines the holding state of the imaging device based on the output signal of the camera-side shake detection unit 134. In this embodiment, the holding state of the imaging device refers to camera work such as a state in which the photographer is taking pictures while walking, a state in which the photographer is taking pictures while panning or tilting, or a state in which the photographer is taking pictures while holding a steady position. Hereinafter, the state in which the photographer is taking pictures while walking may be referred to as the walking state. Hereinafter, the state in which the photographer is taking pictures while panning or tilting may be referred to as the panning state. Details of the determination flow will be described later.

[0060] The tracking area determination unit 1421 determines an area in which the subject is not tracked in the imaging area (hereinafter, sometimes referred to as a dead zone) and an area in which the subject is tracked (hereinafter, sometimes referred to as a tracking area). In this embodiment, the tracking area is determined based on the determination result by the determination unit 1422 and a target position set by the subject target position setting unit 1424 described later. In addition, by setting a dead zone, the remaining area may be set as the tracking area, or conversely, by setting a tracking area, the remaining area (inside the tracking area as viewed from the target position) may be set as the dead zone. Details of the flow for determining the tracking area will be described later.

[0061] The subject target position setting unit 1424 sets a target position in the image of the main subject set by the subject setting unit 143. In this embodiment, the subject target position can be changed by camera settings. The target position can be the center of the imaging range (recorded image), a position specified by the user, a coordinate position stored in advance, or the like. In addition, the position of the subject at the timing when the tracking function of the subject is turned on can be set as the target position. The user can specify the target position by displaying a live view image or a recorded video on a touch panel and touching a location on the touch panel that the user wants to set as the target position. Note that it is not essential that the subject target position can be changed, and the target position can be always fixed. In this case, the subject target position setting unit 1424 can always output the same target position to the tracking region determination unit 1421, or the subject target position setting unit 1424 itself can be eliminated. In this embodiment, for simplicity, the center of the imaging range is set as the subject target position.

[0062] The tracking amount calculation unit 1423 calculates the tracking amount based on the subject target position set by the subject target position setting unit 1424, the current position of the main subject in the image detected by the subject detection unit 141, and the tracking area determined by the tracking area determination unit 1421.

[0063] The image processing circuit 109 processes the image using the tracking amount calculated by the angle of view correction amount calculation unit 1423 as an input, and in this embodiment, performs a geometric conversion process similar to electronic image stabilization. In this manner, the subject tracking process is performed, and the image that has been subjected to the subject tracking process is recorded in the storage memory 113 or displayed on the display unit 111.

[0064] Fig. 3 is a flowchart of the subject tracking process. Fig. 3(a) is a flowchart of the entire subject tracking process, and Fig. 3(b) is a flowchart of the process of determining the holding state of the imaging device and setting the tracking area of ​​the subject. These processes are mainly performed by the subject tracking calculation unit 142. The flowchart shown in Fig. 3 will be described in detail below.

[0065] The overall flow of the subject tracking process will be described with reference to Fig. 3(a). When the subject tracking process is started, the imaging device 1 of this embodiment first sets a target position by the subject target position setting unit 1424 in step S201. Here, the target position is set to the center of the imaging range. Note that in the case where the target position is not changeable, this step is omitted.

[0066] Next, the imaging device 1 sets a dead zone and a subject tracking area in step S202. In this embodiment, as described above, the tracking area determination unit 1421 determines the dead zone and the tracking area based on the holding state of the imaging device 1 and the target position set in step S201. Details of this process will be described in detail with reference to FIG. 3(b).

[0067] Next, imaging device 1 calculates the tracking amount in step S203. In this embodiment, tracking amount calculation unit 1423 calculates the tracking amount based on the difference between the subject target position set by subject target position setting unit 1424 and the current position of the main subject in the image detected by subject detection unit 141. At this time, if the current position of the main subject is within the dead zone, tracking amount calculation unit 1423 sets the tracking amount to a fixed value regardless of the difference between the target position and the main subject position. For example, when the tracking amount is 0 and the position of the main subject is within the dead zone, the tracking amount is set to 0, and when the tracking amount is a predetermined amount and the position of the main subject is within the dead zone, the tracking amount at that time is maintained.

[0068] Next, the imaging device 1 performs a tracking process in step S204. In this embodiment, the tracking amount calculation unit 1423 outputs the tracking amount to the image processing circuit 109, and the image processing circuit 109 performs a geometric transformation based on the tracking amount to bring the position of the main subject in the recorded image closer to the target position. The subject tracking process can be performed by repeatedly performing a series of processes from steps S201 to S204 for each frame. Note that the target position setting process in step S201 and the setting of the dead zone and the tracking area in step S202 do not have to be performed for every frame. For example, the target position setting process in step S201 may be omitted until the photographer inputs an operation to change the target position. Also, the setting of the dead zone and the subject tracking area in step S202 may be performed at regular intervals. For example, when the holding state is a walking shooting state, even if the photographer suddenly stops, there is a time lag until the photographer can hold the imaging device firmly, and step S202 may be performed only once every few frames. Also, after the first step S202 is completed, the process proceeds to step S203, and thereafter, the tracking area setting process in step S202 may be performed in parallel with the tracking amount calculation and tracking control in steps S203 and S204. Only when the size and position of the tracking area are changed by the tracking area setting process, information on the dead zone and tracking area used in the tracking amount calculation process in step S203 may be updated.

[0069] The process of determining the holding state (camera work) of the imaging device and setting the tracking area of ​​the subject will be described with reference to Fig. 3(b). In this embodiment, the shake applied to the imaging device is acquired from the camera shake detection unit 132, and the holding state is determined based on this. Note that these processes are mainly performed by the holding state determination unit 1422 and the tracking area determination unit 1421.

[0070] First, in step S301, the held state determination unit 1422 acquires a detection result from the camera-side shake detection unit 134, and performs a calculation to filter the detection result.

[0071] Here, the filter processing performed in step S301 will be described with reference to FIG. 4. FIG. 4(a) shows the output of the camera-side shake detection unit 134 when the photographer is walking, that is, the angular velocity signal, and FIG. 4(b) shows a signal obtained by filtering (High-pass filter: HPF) the angular velocity signal in FIG. 4(a). Whether the photographer is walking or not can be determined by comparing the signal in FIG. 4(b) with the frequency band when walking. For example, a threshold value and a predetermined number of times are set in advance based on a signal obtained by filtering the angular velocity signal when walking. Then, the number of times that the signal obtained by filtering the angular velocity signal exceeds the threshold value is counted, and if the number of times that the threshold value is exceeded is equal to or greater than the predetermined number, it can be determined that the photographer is walking. Alternatively, only a threshold value may be set, and if the threshold value is exceeded, it is determined that the photographer is walking, and if the threshold value is equal to or less than the threshold value, it is not walking. For this reason, filter processing is performed in step S301.

[0072] The judgment is performed at a fixed cycle (judgment cycle). If the judgment cycle is long, there will be a delay in judgment with respect to the photographer's movements, but if the judgment cycle is short, there will be less delay in judgment with respect to the photographer's movements, but since there is a risk of erroneous judgment, the cycle should be set appropriately. The above-mentioned predetermined number of times is set to a number that corresponds to this judgment cycle. Here, the timing of filter calculation and judgment is shown in Figure 3(c). Figure 3(c) shows that walking judgment is made based on the results of 10 filter calculations and threshold comparisons.

[0073] Incidentally, instead of the filter processing, it may be possible to determine whether or not the photographer is walking by performing a calculation such as frequency analysis. FIG. 4(c) is a graph obtained by performing a fast Fourier transform (FFT) analysis on the angular velocity signal in FIG. 4(a). As described above, when the photographer is walking, the peak of the frequency components of the vibrations that occur with walking (particularly around 2 to 6 Hz in this case) becomes large. Therefore, it is also possible to determine whether or not the photographer is walking by comparing the output of the frequency band during walking in the FFT analysis result of the detected angular velocity signal with an arbitrary threshold value.

[0074] Also, the method using the output of the camera side shake detection unit 134 has been described here, but the output of the lens side shake detection unit 125, the motion vector detection unit 131, or another motion sensor may also be used.

[0075] When the process of step S301 is completed, the process proceeds to step S302. In step S302, it is determined whether or not the walking determination time has elapsed. Since the walking determination is performed periodically, it is determined whether the determination period has elapsed. If the walking determination time has elapsed (step S302, Yes), the process proceeds to step S303. If the walking determination time has not elapsed (step S303, No), the process proceeds to step S307.

[0076] In step S303, walking is judged. For example, the angular velocity signal filtered in step S301 (FIG. 4(b)) is checked as described above, and the number of times the value of the filtered angular velocity signal exceeds a predetermined threshold is counted to make the judgment. Here, if the threshold is exceeded a predetermined number of times within the walking judgment period, it is judged that the user is walking, and if the number of times is less than the predetermined number, the user is judged to be stationary. After the processing of step S303 is completed, the process proceeds to step S304.

[0077] In step S304, the walking determination result in this step S303 is confirmed. If it is determined in step S303 that the user is walking (S304, Yes), the process proceeds to step S305, where the holding state is set to "walking." On the other hand, if it is determined in step S303 that the user is stationary (not walking) (step S304, No), the process proceeds to step S311.

[0078] In step S311, it is determined whether or not panning is currently occurring. Panning determination is a known technique and will not be described in detail, but panning can also be determined using the output of the camera-side shake detection unit 134, a motion vector, or the like. If the photographer is panning (step S311, Yes), the process proceeds to step S312, where the holding state is set to "panning state." On the other hand, if the photographer is not currently panning in step S311 (step S311, No), the process proceeds to step S313, where the holding state is set to "held."

[0079] Next, step S307 will be described. As described above, if it is determined in step S302 that the walking determination time has not elapsed, in step S307, the determination result stored in the previous step S314 is referenced to check whether or not the walking state was determined. If the determination result stored in the previous step S312 is the walking state (step S307, Yes), this flow ends. If the walking state is not determined (step S307, No), the flow proceeds to step S308.

[0080] In step S308, it is determined whether panning is currently occurring. The determination method may be the same as in step S311, or it may be different. As described above, panning determination is a known technique, so detailed description is omitted. If the photographer is panning (step S308, Yes), the process proceeds to step S309, where the holding state is set to "panning state." On the other hand, if the photographer is not currently panning in step S308 (step S308, No), the process proceeds to step S310, where the holding state is set to "held."

[0081] Once the hold state has been set by the processes in steps S305, S312, S313, S309, and S310, the process proceeds to step S306.

[0082] In step S306, the tracking area is set according to the holding state. In this embodiment, when the holding state is determined to be a walking state or a panning state, the dead zone is narrowed to shorten the distance between the tracking area and the target position, compared with when the holding state is determined to be a ready (held) state. This will be described with reference to FIG. 5.

[0083] FIG. 5(a) shows an example of the dead zone and the tracking area when the photographer is holding the imaging device 1 and is stationary, and the holding state is determined to be "held". FIG. 5(b) shows an example of the dead zone and the tracking area when the photographer is walking while holding the imaging device 1 and the holding state is determined to be the "walking" state, or when the photographer is panning or tilting the imaging device 1 and the holding state is determined to be the "panning" state. In FIG. 5(a) and (b), the areas 502 and 505 are the tracking areas, and the areas 503 and 506 are the dead zones. In this embodiment, the upper limit value that can be tracked does not depend on the holding state, so in FIG. 5(a) and (b), Xa of the area 502 and Xb of the area 504, and Ya of the area 502 and Yb of the area 504 are equal to each other. Xa and Xb are the upper limit values ​​that can be tracked in the horizontal direction, and Ya and Yb are the upper limit values ​​that can be tracked in the vertical direction.

[0084] In addition, in FIG. 5(a) and (b), the area 501 and the area 504 are areas beyond the limit of tracking, and if the subject is in these areas, the subject position in the captured image cannot be matched with the target position. If the subject is in the area 501 or the area 504, tracking may be performed so that the subject position approaches the target position, but it may be a dead zone in order to track the subject firmly when the subject enters the subject tracking area (areas 502 and 505). In this embodiment, it is described as a dead zone. In addition, if the range that can be used for tracking can be made large, that is, if tracking is performed by geometric deformation and there are many surplus pixels, the peripheral area can be eliminated or narrowed.

[0085] FIG. 5(c) shows the tracking amount corresponding to FIG. 5(a), and FIG. 5(d) shows the tracking amount corresponding to FIG. 5(b).

[0086] In this embodiment, since the target position of the subject is the center of the imaging range, dead zones are provided in the center of the imaging range and in the peripheral area beyond the tracking limit. The dead zone (first area, areas 503, 506) provided in the center is an area that does not allow subject tracking to be performed sensitively. When the main subject is in this area, the tracking amount is set to 0, as shown in Figures 5(c) and (d). A tracking area (second area, areas 502, 505) is provided outside this dead zone. When the main subject is in this area, the tracking amount increases as the difference between the target position and the main subject position increases.

[0087] As shown in FIG. 5(a), when the photographer is ready, the size of area 503, which is a dead zone near the center arranged to include the target position, is set wide. This makes it possible to avoid sensitive reaction to small movements of the subject to be tracked. Also, it is possible to avoid consuming the movable range (in this embodiment, the width of surplus pixels, Xa and Ya) used for tracking to accommodate small movements. On the other hand, by setting area 502, which is a tracking area, at a position away from the target position outside area 503, subject tracking can be made effective against large movements that may lead to the subject going out of the frame.

[0088] On the other hand, when the photographer is walking or panning, the size of the area 506, which is a dead zone near the center arranged to include the target position, is set small as shown in FIG. 5(b). As a result, the area 505, which is a tracking area, is set close to the target position, and the subject tracking is effective from the target position. When the photographer is walking, he / she cannot concentrate on framing, so by controlling the subject tracking from the target position in this way, the subject image can be stably accommodated within the imaging range. In particular, when the photographer is panning, it is difficult to stabilize the subject and keep it at the same position within the imaging range. This is because a difference is likely to occur between the movement of the subject and the panning speed of the photographer. In this embodiment, the size of the dead zone is set smaller in the panning state than in the ready state, and the subject is tracked, so that the subject image is stabilized because the deviation of the subject position caused by the speed difference can be corrected.

[0089] Here, the holding state is described as a state in which the photographer is holding the camera, a walking state, and a panning state, but the holding state is not limited to these. This embodiment is also effective when the imaging device is held using a gimbal device, for example. Since the gimbal device corrects all angular velocity components applied to the imaging device, even if you try to frame the image by panning or tilting the image according to the movement of the subject, the panning or tilting is not reflected, making it difficult to frame the image. Therefore, when the attachment of the gimbal device is detected, as shown in FIG. 5(b), the dead zone is made smaller than when the gimbal device is not detected, and the tracking area is brought closer to the target position of the subject, so that framing can be performed stably even if the subject image moves. In this case, the gimbal basically corrects the shake (camera shake) applied to the imaging device, so the imaging device may use all of its camera shake correction mechanism for tracking the subject. The mounting of the gimbal equipment (gimbal mode determination) may be determined based on the detection result by the shake detection unit (camera shake detection unit 134) of the imaging device, or may be set by the photographer using the operation unit 114. When the determination is based on the detection result by the shake detection unit, it can be determined that the gimbal equipment is mounted when the shake signal is less than a predetermined value for a predetermined time or more (when a state in which almost no shake is applied continues). The mounting state of the gimbal equipment can also be determined by monitoring the outputs of both sensors using an acceleration sensor in addition to the angular velocity sensor of the shake detection unit. For example, when walking and taking a picture with a gimbal mounted, the angular velocity output becomes small, but a predetermined amount of acceleration output is generated. This characteristic can be used to detect the gimbal mounting state.

[0090] Once the tracking region is set in step S306, the process proceeds to step S314, where the current determination result is stored so that the holding state can be referenced in the next step S307, and this flow ends.

[0091] As in this embodiment, by using the shake detection unit to determine the holding state of the imaging device and setting the position of the subject tracking area based on the result, it is possible to reduce the unnaturalness of the video caused by the influence of the delay time in the process of calculating the tracking amount, which is particularly noticeable when the photographer is holding the camera. In addition, when the photographer is walking or panning, when the subject position is likely to deviate from the target position, it is possible to effectively track the subject. Therefore, it is possible to provide an imaging device that can track the subject with good visibility.

[0092] [Second embodiment] In this embodiment, a mode in which an operation on an imaging device by a photographer is detected and a subject tracking area is changed in response to the operation will be described. This embodiment corresponds to (1) in the table shown in FIG.

[0093] The imaging device according to this embodiment will be described below.

[0094] The configuration of the imaging device of this embodiment is the same as that of the first embodiment described with reference to Figures 1 and 2, so the description will be omitted and only the differences from the first embodiment will be described. For simplicity, the center of the imaging range is also set as the subject target position in this embodiment.

[0095] In this embodiment, the determination unit 1422 functions as an operation detection unit that obtains a signal indicating an operation by the photographer performed via the operation unit 114 and determines the operation content. Then, the tracking region determination unit 1421 determines a subject tracking region based on the determination result of the determination unit 1422.

[0096] Examples of the operation of the photographer performed by the operation unit 114 include an operation to request the start of movie recording (such as pressing the movie recording start button) and an operation to request the stop of movie recording (such as pressing the movie recording start button again). Before the operation to request the start of movie recording is input, the size of the dead zone (first area) is set large, and the tracking area (second area) is set to a position away from the subject target position, as shown in FIG. 5(a). When the determination unit 1422 determines that an operation to request the start of movie recording has been input, the determination result is input to the tracking area determination unit 1421, and the tracking area determination unit 142, which has received the input of the determination result, moves the tracking area closer to the subject target position as shown in FIG. 5(b) than in FIG. 5(a). Also, when the determination unit 1422 determines that an operation to request the stop of movie recording has been input, the determination result is input to the tracking area determination unit 1421, and the tracking area determination unit 142, which has received the input of the determination result, returns the positions of the dead zone and the subject tracking area to the state shown in FIG. 5(a). In this way, by setting the subject tracking area based on the operation to request the start and stop of movie recording, it is possible to reduce the usage rate of the movable range available for subject tracking by having the photographer work on framing before the start of movie recording. In other words, it is possible to reduce the probability of using up the movable range available for subject tracking before movie recording.

[0097] In addition, in this embodiment, the content of changing the tracking area of ​​the subject by the operation of requesting the start and stop of video recording has been described, but the tracking area may be changed by other operations. For example, when live broadcasting / live distribution is performed using an imaging device, when an operation to start broadcasting or distribution is received, the dead zone may be made smaller than when in a standby state, and the tracking area may be brought closer to the target position. In this case, the tracking area is changed by an operation to request the start and stop of broadcasting or distribution. Also, when live broadcasting / live distribution of videos is performed using multiple imaging devices, when one is designated as an imaging device that captures images to be broadcasted / distributed, the dead zone may be set smaller than when it is designated as an imaging device in a standby state. In this case, the tracking area may be set based on the switching operation of the imaging device.

[0098] Alternatively, the tracking area may be changed depending on whether the focus is manual or autofocus. In manual focus, the movement of the subject may cause the focus to shift, which may reduce the accuracy of the subject detection information, so the dead zone is enlarged to prevent erroneous tracking. On the other hand, in the case of autofocus setting, it is possible to always focus on the subject, and the accuracy of the subject detection information is high. Therefore, the dead zone is made smaller than in manual focus to enable better tracking. Also, for example, when a mode is set to perform processing such as trimming an image in a video, the dead zone may be made smaller and the tracking area may be moved closer to the target position. This is because the subject may be cut off by trimming.

[0099] The tracking region determination unit 1421 may determine the tracking region based on the holding state as in the first embodiment, and may further determine the tracking region based on the operation by the photographer. For example, when the holding state is a walking state or a panning state and a moving image is not being recorded, the dead zone may be set larger than when the holding state is a state in which the camera is held firmly and when the holding state is a walking shooting or panning state and a moving image is being recorded. Also, when the holding state is a walking state or a panning state and a moving image is not being recorded, the dead zone may be set larger, when the holding state is a state in which the camera is held firmly, the dead zone may be set smaller, and when the holding state is a walking state or a panning state and a moving image is being recorded, the size of the dead zone may be set intermediate between the two.

[0100] [Third embodiment] In this embodiment, a tracking area is set based on subject information obtained by subject detection. This embodiment corresponds to (2), (3), and (4) in the table shown in FIG.

[0101] The imaging device according to this embodiment will be described below.

[0102] The configuration of the imaging device of this embodiment is the same as that of the first embodiment described with reference to Figures 1 and 2, so the description will be omitted and only the differences from the first embodiment will be described. For simplicity, the center of the imaging range is also set as the subject target position in this embodiment.

[0103] In this embodiment, the tracking region determination unit 1421 acquires subject information of the main subject detected by the subject detection means 141, and determines the tracking region based on the subject information. The subject information may include, for example, the size of the face of the main subject, the moving speed of the main subject, and the subject type of the main subject (for example, information on whether the main subject is a person or not).

[0104] An example of determining a tracking area based on the size of a main subject will be described. Fig. 7 shows an image of the size of the main subject and the tracking area. Areas 702 and 705 are tracking areas (second areas), and areas 703 and 706 are dead zones (first areas). Areas 701 and 704 are areas that exceed the limit of what can be tracked, and are dead zones as in the first embodiment. Fig. 7(a) shows a case where the size of the main subject is less than a predetermined size, and Fig. 7(b) shows a case where the size of the main subject is equal to or greater than the predetermined size.

[0105] When the size of the subject area to be tracked is small as in Fig. 7(a), the movement of the main subject appears small on the image (recorded image during recording, displayed image during live view), so the main subject is less likely to go out of frame, and it is thought that it is easier for the photographer to frame it. Therefore, the photographer is primarily responsible for framing, and the small movements of the main subject are not tracked, and subject tracking is activated when the main subject is about to go out of frame. Therefore, a large dead zone area 703 is provided around the center of the image, which is the target position, and the tracking area 702 is located away from the target position.

[0106] On the other hand, as shown in Fig. 7(b), when the size of the subject area to be tracked is large, the movement of the main subject appears large on the image, and it is considered that the probability of the main subject going out of frame increases even if the photographer frames the image. Therefore, by making area 706, which is a dead zone area around the center of the image that is the target position, smaller than area 703 and locating area 706, which is a tracking area, closer to the target position than area 702, subject tracking is made sensitive to the movement of the main subject that leads to going out of frame.

[0107] Also, the moving speed of the main subject may be calculated from the amount of change in the position information, and the tracking area may be determined based on the moving speed. It is easy to imagine that a subject will easily go out of frame if its moving speed is high. Therefore, when the moving speed of the main subject is below a predetermined speed, the dead zone area is set large relative to the center of the image, which is the target position, as shown in FIG. 7(a), and when the moving speed of the main subject is equal to or higher than the predetermined speed, the dead zone area is set small.

[0108] A case where the tracking area is determined based on the type of the main subject will be described. For example, the tracking area is determined based on information such as whether the main subject is a person, an animal, or another object. In particular, animals are expected to move quickly or unexpectedly, and it is considered that the photographer's reaction cannot keep up with them and they are easily framed out. Therefore, the dead zone area is made small with respect to the center of the image, which is the target position, as shown in FIG. 7(b), and the subject is tracked sensitively to the movement of the tracking target. In addition, when a vehicle can be identified, since the vehicle generally moves quickly, it is considered that the photographer's reaction cannot keep up with them and they are easily framed out, so the dead zone area is similarly made small. The method of determining the tracking area based on the type of the subject is not limited to this. For example, conversely, since the trajectory of a vehicle is fixed, it is assumed that the photographer can easily frame it, and the dead zone area may be set large as shown in FIG. 7(a). In addition, when the type of vehicle (train, car, airplane, etc.) can be determined, the dead zone area may be set larger for vehicles with a fixed trajectory such as trains and airplanes, and smaller for vehicles that may not have a fixed trajectory such as cars and motorcycles than for trains and airplanes. In addition, the tracking area may be determined based on whether the tracking target is a child or an adult, in combination with a personal authentication function. For example, it is possible to set the dead zone smaller when the tracking target subject is a child than when the tracking target subject is an adult, assuming that children move more unpredictably than adults. In addition, it is possible to obtain only the type of subject, whether it is a person or something else, and set the dead zone larger and the tracking area farther away when the main subject is a person than when it is something other than a person.

[0109] As described above, by determining the tracking area in accordance with the subject information acquired from subject detection means 141, it is possible to reduce the probability that the main subject will go out of frame and provide a moving image with a stable subject image.

[0110] Furthermore, in this embodiment, the tracking area is changed depending on the size, speed, and type of the subject. However, the tracking area may be determined based on other subject information.

[0111] [Fourth embodiment] In this embodiment, a form in which a tracking region is determined according to a target position will be described. This embodiment corresponds to (5) in the table shown in FIG.

[0112] The imaging device according to this embodiment will be described below.

[0113] The configuration of the imaging device of this embodiment is the same as that of the first embodiment described with reference to Figs. 1 and 2, so the description will be omitted, and only the differences from the first embodiment will be described. In this embodiment, subject tracking control is performed by setting arbitrary coordinates designated by the photographer as the target position. The photographer sets the arbitrary coordinates in advance via the operation unit 114. The subject tracking calculation unit 142 transmits the coordinates designated by the photographer to the subject target position setting unit 1424, and sets them as the target position.

[0114] The tracking region determination unit 1421 determines the tracking region based on the target position set by the subject target position setting unit 1424. This process will be described with reference to FIG.

[0115] Fig. 8 is a diagram showing an example of the relationship between the target position and the tracking area, where areas 802 and 805 are tracking areas (second areas), and areas 803 and 806 are dead zones (first areas). Areas 801 and 804 are areas beyond the tracking limit. Fig. 8(a) shows the tracking area when the target position is at the center of the image. In contrast, Fig. 8(b) shows the tracking area when the target position is away from the center of the image.

[0116] In this embodiment, the further away the target position of the main subject is from the center of the image, the smaller the dead zone is, and the closer the distance from the target position to the tracking area is. This is because, when a subject moves at a predetermined speed, it is more difficult to frame the subject in the center of the image than in the corner of the image, and the subject is more likely to go out of frame. When the subject is in the corner of the image and is more likely to go out of frame, the subject is tracked more sensitively to the subject's movement than in other cases, which can assist the photographer in framing. This reduces the probability that the subject to be tracked will go out of frame, and a moving image with a stable image of the subject to be tracked can be provided.

[0117] In this way, in the first, third and fourth embodiments, under a photographing condition where framing is assumed to be difficult, the dead zone (first area) is made smaller and the tracking area (second area) is set closer to the target position than under a photographing condition where framing is not assumed to be difficult. Note that the tracking area being closer to the target position means that the shortest distance between the tracking area and the target position is short.

[0118] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0119] (Modification) In the first to fourth embodiments, a lens-interchangeable imaging device has been described, but each embodiment can also be applied to an imaging device with an integrated lens, and can also be applied to devices such as smartphones that have various functions in addition to imaging functions.

[0120] In the first to fourth embodiments, the lens 2 and the camera 1 each have an optical image stabilization mechanism, and the camera 1 further has an electronic image stabilization mechanism, but the present invention is not limited to this configuration. It is sufficient to have at least one of the optical image stabilization mechanism on the lens side, the optical image stabilization mechanism on the camera side, and the electronic image stabilization mechanism on the camera side. When two image stabilization mechanisms are provided, the combination is not particularly limited.

[0121] For example, when neither the lens nor the camera has an optical image stabilization mechanism, and image stabilization and subject tracking are performed only by an electronic image stabilization mechanism, the imaging range may be moved based on a value obtained by adding the outputs of the image stabilization amount calculation unit 1332 and the tracking amount calculation unit 1423. In this case, the image stabilization amount output from the image stabilization amount calculation unit 1332 and the subject tracking amount output from the tracking amount calculation unit 1423 may have different target frequencies. For example, the image stabilization amount calculated based on a shake signal of a predetermined frequency or higher out of the detected shake amount may be added to the tracking amount based on a signal of a predetermined frequency or lower out of the difference from the target position, and the electronic image stabilization mechanism may be controlled based on this.

[0122] In addition, the image stabilization mechanism that moves the imaging range to track the subject is not limited to an electronic image stabilization mechanism, but may be an optical image stabilization mechanism on the lens side or camera side, or may be multiple image stabilization mechanisms.

[0123] Furthermore, in the first to fourth embodiments, the description has been given of a configuration in which camera shake correction and subject tracking are performed, but a configuration in which subject tracking is performed without camera shake correction may also be used.

[0124] In the first to fourth embodiments, the imaging device performs a series of object tracking processes such as setting the object tracking area, calculating the tracking amount, and outputting the tracking amount to the image stabilization mechanism (image processing circuit 109) to control the object tracking, but the present invention is not limited to this. For example, a series of object tracking controls may be performed by a control device of the imaging device that controls the imaging device from the outside, or the object tracking processes may be performed by the imaging device and the control device, or by a plurality of control devices sharing the same. The control device may be a cloud, or the imaging device may be configured to be controlled from the cloud. When the control device performs a series of object tracking controls, the control device may obtain the captured image from the imaging device and perform object detection to obtain object information, or the imaging device may perform object detection and obtain the object detection result to obtain object information.

[0125] In addition, in the first to fourth embodiments, subject tracking is performed in real time, but subject tracking may also be achieved by performing image processing (such as changing the cut-out position) on a video that has already been shot and primarily recorded.

[0126] In addition, although the first to fourth embodiments have been described with respect to video shooting, the same effects can be obtained with respect to continuous shooting of live view images and still images. In the case of a live view image, the range of the image displayed as the live view image can be regarded as the imaging range described in the above embodiments.

[0127] In the first to fourth embodiments, the tracking area determination unit 1421 sets a dead zone (first area) and a tracking area (second area), but instead of the dead zone, an area with a low degree of subject tracking may be set as the first area. The degree of subject tracking refers to how close the subject position is to the target position when the difference between the target position and the current position of the subject is set to 1, and this degree is 0 in the dead zone. If the degree of subject tracking in the first area is lower than the degree of subject tracking in the second area, the same effect as in the above-mentioned embodiments can be obtained. [Explanation of symbols]

[0128] 125 Lens side vibration detection unit 134 Camera side shake detection unit 141 Object detection unit 142 Subject tracking calculation section 1421 Tracking area determination unit 1423 Tracking amount calculation unit 109 Image Processing Circuit

Claims

1. a subject information acquisition means for acquiring information on a subject detected from a captured image; a tracking amount calculation means for calculating a tracking amount based on the position of the subject in the captured image and a target position; a tracking control means for controlling object tracking to move the position of the object in the image closer to the target position based on the tracking amount; an area setting unit that sets a first area and a second area based on at least one of a holding state of an imaging device that captures the captured image, a detection result of an operation on the imaging device by a photographer, a position of a target position within the captured image, and a type of the subject; A control device, characterized in that the first area performs subject tracking to a lower degree than the second area.

2. 2. The control device according to claim 1, wherein when the size of the first area is changed, the position of the second area is changed.

3. 2. The control device according to claim 1, wherein the area setting means sets the first area and the second area based on a holding state of the imaging device.

4. a camerawork determination means for determining camerawork by a photographer based on a detection result of a shake detection means for detecting a movement applied to the imaging device; 2. The control device according to claim 1, wherein the holding state is a result of the determination of the camera work.

5. the holding state is determined based on a detection result by a shake detection means that detects a movement applied to the imaging device, The control device according to claim 3, characterized in that the area setting means sets the first area and the second area based on the holding state by setting the first area and the second area based on the detection result by the shake detection means.

6. 6. The control device according to claim 5, wherein when the magnitude of the shake detected by said shake detection means exceeds a threshold value, the size of said first region is made smaller than when the magnitude is equal to or smaller than the threshold value.

7. The area setting means 7. The control device according to claim 6, wherein, when the number of times that the magnitude of the shake detected by the shake detection means exceeds a threshold is equal to or less than a predetermined number of times, the second region is set at a position farther from the target position than when the number of times that the magnitude of the shake detected by the shake detection means exceeds the threshold is greater than the predetermined number of times.

8. The area setting means The control device according to claim 6, characterized in that, when the number of times that the magnitude of the shake detected by the shake detection means exceeds a threshold is less than a predetermined number of times, the size of the first area is set to be larger than when the number of times that the threshold is exceeded is more than the predetermined number of times.

9. The holding state is a determination result of whether or not a gimbal is attached to the imaging device, The area setting means The control device according to claim 3, characterized in that, when it is determined that the gimbal is attached, the second region is set to a position closer to the target position than when it is determined that the gimbal is not attached.

10. The holding state is a determination result of whether or not a gimbal is attached to the imaging device, The control device according to claim 3, characterized in that the area setting means sets the size of the first area to be smaller when it is determined that the gimbal is attached than when it is not determined that the gimbal is attached.

11. The control device described in Claim 1, characterized in that the area setting means sets the first area and the second area based on the detection result of the operation.

12. detecting an operation by a photographer to start video recording and an operation to end video recording on the imaging device; The area setting means 12. The control device according to claim 11, wherein the second area is set to a position closer to the target position during video recording than when video recording is not being performed.

13. Detecting an operation by a photographer to start video recording and an operation to end video recording; The area setting means 12. The control device according to claim 11, wherein the size of the first area is set smaller when a moving image is being recorded than when a moving image is not being recorded.

14. The control device described in Claim 1, characterized in that the area setting means sets the first area and the second area based on the distance between the target position and the center of the captured image.

15. The control device described in Claim 14, characterized in that the area setting means sets the second area to a position closer to the target position when the distance is a second value greater than the first value compared to when the distance is a first value.

16. The control device according to claim 1, wherein the area setting means sets the first area and the second area based on the type of subject.

17. The area setting means 2. The control device according to claim 1, wherein when the subject size is larger than a predetermined size, the second area is set at a position closer to the target position than when the subject size is equal to or smaller than the predetermined size.

18. The area setting means 2. The control device according to claim 1, wherein when the subject size is larger than a predetermined size, the size of the first region is set smaller than when the subject size is equal to or smaller than the predetermined size.

19. The area setting means 2. The control device according to claim 1, wherein when the amount of change in the subject position is greater than a predetermined amount, the second area is set to a position closer to the target position than when the amount of change is equal to or less than the predetermined amount.

20. The area setting means 2. The control device according to claim 1, wherein when the amount of change in the subject position is greater than a predetermined amount, the size of the first region is set smaller than when the amount of change is equal to or smaller than the predetermined amount.

21. The area setting means 17. The control device according to claim 16, wherein when the type of subject is a person, the second area is set at a position farther away from the target position than when the type of subject is other than a person.

22. The area setting means 17. The control device according to claim 16, wherein when the type of the subject is a person, the size of the first region is set to be larger than when the type of the subject is other than a person.

23. 23. The control device according to claim 1, wherein the first area is a dead zone for the subject tracking.

24. 23. The control device according to claim 1, wherein the first region is set closer to the target position than the second region.

25. 23. The control device according to claim 1, wherein the first region is set inside the second region.

26. a subject detection means for detecting a subject from the captured image; a subject setting means for setting a specific subject from the subjects detected by the subject detection means, 23. The control device according to claim 1, wherein the tracking amount calculation means calculates the tracking amount based on the position of the subject set by the subject setting means in the captured image and the target position.

27. A control device according to any one of claims 1 to 22; an imaging element that captures the captured image; a tracking means for performing subject tracking by controlling the tracking control means to move the position of the subject in the image closer to the target position; An imaging device comprising:

28. a subject information acquisition step of acquiring information about a subject detected from a captured image; a tracking amount calculation step of calculating a tracking amount based on a position of the subject in the captured image and a target position; a tracking control step of controlling object tracking based on the tracking amount to move the position of the object in the image closer to the target position; a region setting step of setting a first region and a second region based on at least one of a holding state of an imaging device that captures the captured image, a detection result of an operation on the imaging device by a photographer, a position of a target position within the captured image, and a type of the subject, A method for controlling an imaging device, wherein the first area is subject to a lower degree of tracking than the second area.

Images