Control device, imaging device, control method, and program

Abstract

To provide a control device that enables proper autofocus operation. [Solution] The control device (204) includes an acquisition means (2041) for acquiring movement information of the subject and a control means (2042) for controlling the focus lens (104) using focus information acquired from the imaging signal. The control means (2042) changes the processing related to the autofocus tracking performance according to the movement information of the subject.

Description

Technical Field

[0001] The present invention relates to a control device, an imaging device, a control method, and a program.

Background Art

[0002] Conventionally, when a user performs a manual focus (MF) operation during an auto focus (AF) operation, a function (full-time MF) that switches AF to MF for focus control is known. Also, when photographing a subject approaching the imaging device, the user may follow the subject by MF operation using full-time MF and stop the MF operation halfway to switch to pin tracking by AF. In such a case, since the subject gets closer while switching from MF to AF, the pin tracking by AF after the switch is delayed.

[0003] Patent Document 1 discloses a method for changing the AF operation after full-time MF, specifically, a method for changing which area on the screen to focus on, depending on whether there is a subject in focus by the MF operation, for the AF after full-time MF.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, with the method disclosed in Patent Document 1, when a pin change in the depth direction occurs, such as when the subject approaches or moves away from the imaging device, an appropriate AF operation cannot be performed.

[0006] Therefore, an object of the present invention is to provide a control device capable of performing an appropriate AF operation.

Means for Solving the Problems

[0007] One aspect of the present invention is a control device comprising an acquisition means for acquiring movement information of a subject and a control means for controlling a focus lens using focus information acquired from an imaging signal, wherein the control means changes the processing related to the autofocus tracking performance according to the movement information of the subject.

[0008] Other objects and features of the present invention are described in the following embodiments. [Effects of the Invention]

[0009] According to the present invention, a control device capable of appropriate AF operation can be provided. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram of the imaging device in the first, second, and fourth embodiments. [Figure 2] This is a pixel arrangement diagram of the image sensor in each embodiment. [Figure 3] This is a diagram illustrating the focus detection process in each embodiment. [Figure 4] This figure shows a pair of image signals obtained from the AF region in each embodiment. [Figure 5] This figure shows the relationship between the shift amount and the correlation amount of a pair of image signals in each embodiment. [Figure 6] This figure shows the relationship between the amount of shift and the amount of correlation change of a pair of image signals in each embodiment. [Figure 7] This flowchart shows the video recording process in each embodiment. [Figure 8] This is a flowchart illustrating the subject movement detection process in the first, second, and fourth embodiments. [Figure 9] This is a flowchart showing the focus driving process in the first and third embodiments. [Figure 10]It is a flowchart showing focus drive processing by AF in each embodiment. [Figure 11] It is a flowchart showing AF execution processing in each embodiment. [Figure 12] It is a diagram showing an example of subject movement detection in each embodiment. [Figure 13] It is an explanation of problems when each embodiment is not applied. [Figure 14] It is a diagram showing the ideal focus position and the actual focus position of the subject in each scene of FIGS. 13(a) to (d). [Figure 15] It is a diagram showing another example of the ideal focus position and the actual focus position of the subject in each scene of FIGS. 13(a) to (d). [Figure 16] It is a diagram showing an example of the effect by applying each embodiment. [Figure 17] It is a diagram showing the ideal focus position and the actual focus position of the subject in each scene of FIGS. 16(a) to (d’). [Figure 18] It is a flowchart showing focus drive processing in the second embodiment. [Figure 19] It is a diagram showing an example of a good case with a waiting time until AF processing is executed after full-time MF operation in the second embodiment. [Figure 20] It is a diagram showing an example of changes in the actual focus position when starting and stopping full-time MF operation in the scenes of FIGS. 19(a) to (c). [Figure 21] It is a diagram showing an example of the ideal focus position and the actual focus position of the subject in each scene of FIGS. 13(a) to (d) with a waiting time until AF start after full-time MF operation. [Figure 22] It is a block diagram of an imaging device in the third embodiment. [Figure 23] It is a flowchart showing subject movement detection processing in the third embodiment. [Figure 24] It is a flowchart showing focus drive processing in the fourth embodiment. [Figure 25] FIG. 16 is a diagram showing the ideal focus position and the actual focus position of the subject position in each scene of FIGS. 16(a) to (d’) when an AF start operation is involved.

BEST MODE FOR CARRYING OUT THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0012] (First Embodiment) First, the first embodiment of the present invention will be described.

[0013] [Configuration of Imaging Device] First, referring to FIG. 1, a functional configuration example of a digital camera 100 as an example of an imaging device according to the present embodiment will be described. FIG. 1 is a block diagram of a digital camera (imaging device) 100. The digital camera 100 of the present embodiment is an interchangeable-lens camera and includes a lens unit 10 having an optical system (imaging optical system) and a camera unit 20. In the present embodiment, the lens unit 10 may be detachable from the camera unit 20 or may be integrally configured with the camera unit 20. When the lens unit 10 is detachable from the camera unit 20, the lens unit 10 constitutes a lens device (interchangeable lens), the camera unit 20 constitutes a camera body (imaging device), and an imaging system is constituted by the lens device and the camera body.

[0014] The lens unit 10 includes an optical system (first lens group 101, aperture (aperture diaphragm) 102, second lens group 103, focus lens (focus lens group) 104, and a drive / control system. As such, the lens unit 10 includes the focus lens 104 and is a photographing lens that forms an optical image of a subject.

[0015] The first lens group 101 is positioned at the tip of the lens unit 10 and is held so as to be movable in the optical axis direction. The aperture 102 has the function of adjusting the amount of light during shooting. The aperture 102 and the second lens group 103 are movable together in the optical axis direction and move in conjunction with the first lens group 101 to realize a zoom function. The focus lens 104 is also movable in the optical axis direction, and the subject distance (focus distance) at which the lens unit 10 is in focus changes depending on its position. Focus adjustment is performed to adjust the focus distance of the lens unit 10 by controlling the position of the focus lens 104 in the optical axis direction.

[0016] The drive / control system includes a zoom actuator 105, an aperture actuator 106, and a focus actuator 107. The drive / control system also includes a zoom drive circuit 108, an aperture drive circuit 109, a focus drive circuit 110, a lens control unit 111, a lens operation unit 112, and a lens memory 113. The zoom drive circuit 108 uses the zoom actuator 105 to drive the first lens group 101 and the second lens group 103 in the optical axis direction, controlling the angle of view of the optical system of the lens unit 10. The aperture drive circuit 109 uses the aperture actuator 106 to drive the aperture 102, controlling the aperture diameter and opening / closing operation of the aperture 102. The focus drive circuit 110 uses the focus actuator 107 to drive the focus lens 104 in the optical axis direction, changing the focus distance of the optical system of the lens unit 10. Furthermore, the focus drive circuit 110 uses the focus actuator 107 to detect the current position of the focus lens 104.

[0017] The lens control unit 111 controls the zoom drive circuit 108, the aperture drive circuit 109, and the focus drive circuit 110. The lens control unit 111 also communicates with the camera control unit (control device) 204. For example, the lens control unit 111 detects the position of the focus lens 104 and notifies the camera control unit 204 of the focus lens position information. The lens control unit 111 also controls the zoom drive circuit 108, the aperture drive circuit 109, and the focus drive circuit 110 in accordance with the processing commands of the camera control unit 204. Furthermore, the lens control unit 111 controls the zoom drive circuit 108 and the focus drive circuit 110 based on information notified by the operation of the lens operation unit 112, which will be described later.

[0018] The lens control unit 112 consists of a zoom ring, a focus ring, and the like. It receives ring operations from the user and notifies the lens control unit 111 of the operation information. This enables the user to perform zoom operations and manual focus operations using the focus ring.

[0019] The lens memory 113 has pre-stored optical information necessary for autofocus detection. The camera control unit 204 controls the operation of the lens unit 10 by executing programs stored in, for example, the built-in non-volatile memory or the lens memory 113.

[0020] The camera unit 20 includes an imaging system (image sensor 201) and a drive / control system. The first lens group 101, aperture 102, second lens group 103, and focus lens 104 of the lens unit 10, along with the image sensor 201 of the camera unit 20, constitute the imaging unit.

[0021] The image sensor 201 consists of a CMOS image sensor and peripheral circuits, and has m pixels in the horizontal direction and n pixels in the vertical direction (m and n are integers of 2 or more). The image sensor 201 in this embodiment has an pupil division function and enables phase-detection autofocus using image data. The image sensor drive circuit 202 controls the operation of the image sensor 201 and performs A / D conversion on the acquired image signal and transmits it to the camera control unit 204.

[0022] The image processing circuit 203 generates data for phase-detection autofocus and display and recording image data from the image data (imaging signal) output by the image sensor 201. It also performs general image processing on the image data acquired by the image sensor 201, such as gamma conversion, white balance adjustment, color interpolation, and compression encoding, which are common processes performed in digital cameras.

[0023] The camera control unit 204 performs all calculations and controls related to the camera unit 20, and controls the image sensor drive circuit 202, image processing circuit 203, image plane phase-detection focus unit 205, display unit 206, camera operation unit 207, and memory 208. The camera control unit 204 is connected to the lens control unit 111 via a signal line between the lens unit 10 and the camera unit 20, and communicates commands and data with the lens control unit 111. The camera control unit 204 requests the lens control unit 111 to acquire the focus lens position, to drive the aperture, zoom lens, and focus lens at a predetermined drive amount, and to acquire optical information specific to the lens unit 10.

[0024] The camera control unit 204 incorporates a ROM 204a, a RAM 204b, and an EEPROM 204c. The ROM (Read Only Memory) 204a stores the program that controls the camera's operation. The RAM (Random Access Memory) 204b stores variables. The EEPROM (Electrically Erasable Programmable Read-Only Memory) 204c stores various parameters and user-configured settings for the camera unit 20.

[0025] The camera control unit 204 includes an acquisition means 2041 and a control means 2042. The acquisition means 2041 acquires information about the movement of the subject. The control means 2042 controls the focus lens using focus information acquired from the imaging signal. The control means 2042 also changes the processing related to autofocus tracking performance according to the information about the movement of the subject. Movement information refers to information about whether or not the subject has moved. For example, if the subject has moved, the control means 2042 changes the processing related to tracking performance to improve tracking performance compared to when the subject has not moved. The direction of movement of the subject is, for example, the direction in which the subject is moving closer or further away. The acquisition means 2041 acquires information about the movement of the subject when, for example, the first operating means for performing manual focus (the focus ring of the lens operating unit 112) is operated.

[0026] The image plane phase-difference focus detection unit 205 performs focus detection processing using a phase-difference detection method with focus detection data (focus information acquired from the imaging signal) obtained by the image processing circuit 203. More specifically, the image processing circuit 203 generates image data for each pair formed by light beams passing through two pairs of pupil regions as focus detection data. The image plane phase-difference focus detection unit 205 then detects the amount of focus shift based on the amount of shift in each pair of image data. In this way, the image plane phase-difference focus detection unit 205 of this embodiment performs phase-difference AF (image plane phase-difference AF) based on the output of the image sensor 201 without using a dedicated AF sensor. The operation of the image plane phase-difference focus detection unit 205 will be described in detail later.

[0027] The display unit 206 consists of an LCD (liquid crystal display) and other components, and displays information related to the camera's shooting mode, a preview image before shooting and a confirmation image after shooting, and an image showing the focus status when focus is detected. The display unit 206 also has a touch operation function, allowing the camera to be operated by touching the display unit 206 directly.

[0028] The camera control unit 207 consists of a power switch, a focus adjustment start switch, a shutter release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, a video recording switch, and the like. The memory (storage unit) 208 is a removable flash memory that stores captured images.

[0029] [Details of the imaging plane phase-contrast focus detection unit 205] Next, the operation of the image plane phase-difference focus detection unit 205 will be described in detail with reference to Figures 2(a) and (b). Figure 2(a) is a pixel arrangement diagram of the image sensor 201 in this embodiment, showing the range of 6 rows vertically (Y direction) and 8 columns horizontally (X direction) of the 2D C-MOS area sensor as observed from the lens unit 10 side. The image sensor 201 is provided with Bayer-arranged color filters, with red (R) and green (G) color filters alternately arranged from left to right for pixels in odd-numbered rows, and green (G) and blue (B) color filters alternately arranged from left to right for pixels in even-numbered rows.

[0030] Referring to Figure 2(b), pixel 211R will be described. The circle 211i represents an on-chip microlens, and the multiple rectangles 211A and 211B located inside the on-chip microlens are photoelectric conversion units, respectively. Pixels 211Gr, 211Gb, and 211B have a similar configuration.

[0031] The image sensor 201 of this embodiment has pixels (211R, 211Gr, 211Gb, 211B) in which the photoelectric conversion section of the image pixel is divided into two in the X direction. The photoelectric conversion signals from each photoelectric conversion section can be used as data for phase-difference autofocus or for generating disparity images that constitute a 3D (3-dimensional) image. In addition, the sum of the photoelectric conversion signals can be used as normal captured image data.

[0032] Here, we will explain the pixel signal when performing phase-detection autofocus. In this embodiment, the microlens 211i and the divided photoelectric conversion units 211A and 211B shown in Figure 2(b) divide the light beam emitted from the imaging optical system into pupils. The photoelectric conversion units 211A and 211B shown in Figure 2(b) are used as a pair. This makes it possible to perform focus detection based on the amount of image shift (phase difference) in the X direction.

[0033] Phase-detection autofocus (AF) based on focus detection using the amount of image shift in the X direction is described below. In Figure 2(b), the signals from the photoelectric converters 211A, which are located on multiple pixels 211R within a predetermined range arranged in the same pixel row, are used as the AF A image. The signal from the photoelectric converter 211B is used as the AF B image. The outputs of the photoelectric converters 211A and 211B use a pseudo-luminance (Y) signal calculated by adding the green, red, blue, and green outputs included in the unit array of the color filter. However, AF A and B images may be organized for each of the red, blue, and green colors. By detecting the relative amount of image shift of the pair of image signals, which are the AF A and B images generated in this way, through correlation calculation, the prediction, which is the degree of correlation between the pair of image signals, can be detected. The camera control unit 204 can detect the amount of defocus in a predetermined area by multiplying the prediction by a conversion coefficient. The sum of the outputs of the photoelectric converters 211A and 211B forms one pixel (output pixel) of the output image.

[0034] Next, the focus detection process will be explained in detail with reference to Figure 3. Figure 3 is an explanatory diagram of the focus detection process. Figure 3 shows an example of the AF region 302 on the pixel array 301 of the image sensor 201 during the focus detection process.

[0035] The shift regions 303 on both sides of the AF region 302 are the regions required for correlation calculation. Therefore, the region 304, which is the sum of the AF region 302 and the shift region 303, is the pixel region required for correlation calculation. In the figure, p, q, s, and t each represent coordinates in the X direction, with p and q representing the X coordinates of the start and end points of the pixel region 304, respectively, and s and t representing the X coordinates of the start and end points of the AF region 302, respectively.

[0036] Figures 4(a) to 4(c) show examples of a pair of AF image signals acquired from multiple pixels included in the AF region 302 shown in Figure 3. The solid line 401 represents one of the AF A images, and the dashed line 402 represents the AF B image. Figure 4(a) shows the AF A and B images before shifting, while Figures 4(b) and 4(c) show the AF A and B images after being shifted in the positive and negative directions, respectively, from the state shown in Figure 4(a). When calculating the correlation between the pair of AF A images 401 and B images 402, both AF A images 401 and B images 402 are shifted by one bit in the direction of the arrows.

[0037] Next, we will explain how to calculate the correlation. First, as shown in Figures 4(b) and (c), the AF A image 401 and B image 402 are each shifted by one bit, and the sum of the absolute values ​​of the differences between the AF A image 401 and B image 402 is calculated. When the shift amount is i, the maximum shift amount in the negative direction is ps, the maximum shift amount in the positive direction is qt, x is the starting coordinate of the AF region 302, and y is the ending coordinate of the AF region 302, the correlation amount COR can be calculated by the following equation (1).

[0038]

number

[0039] Figure 5(a) shows an example of the relationship between the shift amount and the correlation coefficient COR. In Figure 5(a), the horizontal axis represents the shift amount, and the vertical axis represents the correlation coefficient COR. Among the extreme values ​​502 and 503 of the correlation coefficient 501, which changes with the shift amount, the degree of agreement between the pair of AF A and B images is highest at the shift amount corresponding to the smaller correlation coefficient.

[0040] Next, we will explain how to calculate the correlation change. The correlation change is calculated by taking the difference in correlation amounts at every one-shift interval in the waveform of correlation amount 501 shown in Figure 5(a). If the shift amount is i, the maximum shift amount in the negative direction is ps, and the maximum shift amount in the positive direction is qt, then the correlation change ΔCOR can be calculated by the following equation (2).

[0041]

number

[0042] Figure 6(a) shows an example of the relationship between the shift amount and the correlation change ΔCOR. The horizontal axis represents the shift amount, and the vertical axis represents the correlation change ΔCOR. The correlation change 601, which changes with the shift amount, changes from positive to negative in the 602 and 603 parts. The state where the correlation change is 0 is called zero crossing, and the degree of agreement between the pair of AF A and B images is highest. Therefore, the shift amount that gives zero crossing is the image displacement amount.

[0043] Figure 6(b) shows a magnified view of the portion indicated by 602 in Figure 6(a). 604 is a portion of the correlation change amount 601. The method for calculating the image displacement amount will be explained using Figure 6(b).

[0044] The shift amount (k-1+α) that gives the zero crossing can be divided into an integer part β (=k-1) and a fractional part α. The fractional part α can be calculated using the following equation (3) based on the similarity relationship between triangle ABC and triangle ADE in the figure.

[0045]

number

[0046] The integer part β can be calculated from Figure 6(b) using the following equation (4).

[0047]

number

[0048] Then, from the sum of α and β, the amount of image shift, or prediction which is the degree of correlation between a pair of image signals, can be detected.

[0049] As shown in Figure 6(a), if there are multiple zero-crossings of the correlation change ΔCOR, the zero-crossing with the steeper change in the correlation change ΔCOR in that vicinity is designated as the first zero-crossing. This steepness is an indicator of how easy it is to perform AF, and a larger value indicates that it is easier to perform accurate AF at that point. The steepness maxder can be calculated by the following equation (5).

[0050]

number

[0051] In this embodiment, if there are multiple zero-crossings of the correlation change, the first zero-crossing is determined by its steepness, and the shift amount that gives this first zero-crossing is used as the prediction.

[0052] Next, we will explain how to calculate the reliability of the image displacement. The reliability of the image displacement can be defined by the degree of agreement between a pair of AF images A and B (hereinafter referred to as the two-image agreement) fnclvl and the steepness of the correlation change described above. The two-image agreement is an indicator of the accuracy of the image displacement, and in the correlation calculation method of this embodiment, a smaller value indicates better accuracy.

[0053] Figure 5(b) is an enlarged view of the portion indicated by 502 in Figure 5(a), where 504 is a part of the correlation 501. The two-image agreement score fnclvl can be calculated using the following formula (6).

[0054]

number

[0055] [Details of various processes performed by the camera unit 20] The camera control unit 204 of the camera unit 20 performs the following processing according to the imaging processing program, which is a computer program. Figure 7 is a flowchart showing the procedure for video recording processing. The camera control unit 204 performs various video recording controls and AF controls by repeatedly performing the video recording process. In this embodiment, only the video recording process is described, but the camera unit 20 may also be capable of performing still image recording processing.

[0056] First, in step S701, the camera control unit 204 determines whether a video recording start instruction (video recording instruction) has been input by touch operation on the camera operation unit 207 or the display unit 206. A video recording instruction is notified when the video recording switch on the camera operation unit 207 is pressed or when the video recording icon on the display unit 206 is pressed when video recording is not in progress. If a video recording instruction is notified, the camera control unit 204 proceeds to step S702; otherwise, it proceeds to step S703.

[0057] In step S702, the camera control unit 204 performs video recording processing and records the video image in the memory 208, then proceeds to step S706.

[0058] In step S703, the camera control unit 204 determines whether video recording is already in progress. If video recording is in progress, the process proceeds to step S704; otherwise, the process proceeds to step S706.

[0059] In step S704, the camera control unit 204 determines whether a video recording stop instruction has been input by touch operation on the camera operation unit 207 or the display unit 206. A video recording stop instruction is notified when the video recording switch on the camera operation unit 207 is pressed or when the video recording icon on the display unit 206 is pressed during video recording. If a video recording stop instruction is notified, the camera control unit 204 proceeds to step S705, and if no video recording stop instruction is notified, it proceeds to step S702 to continue the video recording process.

[0060] In step S705, the camera control unit 204 performs a video recording stop process to stop recording the video image to the memory 208, and proceeds to step S706.

[0061] In step S706, the camera control unit 204 performs AF area setting processing and proceeds to step S707. In the AF area setting processing, it sets which position of the subject within the image capture screen will be targeted for AF. In this embodiment, even when MF operation (described later) is being performed, the AF area is set in this step and used for AF operation after the MF operation is completed.

[0062] In step S707, the camera control unit 204 instructs the image plane phase-detection focus detection unit 205 to perform focus state detection processing. The details of the focus detection processing are as explained with reference to Figures 3 to 6, and the unit performs processing to acquire the amount of defocus required for image plane phase-detection AF and information on the reliability of the defocus amount, and then proceeds to step S708.

[0063] In step S708, the camera control unit 204 retains the defocus amount, the reliability of the defocus amount, and the focus lens position, and proceeds to step S709. This information is used in subsequent processing for detecting subject movement and for controlling the focus lens by predicting subject movement. The number and size of the history to be retained are determined by factors such as the movement speed of the subject targeted for focus tracking and the size of the installed ROM 204a.

[0064] In step S709, the camera control unit 204 performs subject movement detection processing and proceeds to step S710. Details of the subject movement detection processing will be described later.

[0065] In step S710, the camera control unit 204 performs focus drive processing and terminates the video recording process. Details of the focus drive processing will be described later.

[0066] Next, referring to Figure 8, the subject movement detection process performed by the camera control unit 204 in step S709 of Figure 7 will be explained.

[0067] In step S801, the camera control unit 204 determines whether the number of retained defocus amount history entries is greater than or equal to a predetermined value. If the number of retained history entries is greater than or equal to the predetermined value, the process proceeds to step S802. On the other hand, if the number of retained history entries is less than or equal to the predetermined value, the process proceeds to step S807. This determination corresponds to the determination of whether the history of information retained in the process described in step S708 in the video recording process flowchart of Figure 7 is greater than or equal to a predetermined value. The predetermined number that serves as the threshold can be determined in the subsequent process of detecting the movement of the subject, based on factors such as the movement detection accuracy and the size of the installed ROM 204a.

[0068] In step S802, the camera control unit 204 determines whether the reliability of all retained defocus amounts is above a predetermined level. If the reliability is above a predetermined level, the process proceeds to step S803. On the other hand, if there is a history of reliability not above a predetermined level, the process proceeds to step S807. The reliability threshold for the defocus amount set in step S802 is preferably set so that the calculated defocus amount and direction are reliable. This determination is made because if the reliability of the defocus amount is low, there is a concern that the movement of the subject may not be detected correctly. The reliability of the defocus amount may be determined using both the degree of two-image coincidence and the steepness of the image misalignment, or using only one of them. Other indicators such as signal level may also be used.

[0069] In step S803, the camera control unit 204 calculates subject movement information from the retained defocus amount and the position of the focus lens, and proceeds to step S804. Details will be described later.

[0070] In step S804, the camera control unit 204 determines whether or not there is a change in the movement of the subject in a specific direction. If there is a change in the movement of the subject, the process proceeds to step S805. On the other hand, if there is no change in the movement of the subject, the process proceeds to step S807. Details of this process will be described later.

[0071] In step S805, the camera control unit 204 determines whether the subject's movement is unidirectional or not. If the subject's movement is unidirectional, the process proceeds to step S806. On the other hand, if the subject's movement is not unidirectional, the process proceeds to step S807. Further details will be described later.

[0072] In step S806, the camera control unit 204 determines that it has detected movement of the subject and terminates the subject movement detection process. On the other hand, in step S807, the camera control unit 204 determines that it has not detected movement of the subject and terminates the subject movement detection process.

[0073] As explained with reference to Figure 8, the subject movement detection process determines, depending on the conditions, that subject movement has been detected in step S806, or that subject movement has not been detected in step S807. Figures 12(a) to (c) show an example of subject movement detection. Figures 12(a) to (c) are graphs in which the horizontal axis represents time and the vertical axis represents the focus position. The graphs show the ideal focus position corresponding to the position of the subject and the position of the focus lens 104 acquired by the camera control unit 204 via the lens control unit 111. Furthermore, the graphs show the focus position of the subject position calculated based on the amount of defocus at five points from detected time n-4 to time n.

[0074] Figures 12(a), (b), and (c) show examples of different subject movements. Figure 12(a) is an example where it is determined in step S806 that the subject movement has been detected, and it also shows the predicted focus position of the subject in the future. Figures 12(b) and (c) are examples where it is determined in step S807 that the subject movement has not been detected.

[0075] Figure 12(a) shows an example where, when referring to the ideal focus position of the subject, it can be seen that the subject is moving from infinity to near, indicating that the subject is moving in one direction and that we want to detect the movement of the subject. In response to this, in step S803, based on the actual focus position and the amount of defocus, information on the transition of the subject's position is obtained that is close to the transition of the ideal focus position of the subject, using the focus position information of the subject's position calculated based on the amount of defocus. Based on this information, in step S804, it is determined whether there is a transition of movement of the subject in a specific direction, and in Figure 12(a), it can be determined that there is a transition of movement of the subject in the near direction. To prevent false detection of movement when the subject is not actually moving, it is advisable to set thresholds, such as when the change in focus position is greater than a predetermined value, or when the subject moves in the same direction more than a predetermined number of times, and use these thresholds for determination.

[0076] Furthermore, in step S805, it is determined whether the movement of the subject is unidirectional or not. In Figure 12(a), it is unidirectional towards the nearest object, so based on these determination results, the state in step S806 is set to detect the movement of the subject. Note that when the movement of the subject is detected, in the process described later, the position of the subject is predicted and the focus is driven. Therefore, as information for prediction, Figure 12(a) shows the trajectory of the predicted focus position of the subject in the future.

[0077] Figure 12(b) shows an example where, when referring to the ideal focus position of the subject, there is little change, and it appears that the subject has hardly moved, so we do not want to detect the subject's movement. In such cases, the determination in step S804 whether or not there is a change in the subject's movement in a specific direction is made, and as mentioned above, based on information such as the small change in the focus position, it is determined that there is no change in movement, and in step S807, it is determined that no movement of the subject has been detected.

[0078] Figure 12(c) shows that, when referring to the ideal focus position of the subject, it can be seen that both movement toward infinity and movement toward the near side are occurring, making it difficult to predict the future movement of the subject. In this embodiment, in cases where it is difficult to predict the movement of the subject, that is, when the progression of the subject's movement is not unidirectional, the system is controlled to determine that no movement of the subject has been detected. In Figure 12(c), although it can be determined in step S804 that there is a progression of movement of the subject in a specific direction, it cannot be determined in step S805 that the progression of the subject's movement is unidirectional, so in step S807, the system is set to the state where no movement of the subject has been detected.

[0079] Next, referring to Figure 9, the focus drive process performed by the camera control unit 204 in step S710 of Figure 7 will be explained. Figure 9 is a flowchart of the focus drive process.

[0080] In step S901, the camera control unit 204 determines whether or not full-time manual focus (MF) operation is in progress. If full-time MF operation is in progress, the process proceeds to step S902. On the other hand, if full-time MF operation is not in progress, the process proceeds to step S905.

[0081] In step S902, the camera control unit 204 maintains the full-time MF operation direction and proceeds to step S903.

[0082] In step S903, the camera control unit 204 clears the AF tracking performance improvement state after focus operation and proceeds to step S904. The AF tracking performance improvement state after focus operation is information that is not used during full-time MF operation, as will be described in detail later, so it is preferable that it be in a cleared state.

[0083] In step S904, the camera control unit 204 performs focus drive processing by manual focus (MF) operation and then terminates the focus drive processing. The MF focus drive processing involves rotating the focus ring on the lens operation unit 112 in the infinity direction or the near-focus direction, and performing focus drive according to the amount of rotation. Alternatively, the MF may be performed not by operating the focus ring, but for example, by operating a button on the camera operation unit 207.

[0084] If it is determined in step S901 that full-time manual focus (MF) operation is not in progress, the camera control unit 204 proceeds to step S905, where it determines whether or not the full-time MF operation has ended. If the full-time MF operation was performed in step S901 and has now ended, the process proceeds to step S906. If full-time MF operation was not performed in the first place, or if the process described below, which is performed after the completion of full-time MF operation, has already been completed, the process proceeds to step S909.

[0085] In step S906, the camera control unit 204 determines whether or not it has detected movement of the subject. If it has detected movement of the subject, that is, if it has performed the process in step S806 in Figure 8, it proceeds to step S907. On the other hand, if it has not detected movement of the subject, that is, if it has performed the process in step S807 in the flowchart of Figure 8, it proceeds to step S910.

[0086] In step S907, the camera control unit 204 determines whether the direction of movement of the subject and the full-time MF operation direction (manual focus direction such as near focus or infinity) are the same. Based on the direction of movement of the subject detected in step S806 in Figure 8 and the full-time MF operation direction held in step S902, if both are the same direction, the process proceeds to step S908. On the other hand, if the direction of movement of the subject and the full-time MF operation direction are different, the process proceeds to step S910.

[0087] In step S908, the camera control unit 204 sets the camera to a state of improved AF tracking after focus operation and proceeds to step S909. If the following conditions are met in step S905, when the camera transitions from the full-time MF operation state to the termination state, when the movement of the subject is detected in step S906, and when the direction of movement of the subject matches the direction of full-time MF in step S907, the subsequent AF tracking performance is improved. This is because the user can determine that there is a high probability that the camera is tracking a subject moving in the depth direction during full-time MF operation. In this case, it is preferable that focus tracking of the subject can be maintained without delay even during AF after full-time MF. Details of the processing in the improved AF tracking state will be described later.

[0088] In step S909, the camera control unit 204 performs focus drive using autofocus (AF) and terminates the focus drive process. Details of the AF focus drive process will be described later.

[0089] If no subject movement is detected in step S906, or if the direction of subject movement and full-time MF operation differ in step S907, the camera control unit 204 proceeds to step S910, where it clears the defocus amount, reliability, and focus lens position retention history. Then, it proceeds to step S909. If no subject movement is detected, or if the direction of subject movement and full-time MF differ, no improvement in AF tracking performance is performed after full-time MF operation. Rather, there is a concern that using the information from the full-time MF operation in subsequent AF processing may result in focusing behavior that differs from the user's expectations. For this reason, various information such as the defocus amount retained during full-time MF is reset once.

[0090] Next, referring to Figure 10, the autofocus (AF) focus drive process performed by the camera control unit 204 in step S909 of Figure 9 will be explained. Figure 10 is a flowchart of the autofocus (AF) focus drive process.

[0091] In step S1001, the camera control unit 204 determines whether or not the AF tracking performance is improved after the focus operation. If the AF tracking performance is improved after the focus operation, the process proceeds to step S1002. On the other hand, if the AF tracking performance is not improved after the focus operation, the process proceeds to step S1009. The AF tracking performance improved after the focus operation is the state set in step S908 of Figure 9 when, after full-time MF operation, the direction of movement of the detected subject matches the direction in which the full-time MF operation was performed.

[0092] In step S1002, the camera control unit 204 determines whether or not it has detected a defocus amount within the depth of field for a predetermined time or longer. If it has not detected a defocus amount within the depth of field for a predetermined time or longer, it proceeds to step S1004. On the other hand, if it has detected a defocus amount within the depth of field for a predetermined time or longer, it proceeds to step S1003.

[0093] In step S1003, the camera control unit 204 determines whether the focus lens position has not been within a predetermined range for a predetermined time or longer. If the focus lens position has not been within a predetermined range for a predetermined time or longer, the process proceeds to step S1004. On the other hand, if the focus lens position has been within a predetermined range for a predetermined time or longer, the process proceeds to step S1007.

[0094] In step S1004, the camera control unit 204 determines whether the reliability of the defocus amount is above a predetermined level. If the reliability of the defocus amount is above a predetermined level, the process proceeds to step S1005. On the other hand, if the reliability of the defocus amount is not above a predetermined level, the process proceeds to step S1007. The reliability threshold of the defocus amount set in step S1004 is preferably set so that the calculated defocus amount and direction are reliable.

[0095] In step S1005, the camera control unit 204 sets the system to a state where transition to the focus stop state is prohibited, and proceeds to step S1006.

[0096] In step S1006, the camera control unit 204 sets to the predictive drive permission state and proceeds to step S1008.

[0097] In step S1007, the camera control unit 204 clears the AF tracking performance improvement state after the focus operation and proceeds to step S1008.

[0098] In step S1008, the camera control unit 204 performs AF execution processing and terminates the focus drive processing by AF. Details of the AF execution processing will be described later.

[0099] If step S1001 determines that the AF tracking performance has improved after the focus operation, steps S1002 to S1004 determine whether to continue the state. If it is determined that the state should be continued, step S1005 sets the state to prohibit transition to the focus stop state, step S1006 sets the state to allow predictive drive, and step S1008 performs the AF execution process.

[0100] On the other hand, if it is determined that the improved AF tracking performance state after the focus operation should not be continued, the improved AF tracking performance state after the focus operation is cleared in step S1007, and then the AF execution process is performed in step S1008. Regarding whether or not to continue the improved AF tracking performance state after the focus operation, first in step S1002, if the amount of defocus within the depth of field has not been detected for a predetermined time or longer, that is, if the subject has not yet been fully focused, the policy is to continue the state. Even if the amount of defocus within the depth of field has been detected for a predetermined time or longer, if the focus lens position has not been within a predetermined amount for a predetermined time or longer, that is, if the subject has been focused on but the subject continues to move, the policy is to continue the state. However, if the reliability of the amount of defocus is not above a predetermined level in step S1004, that is, if the subject cannot be captured, or if it is assumed that the subject has changed, tracking will be difficult, and the state will be cleared in step S1007.

[0101] If step S1001 determines that the AF tracking performance is not improved after the focus operation, the camera control unit 204 proceeds to step S1009, where it determines whether or not the subject is moving during AF operation. If the subject is moving, the process proceeds to step S1005. On the other hand, if the subject is not moving, the process proceeds to step S1008. Even if the AF tracking performance is not improved after the focus operation in step S1001, if step S1009 determines that the subject is moving during AF operation, steps S1005 and S1006 are used to set the AF tracking performance to be improved. This determination may be the same as the subject movement detection process in Figure 8, or it may be a different process that assumes AF operation. At least, if the AF tracking performance is not improved after the focus operation, various information such as the amount of defocus is cleared in step S910 in Figure 9, so even if the determination in step S1009 is made, the system is controlled not to improve tracking performance immediately after full-time MF operation.

[0102] Next, with reference to Figure 11, the AF execution process performed by the camera control unit 204 in step S1008 of Figure 10 will be explained. Figure 11 is a flowchart of the AF execution process.

[0103] In step S1101, the camera control unit 204 determines whether or not the predictive drive permission state is active. The predictive drive permission state is set in step S1006 in Figure 10. If the predictive drive permission state is active, the process proceeds to step S1102. On the other hand, if the predictive drive permission state is not active, the process proceeds to step S1104.

[0104] In step S1102, the camera control unit 204 performs predictive driving by setting the lens drive settings for driving the focus lens 104 based on the history of past focus positions and the history of defocus amounts, and then proceeds to step S1103.

[0105] In step S1103, the camera control unit 204 sends a drive command for the focus lens 104 to the lens control unit 111 based on the lens drive setting information set in step S1102, and terminates the AF execution process.

[0106] In step S1104, the camera control unit 204 determines whether or not the camera is in a state where autofocus has stopped. If it is not in a state where autofocus has stopped, the process proceeds to step S1105. On the other hand, if it is in a state where autofocus has stopped, the process proceeds to step S1112.

[0107] In step S1105, the camera control unit 204 determines whether the reliability of the defocus amount is above a predetermined level. If the reliability of the defocus amount is above a predetermined level, the process proceeds to step S1106. On the other hand, if the reliability of the defocus amount is not above a predetermined level, the process proceeds to step S1110. Preferably, the reliability threshold of the defocus amount set in step S1105 is set to the maximum value of the reliability range in which not only the calculated defocus amount but also the defocus direction is unreliable. The reliability of the defocus amount may be determined using both the degree of two-image agreement and the steepness of the image misalignment, or using only one of them. Other indicators such as signal level may also be used.

[0108] In step S1106, the camera control unit 204 determines whether the amount of defocus is within the depth of field. If the amount of defocus is within the depth of field, the process proceeds to step S1107. On the other hand, if the amount of defocus is not within the depth of field, the process proceeds to step S1108.

[0109] In step S1107, the camera control unit 204 considers the amount of defocus to be within the depth of field and transitions to the focus stop state, and terminates the AF execution process.

[0110] In step S1108, the camera control unit 204 assumes that focus has not yet been achieved, sets the lens drive settings to drive the focus lens 104 based on the amount of defocus, and proceeds to step S1109.

[0111] In step S1109, the camera control unit 204 sends a drive command for the focus lens 104 to the lens control unit 111 based on the defocus amount and the lens drive setting information set in step S1108, and terminates the AF execution process.

[0112] In step S1110, the camera control unit 204 cannot use the defocus amount to drive the focus lens 104 because the defocus amount is unreliable. Therefore, the camera control unit 204 performs a search drive to calculate the defocus amount while moving the focus lens 104 toward its movable end in order to detect the position of the focus lens 104 where a highly reliable defocus amount can be obtained. For this reason, the camera control unit 204 first sets the lens drive settings for the search drive. The lens drive settings for the search drive include settings such as the drive speed of the focus lens 104 and the direction in which the drive starts. After setting these, the process proceeds to step S1111.

[0113] In step S1111, the camera control unit 204 sends a control command for the focus lens 104 to the lens control unit 111 based on the lens drive settings for search drive set in step S1110, and terminates the AF execution process.

[0114] In step S1112, the camera control unit 204 determines whether the amount of defocus is within the depth of field. If the amount of defocus is within the depth of field, the process proceeds to step S1113, and the focus-stopped state is maintained. On the other hand, if the amount of defocus is not within the depth of field, the process proceeds to step S1114.

[0115] In step S1113, the camera control unit 204 maintains the focus stop state and terminates the AF execution process.

[0116] In step S1114, the camera control unit 204 determines whether the state in which the amount of defocus is not within the depth of focus has continued for a predetermined time. If this condition is met, the process proceeds to step S1115. On the other hand, if this condition is not met, the process proceeds to step S1113.

[0117] In step S1115, the camera control unit 204 terminates the AF execution process by releasing the focus stop state in order to track the focus change if the amount of defocus remains outside the depth of field for a predetermined period of time.

[0118] In this embodiment, if the system is set to the AF tracking improvement state after focus operation in step S908 of Figure 9, and the system is set to the predictive drive permission state in step S1006 through the process in Figure 10, the predictive drive process is performed in steps S1101 to S1103. As a result, even when switching to AF after full-time MF operation, the predictive drive process is performed immediately, thereby improving subject tracking performance. Furthermore, since the transition to the focus stop state in step S1107 is not performed when the predictive drive process is executed, subject tracking performance can be improved in this respect as well.

[0119] Figures 13(a) to 13(d) illustrate the problems that arise when this embodiment is not applied. Figures 13(a) to 13(d) depict a scene in which a person is approaching. Figures 13(a) to 13(d) also show the chronological changes in the same scene, with the state progressing sequentially from Figure 13(a) to Figure 13(d). Figures 13(a) to 13(d) also show the focus state of the person.

[0120] Figure 14 shows the ideal and actual focus positions of the subject in each scene of Figures 13(a) to (d). Figures 13(a) to (d) correspond to the time ranges (a) to (d) in Figure 14. Figures 13(a) to (d) will be explained in conjunction with Figure 14.

[0121] Figure 13(a) shows a state where the person is stationary and has not yet started moving. The person is in focus, and for the range (a) in Figure 14, the ideal focus position of the subject position is the same as the actual focus position, resulting in a focused state. At this point, the focus is stopped, as in step S1107 of Figure 11.

[0122] Figure 13(b) shows the state in Figure 13(a) after the person suddenly starts moving towards the near side. In the range (b) of Figure 14, because the person suddenly started moving towards the near side, the ideal focus position of the subject has also shifted towards the near side. On the other hand, especially in video recording as explained in Figure 7, the behavior of the focus lens when focusing is recorded as part of the video. Therefore, control is taken to prevent undesirable behavior of the focus lens, such as the focus immediately shifting when a subject other than the subject of filming crosses in front of the person. From the state in which the focus is stopped in step S1107 of Figure 11, the focus is stopped in step S1105 when the amount of defocus is not within the depth of field for a predetermined period of time, as determined in steps S1112 and S1114. After the focus is stopped, the AF drives the focus lens to focus in the processes of steps S1103, S1109 and S1111.

[0123] Figure 13(b) shows that the focus has not been able to track the change in subject distance caused by the person suddenly moving closer, because the focus has remained locked. Therefore, in the range (b) of Figure 14, the actual focus position has not changed, and a difference has occurred between the subject's position and the ideal focus position, meaning that the image is out of focus.

[0124] Figure 13(c) shows the state in Figure 13(b) after the user has recovered the focus tracking delay using full-time manual focus (MF) operation and focused on a person who is continuously moving toward the near side. As mentioned above, in the AF execution process in Figure 11, it is possible to expect to track focus on the person by waiting for the state to change from the focus stopped state to the state in which the focus lens is driven. However, one way for the user to track focus at an earlier stage is to use full-time manual focus operation. This is the state after executing step S904 of the focus drive process in Figure 9.

[0125] In the area (c) of Figure 14, although the ideal focus position of the subject is constantly shifting towards the near side because the person is continuously moving towards the near side, the actual focus position catches up to the ideal focus position of the subject with full-time manual focus operation.

[0126] Figure 13(d) shows the state in Figure 13(c) after the user has stopped full-time manual focus operation and switched to autofocus operation for a person who is continuously moving toward the near side.

[0127] In the area (d) of Figure 14, although the ideal focus position of the subject is constantly shifting toward the near side because the person is continuously moving toward the near side, there is a difference in the actual focus position, meaning that blurring occurs. This blurring is as shown in Figure 13(d).

[0128] Without applying this embodiment, a delay in focus tracking is a concern. This means that the AF tracking improvement state after the focus operation in step S1001 of Figure 10 is absent, and the system always operates in a state equivalent to not meeting the conditions. In such a state, when transitioning from full-time MF operation to AF operation, the AF distance measurement information from the full-time MF operation, i.e., information on whether the person is moving to the near side, cannot be utilized, and predictive drive processing like that in step S1103 of Figure 11 cannot be performed. In particular, if the person is moving at a high speed, and the amount of drive required for focus tracking of the focus lens 104 is large, the focus tracking may lag behind the person, as shown in range (d) of Figure 13(d) and Figure 14(d), potentially causing the person to become blurred.

[0129] In the explanation of Figures 13(a) to (d), the change in subject distance of a person was used as an example, but the type of subject is not limited to people; any subject in which a change in subject distance occurs can be used. For example, it could be a scene where an animal approaches, or a scene where someone is holding something and introducing it. Also, it could be a case where the subject moves away instead of approaching.

[0130] Figure 15 shows another example of the ideal and actual focus positions of the subject in each scene of Figures 13(a) to (d). The ranges (a) to (d) in Figure 15 correspond to Figures 13(a) to (d), respectively, as in Figure 14. Note that the ranges (a) to (c) in Figure 15 are the same as the ranges (a) to (c) in Figure 14, so their explanation is omitted.

[0131] In range (d) of Figure 15, the actual focus position catches up to the ideal focus position of the subject after the user stops full-time MF operation and switches to AF operation, meaning that focus tracking was possible from the start of AF movement. The ability of AF to track focus varies depending on the performance of the focus actuator 107 of the lens unit 10 attached to the camera unit 20, and the focus speed information set in the lens drive setting for defocus amount drive set in step S1108 of Figure 11. Therefore, in some cases, as shown in range (d) of Figure 15, it may be possible to track focus on the subject from the start of AF movement.

[0132] However, range (d) in Figure 15 shows the case where, in step S1106 of Figure 11, it is determined that the amount of defocus is within the depth of field, and the system transitions to the focus stop state in step S1107. When the system transitions to the focus stop state in step S1107, the focus lens drive is temporarily suspended until it is determined in step S1112 that the amount of defocus is not within the depth of field, or in step S1114 that this state has continued for a predetermined time.

[0133] In scenes where the subject being filmed is not moving, the stability of focus during video recording can be improved. On the other hand, in scenes like this one, where a person is continuously moving towards the viewer, a focus delay occurs. As shown in Figures 13(a) to (d), Figure 14, and Figure 15, without applying this embodiment, when filming a subject moving in the depth direction, there is a possibility of a delay in focus tracking during AF operation after full-time MF operation.

[0134] Figures 16(a), (b), (c), and (d') illustrate an example of the effect of applying this embodiment. Figures 16(a), (b), (c), and (d') show a scene in which a person approaches, similar to Figures 13(a) to (d). Figures 16(a), (b), (c), and (d') show the chronological changes of the same scene, with time progressing in the order of Figures 16(a), (b), (c), and (d'). Figures 16(a), (b), (c), and (d') also show the focus state of the person. Note that Figures 16(a), (b), and (c) are the same as Figures 13(a), (b), and (c), so their explanations are omitted.

[0135] Figure 17 shows the ideal and actual focus positions of the subject in each scene of Figure 16(a), (b), (c), and (d'). Figures 16(a), (b), (c), and (d') correspond to the time ranges (a), (b), (c), and (d') in Figure 17, respectively. The ranges (a), (b), and (c) in Figure 17 are basically the same as the ranges in Figure 14(a), (b), and (c), so their explanations are omitted. However, it differs in that the focus position of the subject, calculated based on the amount of defocus, is retained before the range in Figure 17(d'), which will be explained later.

[0136] Figure 16(d') shows the state in Figure 16(c) after the user has stopped full-time manual focus operation and switched to autofocus operation for a person who is continuously moving toward the near side. In the range of Figure 17 (d'), because the person is continuously moving toward the near side, the ideal focus position of the subject is constantly shifting toward the near side, and the actual focus position is able to follow this, that is, the focus is being tracked near the point of focus.

[0137] By applying this embodiment, the focus tracking delay occurring in the range (d) of Figures 13(d) and 14 can be improved. This is because, in step S901 of Figure 9, the full-time MF operation was performed, and in step S905, the full-time MF operation is terminated. Furthermore, in step S906, as explained in Figure 12(a), movement of the subject toward the nearest object is detected. Furthermore, in step S907, since the full-time MF operation is performed toward the nearest object, it coincides with the aforementioned movement of the subject toward the nearest object, so in step S908, the system is set to the AF tracking improvement state after focus operation. Then, it is determined that the system is in the AF tracking improvement state after focus operation as shown in step S1001 of Figure 10. Furthermore, after the cancellation determination from steps S1002 to S1004, the system is set to the state where transition to the focus stop state is prohibited in step S1005, and to the state where predictive drive is permitted in step S1006.

[0138] As a result, after the determination in step S1101 in Figure 11, the predictive drive processing in steps S1102 and S1103 is performed. In this way, in the AF operation after the end of full-time MF operation, focus tracking can be continued by predictive drive AF based on the focus position of the subject position calculated from the amount of defocus held in the range before range (d') in Figure 17, i.e., the movement history information of the person during MF operation. Furthermore, since the transition to the focus stop state is prohibited in step S1005 in Figure 10, the focus tracking delay caused by the transition to the focus stop state can also be suppressed, as explained in range (d) in Figure 15.

[0139] Furthermore, regarding the cancellation process in steps S1002 to S1004 in Figure 10, in the case of Figure 17, even if the amount of defocus within the depth of field is detected for a predetermined time in step S1002, the focus lens position continues to change. Therefore, the conditions for step S1003 are not met. As a result, the improved AF tracking performance after the focus operation can be maintained.

[0140] Furthermore, if any changes occur after the range (d') in Figure 17, such as the human subject stopping or the subject changing, the AF tracking performance enhancement state after focus operation will be canceled according to the conditions in steps S1002 to S1004. Subsequently, when full-time MF operation is performed again, it is determined again in Figure 9 whether or not to set the AF tracking performance enhancement state after focus operation.

[0141] In the explanation of Figures 16(a), (b), (c), and (d'), the change in the subject distance of a person was used as an example. However, as with Figures 13(a) to (d), the type of subject is not limited to people; any subject that causes a change in subject distance is acceptable. For example, it could be a scene where an animal approaches, or a scene where someone is holding something and introducing it. Furthermore, this embodiment is not limited to cases where the subject approaches, but can also be applied when the subject moves away.

[0142] In this embodiment, preferably, the control means 2042 changes the tracking performance in autofocus after manual focusing is completed, depending on whether the direction of manual focus and the direction of movement of the subject are the same. Also preferably, the acquisition means 2041 acquires subject movement information using the defocus amount and reliability information of the defocus amount acquired from the focus information. Also preferably, the acquisition means 2041 determines that the subject has moved if the subject is moving in one direction within a predetermined time.

[0143] Preferably, when the subject moves, the control means 2042 improves tracking performance compared to when the subject is not moving by making it less likely for the focus lens, which is driven in the direction of the subject's movement, to stop. Preferably, after deciding to improve tracking performance, the control means 2042 restores the tracking performance to its original state (cancels the improvement in tracking performance) if the position of the focus lens remains within a predetermined amount for a predetermined time and the amount of defocus remains within a predetermined amount for a predetermined time. Preferably, after deciding to improve tracking performance, the control means 2042 restores the tracking performance to its original state if the reliability of the amount of defocus falls below a predetermined value.

[0144] Preferably, when the subject moves, the control means 2042 predicts the movement of the subject using the past position information and focus information of the focus lens, and operates in a way that makes it easier to drive the focus lens, thereby improving the tracking performance compared to when the subject is not moving. Preferably, when the subject moves, the control means 2042 predicts the movement of the subject using the past position information and focus information of the focus lens, thereby improving the tracking performance compared to when the subject is not moving.

[0145] As described above, in this embodiment, after full-time MF operation, it is determined whether or not to enter a state of improved AF tracking performance after focus operation. If movement of the subject is detected and the direction of movement of the subject matches the direction of the full-time MF operation, the system is set to the state of improved AF tracking performance after focus operation. When the system is set to the state of improved AF tracking performance after focus operation, the system prevents transition to the focus stop state and enables predictive drive during AF operation after full-time MF operation. This improves focus tracking performance even if there is a delay in focusing on the subject during AF operation after full-time MF operation.

[0146] (Second Embodiment) Next, a second embodiment of the present invention will be described. In this embodiment, the same configuration as in the first embodiment will not be described.

[0147] [Details of various processes performed by the camera unit 20] In this embodiment, the camera unit 20 performs the focus drive process shown in Figure 18 in step S710 of Figure 7. Figure 18 is a flowchart of the focus drive process in this embodiment. Note that the processes in steps S1801 to S1808, S1811, and S1812 in Figure 18 are the same as the processes in steps S901 to S910 in Figure 9, respectively, and therefore a detailed explanation is omitted.

[0148] In step S1809 of Figure 18, the camera control unit 204 sets the waiting time until AF starts to 0 if the condition set in step S1808 to improve AF tracking after focus operation, and proceeds to step S1810.

[0149] In step S1810, the camera control unit 204 determines whether the waiting time before starting AF has elapsed. If the waiting time has elapsed, the process proceeds to step S1811, and the AF focus drive process is executed. On the other hand, if the waiting time has not elapsed, the focus drive process is terminated without executing the AF focus drive process in step S1811.

[0150] In step S1813, the camera control unit 204, under conditions where it is not set to the AF tracking performance improvement state after focus operation, sets the waiting time until AF starts to X and proceeds to step S1813. This waiting time X is set to a value greater than 0.

[0151] In this embodiment, the camera control unit 204 has a process for setting a waiting time and delaying the start of AF processing during the set waiting time when performing AF processing after full-time MF operation. The waiting time is set to 0 when setting the AF tracking performance improvement state after focus operation, and to X (greater than 0) otherwise.

[0152] Referring to Figures 19(a) to (c) and Figure 20, we will explain when it is advisable to allow a waiting time between full-time MF operation and AF processing. Figure 19 shows an example of a case where it is advisable to allow a waiting time between full-time MF operation and AF processing.

[0153] Figures 19(a) to (c) show scenes of animals inside cages. In Figure 19(a), the cage is in focus, while the animal is out of focus. In Figure 19(b), the focus is set to an intermediate position between the cage and the animal, resulting in both the cage and the animal being out of focus. In Figure 19(c), the animal is in focus, while the cage is out of focus. Figure 20 shows examples of how the actual focus position changes when full-time manual focus operation is started and stopped in the scenes shown in Figures 19(a) to (c). In Figure 20, the vertical axis represents the focus position, showing the focus positions for the cage and the animal, respectively. The focus position for the cage corresponds to the state in Figure 19(a), and the focus position for the animal corresponds to the state in Figure 19(c). Additionally, there is a state corresponding to Figure 19(b) between the focus position for the cage and the focus position for the animal. In Figure 20, the horizontal axis represents time, and the changes in the focus position will be described later, focusing on each of the times l, m, and n.

[0154] As a use case for full-time manual focus, we have given examples in Figures 13(a)-(d) and 16(a)-(d') where the focus tracks the subject as it moves closer or further away. In Figures 19(a)-(c), neither the cage nor the animal moves closer or further away. However, there may be cases where the user wants to focus on the animal, but the AF processing causes the focus to shift to the cage, which is closer. In this case, one use case is to use full-time manual focus to change the focus point, thereby shifting the focus from the cage to the animal.

[0155] At time l in Figure 20, the camera starts full-time manual focus (MF) operation in the direction of the animal, i.e., infinity, from a state where the focus is on the cage, as in Figure 19(a). However, during full-time MF operation, if the amount of rotation required for the focus ring of the lens control unit 112 is large, it may not be possible to bring the camera into focus on the animal in a single full-time MF operation, as in Figure 19(c). At time m in Figure 20, assuming such a case, the camera stops the first full-time MF operation and temporarily brings the camera into focus between the cage and the animal, as in Figure 19(b). From this state, by preparing for a second full-time MF operation and re-rotating the focus ring of the lens control unit 112, it is possible to move from the state in Figure 19(b) closer to the state in Figure 19(c).

[0156] However, after time m, while preparing to turn the focus ring again, the AF process may operate, causing the focus to return from the state in Figure 19(b) to the state in Figure 19(a) where the focus is on the cage. At time n, as mentioned above, the AF process has returned the focus to the cage in Figure 19(a). To avoid this problem, by adding a waiting time between full-time MF operation and the start of AF processing, it is possible to prevent the focus position from immediately returning to the state in Figure 19(a) as shown from time m to n in Figure 20. Thus, when the subject is not moving in the depth direction and you want to change from one subject to another, it is preferable to add a waiting time between full-time MF operation and the start of AF processing.

[0157] Steps S1810 and S1813 in Figure 18 represent the process assuming such a case. The waiting time X set in step S1813 should be determined by considering, for example, the time between performing a full-time MF operation and performing another full-time MF operation.

[0158] On the other hand, in the case of scenes that are problematic in this embodiment, such as those shown in Figures 13(a) to (d), where the subject is moving in the depth direction, the process of adding a waiting time between the start of AF after the full-time MF operation is undesirable. Figure 21 shows examples of the ideal focus position and the actual focus position of the subject in each of the scenes in Figures 13(a) to (d), with a waiting time added between the start of AF after the full-time MF operation. Note that the ranges (a), (b), and (c) in Figure 21 are the same as the ranges (a), (b), and (c) in Figure 14 described in the first embodiment, so their explanation is omitted.

[0159] Area (d) in Figure 21 shows the state after focusing on the subject using full-time MF operation in area (c) of Figure 21, and then handing over to AF processing. In this case, if the start of AF processing is delayed by the waiting time set in step S1813 of Figure 18, it will not be possible to immediately track a subject that is continuously approaching, as shown in area (d) of Figure 21. In this embodiment, even if there is a mechanism to delay the start of AF processing after full-time MF operation, step S1806 of Figure 18 detects the movement of the subject, and step S1807 determines whether the direction of movement of the subject is the same as the direction of full-time MF operation. If this condition is met, that is, if it is assumed that the subject is moving in the depth direction and being tracked with full-time MF, then in step S1809, the waiting time until the start of AF after full-time MF operation is set to 0.

[0160] In this embodiment, preferably, the control means 2042 improves tracking performance compared to when the subject is stationary by shortening the waiting time until autofocus starts when the subject moves.

[0161] As a result, according to this embodiment, similar to the first embodiment, it is possible to suitably track focus on a subject moving in the depth direction during AF processing after full-time MF operation. In this embodiment, the waiting time until AF starts is set to 0 in step S1809, but the waiting time does not have to be 0; it should be shorter than X set in step S1813.

[0162] (Third embodiment) Next, a third embodiment of the present invention will be described. Note that the same configuration as in the first embodiment will not be described.

[0163] [Configuration of the imaging device] First, with reference to Figure 22, an example of the functional configuration of a digital camera 100a as an example of an imaging device according to this embodiment will be described. Figure 22 is a block diagram of a digital camera (imaging device) 100a. The digital camera 100a is composed of a lens unit 10 and a camera unit 30. In Figure 22, the lens unit 10 and its internal configuration are the same as those of the lens unit 10 and its internal configuration in Figure 1, so a detailed explanation is omitted. Also, parts 201 to 208 of the camera unit 30 in Figure 22 are the same as parts 201 to 208 of the camera unit 20 in Figure 1, so a detailed explanation is omitted.

[0164] The camera unit 30 of the digital camera 100a differs from the camera unit 20 of the digital camera 100 in that it has a subject detection unit 309. The camera control unit 204 controls the subject detection unit 309 to exchange information.

[0165] The subject detection unit 309 performs subject detection based on image data obtained by the image processing circuit 203. Subject detection, which estimates the position of the target subject in the image data, is used by the camera control unit 204 to select the focus adjustment result of the imaging plane phase-difference focus detection unit 205 for driving the focus lens 104 via the lens control unit 111. Subjects to be detected include, for example, a person's face and its eyes, an animal's torso and its face / eyes, and the entire vehicle and its characteristic parts (such as the driver or cockpit). Furthermore, it is possible to detect the subject's movement not only from the face but also from various parts such as the torso, arms, and legs, as well as information on their arrangement and shape. For example, it is possible to determine whether a person is running based on the posture of their arms and legs positioned forward and backward. In addition, subjects located at a position specified by the user within the imaging screen are detected via user touch operation on the display unit 206. Various information related to subject detection, such as the size of the detected subject, is also used to control the camera control unit 204.

[0166] [Details of various processes performed by the camera unit 30] In this embodiment, the subject movement detection process in step S709 of Figure 7 follows the flowchart shown in Figure 23. Figure 23 is a flowchart of the subject movement detection process in this embodiment. Note that the processes in steps S2301 to S2306 and S2309 of Figure 23 are the same as the processes in steps S801 to S807 of Figure 8, respectively, so a detailed explanation is omitted.

[0167] In step S2307 of Figure 23, the camera control unit 204 determines, based on the information from the subject detection unit 309, whether the subject detection frame size is changing to be larger or smaller. If the subject detection frame size is continuously increasing or decreasing, the process proceeds to step S2306. Otherwise, the process proceeds to step S2308.

[0168] In step S2308, the camera control unit 204 determines, based on the information from the subject detection unit 309, whether the subject is moving or in a position where it may move. If the subject is moving or in a position where it may move, the process proceeds to step S2306. Otherwise, the process proceeds to step S2309.

[0169] In this embodiment, in the subject movement detection process, in addition to the change in the defocus amount in steps S2301 to S2305, information from the subject detection unit 309 (subject movement information) is also used. If the subject detection size changes significantly in step S2307, it is possible that the subject is approaching or moving away. In such cases, it is determined in step S2306 that the movement of the subject has been detected. In step S2308, it is determined whether the subject is in a moving posture or a posture that could potentially move, and if so, it is also determined in step S2306 that the movement of the subject has been detected. Examples of a moving posture include running or dribbling in soccer or basketball. An example of a posture that could potentially move is a crouching start posture, where the subject is about to start running. When such a posture or action is detected, it is determined in step S2306 that the movement of the subject has been detected. On the other hand, if a posture is detected in which there is no movement in the depth direction, or the movement is small, such as the posture of shooting in soccer or basketball, or the posture of an animal sitting, it is determined in step S2309 that no movement of the subject has been detected.

[0170] In this embodiment, preferably, the acquisition means 2041 acquires movement information of the subject using information about changes in the subject's size. Alternatively, preferably, the acquisition means 2041 acquires movement information of the subject using motion information of the subject.

[0171] (Fourth Embodiment) Next, a fourth embodiment of the present invention will be described. Note that the same configuration as in the first embodiment will not be described.

[0172] [Details of various processes performed by the camera unit 20] In this embodiment, the focus drive process in step S710 of Figure 7 follows the flowchart in Figure 24. Figure 24 is a flowchart of the focus drive process in this embodiment. Note that the processes in steps S2401 to S2809 and S2812 in Figure 24 are the same as the processes in steps S901 to S910 in Figure 9, respectively, so a detailed explanation is omitted.

[0173] In step S2410 of Figure 24, the camera control unit 204 determines whether or not an AF execution start operation has been performed. If an AF execution start operation has been performed, the process proceeds to step S2406. On the other hand, if an AF execution start operation has not been performed, the process proceeds to step S2409. An AF execution start operation indicates that the focus adjustment start switch on the camera operation unit 207 of the camera unit 20 has been pressed, and that an AF execution start instruction has been given by the user.

[0174] In step S2411, the camera control unit 204 determines whether or not the AF execution start operation has been performed, similar to step S2410. If the AF execution start operation has been performed, the process proceeds to step S2408. On the other hand, if the AF execution start operation has not been performed, the process proceeds to step S2412.

[0175] In this embodiment, even when the user initiates AF execution rather than performing full-time MF operation, if movement of the subject is detected in step S2406, the system is set to the AF tracking improvement state after focus operation in step S2408. However, unlike full-time MF operation, the AF execution start operation generally does not allow specifying the direction in which the focus lens 104 should be driven. Therefore, as in step S2407, it is not possible to determine whether the direction of movement of the subject and the direction in which the focus lens 104 should be driven coincide. For this reason, when the AF execution start operation is performed, in step S2411, the system is set to the AF tracking improvement state after focus operation in step S2408, regardless of the conditions in step S2407.

[0176] Figure 25 shows the ideal focus position and the actual focus position of the subject in each scene of Figures 16(a) to (d') when the AF execution start operation is performed in this embodiment.

[0177] Figures 16(a), (b), (c), and (d') correspond to the ranges (a), (b), (c'), and (d') in Figure 25, respectively. Since the ranges (a), (b), and (d') in Figure 25 are basically the same as the ranges (a), (b), and (d') in Figure 17 described in the first embodiment, their explanation is omitted.

[0178] In this embodiment, Figure 16(c) shows the state shown in Figure 16(b), that is, the state in which the person has started moving to the near side and the AF tracking is lagging, and the user has initiated AF execution. In the range (b) of Figure 25, the movement of the subject has already been detected based on the focus position of the subject position calculated from the amount of defocus. When the user initiates AF execution at the timing of the transition from Figure 25(b) to (c'), it is determined in step S2410 of Figure 24 that the operation to initiate AF execution has been performed. As mentioned above, since the movement of the subject has already been detected, the process proceeds from step S2406 to step S2407.

[0179] In step S2407, since full-time MF operation is not performed and it is not possible to determine if the direction of movement of the subject matches, the process proceeds to step S2411. When the AF execution start operation is performed, the matching of the direction is not determined, and the process proceeds to step S2408, setting the system to the state of improved AF tracking after focus operation. As a result, looking at the range (c') in Figure 25, the actual focus position catches up to the ideal focus position of the subject position at an early stage, meaning that the delay in focus tracking is recovered at an early stage, and the AF tracking performance is improved. Note that even if the AF execution start operation is canceled after the AF execution start operation has been performed and the system has been set to the state of improved AF tracking after focus operation in step S2408, the AF processing continues. Therefore, unless the conditions for clearing the state of improved AF tracking after focus operation are met by the processing in steps S1002 to S1004 and S1007 in Figure 10, the state of improved AF tracking is maintained regardless of whether the AF execution start operation is ongoing or stopped.

[0180] In this embodiment, preferably, the acquisition means 2041 acquires subject movement information when the second operating means (focus adjustment start switch of the camera operating unit 207) that instructs the start of autofocus is operated. Also preferably, when the second operating means is operated, the control means 2042 changes the autofocus tracking performance according to the subject movement information.

[0181] As mentioned above, unlike full-time manual focus (MF) operation, the instruction to start AF execution generally cannot be set to move the focus lens 104 in the desired direction. Therefore, it is not possible to determine whether the direction of subject movement and the direction in which AF execution is desired match. Consequently, the improvement in AF tracking performance after full-time MF operation is more accurate in terms of the degree to which the direction of subject movement and the direction of focus lens drive match than the improvement in AF tracking performance by instructing the start of AF execution. On the other hand, instructing the start of AF execution is easier to use because it does not require manual focus operation, which requires a certain level of skill, and is completed with AF operation.

[0182] Furthermore, if there is a function that allows the driver to determine the direction in which the focus lens 104 should move when instructing the start of AF execution, the accuracy can be further improved by taking into account the drive direction of the focus lens as a condition for improving AF tracking performance. For example, by excluding the processing in step S2411 in Figure 24, the control in step S2407 can be configured to check whether not only the full-time MF operation direction but also the AF direction at the start of AF execution matches the direction of movement of the subject.

[0183] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0184] According to each embodiment, the autofocus tracking performance can be improved. Therefore, according to each embodiment, a control device, imaging device, control method, and program that enable appropriate AF operation can be provided.

[0185] Each embodiment of the disclosure includes the following configuration and method. (Composition 1) A means for acquiring information on the movement of the subject, It includes a control means for controlling the focus lens using focus information acquired from the imaging signal, The control means is characterized by changing the processing related to the autofocus tracking performance according to the movement information of the subject. (Configuration 2) The aforementioned movement information is information regarding whether or not the subject has moved. The control means is The control device according to configuration 1, characterized in that when the subject moves, the tracking process is changed to improve the tracking performance compared to when the subject is not moving. (Composition 3) The control device according to configuration 2, characterized in that the direction of movement of the subject is the direction in which the subject approaches or moves away from it. (Composition 4) The control device according to any one of configurations 1 to 3, characterized in that the acquisition means acquires the movement information of the subject when the first operating means for performing manual focus is operated. (Composition 5) The control device according to configuration 4, characterized in that the control means changes the tracking performance in the autofocus after the completion of manual focusing by the first operating means, depending on whether the direction of manual focusing by the first operating means and the direction of movement of the subject are the same. (Composition 6) The control device according to any one of configurations 1 to 5, characterized in that the acquisition means acquires the movement information of the subject when the second operating means for instructing the start of autofocus is operated. (Composition 7) The control device according to configuration 6, characterized in that when the second operating means is operated, the control means changes the tracking performance in the autofocus according to the movement information of the subject. (Composition 8) The control device according to any one of configurations 1 to 7, characterized in that the acquisition means acquires the movement information of the subject using the amount of defocus and reliability information of the amount of defocus acquired from the focus information. (Composition 9) The control device according to any one of configurations 1 to 8, characterized in that the acquisition means acquires the movement information of the subject using information regarding the change in the size of the subject. (Composition 10) The control device according to any one of configurations 1 to 9, characterized in that the acquisition means acquires the movement information of the subject using the motion information of the subject. (Composition 11) The control device according to any one of configurations 1 to 10, characterized in that the acquisition means determines that the subject has moved if the subject is moving in one direction during a predetermined period of time. (Composition 12) The control device according to any one of configurations 1 to 11, characterized in that the control means makes it difficult to stop the focus lens which is driven in the direction of movement of the subject when the subject moves, thereby improving the tracking performance compared to when the subject is not moving. (Composition 13) The control device according to any one of configurations 1 to 12, characterized in that, when the subject moves, the control means predicts the movement of the subject using the past position information and focus information of the focus lens, and makes it easier to drive the focus lens, thereby improving the tracking performance compared to when the subject is not moving. (Composition 14) The control device according to any one of configurations 1 to 13, characterized in that when the subject moves, the control means predicts the movement of the subject using the past position information of the focus lens and the focus information, thereby improving the tracking performance compared to when the subject is not moving. (Composition 15) The control device according to any one of configurations 1 to 14, characterized in that the control means improves the tracking performance compared to when the subject is not moving by shortening the waiting time until the autofocus starts when the subject moves. (Composition 16) The control device according to any one of configurations 1 to 15, characterized in that, after deciding to improve the tracking performance, the control means restores the tracking performance to its original state if the position of the focus lens remains within a predetermined amount for a predetermined time and the amount of defocus remains within a predetermined amount for a predetermined time, or if the reliability of the amount of defocus falls below a predetermined value. (Composition 17) An imaging apparatus characterized by having a control device according to any one of configurations 1 to 16 and an image sensor. (Method 1) Steps to acquire information about the subject's movement, The process includes the step of controlling the focus lens using focus information acquired from the imaging signal, A control method characterized in that, in the step of controlling the focus lens, the process relating to the autofocus tracking performance is changed according to the movement information of the subject. (Composition 18) A program characterized by causing a computer to execute the control method described in Method 1.

[0186] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist. [Explanation of Symbols]

[0187] 204 Camera Control Unit (Control Device) 2041 Acquisition method 2042 Control means

Claims

1. A means for acquiring information on the movement of the subject, It includes a control means for controlling the focus lens using focus information acquired from the imaging signal, The control means is characterized by changing the processing related to the autofocus tracking performance according to the movement information of the subject.

2. The aforementioned movement information is information regarding whether or not the subject has moved. The control means is The control device according to claim 1, characterized in that when the subject moves, the tracking process is modified to improve the tracking performance compared to when the subject is not moving.

3. The control device according to claim 2, characterized in that the direction of movement of the subject is the direction in which the subject approaches or moves away from the subject.

4. The control device according to claim 1, characterized in that the acquisition means acquires the movement information of the subject when the first operating means for performing manual focus is operated.

5. The control device according to claim 4, characterized in that the control means changes the tracking performance in the autofocus after the completion of manual focusing by the first operating means, depending on whether the direction of manual focusing by the first operating means and the direction of movement of the subject are the same.

6. The control device according to claim 1, characterized in that the acquisition means acquires the movement information of the subject when the second operating means for instructing the start of autofocus is operated.

7. The control device according to claim 6, characterized in that when the second operating means is operated, the control means changes the tracking performance in the autofocus according to the movement information of the subject.

8. The control device according to claim 1, wherein the acquisition means acquires the movement information of the subject using the amount of defocus and reliability information of the amount of defocus acquired from the focus information.

9. The control device according to claim 1, wherein the acquisition means acquires the movement information of the subject using information regarding the change in the size of the subject.

10. The control device according to claim 1, characterized in that the acquisition means acquires the movement information of the subject using the motion information of the subject.

11. The control device according to claim 1, characterized in that the acquisition means determines that the subject has moved if the subject is moving in one direction during a predetermined period of time.

12. The control device according to claim 1, characterized in that the control means makes it difficult to stop the focus lens which is driven in the direction of movement of the subject when the subject moves, thereby improving the tracking performance compared to when the subject is not moving.

13. The control device according to claim 1, characterized in that, when the subject moves, the control means predicts the movement of the subject using the past position information and focus information of the focus lens, and operates in a way that makes it easier to drive the focus lens, thereby improving the tracking performance compared to when the subject is not moving.

14. The control device according to claim 1, characterized in that, when the subject moves, the control means predicts the movement of the subject using the past position information of the focus lens and the focus information, thereby improving the tracking performance compared to when the subject is not moving.

15. The control device according to claim 1, characterized in that the control means improves the tracking performance compared to when the subject is not moving by shortening the waiting time until the autofocus starts when the subject moves.

16. The control device according to claim 1, characterized in that, after deciding to improve the tracking performance, the control means restores the tracking performance to its original state if the position of the focus lens remains within a predetermined amount for a predetermined time and the amount of defocus remains within a predetermined amount for a predetermined time, or if the reliability of the amount of defocus falls below a predetermined value.

17. An imaging device characterized by having a control device according to any one of claims 1 to 16 and an image sensor.

18. Steps to acquire information about the subject's movement, The process includes the step of controlling the focus lens using focus information acquired from the imaging signal, A control method characterized in that, in the step of controlling the focus lens, the process relating to the autofocus tracking performance is changed according to the movement information of the subject.

19. A program characterized by causing a computer to execute the control method described in claim 18.

Images