Control device, control method and storage medium

Abstract

The present invention discloses a control device, a control method, and a storage medium. The control device controls a focus adjustment operation of an imaging optical system, and includes: a calculation unit configured to calculate a focus evaluation value for each of images continuously captured by the imaging optical system; a determination unit configured to determine whether a change in the focus evaluation value calculated by the calculation unit relative to a reference value is greater than or equal to a threshold value; and a control unit configured to control the focus adjustment operation to be performed by the imaging optical system if the number of times the determination unit determines that the change is greater than or equal to the threshold value is greater than or equal to a first value.

Description

Technical Field

[0001] The present invention relates to a control device, a control method and a storage medium. Background Art

[0002] There is known a technique for setting a reactivation time interval and a change amount of a focus evaluation value from a reference value during reactivation based on imaging conditions, camera status, and other conditions when repeating a focusing operation (Japanese Patent Application Laid-Open No. 2006-301005). Summary of the Invention

[0003] According to one aspect of the present invention, there is provided a control device for controlling a focus adjustment operation of an imaging optical system, the control device including: a calculation unit configured to calculate a focus evaluation value for each of images continuously captured by the imaging optical system; a determination unit configured to determine whether an amount of change of the focus evaluation value calculated by the calculation unit relative to a reference value is greater than or equal to a threshold value; and a control unit configured to control the focus adjustment operation to be performed by the imaging optical system, if the number of times the determination unit determines that the amount of change is greater than or equal to the threshold value is greater than or equal to a first value.

[0004] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 is a diagram showing a functional configuration.

[0006] Figure 2 is a flowchart illustrating a series of steps of autofocus (AF) processing according to the first exemplary embodiment.

[0007] Figure 3 is a flowchart illustrating focusing processing in AF according to the first exemplary embodiment.

[0008] Figure 4 is a graph illustrating an example of focus evaluation values ​​according to the first exemplary embodiment.

[0009] Figure 5 is a flowchart illustrating AF execution determination processing according to the first exemplary embodiment.

[0010] Figure 6 is a flowchart illustrating AF execution determination processing according to the second exemplary embodiment.

[0011] Figure 7 is a flowchart illustrating AF execution determination processing according to the third exemplary embodiment.

[0012] Figure 8A and Figure 8B is a graph illustrating changes in focus evaluation values ​​according to the first exemplary embodiment.

[0013] Figure 9A and Figure 9B is a graph illustrating changes in focus evaluation values ​​according to the second exemplary embodiment.

[0014] Figure 10A and Figure 10B is a graph illustrating changes in focus evaluation values ​​according to the third exemplary embodiment.

[0015] Figure 11 is a diagram showing a hardware configuration of a control device according to each exemplary embodiment. DETAILED DESCRIPTION

[0016] Hereinafter, some exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0017] Figure 1 is a diagram illustrating a functional configuration of a control device (imaging device) according to the first exemplary embodiment.

[0018] Light that has passed through the lens group 101 (imaging optical system) is incident on the image sensor 105 via the optical filter 102 , the aperture 103 , and the color filter 104 .

[0019] The lens group 101 is an optical system that focuses incident light from a subject on the image sensor 105. The lens group 101 includes a focus lens for bringing the subject into focus and a zoom lens for adjusting the angle of view.

[0020] For example, an infrared cut filter (IRCF) is arranged as the optical filter 102 .

[0021] The aperture 103 adjusts the amount of light incident on the image sensor 105 through the lens group 101 and the filter 102 .

[0022] The color filter 104 is arranged in a predetermined order at each pixel of the light receiving surface of the image sensor 105 .

[0023] The image sensor 105 outputs captured image information about the imaging target as an analog signal. Here, the image sensor 105 continuously performs imaging and frame output at a predetermined number of times per second (at a frame rate).

[0024] The video image formed in the image sensor 105 is subjected to gain control in an automatic gain controller (AGC) 106. Subsequently, an analog-to-digital (A / D) conversion unit 107 converts the analog imaging signal into a digital imaging signal.

[0025] The video signal processing unit 108 performs predetermined processing on the digital imaging signal from the A / D conversion unit 107 and outputs a luminance signal and a color signal for each pixel. In addition, the video signal processing unit 108 generates a video for output and generates parameters for camera control.

[0026] Here, the parameters used for camera control include parameters for aperture control, focus evaluation values ​​(autofocus (AF) evaluation values) that are frequency component values ​​used for focusing (focus operation), and parameters used in white balance control for adjusting color tones. These parameters are generated for each frame.

[0027] The video signal output unit 109 outputs the video signal generated by the video signal processing unit 108 to the outside.

[0028] The exposure control unit 110 calculates the brightness information of the captured image based on the brightness information output from the video signal processing unit 108. The exposure control unit 110 then controls the aperture 103 and the AGC 106 to adjust the captured image to an appropriate brightness. Brightness can also be adjusted based on the storage time of the image sensor 105, which is adjusted by the shutter speed.

[0029] The focusing operation in this exemplary embodiment involves calculating a contrast value or high-frequency component intensity based on the video signal generated by the video signal processing unit 108, and using the calculated contrast value or high-frequency component intensity as a focus evaluation value. The optical control unit 111 controls the lens group 101 to a position (in-focus position) that maximizes the focus evaluation value. This is a so-called contrast AF system.

[0030] The external setting unit 112 is used for typical camera operations including focusing, brightness designation, and zoom factor designation.

[0031] The control setting unit 113 sets the camera control command transmitted from the external setting unit 112 and performs settings such as exposure control, lens control, etc. In one or more embodiments, the above functions mentioned by reference numerals 108 to 113 are included in a device connected to the imaging device via a network.

[0032] Below, we will refer to Figure 2 The shown flowchart describes the processing procedure of this exemplary embodiment.

[0033] Here, each time the video signal processing unit 108 generates a focus evaluation value, Figure 2 in the processing.

[0034] In step S201, the optical control unit 111 determines whether an AF execution instruction issued by the user at the external setting unit 112 is received via the control setting unit 113. If the instruction is received ("Yes" in step S201), the process proceeds to step S204. Otherwise ("No" in step S201), the process proceeds to step S202.

[0035] In step S202 , the optical control unit 111 performs a process of determining whether to execute AF processing (ie, AF execution determination processing). Details of this process will be described below.

[0036] In step S203, the optical control unit 111 determines whether the AF execution flag, which issues an instruction to execute AF, is true based on the processing result in step S202. If the AF execution flag is true ("Yes" in step S203), the process proceeds to step S204. Otherwise ("No" in step S203), the process ends.

[0037] In step S204 , the optical control unit 111 performs focus processing by AF.

[0038] Here, we will refer to Figure 3 The details of the processing in step S204 are described with reference to the flowchart shown.

[0039] First, in step S301, the optical control unit 111 starts a focus search operation. In the focus search operation, while driving the focus lens of the lens group 101 in a predetermined direction (closest approach direction or infinity direction), the optical control unit 111 acquires a focus evaluation value in step S302 and acquires the position of the focus lens (hereinafter referred to as focus position) in step S303.

[0040] In step S304, the optical control unit 111 determines whether the focus evaluation value acquired in step S302 is greater than the focus evaluation value acquired previously. If so ("Yes" in step S304), the process proceeds to step S305. Otherwise ("No" in step S304), the process proceeds to step S306.

[0041] In step S305, the optical control unit 111 updates the peak value of the focus evaluation value. Specifically, if the current focus evaluation value is greater than the peak focus evaluation value (the focus evaluation value stored as the peak value), the current focus evaluation value is set (stored) as the peak focus evaluation value. Furthermore, the current focus position is set (stored) as the peak focus position (the focus lens position corresponding to the peak value of the focus evaluation value).

[0042] In step S306, the optical control unit 111 sets the lens driving direction in the focus search operation to the direction opposite to the current direction. In other words, if the optical control unit 111 is currently driving the lens in the closest approach direction, the optical control unit 111 sets the lens driving direction to the infinite distance direction; otherwise, the optical control unit 111 sets the lens driving direction, which is currently in the infinite distance direction, to the closest approach direction.

[0043] If the focus evaluation value increases during the current focus search, the optical control unit 111 updates the peak value without changing the driving direction of the focus lens, assuming that the focus position is in the current driving direction. On the other hand, if the focus evaluation value becomes smaller than the previous focus evaluation value, the optical control unit 111 determines that the focus lens has moved away from the focus position, and then drives the focus lens in the direction opposite to the current movement direction.

[0044] In step S307, the optical control unit 111 determines whether the focus lens has reciprocated a predetermined number of times or more within the same area (focus position). If the focus lens has reciprocated a predetermined number of times or more ("Yes" in step S307), the process proceeds to step S308. Otherwise ("No" in step S307), the process returns to step S301 to continue the focus search. This determination is made based on the assumption that the focus lens's stay within the same drive range means that the in-focus position will be within that range.

[0045] In step S308 , the optical control unit 111 drives the focus lens to the peak focus position set in step S305 , and then stops the focus lens (focus stop).

[0046] In step S309, the optical control unit 111 stores the current focus evaluation value as the focus evaluation value at which focus is stopped, and the process ends. This focus evaluation value (AF evaluation value) will be used in the AF execution determination process in step S202 described below.

[0047] Figure 4 The relationship between the focus evaluation value and the focus position is shown. Figure 3 When a series of steps in the processing in FIG. 4 are completed, the focus lens stops at the position of the peak focus evaluation value (evaluation value peak 401 ) (in-focus position 402 ).

[0048] Next, we will refer to Figure 5 The shown flowchart describes the processing in step S202.

[0049] In step S501, the optical control unit 111 calculates the amount of change in the focus evaluation value. The amount of change is the absolute value difference between the focus evaluation value (reference value) at the focus stop point stored in step S309 and the focus evaluation value of the current frame.

[0050] In step S502, the optical control unit 111 determines whether the amount of change in the focus evaluation value calculated in step S501 is greater than or equal to a threshold value. If the amount of change is greater than or equal to the threshold value ("Yes" in step S502), the process proceeds to step S503. Otherwise ("No" in step S502), the process proceeds to step S504. The threshold value is a value used to adjust the sensitivity to changes in the subject. When the threshold value is set to a low value, a response to even slight changes in the subject is likely to occur, and a response to interference such as noise may also occur. On the other hand, when the threshold value is set to a high value, a response to interference such as noise is unlikely to occur, and the response to changes in the subject may be weak. Therefore, it is more appropriate to pre-adjust the threshold value based on the actual imaging scene.

[0051] The processing in step S503 and step S504 is accumulation processing of the determination result in step S502. In this processing, the optical control unit 111 holds the determination results for the latest predetermined number of past frames using a buffer that stores the determination result of each past frame.

[0052] In step S503 , the optical control unit 111 sets the determination result corresponding to the current frame to 1.

[0053] In step S504 , the optical control unit 111 sets the determination result corresponding to the current frame to 0.

[0054] In step S505 , the optical control unit 111 calculates the total value of the determination results of the latest predetermined number of past frames.

[0055] In step S506, the optical control unit 111 determines whether the total value calculated in step S505 is greater than or equal to a threshold value (first value / AF execution threshold value). If the total value is greater than or equal to the threshold value ("Yes" in step S506), the process proceeds to step S507. Otherwise ("No" in step S506), the process proceeds to step S509. The threshold value (first value) in step S506 is a parameter used to control responsiveness, which is the time interval between the occurrence of a subject change and the execution of AF. When the threshold value (first value) is set to a high value, the response to temporary subject changes due to, for example, a person cutting in from the front of the subject is reduced, and the responsiveness is also correspondingly low. When the threshold value is set to a low value, the responsiveness is increased, but the response to temporary subject changes may also be more likely to occur. Therefore, it is more appropriate to set the threshold value (first value) based on the imaging scene.

[0056] In step S505 , the optical control unit 111 further sets a predetermined number of frames (second value) for calculating the total value in association with the threshold value (first value) so as to satisfy the threshold value ≤ predetermined number of frames.

[0057] The specific setting examples are as follows.

[0058] First, the optical control unit 111 determines a threshold value (first value / AF execution threshold value) to achieve appropriate responsiveness. For example, to achieve a response to a subject change occurring within one second, the optical control unit 111 sets the threshold value to 60 frames for an imaging device with a frame rate of 60 fps. This is because a frame rate of 60 fps means that each frame lasts for 1 / 60 second.

[0059] Next, the optical control unit 111 sets the predetermined number of frames to a value greater than or equal to the threshold value. For example, when the predetermined number of frames is set to 60 (i.e., the same as the threshold value), the AF operation (focus adjustment operation) is performed (started) only when the change in the focus evaluation value of each of all 60 frames is greater than or equal to the threshold value of the focus evaluation value in step S502.

[0060] The focus evaluation value is sensitive to noise and other factors in the image. This results in fewer AF operations being performed in environments prone to noise generation at high gain (such as low-light imaging). This is because the focus evaluation value based on subject changes, to which a response should occur, becomes closer to the focus evaluation value based on noise, thereby reducing the amount of change in the overall focus evaluation value. Therefore, if the predetermined number of frames is set to a value greater than a threshold, this situation is less likely to occur. In other words, the predetermined number of frames to be set depends on the acceptable amount of noise affecting the focus evaluation value. For example, the relationship between the predetermined number of frames and the threshold is determined based on the amount of noise in the image.

[0061] In step S507 , the optical control unit 111 clears (deletes) the buffer for storing the determination result.

[0062] In step S508 , the optical control unit 111 sets the AF execution flag for issuing an instruction to execute (start) the AF operation to true, and the processing ends.

[0063] In step S509 , the optical control unit 111 sets the AF execution flag to false, and the process ends.

[0064] Figure 8A Changes in the focus evaluation value in the present exemplary embodiment are illustrated. Figure 8BThe change in the total value calculated in step S505 is illustrated. In this example, the predetermined number of frames for calculating the total value is 6 frames (frame 803), and the threshold value (first value / AF execution threshold value) in step S506 is set to 5. In other words, when the change in the focus evaluation value of 5 frames out of 6 frames is greater than the threshold value, the AF operation is performed. In addition, a fixed frame rate (fps) means a fixed time per frame. At a fixed frame rate, the predetermined number of frames corresponds to a predetermined time. For example, if the frame rate is 50 fps, the time per frame is 1 / 50 second = 20 milliseconds. Therefore, the 5 frames out of the above 6 frames mean 100 milliseconds out of 120 milliseconds. Therefore, when the focus evaluation value exceeds the threshold value of the predetermined time (period), the AF operation is performed.

[0065] The amount of change in the focus evaluation value temporarily becomes high due to a temporary subject change, such as a person cutting in from the front of the subject, as shown in segment 801. However, as Figure 8B As shown in , such a temporary subject change does not cause the total value to reach the threshold, and thus AF execution is not initiated. On the other hand, continuous subject change causes a continuous high change in the focus evaluation value, as shown in segment 802. In this case, the total value exceeds the threshold, and AF execution is initiated.

[0066] As described above, according to this exemplary embodiment, AF execution is determined based on past changes in focus evaluation values, thereby reducing responsiveness to temporary subject changes, such as a person cutting in from the front of the subject. Consequently, this configuration reduces video image blur or changes in angle of view caused by additional AF execution.

[0067] Next, a second exemplary embodiment will be described. In the first exemplary embodiment, a method has been described in which determination results for a predetermined number of past frames are stored, a total value is calculated each time the results are stored, and AF execution is determined based on the calculated total value. In the second exemplary embodiment, AF execution will be determined using a simpler method that does not require storing determination results for past frames or calculating a total value.

[0068] Below, we will refer to Figure 6 The flowchart shown describes the AF execution determination processing according to the present exemplary embodiment. The timing of starting the processing is similar to that in the first exemplary embodiment, so redundant description will be omitted.

[0069] In step S601, the optical control unit 111 determines whether a predetermined number of frames has been passed. The number of frames passed is calculated based on the frames previously determined to have been passed in step S601. If the predetermined number of frames has been passed ("Yes" in step S601), the process proceeds to step S602. Otherwise ("No" in step S601), the process proceeds to step S603. During the initial execution, the determination in step S601 is Yes.

[0070] In step S602, the optical control unit 111 clears the count value. Next, the count value will be described in detail.

[0071] In step S603, the optical control unit 111 calculates the amount of change in the focus evaluation value. The calculation method is similar to that in step S501 of the first exemplary embodiment.

[0072] In step S604, the optical control unit 111 determines whether the amount of change in the focus evaluation value calculated in step S603 is greater than or equal to a threshold value (count threshold value). The determination method is similar to that in step S502 of the first exemplary embodiment. If the optical control unit 111 determines that the amount of change is greater than or equal to the threshold value ("Yes" in step S604), the process proceeds to step S605. Otherwise ("No" in step S604), the process proceeds to step S606.

[0073] In step S605 , the optical control unit 111 accumulates a count value indicating that a change has occurred in the focus evaluation value (ie, a change in the subject).

[0074] In step S606, the optical control unit 111 determines whether the count value is greater than or equal to a threshold value (first value / AF execution threshold value). If the count value is greater than or equal to the threshold value ("YES" in step S606), the process proceeds to step S607. Otherwise ("NO" in step S606), the process proceeds to step S609.

[0075] In step S607 , the optical control unit 111 clears the count value.

[0076] In step S608 , the optical control unit 111 sets the AF execution flag for issuing an instruction to perform AF execution to true, and the processing ends.

[0077] In step S609 , the optical control unit 111 sets the AF execution flag for issuing an instruction to perform AF execution to false, and the processing ends.

[0078] Both the predetermined number of frames in step S601 and the threshold value in step S604 can be set in a similar manner to that in the first exemplary embodiment.

[0079] Figure 9A Changes in the focus evaluation value in the present exemplary embodiment are illustrated. Figure 9BThe change of the count value calculated in step S605 is illustrated. In this example, the predetermined number of frames (second value) for clearing (deleting) the count value is 6 frames (frame 901), and the threshold value (first value) in step S606 is set to 5. As in the first exemplary embodiment, a temporary subject change due to, for example, a person cutting in from the front of the subject produces a temporary high amount of change in the focus evaluation value, as shown in segment 801. In this case, as shown in FIG. Figure 9B As shown in , the count value does not reach the 6-frame threshold for clearing the count value, and thus AF execution is not started. On the other hand, continuous subject changes produce continuous high changes in the focus evaluation value, as shown in segment 802. In this case, Figure 9B The count value in exceeds the threshold, thus starting AF execution.

[0080] As described above, according to the present exemplary embodiment, since the operations of storing the determination results corresponding to each of the past frames and calculating the total value are unnecessary, AF execution is determined in a simple manner compared to the first exemplary embodiment.

[0081] Next, a third exemplary embodiment will be described. In the second exemplary embodiment, a simple method was described in which a count value is accumulated each time a subject change is detected and AF execution is determined by comparing the count value with a threshold. However, depending on the timing of the subject change, clearing the count value each time a predetermined number of frames passes may cause AF execution to be delayed. For example, if the count value is accumulated as a result of continuous subject change detection but is cleared in response to reaching the predetermined number of frames, AF execution may be delayed.

[0082] In the third exemplary embodiment, a method of reducing the delay with a simple process as in the second exemplary embodiment will be described.

[0083] Figure 7 is a flow chart of this exemplary embodiment. Figure 7 The flowchart in describes the AF execution determination processing according to the present exemplary embodiment. The processing in steps similar to those in the above exemplary embodiment will not be described again.

[0084] In step S702, the optical control unit 111 determines whether the number of consecutive counts (described below) is less than a predetermined value. If the number of consecutive counts is less than the predetermined value ("Yes" in step S702), the process proceeds to step S703. Otherwise ("No" in step S702), the process proceeds to step S705.

[0085] In step S704 , the optical control unit 111 clears the number of consecutive counts.

[0086] In step S708 , the optical control unit 111 counts up the number of consecutive counts indicating that a change in the focus evaluation value (ie, a subject change) has been continuously detected.

[0087] In step S709 , the optical control unit 111 clears the number of consecutive counts.

[0088] In step S712 , the optical control unit 111 clears the number of consecutive counts.

[0089] Figure 10A Changes in the focus evaluation value in the present exemplary embodiment are illustrated. Figure 10B The change of the count value calculated in step S707 is illustrated. In this example, the predetermined number of frames for clearing the count value (second value) is 6 frames (frame 1001), and the threshold value (first value) in step S710 is set to 5. In addition, the predetermined value of the number of consecutive counts in step S702 is set to 1. As in the first exemplary embodiment, because of a temporary subject change due to, for example, a person cutting in from the front of the subject, the amount of change in the focus evaluation value temporarily becomes high, as shown in segment 801. In this case, as shown in FIG. Figure 10B As shown in , the count value does not reach the threshold value, and thus AF execution is not started. On the other hand, as the continuous subject change occurs, the change amount of the focus evaluation value continues to maintain a high level, as shown in segment 802. In this case, Figure 10B The count value in exceeds the threshold value, thereby starting AF execution. In addition, although the second exemplary embodiment is referred to Figure 9B In the example described above, the count value is cleared twice, but in this example, the count value is not cleared a second time. This is because the number of consecutive counts exceeds 1 at this time. Therefore, compared with the example of the second exemplary embodiment, the threshold value is reached more quickly and the AF operation is thus performed earlier.

[0090] According to the above processing, when the number of consecutive counts is greater than or equal to a predetermined value, that is, when a continuous subject change is detected, the count value is not cleared (maintained) in step S703. This reduces the delay in AF execution of the second exemplary embodiment.

[0091] (Other exemplary embodiments)

[0092] The present invention can also be implemented by providing a program for implementing one or more functions of the above exemplary embodiments to a system or device via a network or storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. The present invention can also be implemented by a circuit (e.g., an application-specific integrated circuit (ASIC)) that implements one or more functions.

[0093] Figure 11is a block diagram illustrating a hardware configuration example of a computer that executes the processing according to each of the above-described exemplary embodiments using a program.

[0094] The control device 1100 includes a central processing unit (CPU) 1101 , a read-only memory (ROM) 1102 , a random access memory (RAM) 1103 , an external memory 1104 , a communication interface (I / F) 1105 , and a system bus 1106 .

[0095] The CPU 1101 generally controls operations in the control device 1100 , and controls each component (ROM 1102 , RAM 1103 , external memory 1104 , and communication I / F 1105 ) via a system bus 1106 .

[0096] The ROM 1102 is a nonvolatile memory for storing control programs for processing executed by the CPU 1101. In one or more embodiments, these programs are stored in the external memory 1104 or a removable storage medium.

[0097] The RAM 1103 functions as a main memory, a work area, etc. of the CPU 1101. In other words, the CPU 1101 loads a program for executing processing from the ROM 1102 into the RAM 1103 and runs the loaded program, thereby realizing various functional operations.

[0098] The external memory 1104 stores, for example, various types of data and various types of information used for the CPU 1101 to execute processing using a program. The external memory 1104 also stores, for example, various types of data and various types of information obtained by the CPU 1101 executing processing using a program.

[0099] The communication I / F 1105 is an interface for communicating with an external device and is, for example, a local area network (LAN) interface.

[0100] The system bus 1106 communicatively connects the CPU 1101 , the ROM 1102 , the RAM 1103 , the external memory 1104 , and the communication I / F 1105 .

[0101] The above-described exemplary embodiments of the present invention do not limit the present invention, and various modifications and variations can be made within the scope of the gist thereof.

[0102] The embodiments of the present invention can also be implemented by the following method, that is, providing software (program) that performs the functions of the above-mentioned embodiments to a system or device through a network or various storage media, and the computer or central processing unit (CPU) or microprocessing unit (MPU) of the system or device reads and executes the program.

[0103] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. A control device for controlling a focus adjustment operation of an imaging optical system, the control device comprising: a calculation unit configured to calculate a focus evaluation value for each of a plurality of frames captured by the imaging optical system; a determining unit configured to determine whether a change amount of the focus evaluation value calculated by the calculating unit relative to a reference value is greater than or equal to a count threshold; a counting unit configured to count the number of times the determination unit determines that the change amount of the focus evaluation value is greater than or equal to the counting threshold; a reset unit configured to clear a count value indicating the number of times counted by the counting unit when a predetermined time has passed; as well as A control unit configured to, when a count value indicating the number of times counted by the counting unit is greater than or equal to a threshold value for the focus adjustment operation, control the imaging optical system so as to perform the focus adjustment operation. 2 . The control device according to claim 1 , further comprising a storage unit configured to store the focus evaluation value.

3. The control device according to claim 1, wherein: The reference value is a focus evaluation value when a predetermined object focus is in an in-focus state.

4. A method for controlling a focus adjustment operation of an imaging optical system, the method comprising: a calculation step of calculating a focus evaluation value for each of a plurality of frames captured by the imaging optical system; a determining step of determining whether a change in the calculated focus evaluation value relative to a reference value is greater than or equal to a counting threshold; a counting step of counting the number of times that the change in the focus evaluation value is determined to be greater than or equal to the counting threshold; a clearing step of clearing a count value indicating the number of counts when a predetermined time has passed; as well as A control step of controlling the imaging optical system so as to perform the focus adjustment operation when a count value indicating the number of times counted is greater than or equal to a threshold value for the focus adjustment operation. The method of claim 4 , further comprising storing the focus evaluation value.

6. A non-transitory computer-readable storage medium storing a program for causing a computer to execute an image processing method, the method comprising: a calculation step of calculating a focus evaluation value for each of a plurality of frames captured by the imaging optical system; a determining step of determining whether a change in the calculated focus evaluation value relative to a reference value is greater than or equal to a counting threshold; a counting step of counting the number of times that the change in the focus evaluation value is determined to be greater than or equal to the counting threshold; a clearing step of clearing a count value indicating the number of counts when a predetermined time has passed; as well as A control step of controlling the imaging optical system so as to perform the focus adjustment operation when a count value indicating the number of times counted is greater than or equal to a threshold value for the focus adjustment operation.

Images