Control device, imaging device, control method, and program

Abstract

This invention provides a control device that enables appropriate focus control when repeatedly photographing moving subjects. [Solution] The control device (212) includes a subject detection means (2121) that detects a subject using a signal output from the image sensor (201), and a control means (2122) that controls the focus lens (103) according to the focus state of the subject. The control means stores a first position of the focus lens according to the focus state of the subject in a first region, and when shooting, if the focus lens moves from the first position to the second position and the subject moves outside the first region, it moves the focus lens from the second position to the first position.

Description

Technical Field

[0001] The present invention relates to a control device, an imaging device, a control method, and a program.

Background Art

[0002] Patent Document 1 discloses a method of moving a focus lens to a predetermined position when detecting that a preset subject has entered the frame. Patent Document 2 discloses a method of moving a focus lens to a position estimated when the subject has exited the frame.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the methods disclosed in Patent Document 1 and Patent Document 2, when repeatedly photographing a moving subject, it is impossible to improve the time lag and focus accuracy of the next photographing.

[0005] Therefore, an object of the present invention is to provide a control device capable of realizing appropriate focus control when repeatedly photographing a moving subject.

Means for Solving the Problems

[0006] A control device as one aspect of the present invention includes a subject detection means for detecting a subject using a signal output from an image sensor, and a control means for controlling a focus lens according to the focus state of the subject, wherein the control means stores a first position of the focus lens according to the focus state of the subject in a first region, and when taking a picture, if the focus lens moves from the first position to a second position and the subject moves outside the first region, the control means moves the focus lens from the second position to the first position.

[0007] Other objects and features of the present invention are described in the following embodiments. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a control device that can achieve appropriate focus control when repeatedly photographing a moving subject. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram of the imaging system in each embodiment. [Figure 2] This is a timing chart used as a comparative example. [Figure 3] This is a pixel arrangement diagram of the image sensor in each embodiment. [Figure 4] This is a flowchart of the AF operation in the first embodiment. [Figure 5] This is a flowchart of the AF start position drive determination in each embodiment. [Figure 6] This is a timing chart showing the effects in each embodiment. [Figure 7] This is a flowchart of the focus detection process in each embodiment. [Figure 8] This is an explanatory diagram of the distance measuring area in each embodiment. [Figure 9] This is an explanatory diagram of the image signal in each embodiment. [Figure 10]This diagram illustrates the relationship between the shift amount and the correlation amount in each embodiment. [Figure 11] This diagram illustrates the relationship between the shift amount and the correlation change amount in each embodiment. [Figure 12] This is a flowchart of the AF operation in the second embodiment. [Figure 13] This is a timing chart showing the effects of the second embodiment. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0011] First, before describing each embodiment, let's explain one of the problems with each embodiment as a comparative example. When repeatedly shooting with a fixed composition, in scenes with large image plane movement, such as track and field events or trains, the focus point shifts significantly during shooting. As a result, if the subject being shot moves out of the field of view, no subject will be in focus, and the entire shooting screen will be greatly blurred. For this reason, before starting the second or subsequent shots, it is necessary to refocus on a different subject before framing. Also, if the interval until the next shot is short and there is no time to refocus, the focus will have to be refocused from a blurred state, leading to a time lag and a decrease in focus accuracy.

[0012] FIG. 2 is a diagram showing the movement of the subject and the lens in time series as a comparative example. After the subject to be photographed starts moving at time t0, it is assumed that the subject frames in within the angle of view at time t1. Usually, the user instructs the start of the AF operation at the timing when the subject enters the distance measurement frame at time t2, and starts photographing when the focus is achieved at time t3. When the subject frames out of the angle of view at time t4, the photographing and AF operations are terminated. However, at this point, the focus lens has stopped moving in the closest direction after a large movement, and the AF operation has stopped. Therefore, when the subject frames in within the angle of view again at time t1', the AF operation starts from a largely blurred state, and as described above, the time lag and focus accuracy deteriorate.

[0013] Hereinafter, each embodiment of the present invention will be described in detail.

[0014] (First Embodiment) First, referring to FIG. 1, the imaging system 1 in the first embodiment of the present invention will be described. The imaging system 1 is an interchangeable-lens camera system including a camera body (imaging device) 20 and a lens unit (lens device) 10 detachably attached to the camera body 20. A lens control unit 106 that comprehensively controls the operation of the entire lens and a camera control unit (control device) 212 that comprehensively controls the operation of the entire camera system including the lens unit 10 can communicate with each other through terminals provided on the lens mount. However, this embodiment is not limited thereto, and is also applicable to an imaging device in which the camera body and the lens unit are integrally configured.

[0015] First, the configuration of the lens unit 10 will be described. The fixed lens 101, the aperture (iris diaphragm) 102, and the focus lens 103 constitute the imaging optical system. The aperture 102 is driven by an aperture driving unit 104 to control the amount of incident light to the image sensor 201 described later. The focus lens 103 is driven by a focus lens driving unit 105, and the combined focal length of the imaging optical system changes according to the position of the focus lens 103. The aperture driving unit 104 and the focus lens driving unit 105 are controlled by a lens control unit 106 to determine the aperture amount of the aperture 102 and the position of the focus lens 103, respectively.

[0016] The lens operation unit 107 is a group of input devices for the user to set the operations of the lens unit 10, such as switching between AF (autofocus) / MF (manual focus) modes, adjusting the position of the focus lens by MF, and setting the shake correction mode. When the lens operation unit 107 is operated, the lens control unit 106 performs control according to the operation.

[0017] The lens control unit 106 controls the aperture driving unit 104 and the focus lens driving unit 105 according to the control commands and control information received from the camera control unit 212 described later. Also, the lens control unit 106 transmits lens control information to the camera control unit 212.

[0018] Next, the configuration of the camera body 20 will be described. The camera body 20 is configured to be able to acquire an imaging signal from the light beam that has passed through the imaging optical system of the lens unit 10.

[0019] The image sensor 201 is composed of a photoelectric conversion element such as a CCD sensor or a CMOS sensor. The light beam incident from the imaging optical system of the lens unit 10 forms an image on the light receiving surface of the image sensor 201, and is converted into signal charges corresponding to the amount of incident light by the photodiodes provided in the pixels arranged in the image sensor 201. The signal charges accumulated in each photodiode are sequentially read out from the image sensor 201 as voltage signals corresponding to the signal charges according to the driving pulses output by the timing generator 214 according to the commands of the camera control unit 212.

[0020] Each pixel of the image sensor 201 used in this embodiment is composed of two (a pair) photodiodes A and B, and one microlens provided for the pair of photodiodes A and B. Each pixel splits the incident light with the microlens to form a pair of optical images on the pair of photodiodes A and B, and outputs a pair of pixel signals (A signal and B signal) used for AF signals, which will be described later, from the pair of photodiodes A and B. Furthermore, an imaging signal (A+B signal) can be obtained by adding the outputs of the pair of photodiodes A and B.

[0021] By combining multiple A signals and multiple B signals output from multiple pixels, a pair of image signals are obtained to be used as AF signals (focus detection signals) for AF using the image plane phase-difference detection method (hereinafter referred to as image plane phase-difference AF). The AF signal processing unit 204, described later, performs a correlation calculation on the pair of image signals to calculate the phase difference (hereinafter referred to as image shift amount), which is the amount of shift between these pair of image signals, and further calculates the amount of defocus (and defocus direction) of the imaging optical system from this image shift amount.

[0022] Figure 3(a) shows the pixel configuration of an image sensor that does not support on-sensor phase-detection autofocus (AF). Figure 3(b) shows the pixel configuration of image sensor 201 that supports on-sensor phase-detection AF. In both Figures 3(a) and (b), a Bayer array is used, where R represents a red color filter, B represents a blue color filter, and Gr and Gb represent green color filters. In the pixel configuration that supports on-sensor phase-detection AF (Figure 3(b)), two photodiodes A and B, which are divided horizontally in Figures 3(a) and (b), are provided within the pixel corresponding to one pixel (solid line area) in the pixel configuration that does not support on-sensor phase-detection AF (Figure 3(b)). Note that the pixel division method shown in Figure 3(b) is merely an example; the pixels may be divided vertically, or divided into two horizontally and two vertically (a total of four divisions). Furthermore, multiple types of pixels divided using different division methods may be included within the same image sensor.

[0023] The converter (CDS / AGC / AD converter) 202 performs correlated double sampling, gain adjustment, and AD conversion on the AF signal and imaging signal read from the image sensor 201 to remove reset noise. The converter 202 outputs the processed imaging signal and AF signal to the image input controller 203 and the AF signal processing unit 204, respectively.

[0024] The image input controller 203 stores the imaging signal output from the converter 202 as an image signal in the SDRAM 209 via the bus 21. The image signal stored in the SDRAM 209 is read out by the display control unit 205 via the bus 21 and displayed in the display unit (display means) 206. In recording mode, where the image signal is recorded, the image signal stored in the SDRAM 209 is recorded in a recording medium 208 such as a semiconductor memory by the recording medium control unit 207.

[0025] ROM 210 stores control programs and processing programs executed by the camera control unit 212, as well as various data necessary for their execution. Flash ROM 211 stores various setting information related to the operation of the camera body 20, as set by the user.

[0026] The subject detection unit (subject detection means) 2121 within the camera control unit 212 detects a specific subject based on the imaging signal input from the image input controller 203 and determines the position of the specific subject within the imaging signal. The subject detection unit 2121 also continuously receives imaging signals from the image input controller 203, determines the destination position if the detected specific subject moves, and tracks the position of the specific subject. Examples of a specific subject include a face or a subject located at a position specified by the user within the imaging screen using the camera operation unit (operation means) 213. As will be described later, information regarding the position and size of the detected specific subject is mainly used to set the area for autofocus (AF).

[0027] The AF signal processing unit 204, acting as a focus detection device, performs correlation calculations on a pair of image signals, which are AF signals output from the converter 202, and calculates the amount of image shift and reliability of these two image signals. Reliability is calculated using the degree of two-image match and the steepness of the correlation change, which will be described later. The AF signal processing unit 204 also sets the position and size of the distance measurement area, which is the area within the imaging screen where focus detection and AF are performed. The AF signal processing unit 204 outputs the image shift amount (detection amount) and reliability information calculated in the distance measurement area to the camera control unit 212. Details of the processing performed by the AF signal processing unit 204 will be described later.

[0028] The AF control unit (control means) 2122 within the camera control unit 212 modifies the settings of the AF signal processing unit 204 as needed, based on the amount of image shift, reliability, and information indicating the status of the lens unit 10 and the camera body 20, as determined by the AF signal processing unit 204. For example, if the amount of image shift is greater than or equal to a predetermined amount for the AF signal processing unit 204, the area in which correlation calculations are performed is widened, and the type of bandpass filter is changed according to the contrast of the pair of image signals. The AF control unit 2122 also sets the position and range of the distance measurement area for setting the distance measurement area in the AF signal processing unit 204, using information regarding the position of a specific subject detected by the subject detection unit 2121, or a position specified by the user within the image capture screen using the camera operation unit 213.

[0029] In this embodiment, a total of three signals are obtained from the image sensor 201: the imaging signal and a pair of image signals which are AF signals. However, considering the load on the image sensor 201, for example, two signals, the imaging signal and one AF image signal, may be extracted, and the difference between the extracted imaging signal and the AF image signal may be used as the other AF image signal.

[0030] The camera control unit 212 controls each part of the camera body 20 while exchanging information with it. The camera control unit 212 also performs various processes corresponding to user operations, such as turning the power on / off, changing various settings, image capture, AF processing, and playback of recorded images, in response to input from the camera operation unit 213 based on user operations. Furthermore, the camera control unit 212 transmits control commands to the lens unit 10 (lens control unit 106) and information from the camera body 20 to the lens control unit 106, and also obtains information from the lens unit 10 from the lens control unit 106. The camera control unit 212 is composed of a microcomputer and controls the entire camera system, including the lens unit 10, by executing computer programs stored in the ROM 210. The camera control unit 212 calculates the defocus amount using the image shift amount in the distance measurement area calculated by the AF signal processing unit 204, and controls the drive of the focus lens 103 through the lens control unit 106 based on the defocus amount.

[0031] Next, the processing performed by the camera body 20 will be described. The camera control unit 212 performs the following processing according to the imaging processing program, which is a computer program.

[0032] Referring to Figure 4, the procedure for AF operation, assuming a workflow mainly for still image shooting performed by the camera body 20, particularly the camera control unit 212, will be explained. Figure 4 is a flowchart of the AF operation in this embodiment.

[0033] First, in step S401, the camera control unit 212 detects a subject to be focused on from the captured image using the subject detection unit 2121, and monitors the direction of movement of the detected subject within the screen. The camera control unit 212 then monitors whether the detected subject has entered the frame into the area targeted for focus detection. The subject can be of a type, such as a person, a dog, or a wild bird, or a vehicle such as a motorcycle or car, and major parts of the subject can also be detected. Major parts refer to the pupils, face, or body of a person or animal, or localized body parts of a vehicle. These detection methods utilize known technologies such as deep learning techniques and image processing methods, but since these are not the main technologies in this proposal, details are omitted.

[0034] Next, in step S402, the camera control unit 212 determines whether or not there is an instruction from the camera operation unit 213 to start AF operation by half-pressing the shutter button (whether or not SW1 is turned on). If there is an instruction to start AF operation (SW1 is turned on), the system transitions to step S403, where the shutter button is half-pressed (state B). On the other hand, if there is no instruction to start AF operation (SW1 is off), the system transitions to step S401, and the camera control unit 212 continues to monitor the subject's movement into the frame.

[0035] In state B, first in step S403, the camera control unit 212 determines whether or not there is an instruction from the camera operation unit 213 to terminate the AF operation by half-releasing the shutter button. If there is no instruction to terminate the AF operation and the AF operation continues, the system proceeds to step S404. On the other hand, if there is an instruction to terminate the AF operation, the system proceeds to step S411.

[0036] In step S404, the camera control unit 212 drives the focus lens 103 via the lens control unit 106 based on the output result of the focus detection process by the AF signal processing unit 204, and performs servo AF to continuously adjust the focus on the aforementioned subject. In this embodiment, during the execution of both the servo AF in step S404 and the continuous shooting servo AF in step S415 (described later) (during the execution of the first process), focus detection is performed on the entire shooting screen. The focus detection process is a process that acquires information on the amount of defocus and reliability for performing image plane phase-detection AF. The area within the image screen from which information is acquired is also set according to the state of the camera body 20. Further details will be described later.

[0037] Next, in step S405, the camera control unit 212 determines whether or not it was able to focus on the subject initially using the servo AF in step S404 (i.e., whether or not focus has been achieved, i.e., whether or not focus adjustment has been completed). If it was able to focus on the subject even once, the process proceeds to step S406. On the other hand, if it has not yet been able to focus on the subject even once, the process proceeds to step S403 to continue the servo AF operation.

[0038] In step S406, the camera control unit 212 determines whether the AF start position, i.e., the position of the focus lens 103 (first position) based on the subject that has entered the frame, has already been set. If the AF start position has already been set, the process proceeds to step S409. On the other hand, if the AF start position has not been set, the process proceeds to step S407.

[0039] In step S407, the camera control unit 212 sets the position of the focus lens 103 when it was focused by the servo AF in step S404 as the AF start position (first position). Here, an example has been described in which the position where the focus was first achieved is set as the AF start position, but the AF start position set using the history of the focus lens 103 in the servo AF in step S404 may be changed as appropriate.

[0040] Next, in step S408, the camera control unit 212 notifies the user that the AF start position has been set (information regarding the AF start position) by sound or display. In this embodiment, an example has been described in which the user is notified by sound or display at the time the AF start position is set, but it is not limited to this. Other methods or forms may be used as long as it is possible to notify the user of information regarding the set AF start position, such as displaying information regarding the subject distance corresponding to the AF start position on the display unit 206. The camera control unit 212 may also notify the user of information regarding the timing when the AF start position was set (the timing when the focus adjustment was first completed).

[0041] Next, in step S409, the camera control unit 212 monitors how much the target subject has moved in the optical axis direction while the servo AF is being executed in step S404. Then, in step S410, the camera control unit 212 determines whether or not there has been an instruction from the camera operation unit 213 to start continuous shooting by fully pressing the shutter button (whether or not SW2 is turned on). If there has been an instruction to start continuous shooting (SW2 is turned on), the system transitions to step S415, where the shutter button is fully pressed (state C). On the other hand, if there has been no instruction to start continuous shooting (SW2 is off), the system transitions to step S403 and continues the operation in the half-pressed state of the shutter button (state B).

[0042] If it is determined in step S403 that an instruction to terminate the AF operation has been given, in step S411 the camera control unit 212 will determine whether or not to drive the focus lens 103 to the AF start position set in step S407. Further details will be described later.

[0043] Next, in step S412, the camera control unit 212 determines the result of the AF start position drive determination in step S411. If it is determined that the camera should be driven to the AF start position, the process proceeds to step S413. On the other hand, if it is determined that the camera should not be driven to the AF start position, the process proceeds to step S414.

[0044] In step S413, the camera control unit 212 drives the focus lens 103 to the AF start position set in step S407. This locks the focus at the position where it was first focused on the subject that was photographed immediately before. As a result, the camera can wait in a good focus state for subjects that similarly enter the frame area for focus detection in subsequent steps, improving the time lag and focus accuracy of AF and shooting. Next, in step S414, the camera control unit 212 initializes the AF start position information set in step S407. After that, it transitions to step S401, where the shutter button is not pressed (state A).

[0045] In state C, first in step S415, the camera control unit 212 performs continuous shooting servo AF. That is, while shooting in continuous shooting mode, the camera control unit 212 drives the focus lens 103 via the lens control unit 106 based on the output result of the focus detection process by the AF signal processing unit 204, and performs AF that continuously adjusts the focus on the subject. The focus detection process is basically the same as the servo AF in step S404, so a detailed explanation is omitted.

[0046] Next, in step S416, the camera control unit 212 monitors how much the target subject has moved in the optical axis direction while the servo AF in step S404 and the continuous shooting servo AF in step S415 are being executed. Next, in step S417, the camera control unit 212 monitors whether the subject being focused has gone out of frame. In this embodiment, the area to be monitored for going out of frame has been described as the entire shooting screen (the area in which the subject can be detected), but it is not limited to this. If the area to be focused is not the entire shooting screen, the area to be monitored for going out of frame may be the area to be focused (the area used to acquire the focus state of the subject (distance measurement area)). The camera control unit 212 can, for example, set the area to be focused in any area within the imaging screen.

[0047] Next, in step S418, the camera control unit 212 determines whether or not there is an instruction from the camera operation unit 213 to end the continuous shooting operation by fully releasing the shutter button (whether or not SW2 has been turned off). If there is no instruction to end the continuous shooting operation and the continuous shooting operation continues (SW2 is on), the system proceeds to step S415. On the other hand, if there is an instruction to end the continuous shooting operation (SW has been turned off), the system proceeds to step S403, where the shutter button is half-pressed (state B).

[0048] Next, with reference to Figure 5, the procedure for determining the AF start position drive in step S411 will be explained. Figure 5 is a flowchart of the AF start position drive determination.

[0049] First, in step S501, the camera control unit 212 determines whether there is a history of performing a continuous shooting operation via the fully pressed state (state C). If there is a history of performing a continuous shooting operation, the process proceeds to step S502. On the other hand, if there is no history of performing a continuous shooting operation, the process proceeds to step S507.

[0050] In step S502, the camera control unit 212 determines, based on the monitoring results from step S401, whether the subject to be focused has entered the frame into the area subject to focus detection. If the subject has entered the frame, the process proceeds to step S503. On the other hand, if the subject has not entered the frame, the process proceeds to step S507.

[0051] In step S503, the camera control unit 212 determines, based on the monitoring results from steps S409 and S416, whether the subject moved in the same direction during the execution of servo AF in step S404 and continuous servo AF in step S415. For example, the camera control unit 212 determines whether the subject was moving in the same direction with respect to the optical axis, such as consistently approaching or consistently moving away. If the subject was moving in the same direction, the process proceeds to step S504. On the other hand, if the subject was not moving in the same direction, the process proceeds to step S507.

[0052] In step S504, the camera control unit 212 determines, based on the monitoring results from steps S409 and S416, whether the target subject has moved by a predetermined amount or more during the execution of the servo AF in step S404 and the continuous shooting servo AF in step S415. If the target subject has moved by a predetermined amount or more, the process proceeds to step S505. On the other hand, if the target subject has not moved by a predetermined amount or more, the process proceeds to step S507. Here, it is desirable to determine the threshold amount as the amount of image plane change based on a ratio to the depth of field, but the threshold may also be determined as the actual distance the subject has moved.

[0053] In step S505, the camera control unit 212 determines, based on the monitoring results from step S417, whether the subject has gone out of frame outside the shooting screen. If the subject has gone out of frame, the process proceeds to step S506. On the other hand, if the subject has not gone out of frame, the process proceeds to step S507. In this embodiment, the area for monitoring out-of-frame shots has been described as the entire shooting screen, but it is not limited to this. If the area subject to focus detection is not the entire shooting screen, the area for monitoring out-of-frame shots may be the area subject to focus detection.

[0054] In step S506, based on the determinations in steps S501 to S505, the camera control unit 212 decides in step S413 that it may drive the focus lens 103 to the AF start position set in step S407. In step S507, based on the determinations in steps S501 to S505, the camera control unit 212 decides in step S413 that it should not drive the focus lens 103 to the AF start position set in step S407.

[0055] Next, referring to Figure 6, we will describe the movement of the subject and lens in chronological order when the AF operation described with reference to Figures 4 and 5 is applied. Figure 6 is a timing chart showing the effect in this embodiment.

[0056] In Figure 6, the times t0 to t5 in the first shot are the same as in Figure 2. On the other hand, when shooting and AF are stopped at time t5, the focus lens 103 is driven to the position where it was initially focused, so that when the subject comes back into the frame at t1' in the second and subsequent shots, it can wait in a good focused state. Therefore, it is possible to improve the time lag and focus accuracy of AF and shooting at times t2' to t3'.

[0057] Next, with reference to Figure 7, the details of the focus detection process performed by the AF signal processing unit 204 in the servo AF of step S404 and the continuous shooting servo AF of step S415 will be described. Figure 7 is a flowchart of the focus detection process.

[0058] First, in step S701, the AF signal processing unit 204 acquires a pair of image signals as AF signals from multiple pixels included in the distance measurement area (focus detection area, AF area) of the image sensor 201.

[0059] Figure 8 is an explanatory diagram showing an example of a distance measuring area 802 on the pixel array 801 of the image sensor 201. The shift areas 803 on both sides of the distance measuring area 802 are areas necessary for correlation calculation. Therefore, the area 804, which is the sum of the distance measuring area 802 and the shift areas 803, is the pixel area necessary for correlation calculation. In Figure 8, p, q, S, and t each represent coordinates in the horizontal direction (x-axis direction), with p and q representing the x-coordinates of the start and end points of the area (pixel area) 804, respectively, and S and t representing the x-coordinates of the start and end points of the distance measuring area 802, respectively.

[0060] Figures 9(a) to 9(c) are explanatory diagrams illustrating examples of a pair of image signals for autofocus (AF) acquired from multiple pixels included in the distance measurement area 802 shown in Figure 8. The solid line 901 represents one image signal A, and the dashed line 902 represents the other image signal B. Figure 9(a) shows image signals A and B before shifting. Figures 9(b) and 9(c) show image signals A and B shifted in the positive and negative directions, respectively, from the state shown in Figure 9(a).

[0061] Next, in step S702 of Figure 7, the AF signal processing unit 204 calculates the correlation amount of the pair of image signals while relatively shifting the acquired pair of image signals one pixel (one bit) at a time. For each of the multiple pixel lines (scanning lines) provided within the distance measurement area, the correlation amount of the pair of image signals A901 and B902 is calculated by shifting both image signals A901 and B902 one bit at a time in the direction of the arrows, as shown in Figures 9(b) and (c). Then, the correlation amounts are added together and averaged to calculate a single correlation amount. Here, the pair of image signals are shifted one pixel at a time in order to calculate the correlation amount, but it is also acceptable to shift in units of more pixels, such as two pixels at a time. Furthermore, although a single correlation amount is calculated by adding together the correlation amounts of each scanning line, it is also acceptable to perform an add-and-average on the pair of image signals of each scanning line, and then calculate the correlation amount on the added-and-averaged pair of image signals. When the shift amount is i, the minimum shift amount is pS, the maximum shift amount is qt, x is the starting coordinate of the distance measurement area 802, and y is the ending coordinate of the distance measurement area 802, the correlation amount COR can be calculated by the following equation (1).

[0062]

number

[0063] Figure 10(a) is an explanatory diagram illustrating an example of the relationship between the shift amount and the correlation amount COR. In Figure 10(a), the horizontal axis represents the shift amount and the vertical axis represents the correlation amount COR. Among the regions 1002 and 1003 near the extreme values ​​of the correlation amount 1001, which changes with the shift amount, the degree of agreement between the pair of image signals A and B is highest at the shift amount corresponding to the smaller correlation amount.

[0064] Next, in step S703 of Figure 7, the AF signal processing unit 204 calculates the correlation change amount from the correlation amount calculated in step S702. The correlation change amount is calculated as the difference in the correlation amount every one shift in the waveform of correlation amount 1001 shown in Figure 10(a). If the shift amount is i, the minimum shift amount is pS, and the maximum shift amount is qt, then the correlation change amount ΔCOR can be calculated by the following equation (2).

[0065]

number

[0066] Next, in step S704, the AF signal processing unit 204 calculates the image shift amount using the correlation change amount calculated in step S703. Figure 11(a) is an explanatory diagram showing an example of the relationship between the shift amount and the correlation change amount ΔCOR. In Figure 11(a), the horizontal axis is the shift amount and the vertical axis is the correlation change amount ΔCOR. The correlation change amount 1101, which changes with the shift amount, changes from positive to negative in regions 1102 and 1103. The state in which the correlation change amount becomes 0 is called zero crossing, and the degree of agreement between the pair of image signals A and B is highest. Therefore, the shift amount that gives zero crossing is the image shift amount.

[0067] Figure 11(b) is an enlarged view of region 1102 in Figure 11(a). 1104 is a portion of the correlation change 1101. The shift amount (k-1+α) that gives zero crossing can be divided into an integer part β (=k-1) and a fractional part α. The fractional part α can be calculated by the following equation (3) based on the similarity relationship between triangle ABC and triangle ADE in the figure.

[0068]

number

[0069] Furthermore, the integer part β can be calculated from Figure 11(b) using the following equation (4).

[0070]

number

[0071] In other words, the image displacement PRD can be calculated from the sum of α and β. As shown in Figure 11(a), if there are multiple zero-crossings of the correlation change ΔCOR, the one with the steeper change in the correlation change ΔCOR in its vicinity is designated as the first zero-crossing. This steepness is an indicator of how easy it is to perform autofocus (AF), and a larger value indicates that it is easier to perform accurate AF at that point. The steepness maxder can be calculated using the following equation (5).

[0072]

number

[0073] In this embodiment, if there are multiple zero-crossings of the correlation change amount, the first zero-crossing is determined by its steepness, and the shift amount that gives the first zero-crossing is defined as the image displacement amount.

[0074] Next, in step S707, the AF signal processing unit 204 calculates reliability, which represents the reliability of the image shift amount calculated in step S704. The reliability of the image shift amount can be defined by the degree of agreement between a pair of image signals A and B (called the two-image agreement degree) fnclvl and the steepness of the correlation change amount described above. The two-image agreement degree is an indicator of the accuracy of the image shift amount, and here, a smaller value means better accuracy. Figure 10(b) is an enlarged view of region 1002 in Figure 10(a), where 1004 is a part of the correlation amount 1001. The two-image agreement degree fnclvl can be calculated by the following equation (8).

[0075]

number

[0076] Finally, in step S708, the AF signal processing unit 204 calculates the amount of defocus in the distance measurement area using the amount of image shift in the distance measurement area calculated in step S704.

[0077] As described above, in this embodiment, when the focus lens 103 moves from the first position to the second position and the subject moves outside the first region during shooting, the AF control unit 2122 moves the focus lens 103 from the second position to the first position. For example, the second position is any position different from the first position. Preferably, the AF control unit 2122 stores the first position of the focus lens based on the subject that has entered the first region. Then, when the focus lens 103 moves from the first position to the second position by moving a predetermined distance or more during shooting and the subject moves out of the first region, the AF control unit 2122 moves the focus lens 103 to the first position at the end of shooting. The first position can be stored in the internal memory of the camera control unit 212, for example, but is not limited to this, and may be stored in other storage means.

[0078] According to this embodiment, by using the shooting history of the previous subject to prepare for the next shooting, it becomes possible to achieve appropriate focusing, especially when repeatedly shooting moving subjects.

[0079] Furthermore, while this embodiment describes the AF operation procedure assuming a still image shooting workflow, it is not limited to this. This embodiment can also be applied to automatic shooting without user intervention.

[0080] For example, the AF control unit 2122 may be capable of performing a first process (first focus adjustment control) and a second process (second focus adjustment control). The first process is a process that starts controlling the focus lens 103 in response to instructions from the camera operation unit 213 by user operation, and is, for example, a process related to still image AF. The second process is a process that starts controlling the focus lens 103 automatically for a subject without instructions from the camera operation unit 213, and is, for example, a process related to video AF.

[0081] In this embodiment, preferably, the first position is the position of the focus lens 103 when the initial focus adjustment is completed. Preferably, in the first process, the focus state can be acquired across the entire imaging screen. Preferably, the AF control unit 2122 notifies the user of information regarding the timing when the initial focus adjustment is completed. Preferably, if the AF control unit 2122 is automatically instructed to start shooting a still image or video, it moves the focus lens 103 to the first position when shooting is completed.

[0082] (Second Embodiment) Next, a second embodiment of the present invention will be described. In this embodiment, descriptions of configurations and operations common to the first embodiment will be omitted.

[0083] First, let's explain the processing performed by the camera body 20. The camera control unit 212 performs the following processing according to the imaging processing program, which is a computer program.

[0084] Referring to Figure 12, the procedure for AF operation, which is mainly assumed to be performed in the camera control unit 212 during the shooting process of the camera body 20, will be explained. Figure 12 is a flowchart of the AF operation in this embodiment.

[0085] First, in step S1201, the camera control unit 212 determines whether or not there is an instruction from the camera operation unit 213 to start video recording by pressing the video recording button. If there is an instruction to start video recording, the process proceeds to step S1202. On the other hand, if there is no instruction to start video recording, the camera control unit 212 waits for an instruction to start video recording.

[0086] In step S1202, the camera control unit 212 detects a subject to be focused on from the captured image using the subject detection unit 2121. The camera control unit 212 also monitors the direction of movement of the subject within the screen and checks whether the subject has entered the frame into the area to be focused. Here, the subject can be of a type, such as a person, a dog, or an animal such as a wild bird, or a vehicle such as a motorcycle or car, and major parts within the subject can be detected. Major parts refer to the pupils, face, and body of a person or animal, or local parts and the body of a vehicle. These detection methods utilize known technologies such as deep learning learning methods and image processing means, but since they are not the main technologies in this proposal, details are omitted. In this embodiment, focus detection is performed on the entire captured screen, but it is not limited to this.

[0087] Next, in step S1203, the camera control unit 212 determines, based on the monitoring results from step S1201, whether the subject to be focused has entered the frame into the area subject to focus detection. If the subject has entered the frame, the system proceeds to step S1204. On the other hand, if the subject has not entered the frame, the system proceeds to step S1202 to continue monitoring the subject's movement into the frame.

[0088] In step S1204, the camera control unit 212 determines whether the video AF operation is temporarily suspended by step S1217, which will be described later. If the video AF operation is temporarily suspended, the system proceeds to step S1204. On the other hand, if the video AF operation is not temporarily suspended, the system proceeds to step S1206 of state B. In step S1205, the camera control unit 212 resumes the video AF operation that was temporarily suspended by step S1217, which will be described later, and proceeds to S1206 of state B.

[0089] In state B, first, in step S1206, the camera control unit 212 executes video AF (second processing). That is, the camera control unit 212 drives the focus lens 103 via the lens control unit 106 based on the output result of the focus detection processing by the AF signal processing unit 204, and performs AF that continuously adjusts the focus on the target subject. The focus detection processing is a process that acquires information on the amount of defocus and reliability for performing image plane phase-detection AF, and the area within the image capture screen from which the information is acquired is also set according to the state of the camera body 20. Details here are the same as in the first embodiment and are therefore omitted.

[0090] Next, in step S1207, the camera control unit 212 determines whether or not it was able to focus on the subject initially using the video AF in step S1206 (i.e., whether or not focus was achieved, i.e., whether or not focus adjustment was completed). If it was able to focus even once, the process proceeds to step S1208. On the other hand, if it has not yet been able to focus even once, the process proceeds to step S1206, and the camera control unit 212 continues the video AF operation.

[0091] In step S1208, the camera control unit 212 determines whether the AF start position (first position) has been set in step S1209, which will be described later. If the AF start position has been set, the process proceeds to step S1211. On the other hand, if the AF start position has not been set, the process proceeds to step S1209.

[0092] In step S1209, the camera control unit 212 sets the position of the focus lens 103 when it was focused by the video AF in step S1206 as the AF start position (first position). In this embodiment, an example has been described in which the position where focus was first achieved is set as the AF start position, but it is not limited to this. The AF start position set using the history of the focus lens 103 in the video AF in step S1206 may be changed as appropriate.

[0093] Next, in step S1210, the camera control unit 212 notifies the user that the AF start position has been set by sound or display. In this embodiment, an example has been described in which the user is notified by sound or display when the AF start position is set, but it is not limited to this. Other means or forms may be used as long as it is possible to notify the user of information regarding the set AF start position, such as by displaying the corresponding subject distance information on the display unit 206.

[0094] Next, in step S1211, the camera control unit 212 monitors how much the target subject has moved in the optical axis direction during the execution of video AF in step S1206. Next, in step S1212, the camera control unit 212 monitors whether the subject being focused has moved out of frame. In this embodiment, the area monitored for frame-out has been described as the entire shooting screen, but it is not limited to this. If the area to be focused is not the entire shooting screen, the area monitored for frame-out (first area) may be used as the area to be focused.

[0095] Next, in step S1213, the camera control unit 212 determines whether or not there is an instruction to end the video recording operation (an instruction to stop video recording) by pressing the video recording button from the camera operation unit 213. If there is no instruction to end the video recording operation and the video recording operation continues, the system proceeds to step S1206. On the other hand, if there is an instruction to end the video recording operation, the system proceeds to step S1214 of state C.

[0096] In state C, first, in step S1214, the camera control unit 212 determines whether or not to drive the focus lens 103 to the AF start position set in S1209. Note that this process is the same as in the first embodiment, so its explanation is omitted.

[0097] Next, in step S1215, the camera control unit 212 determines whether or not to drive to the AF start position based on the determination result of the AF start position drive determination in step S1214. If it is determined that the camera should drive to the AF start position, the process proceeds to step S1216. If it is determined that the camera should not drive to the AF start position, the process proceeds to step S1218.

[0098] In step S1216, the camera control unit 212 drives the focus lens 103 to the AF start position set in step S1209. Subsequently, in step S1217, the camera control unit 212 pauses the video AF (second processing). This prevents the camera from unintentionally focusing on an unintended subject while waiting for the next subject to be photographed to enter the frame. This fixes the focus at the position where the focus was initially set on the subject that was photographed immediately before. As a result, the camera can wait in a good focused state for subjects that similarly enter the area to be focused in the next time, improving the AF time lag and focus accuracy. Subsequently, in step S1218, the camera control unit 212 initializes the AF start position information set in step S1209. After that, it transitions to step S1201 of state A.

[0099] Next, referring to Figure 13, we will describe the movement of the subject and lens in chronological order when the AF operation described with reference to Figure 12 is applied. Figure 13 is a timing chart showing the effect in this embodiment.

[0100] In the first shot, the position where the focus was first applied when the subject entered the frame at time t2 is set as the AF start position (first position). When shooting is stopped at time t4, the focus lens 103 is driven to the aforementioned AF start position, and then the video AF operation is stopped. This allows the camera to wait in a good focus state when the subject enters the frame again at time t2' in subsequent shots. In other words, in this embodiment, in video AF (second processing), the AF control unit 2122 moves the focus lens 103 to the first position when shooting is finished, temporarily stops video AF, and starts video AF when the subject enters the first area again. This makes it possible to improve AF time lag and focus accuracy.

[0101] As explained above, this embodiment allows for proper focusing, especially when repeatedly photographing moving subjects, by using the shooting history of the previous subject to prepare for the next shot. Furthermore, although this embodiment describes the AF operation procedure assuming a video shooting workflow, this embodiment can also be applied to automatic shooting without user intervention.

[0102] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0103] Each embodiment provides a control device, an imaging device, a control method, and a program that can achieve appropriate focus control when repeatedly photographing a moving subject.

[0104] Each embodiment of the disclosure includes the following configuration and method. (Composition 1) A subject detection means that detects a subject using a signal output from an image sensor, The system includes a control means for controlling the focus lens according to the focal state of the subject, The control means is The system stores the first position of the focus lens corresponding to the focal state of the subject in the first region. A control device characterized in that, during shooting, if the focus lens moves from the first position to the second position and the subject moves outside the first area, the focus lens moves from the second position to the first position. (Configuration 2) The control means is The first position of the focus lens based on the subject that has entered the first region is stored, The control device according to configuration 1, characterized in that, if the focus lens moves from the first position to the second position by moving a predetermined distance or more during shooting, and the subject moves out of frame outside the first area, the focus lens is moved back to the first position when shooting is completed. (Composition 3) The control device according to configuration 1 or 2, characterized in that the first region is a region used to acquire the focus state of the subject. (Composition 4) The control device according to configuration 1 or 2, characterized in that the first region is a region in which the subject can be detected. (Composition 5) The control means is A first process in which control of the focus lens is initiated in response to instructions from a user-operated operating means, A control device according to any one of configurations 1 to 4, characterized in that it can perform a second process of automatically initiating the control of the focus lens with respect to the subject. (Composition 6) The control device according to any one of configurations 1 to 5, characterized in that the first position is the position of the focus lens when the focus adjustment is first completed. (Composition 7) The control device according to configuration 5, characterized in that, in the first process, the focus state can be acquired across the entire imaging screen. (Composition 8) The control means is In the second process, after moving the focus lens to the first position at the end of shooting, the second process is temporarily stopped. The control device according to configuration 5, characterized in that the second process is started when the subject enters the first region again. (Composition 9) The control device according to any one of configurations 1 to 8, characterized in that the control means moves the focus lens to the first position when shooting is completed, if the focus lens moves in the same direction during shooting. (Composition 10) The control device according to any one of configurations 1 to 9, characterized in that the control means notifies the user of information regarding the first position. (Composition 11) The control device according to any one of configurations 1 to 10, characterized in that the control means notifies the user of information regarding the timing when the focus adjustment is first completed. (Composition 12) The control device according to any one of configurations 1 to 11, characterized in that the control means displays the subject distance corresponding to the first position on the display means. (Composition 13) The control device according to any one of configurations 1 to 12, characterized in that when the start of shooting is automatically instructed, the focus lens is moved to the first position when shooting is finished. (Composition 14) An imaging device characterized by having a control device according to any one of configurations 1 to 13 and the image sensor. (Composition 15) The imaging apparatus according to configuration 14, further comprising an optical system including the aforementioned focusing lens. (Method 1) A step of detecting a subject using the signal output from the image sensor, The system includes the step of controlling the focus lens according to the focal state of the subject, In the step of controlling the focus lens, The system stores the first position of the focus lens corresponding to the focal state of the subject in the first region. A control method characterized in that, during shooting, if the focus lens moves from the first position to the second position and the subject moves outside the first area, the focus lens is moved from the second position to the first position. (Composition 16) A program characterized by causing a computer to execute the control method described in Method 1.

[0105] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist. [Explanation of Symbols]

[0106] 103 Focus Lens 201 Image sensor 212 Camera Control Unit (Control Device) 2121 Subject detection unit (subject detection means) 2122 AF control unit (control means)

Claims

1. A subject detection means that detects a subject using a signal output from an image sensor, The system includes a control means for controlling the focus lens according to the focal state of the subject, The control means is The first position of the focus lens corresponding to the focal state of the subject in the first region is stored, A control device characterized in that, during shooting, if the focus lens moves from the first position to the second position and the subject moves outside the first area, the focus lens moves from the second position to the first position.

2. The control means is The first position of the focus lens based on the subject that has entered the first region is stored, The control device according to claim 1, characterized in that, if the focus lens moves from the first position to the second position by moving a predetermined distance or more during shooting, and the subject moves out of frame outside the first area, the focus lens is moved back to the first position when shooting is completed.

3. The control device according to claim 1, characterized in that the first region is a region used to acquire the focal state of the subject.

4. The control device according to claim 1, characterized in that the first region is a region in which the subject can be detected.

5. The control means is A first process that starts controlling the focus lens in response to instructions from a user-operated operating means, The control device according to claim 1, characterized in that it is capable of performing a second process, which is to automatically start the control of the focus lens with respect to the subject.

6. The control device according to claim 1, characterized in that the first position is the position of the focus lens when the focus adjustment is first completed.

7. The control device according to claim 5, characterized in that, in the first process, the focus state can be acquired across the entire imaging screen.

8. The control means is In the second process, after moving the focus lens to the first position at the end of shooting, the second process is temporarily stopped. The control device according to claim 5, characterized in that the second process is started when the subject enters the first region again.

9. The control device according to claim 1, characterized in that the control means moves the focus lens to the first position when the focus lens moves in the same direction during shooting, at the end of shooting.

10. The control device according to claim 1, characterized in that the control means notifies the user of information regarding the first position.

11. The control device according to claim 1, characterized in that the control means notifies the user of information regarding the timing when the focus adjustment is first completed.

12. The control device according to claim 1, characterized in that the control means displays the subject distance corresponding to the first position on the display means.

13. The control device according to claim 1, characterized in that the control means moves the focus lens to the first position when shooting is automatically instructed to start, or when shooting is finished.

14. An imaging device comprising a control device according to any one of claims 1 to 13 and the image sensor.

15. The imaging apparatus according to claim 14, further comprising an optical system including the aforementioned focusing lens.

16. A step of detecting a subject using the signal output from the image sensor, The system includes the step of controlling the focus lens according to the focal state of the subject, In the step of controlling the focus lens, The first position of the focus lens corresponding to the focal state of the subject in the first region is stored, A control method characterized in that, during shooting, if the focus lens moves from the first position to the second position and the subject moves outside the first region, the focus lens is moved from the second position to the first position.

17. A program characterized by causing a computer to execute the control method described in claim 16.

Images