Processing device and control method thereof

Abstract

To allow for more accurately assigning attribute information on the focusing state when shooting a moving object.SOLUTION: A camera control unit 212 provides control to store image data in association with information on the focusing state of the image data. The information on the focusing state is based on a result of moving object prediction calculation of a target object using a focus detection result computed from a signal corresponding to the image data.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to a process for adding attribute information based on focus detection to an image. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a phase difference detection method has been generally known as a focus detection method for image pickup apparatuses.

[0003] In recent years, the frame speed of cameras has improved and the number of shots taken has increased dramatically, resulting in a problem that checking the focus after shooting requires a great deal of time and effort.

[0004] As one of the solutions, Patent Document 1 discloses storing captured image data in association with attribute information on the focus state of the captured image data. The attribute information on the focus state in Patent Document 1 is information indicating in-focus or out-of-focus calculated based on a signal obtained by an imaging sensor using a phase difference detection method. By checking an image whose attribute information indicates in-focus, the photographer can improve the efficiency of the focus check task. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2011-209450 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, the conventional techniques disclosed in the above-mentioned patent documents do not take into consideration the movement of the subject, and therefore, when a photographer photographs a moving object, there are cases where attribute information of the photographed image data that should be assigned with attribute information indicating focus is assigned with attribute information indicating out of focus.

[0007] SUMMARY OF THE PRESENT EMBODIMENTS An object of the present invention is to provide a processing device and a control method thereof that can assign attribute information relating to the focus state with higher accuracy when a moving object is photographed. [Means for solving the problem]

[0008] In order to achieve the above object, the present invention provides a method for obtaining an image of a subject by imaging the subject through an imaging optical system, the method comprising: acquiring image data corresponding to an imaging signal obtained from an imaging sensor that captures an object image incident thereon through an imaging optical system and outputs an imaging signal; based on a comparison between a first moving object prediction result using a first focus detection result based on an imaging signal acquired from the imaging sensor before exposure of the imaging sensor to obtain the image data, and a second moving object prediction result using a second focus detection result based on an imaging signal acquired from the imaging sensor after exposure of the imaging sensor to obtain the image data. The information on the focus state is stored in association with Ruko It is characterized by: Effect of the Invention

[0009] According to the present invention, motion When a body is photographed, attribute information relating to the focus state can be assigned with higher accuracy. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the configuration of an interchangeable lens camera including a lens and a camera body according to a first embodiment; [Diagram 2] Camera sequence for continuous still image shooting [Diagram 3] Diagram explaining the principle of focus detection and defocus amount detection by the AF sensor [Figure 4] Image signal captured by the AF sensor [Diagram 5] An example of an area for acquiring an image signal indicating the focus detection range used in focus detection processing [Figure 6] Correlation amount shown as a waveform [Figure 7] A schematic diagram showing the relative positions of the subject, lens, and image sensor [Figure 8] Flow of focus detection by the AF sensor [Figure 9] Focus adjustment control sequence chart [Figure 10] Overview of calculation of focus attribute information [Figure 11]Sequence chart for calculating focus attribute information [Figure 12] Overview of calculation of focus attribute information [Figure 13] FIG. 11 is a block diagram showing the configuration of a lens-interchangeable camera including a lens and a camera body according to a second embodiment; [Figure 14] Image sensor with image-plane phase-difference AF [Figure 15] 13 is a sequence chart showing calculation of attribute information on whether focus is possible or not in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. EXAMPLES

[0012] (Configuration of the present invention) FIG. 1 is a block diagram showing the configuration of an interchangeable lens camera including a lens and a camera body according to a first embodiment of the present invention.

[0013] As shown in FIG. 1, this embodiment is composed of a lens 10 and a camera 20, and information is communicated between a lens control unit 106 that controls the overall operation of the lens and a camera control unit 212 that controls the overall operation of the camera.

[0014] First, the configuration of the lens 10 will be described.

[0015] The lens 10 includes a fixed lens 101 , an aperture 102 , a focus lens 103 , an aperture driver 104 , a focus lens driver 105 , a lens control 106 , and a lens operation unit 107 .

[0016] Reference numerals 101 to 103 denote an imaging optical system, in which 101 denotes a fixed first lens group, 102 denotes an aperture, and 103 denotes a focus lens. The aperture 102 is driven by an aperture control unit 104 to control the amount of light incident on an image sensor 201, which will be described later.

[0017] The focus lens 103 is driven by a focus lens driver 105 and adjusts the focus of an image formed on an image sensor 201, which will be described later.

[0018] Aperture driver 104 and focus lens driver 105 are controlled by lens controller 106 , which determines the opening amount of aperture 102 and the position of focus lens 103 .

[0019] The lens control unit 106 controls the aperture driving unit 104 and the focus lens driving unit 105 in accordance with control commands and control information received from a camera control unit 212 (described later), and also transmits lens control information to the camera control unit 212.

[0020] Next, the configuration of the camera 20 will be described.

[0021] The camera 20 is configured so as to be able to obtain an image signal from a light beam that has passed through the imaging optical system of the lens 10. The light beam that has passed through the imaging optical system on the lens side is guided to a quick return mirror 252 that is rotatable.

[0022] The center of the quick-return mirror 252 is a half mirror, through which a part of the light beam passes when the quick-return mirror 252 is lowered (lowered in the state shown in FIG. 1).

[0023] This transmitted light beam is reflected by a sub-mirror 253 attached to the quick return mirror 252, and is guided to a phase difference AF sensor 254 which is an automatic focus adjustment means.

[0024] The AF sensor 254 is controlled by a focus detection circuit 255 .

[0025] As will be mentioned in the explanation of FIG. 6, the screen may have multiple separator lenses and light receiving elements arranged vertically and multiple separator lenses and light receiving elements arranged horizontally.

[0026] In this embodiment, this focus detection method is called "focus detection by an AF sensor."

[0027] On the other hand, the photographing light beam reflected by the quick return mirror 252 forms an image on the matte screen 250, which can be observed by the photographer from above through the pentaprism 251 and eyepiece 256.

[0028] Furthermore, when the quick return mirror 252 moves up (rising towards the pentaprism 251 as indicated by the arrow, though not shown), the light beam from the lens 10 is focused on the image sensor 201 via a filter 259 and a focal plane shutter 258, which is a mechanical shutter. That is, an image of the subject is focused on the image sensor 201. The filter 259 has two functions: one is to cut out infrared rays and ultraviolet rays and to guide only visible light to the image sensor 201, and the other is to function as an optical low-pass filter. The focal plane shutter 258 includes a front curtain and a rear curtain, and is a light blocking means that controls the transmission and blocking of the light beam from the lens 10.

[0029] The light beam passing through the photographing optical system on the lens side forms an image on the light receiving surface of the image sensor 201 and is converted by the photodiodes into signal charges according to the amount of incident light. The signal charges accumulated in the photodiodes are sequentially read out from the image sensor 201 as voltage signals according to the signal charges, based on drive pulses provided by a timing generator 215 in accordance with commands from a camera control unit 212.

[0030] The image signal read out from the image sensor 201 is input to a CDS / AGC / AD converter 202, which performs correlated double sampling to remove reset noise, adjusts the gain, and digitizes the signal. The CDS / AGC / AD converter 202 outputs the image signal to an image input controller 203.

[0031] The image input controller 203 stores the image signal output from the CDS / AGC / AD converter 202 in the SDRAM 209 .

[0032] The image signal stored in the SDRAM 209 is displayed on the display unit 206 by the display control unit 205 via the bus 21. In addition, in a mode for recording an imaging signal, the image signal is recorded on the recording medium 208 by the recording medium control unit 207. In addition, the ROM 210 connected via the bus 21 stores a control program executed by the camera control unit 212 and various data necessary for control, and the flash ROM 211 stores various setting information related to the operation of the camera 20, such as user setting information.

[0033] The camera control unit 212 exchanges information with the entire camera 20 to perform control. In addition to processing within the camera 20, it executes various camera functions operated by the user, such as turning the power on / off, changing settings, starting still image / video recording, starting AF control, and checking recorded video, in response to input from the camera operation unit 214. It also exchanges information with the lens control unit 106 in the lens 10, as described above, to send lens control commands and control information, and to obtain information within the lens.

[0034] (Basic operation) Next, the operation of the camera 20 in FIG. 1 according to this embodiment will be described.

[0035] In this embodiment, a sequence in which focus detection is performed only by the AF sensor 254 is used.

[0036] The camera sequence when performing continuous still image shooting will be described with reference to FIG.

[0037] In S201, it is determined whether or not the shutter half-press (SW1) of the camera operation unit 214 is ON. If it is ON, the process proceeds to S202. S201 is repeated until it is ON.

[0038] In S202, the light beam that passes through the lens 10, is reflected by the main mirror 202, and passes through the pentaprism 251, and is then measured by a photometry circuit (not shown).

[0039] In S203, the camera control unit 212 performs focus detection using the AF sensor 254 and the focus detection circuit 255. Details of focus detection by the AF sensor 254 will be described later.

[0040] In S204, focus adjustment control using predictive AF is executed. Details of focus detection control will be described later. 。 In S205, based on the focus adjustment result obtained in S204, the camera control unit 212 transmits a lens driving amount to the lens control unit 106. The lens control unit 106 controls the focus lens driving unit 105 based on the transmitted lens driving amount, and the focus lens driving unit 105 drives the photographing lens 103 to a focus position.

[0041] In S206, it is determined whether or not the shutter button (SW2) of the camera operation unit 214 is fully pressed ON. If it is ON, the process proceeds to S207, where continuous shooting is started. Until it is ON, the process returns to S201.

[0042] In S207, the camera control unit 212 controls the mirror 252 to flip up the mirror.

[0043] In S208, the camera control unit 212 transmits the aperture value information set in S202 to the lens control unit 106, which drives the aperture drive unit 104 to narrow the aperture to the set aperture value. This is performed almost simultaneously with the mirror lock-up in S207.

[0044] In S209, the camera control unit 212 controls the focal plane shutter 258 to open.

[0045] In S210, the light beam from the photographing lens 103 is accumulated in the image sensor 201 for a predetermined period of time.

[0046] In S211, the focal plane shutter 258 is closed. The camera control unit 212 performs a charging operation of the focal plane shutter 258 in preparation for the next operation.

[0047] In S212, the camera control unit 212 instructs the image input controller 203 to read out the image data from the image sensor 201. Simultaneously with the reading, the image input controller 203 receives image data from the image sensor 201, compresses the image, and records it in the storage medium 208.

[0048] In S213, the camera control unit 212 instructs the lens control unit 106 to open the aperture, and drives the aperture drive unit 104 to open the aperture.

[0049] In S214, the camera control unit 212 drives the mirror 252 downward.

[0050] In S215, since the mirror 252 has been lowered in S214, the camera control unit 212 performs focus detection using the AF sensor 254 and the focus detection circuit 255. The same operation as in S203 Therefore, the description will be omitted. In S216, focus adjustment control using moving object predictive AF is executed.

[0051] In S217, calculation of attribute information on whether or not focusing is possible, which is a feature of this embodiment, is executed. P The calculation of the attribute information on whether or not a user can participate will be described in detail later.

[0052] The basic sequence is now complete, but as long as the shutter button (SW2) of the camera operation unit 214 remains fully pressed ON, continuous shooting is continued and the basic sequence is repeated again.

[0053] (Explanation of focus detection by AF sensor) The focus detection process for calculating the defocus amount for driving the lens from the image signal acquired from the AF sensor 254 will be described according to the flow of FIG.

[0054] In step S801, the image signal of the focus detection range set by the imaging device is acquired. 。 (See Figure 4 below.) S802 calculates the correlation amount from the image signal 。 (See Figure 4 below.) In step S803, the correlation change amount is calculated from the correlation amount calculated in step S802. 。 (See Figure 6 below.) In step S804, the amount of focus deviation is calculated from the amount of correlation change calculated in step S803. 。 (See Figure 6 below.) These processes are performed the same number of times as the number of focus detection areas.

[0055] Then, in S805, the amount of focus deviation is converted into a defocus amount for each focus detection area.

[0056] The focus detection process ends.

[0057] Each step is explained in detail below.

[0058] The principles of focus detection and defocus amount detection (detection of the amount of deviation of the focal position) by the AF sensor 254 will be described with reference to FIGS. 3(a) and 3(b).

[0059] 3(a) and 3(b) are diagrams explaining the principle of defocus amount detection. As shown in the figures, when the image sensor is in focus, the distance between two images on the line sensor is a certain value.

[0060] This value can be determined by design, but in reality, it will not be the same as the design value due to component dimensions, variations, and assembly errors. Therefore, it is difficult to determine this two-image interval (reference two-image interval Lo) without actually measuring it. As is clear from Figure 3(a), if the two-image interval is narrower than this reference two-image interval Lo, it is front-focused, and if it is wider than Lo, it is rear-focused.

[0061] FIG. 3B is a diagram showing a model in which a condenser lens is omitted from the optical system of an AF sensor module (not shown).

[0062] As shown in the figure, if the angle of the principal ray is θ, the magnification of the separator lens is β, and the amount of movement of the image is ΔL and ΔL', the amount of defocus L can be calculated by the following formula.

[0063]

number

[0064] ΔL' can be calculated from the reference two-image interval (Lo) and the current two-image interval (Lt). In actual calculation, the images of the light receiving elements in FIG. 3(a) correspond to 401 and 402 in FIG. 4, and the correlation amount COR is calculated.

[0065] The AF sensor 254 has a plurality of the above configurations so as to be able to detect the focus at a plurality of positions on the photographic screen.

[0066] FIG. 5 is a diagram showing an example of an area for acquiring an image signal indicating a focus detection range used in focus detection processing.

[0067] 1A is a diagram showing the focus detection range on the shooting screen of the AF sensor 254. Reference numeral 501 denotes the shooting screen, and 502 denotes the focus detection range. Reference numeral 503 denotes a shift area required for correlation calculation. Reference numeral 504 is a combination of 502 and 503, and is an area required for correlation calculation.

[0068] In the figure, p, q, s, and t each represent coordinates in the x-axis direction, with p to q representing 504 and s to t representing 502.

[0069] 1B is a diagram showing a focus detection area made up of five light receiving elements. As an example, in this embodiment, the amount of focus deviation is calculated for each focus detection area, and focus detection is performed.

[0070] 505 to 509 are one focus detection area consisting of five light receiving elements in the focus detection range 502. As an example of the embodiment, the focus detection result of the most reliable area is selected from the divided focus detection areas, and the focus deviation amount calculated in that area is used.

[0071] 1C is a diagram showing a provisional focus detection area formed by connecting the focus detection areas in FIG.

[0072] Reference numeral 510 denotes a region obtained by connecting the focus detection regions 505 to 509 .

[0073] As an example of an embodiment, the amount of focus deviation calculated from an area obtained by connecting focus detection areas in this manner may be used.

[0074] The layout of the focus detection areas, the size of the areas, etc. are not limited to those described in this embodiment, and may of course be of any form within the scope of the invention without departing from the spirit of the invention.

[0075] 4 shows an image signal acquired by the AF sensor 254. s to t represent the focus detection range, and p to q represent the range required for focus detection calculation based on the shift amount. Also, x to y represent one divided focus detection area.

[0076] 4A is a diagram showing the image signal before shifting as a waveform. A solid line 401 is image signal A, and a dashed line 402 is image signal B. 405 to 409 represent the divided light receiving elements in FIG.

[0077] (B) is a diagram showing the image waveform before (A) shifted in the positive direction, and (C) is a diagram showing the image waveform before (A) shifted in the negative direction. When calculating the correlation amount, 401 and 402 are shifted by one bit in the direction of the arrow. Next, a method for calculating the correlation amount COR will be explained.

[0078] First, as explained in Figures 5(B) and (C), image signals A and B are shifted one bit at a time, and the sum of the absolute values ​​of the differences between image signals A and B at that time is calculated. In this case, the shift amount is represented by i, the minimum shift number is ps in Figure 8, and the maximum shift number is qt in Figure 8. Also, x is the start coordinate of the focus detection area, and y is the end coordinate of the focus detection area. Using these, the following formula can be used for the calculation.

[0079]

number

[0080] FIG. 6(a) is a diagram showing the correlation amount as a waveform. The horizontal axis of the graph indicates the shift amount, and the vertical axis indicates the correlation amount. 6a01 is the correlation amount waveform, and 6a02 and 6a03 indicate the vicinity of the extreme value. Among these, the smaller the correlation amount is, the higher the degree of match between image A and image B is. Next, a method for calculating the correlation change amount ΔCOR will be explained.

[0081] First, from the correlation amount waveform in Figure 6(a), the correlation change amount is calculated from the difference in correlation amount for every shift. At this time, the shift amount is represented as i, the minimum shift number is ps in Figure 4, and the maximum shift number is qt in Figure 4. Using these, the correlation change amount can be calculated using the following formula.

[0082]

number

[0083] Figure 6(b) is a diagram showing the correlation change amount ΔCOR as a waveform. The horizontal axis of the graph indicates the shift amount, and the vertical axis indicates the correlation change amount. 6b01 is the correlation change amount waveform, and 6b02 and 6b03 are the areas where the correlation change amount goes from positive to negative. The point where the correlation change amount becomes 0 from 6b02 is called the zero crossing, where the degree of match between image A and image B is the highest, and the shift amount at this time causes the image to be out of focus.

[0084] Fig. 6(c) is an enlarged view of 6b02 in Fig. 6(b), and 6b01 is a portion of the correlation change amount waveform 6a01. Next, a method for calculating the focus deviation amount PRD will be described.

[0085] First, the amount of focus deviation is divided into an integer part β and a decimal part α. The decimal part α can be calculated using the following formula based on the similarity relationship between triangles ABC and ADE in the figure.

[0086]

number

[0087] Next, the decimal part β can be calculated from FIG. 6(c) using the following formula: β=k-1 As described above, the focus deviation amount PRD can be calculated from the sum of α and β.

[0088] In reality, the defocus amount must be calculated by multiplying the PRD amount by a coefficient K, which actually calculates the lens drive amount. This coefficient K is explained in the next figure.

[0089] 7 is a schematic diagram showing the positional relationship between the subject, lens 10, and image sensor 201. The calculation method for coefficient K, which will be described later, is shown with reference to this diagram. The shapes of pupils 701 and 702 differ whether AF is performed by AF sensor 254 or AF by sensor 201, but the calculation method is the same. Lens 10 is made up of one or more lenses, but is represented in the diagram as a single convex lens. Exit pupil distance: A is a value specific to the lens. Baseline length:B The image shift amount: C is the amount shown in the graph of FIG.

[0090] Then, the defocus amount: D can be calculated from the similarity of the two triangles as follows: D=A / B*C (10)

[0091] The aforementioned coefficient K can be calculated as follows: K=A / B (11) The defocus amount can be calculated by multiplying the image shift amount by this coefficient.

[0092] (Focus Control) The focus adjustment control S204 in FIG. 2 will be described with reference to the sequence chart in FIG.

[0093] In S901, moving object detection is performed using the result of focus detection by the AF sensor, and the process proceeds to S902. As a moving object detection method, for example, a case where the focus detection result detects movement in a certain direction is included.

[0094] In S902, the result of moving object detection is determined using the result of focus detection by the AF sensor calculated in S901. If it is determined that the object is moving, the process proceeds to S903, and if it is determined that the object is not moving, the process ends without doing anything.

[0095] In S903, since the result of focus detection by the AF sensor is determined to be a moving object, a moving object prediction calculation is performed by the AF sensor. The moving object prediction calculation is a process of acquiring a focus detection result corresponding to the future position of the object using past focus detection results for the object, and specifying the position of the focus lens that will focus on the object at the future position of the object from the focus detection result. If the object is a moving object, the latest focus detection result and the timing of actually capturing an image are different, so the predicted position of the object to be reached during capture is calculated using an approximation curve or the like from the focus detection results of multiple past points and the latest focus detection result. FIG. 10 shows that the lens position at the timing of exposure 1003 to the image sensor is predicted by calculating a prediction curve 1004 from the latest focus point detection result 1001 and the focus detection results of multiple past points 1002.

[0096] This calculation allows the image sensor in S210 of FIG. 2 to focus on the subject during exposure.

[0097] (Calculation of attribute information on whether or not focus is possible) The calculation of attribute information of focus possible or not in Fig. 12, which is a feature of this embodiment, will be described below with reference to the sequence chart of Fig. 11. Note that this embodiment illustrates an example in which information indicating whether focus possible or not is recorded as attribute information related to the focus state in association with image data. The attribute information related to the focus state is not limited to this, and for example, a rating according to the focus state may be stored in association with image data.

[0098] In S1101, the prediction curve 1004 used in the motion prediction calculation during exposure to the image sensor in S204 and S210 is read out.

[0099] In S1102, a predictive calculation process is performed to read out the predictive curve 1204 used in the focus adjustment control S216 using the focus detection S215 that was resumed after the mirror down process was executed in S204.

[0100] In S1103, the motion prediction results obtained in S1101 and S1102 are compared, and it is determined whether the difference in lens position during exposure to the image sensor of S210 is equal to or less than a predetermined value. If it is equal to or less than the predetermined value, it is determined that there is no significant change in the prediction curve before and after exposure to the image sensor of S210 and that the subject can be tracked, and the process proceeds to S1104. If it is greater than the predetermined value, a significant change appears in the prediction curve before and after exposure to the image sensor of S210, it is found that there is a change in the movement of the subject, it is determined that the subject cannot be tracked, and the process proceeds to S1105.

[0101] In S1104, since it is determined in S1103 that the subject can be tracked, the attribute information is set to focusable.

[0102] In S1105, since it is determined in S1103 that the subject cannot be tracked, the attribute information is set to out of focus.

[0103] This feature makes it possible to provide attribute information on whether or not the focus is in focus with high accuracy even in an imaging device that does not have an imaging surface phase difference detection method using an imaging sensor. EXAMPLES

[0104] Fig. 13 is a block diagram showing the configuration of a lens-interchangeable camera including a lens and a camera body according to a first embodiment of the present invention. In Fig. 13, an image sensor 201 equipped with an image plane phase difference detection method and an AF signal processor 1301 are added to Fig. 1 used in the first embodiment. Explanation of common parts will be omitted.

[0105] The image sensor 201 in the second embodiment has two photodiodes 1403 and 1404 per pixel in one microlens (1402 in FIG. 14) in order to perform image plane phase difference AF as shown in FIG. 14(b).

[0106] The light beam is separated by a microlens and focused on the two photodiodes, making it possible to extract two signals, one for imaging and one for AF. The signal (A+B) obtained by adding the signals from the two photodiodes is the imaging signal, and the signals from the individual photodiodes (A, B) are the two image signals for AF. Based on the AF signal, the AF signal processor 1301, which will be described later, performs correlation calculations on the two image signals to calculate the amount of image shift and various reliability information.

[0107] In this embodiment, this focus detection method is called "focus detection by the image sensor 201."

[0108] The imaging signal and AF signal read out from the imaging sensor 201 are input to a CDS / AGC / AD converter 202, which performs correlated double sampling to remove reset noise, adjusts the gain, and digitizes the signal. The CDS / AGC / AD converter 202 outputs the imaging signal to an image input controller 203.

[0109] The image input controller 203 stores the imaging signal output from the CDS / AGC / AD converter 202 in an SDRAM 209 , and also outputs an AF signal to an AF signal processing unit 1301 .

[0110] The AF signal processing unit 1301 performs pixel addition and correlation calculation of the AF signals to calculate the image shift amount and reliability information (degree of coincidence between two images, steepness of two images, contrast information, saturation information, scratch information, etc.). The calculated image shift amount and reliability information are output to the camera control unit 212. Furthermore, the camera control unit 212 notifies the AF signal processing unit 1301 of a change in the settings for calculating these based on the acquired image shift amount and reliability information. For example, when the image shift amount is large, the area for performing the correlation calculation is set to be wide, or the type of band pass filter is changed according to the contrast information. Details of the correlation calculation will be described later.

[0111] In this embodiment, a total of three signals, an imaging signal and an image signal for AF, are extracted from the imaging sensor 201, but the present invention is not limited to this method. Taking into consideration the load on the imaging sensor 201, for example, control may be performed such that a total of two signals, an imaging signal and an image signal for AF, are extracted, and another image signal for AF is generated by calculating the difference between the imaging signal and the signal for AF in the image input controller 203.

[0112] (Explanation of focus detection by image sensor) The focus detection by the AF sensor shown in the first embodiment is omitted because focus detection is performed under the same control after the AF image signal is generated.

[0113] As described above, in the second embodiment, in S212, an image pickup signal and an AF signal are read out from the image sensor 201. The AF signal that has been read out is used to perform focus detection by the image sensor.

[0114] This makes it possible to obtain the focus detection result by the image sensor at the timing of exposure 1003.

[0115] (Calculation of attribute information on whether or not focus is possible) Hereinafter, calculation of attribute information on whether or not focusing is possible, which is a feature of the second embodiment, will be described with reference to a sequence chart of FIG.

[0116] In S1501, the prediction curve 1004 used in the motion prediction calculation during exposure to the image sensor in S204 and S210 is read out.

[0117] In S1502, a predictive calculation process is performed to read out the predictive curve 1204 used in the focus adjustment control S216 using the focus detection S215 that was resumed after the mirror down process was executed in S204.

[0118] In S1503, it is determined whether the focus detection result obtained by the image sensor at the timing of exposure 1003 is reliable. If it is determined that the focus detection result is reliable, the process proceeds to S1504, and if not, the process proceeds to S1507.

[0119] In S1504, it is determined whether the focus detection result obtained by the focus detection result by the image sensor at the timing of exposure 1003 is equal to or less than a predetermined value. If it is determined that the focus detection result is equal to or less than the predetermined value, the process proceeds to S1505, and if not, the process proceeds to S1506.

[0120] In S1505, the focus detection result obtained by the image sensor at the timing of exposure 1003 is reliable, and since it is determined that the focus is good because the focus detection result is equal to or less than a predetermined value, the attribute information is set to in focus.

[0121] In S1506, the focus detection result obtained by the image sensor at the timing of exposure 1003 is reliable, and since it is determined that the focus is poor because the focus detection result is greater than a predetermined value, the attribute information is set to out of focus.

[0122] In S1507, it is determined that the focus detection result obtained by the image sensor at the timing of exposure 1003 is unreliable, and it is determined whether the difference in lens position during exposure to the image sensor of S210 obtained in S1501 and S1502 is equal to or less than a predetermined value. If it is equal to or less than the predetermined value, even if the focus detection result obtained by the image sensor is unreliable, it is determined that there is no significant change in the prediction curve before and after exposure to the image sensor of S210 and that the subject can be tracked, and the process proceeds to S1508. If it is greater than the predetermined value, a significant change appears in the prediction curve before and after exposure to the image sensor of S210, it is understood that there is a change in the movement of the subject, and it is determined that the subject cannot be tracked. S1509 Proceed to.

[0123] According to the present invention, attribute information relating to the focus state is added to image data based on a moving object prediction result using a focus detection result based on an image data signal obtained from an imaging sensor. This makes it possible to add attribute information relating to the focus state with higher accuracy when a moving object is captured.

[0124] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0125] For example, the order of steps performed in the operations described using the flowcharts in the above embodiments can be changed as appropriate so as to achieve the same purpose.

[0126] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions. [Explanation of symbols]

[0127] 201 Image sensor 204 AF signal processing section 208 Recording media 212 Camera control unit

Claims

1. a focus detection unit that performs focus detection by phase difference detection using an image pickup signal acquired from an image pickup sensor that picks up an object image incident via an image pickup optical system and outputs an image pickup signal; a moving object prediction means for calculating a lens position corresponding to a future position of a subject from a plurality of focus detection results; a storage means for storing image data corresponding to an image signal acquired from the image sensor and information on a focus state determined based on the image signal acquired from the image sensor in association with each other, a processing device characterized in that the information regarding the focus state is information based on a comparison between a first moving object prediction result using a first focus detection result based on an imaging signal obtained from the imaging sensor before exposure of the imaging sensor to obtain the image data, and a second moving object prediction result using a second focus detection result based on an imaging signal obtained from the imaging sensor after exposure of the imaging sensor to obtain the image data.

2. The processing device according to claim 1, characterized in that, when a difference between the first moving object prediction result and the second moving object prediction result is equal to or less than a predetermined value, the storage means stores information indicating focus as information regarding the focus state to be associated with the image data.

3. The processing device according to claim 1, characterized in that when the reliability of a third focus detection result based on an image signal acquired from the image sensor by exposing the image sensor to obtain the image data is high, the storage means stores information regarding the focus state based on a comparison of the third focus detection result with a predetermined value in correspondence with the image data, regardless of the first moving object prediction result and the second moving object prediction result, and when the reliability of the third focus detection result is low, the storage means stores information regarding the focus state based on a comparison of the first moving object prediction result and the second moving object prediction result in correspondence with the image data.

4. 4. The processing device according to claim 3, wherein when the reliability of the third focus detection result is determined to be high, if the third focus detection result is equal to or less than a predetermined value, the storage means stores information indicating in-focus as information regarding the focus state to be associated with the image data, and if the third focus detection result is greater than the predetermined value, the storage means stores information indicating out-of-focus as information regarding the focus state to be associated with the image data.

5. a focus detection step of performing focus detection by phase difference detection using an image pickup signal acquired from an image pickup sensor that picks up an object image incident via an image pickup optical system and outputs an image pickup signal; a moving object prediction step of calculating a lens position corresponding to a future position of a subject from a plurality of focus detection results; a storage step of storing image data corresponding to the imaging signal acquired from the imaging sensor and information on a focus state determined based on the imaging signal acquired from the imaging sensor in association with each other, A processing control method characterized in that the information regarding the focus state is information based on a comparison between a first moving object prediction result using a first focus detection result based on an imaging signal acquired from the imaging sensor before exposure of the imaging sensor to obtain the image data, and a second moving object prediction result using a second focus detection result based on an imaging signal acquired from the imaging sensor after exposure of the imaging sensor to obtain the image data.

Images