Focusing device and method, and electronic apparatus

Abstract

To perform focusing according to a photographer's intention and considering the influence of curvature of a field.SOLUTION: A focusing device has: focus detection means that detects a focus state in one or more areas on the basis of image data obtained from imaging means; subject detection means that detects a subject from an image of the image data and output subject information; determination means that determines whether one or more predetermined conditions are satisfied on the basis of at least one of photographing setting information set by a user and the subject information, and determines whether to perform adjustment processing of correcting the influence of curvature of a field on the focus state; adjustment means that, when the adjustment processing is determined to be performed, adjusts the focus state detected by the focus detection means; and arithmetic means that obtains the amount of drive of a focus lens on the basis of the focus state adjusted by the adjustment means or the focus state detected by the focus detection means when the focus state is not adjusted by the adjustment means.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a focus adjustment device and method, and to an electronic device equipped with a focus adjustment device. [Background technology]

[0002] In recent years, the pursuit of smaller and thinner digital cameras and higher magnification of zoom lenses has made it more difficult to suppress the residual aberration of the photographing lens. In general, in an imaging optical system, aberration fluctuations occur and the performance changes when the photographing conditions such as the distance to the subject change, so a balanced design is performed to keep the aberration within a predetermined range. For example, among the aberrations, the curvature of field varies depending on the photographing distance.

[0003] Patent Document 1 discloses a technique for removing error components due to the field curvature according to the position of the image plane from the distance information of each subject, taking into consideration the designed amount of field curvature, and adjusting the focus based on the obtained distance information so that the error in the amount of focus deviation is minimized over the entire photographed screen. This controls the entire screen to be in focus. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2014-202922 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the conventional technology disclosed in Patent Document 1, it is necessary to determine the arrangement state of multiple subjects based on the distance to the subject and the position on the screen, but there is a problem that it is difficult to detect the subject in landscape photography where there is no specific subject. In addition, when the photographer sets a focus detection frame at an arbitrary point, it is required to focus the subject within the focus detection frame set at the arbitrary point, rather than focusing the entire screen. However, in Patent Document 1, only the state of the subject is used as a method to remove the influence of the field curvature from the distance information of the subject, and the photographer's intention is not taken into account from the camera settings, etc., so there is a problem that the photographer's demands mentioned above cannot be met.

[0006] The present invention has been made in consideration of the above problems, and has an object to enable focus adjustment that takes into account the effects of curvature of field in accordance with the photographer's intentions. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the focus adjustment device of the present invention has a focus detection means for detecting a focus state in one or more regions based on image data obtained from an imaging means, a subject detection means for detecting a subject from an image of the image data and outputting subject information, a determination means for determining whether or not to perform an adjustment process to correct the effect of field curvature on the focus state by determining whether or not one or more predetermined conditions are satisfied based on at least one of shooting setting information set by a user and the subject information, an adjustment means for adjusting the focus state detected by the focus detection means when it is determined by the determination means that the adjustment process should be performed, and a calculation means for calculating the amount of drive of the focus lens based on the focus state adjusted by the adjustment means or, if no adjustment has been made by the adjustment means, the focus state detected by the focus detection means. Effect of the Invention

[0008] According to the present invention, it is possible to perform focus adjustment that takes into account the influence of curvature of field in accordance with the photographer's intention. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of an imaging apparatus according to an embodiment of the present invention. [Diagram 2] 4A and 4B are diagrams showing the configuration of a pixel according to the embodiment and the corresponding relationship between the exit pupil of an optical system and a photoelectric conversion unit of the pixel. [Diagram 3] 10A and 10B are diagrams showing another configuration of a pixel according to the embodiment and a correspondence relationship between the exit pupil of the optical system and the aperture of the pixel. [Figure 4] 4 is a flowchart illustrating a flow of imaging processing according to the embodiment. [Diagram 5] 5A and 5B are diagrams for explaining focus detection areas according to the embodiment. [Figure 6] 5A to 5C are diagrams showing examples of detected subjects and subject detection frames according to the embodiment. [Figure 7] 5A to 5C are diagrams showing examples of a method for displaying a focus state according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] [Configuration of imaging device] 1 is a block diagram showing the configuration of an imaging device according to an embodiment of the present invention. In the following description, a single-lens reflex digital camera with interchangeable lenses is used as the imaging device, but the present invention is also applicable to digital cameras with non-interchangeable lenses and video cameras. The present invention is also applicable to any electronic device equipped with a camera, such as a mobile phone, a personal computer (laptop, tablet, etc.), or a game console, or to a camera system consisting of a camera and a control device capable of remotely controlling the camera.

[0012] 1, lens unit 150 is a lens unit equipped with a photographing lens, is configured to be detachable from camera body 100, and includes aperture 102, lens 103, aperture drive circuit 2, lens drive circuit 3, lens system control circuit 4, and communication terminal 6. Lens 103 is usually composed of multiple lenses including a focus lens, zoom lens, etc., but is shown here simply as a single lens.

[0013] The lens system control circuit 4 communicates with the system control unit 50 via a communication terminal 6 and a communication terminal 10 on the camera body 100 side. The lens system control circuit 4 controls the aperture 102 via the aperture drive circuit 2 based on instructions from the system control unit 50, and adjusts the focus by moving the position of a focus lens included in the lens 103 via the lens drive circuit 3. At this time, the focus lens is controlled by the system control unit 50 using a focus detection result obtained by a focus detection method in this embodiment, which will be described later.

[0014] In the camera body 100 , the shutter 101 is a focal plane shutter that can control the exposure time of the imaging unit 22 under the control of the system control unit 50 . The imaging unit 22 is an imaging element that converts an optical image incident through the lens unit 150 into an electrical signal and is composed of a CCD, a CMOS element, or the like. The detailed configuration of the imaging unit 22 will be described later with reference to FIG. 2, but a plurality of pixels each having a plurality of photoelectric conversion units are arranged for one microlens. With this configuration, the entrance pupil of the lens 103 is divided, so that a focus detection signal (hereinafter referred to as a "focus detection signal") having a phase difference can be obtained from each photoelectric conversion unit. In addition, an imaging signal can be obtained by adding up the signals of the plurality of photoelectric conversion units for each pixel. The focus detection signal and imaging signal obtained in this manner can be used not only as an image but also for exposure control and focus adjustment control.

[0015] The A / D converter 23 converts the analog signal output from the imaging unit 22 into a digital signal. The image processing unit 24 performs predetermined pixel interpolation, resizing such as reduction, and color conversion processing on the digital signal output from the A / D converter 23 or image data read from the memory control unit 15. The image processing unit 24 also performs predetermined arithmetic processing using the image data (focus detection signal and image capture signal) obtained by capturing an image, and the system control unit 50 performs exposure control and focus adjustment control based on the obtained arithmetic results. This allows TTL (through-the-lens) type AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-flash) processing to be performed. The image processing unit 24 also performs predetermined arithmetic processing using the image data obtained by capturing an image, and also performs TTL type AWB (auto white balance) processing based on the obtained arithmetic results.

[0016] The output data from the A / D converter 23 is written into the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. The memory 32 stores image data obtained by the imaging unit 22 and converted into digital signals by the A / D converter 23, and image data to be displayed on the display unit 28.

[0017] The memory 32 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio. The memory 32 also serves as a memory (video memory) for image display. The D / A converter 19 converts the image data for display stored in the memory 32 into an analog signal and supplies it to the display unit 28. The image data for display written in the memory 32 is displayed by the display unit 28 via the D / A converter 19. The display unit 28 performs display according to the analog signal from the D / A converter 19 on a display such as an LCD. In this way, the digital signal once A / D converted by the A / D converter 23 and stored in the memory 32 is converted to analog in the D / A converter 19 and sequentially transferred to the display unit 28 for display, thereby functioning as an electronic viewfinder and performing through image display (live view display). The display unit 28 may be provided as an electronic viewfinder that is viewed through an eyepiece (not shown), or may be provided as a display on the back of the digital camera 100. In addition, both an electronic viewfinder and a display on the back may be provided.

[0018] The non-volatile memory 56 is an electrically erasable and recordable memory, and may be, for example, an EEPROM. Constants, programs, and the like for operating the system control unit 50 are stored in the non-volatile memory 56. The programs referred to here are programs for executing various flowcharts described later in this embodiment.

[0019] The system control unit 50 controls the entire imaging device. Moreover, each process of this embodiment, which will be described later, is performed under the control of the system control unit 50. At that time, the system control unit 50 executes a program recorded in a non-volatile memory 56 to realize each process of this embodiment, which will be described later. A RAM is used for the system memory 52, and constants and variables for the operation of the system control unit 50, programs read from the non-volatile memory 56, etc. are expanded therein. The system control unit 50 also performs display control by controlling the memory 32, the D / A converter 19, the display unit 28, etc.

[0020] The system timer 53 is a timing unit that measures the time used for various controls and the time of a built-in clock. The power switch 72 is an operating member for switching the power of the digital camera 100 ON / OFF.

[0021] The mode changeover switch 60, the shutter button 61, and the operation unit 70 are operation units for inputting various operation instructions to the system control unit 50. The mode changeover switch 60 switches the operation mode of the system control unit 50 to one of a still image recording mode, a video shooting mode, a playback mode, etc. Modes included in the still image recording mode include, for example, an auto shooting mode, an auto scene discrimination mode, a manual mode, an aperture priority mode (Av mode), and a shutter speed priority mode (Tv mode). In addition, there are various scene modes, which are shooting settings according to shooting scenes, a program AE mode, a custom mode, etc. The mode changeover switch 60 can directly switch to one of these modes included in the menu button. Alternatively, after switching to the menu button once with the mode changeover switch 60, it is also possible to switch to one of these modes included in the menu button using another operation member. Similarly, the video shooting mode may also include multiple modes.

[0022] The first shutter switch 62 is turned on when the shutter button 61 is pressed halfway (instruction to prepare for shooting) during operation, and generates a first shutter switch signal SW1. The first shutter switch signal SW1 starts operations such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing.

[0023] The second shutter switch 64 is turned on when the shutter button 61 is fully pressed (photographing instruction) and generates a second shutter switch signal SW2. The system control unit 50 starts a series of photographing processing operations from reading a signal from the imaging unit 22 to writing image data to the recording medium 200 in response to the second shutter switch signal SW2.

[0024] The operation unit 70 is a variety of operation members that serve as an input unit for accepting operations from a user. The operation unit 70 is provided with buttons, a cross key, a touch panel, and the like for performing menu selection, mode selection, playback of captured video images, and the like. Each operation member of the operation unit 70 is assigned a function for each scene by selecting and operating various function icons displayed on the display unit 28, and acts as various function buttons. Examples of the function buttons include an end button, a back button, an image forward button, a jump button, a filter button, and an attribute change button. For example, when the menu button is pressed, a menu screen in which various settings can be made is displayed on the display unit 28. The user can intuitively make various settings using the menu screen displayed on the display unit 28, the four directional buttons (up, down, left, and right), and the SET button. Also, by arranging a touch panel on the display unit 28, various settings can be made by touching various displays on the display unit 28.

[0025] The power supply control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to which electricity is applied, and detects whether a battery is attached, the type of battery, and the remaining battery power. The power supply control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the necessary voltage to each unit including the recording medium 200 for the necessary period. The power supply unit 30 is composed of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like.

[0026] The recording medium I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium for recording captured images, and is composed of a semiconductor memory such as a memory card, a magnetic disk, or the like.

[0027] The communication unit 54 is connected to an external device wirelessly or via a wired cable, and transmits and receives video and audio signals. The communication unit 54 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 54 can transmit images (including through images) captured by the imaging unit 22 and images recorded on the recording medium 200, and can also receive image data and various other information from the external device.

[0028] The attitude detection unit 55 detects the attitude of the digital camera 100 with respect to the direction of gravity. Based on the attitude detected by the attitude detection unit 55, it is possible to determine whether the image captured by the imaging unit 22 was captured with the digital camera 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the attitude detected by the attitude detection unit 55 to the image file of the image captured by the imaging unit 22, or rotate and record the image. The attitude detection unit 55 can be an acceleration sensor, a gyro sensor, or the like.

[0029] The camera body 100 described above allows shooting using AF method settings such as "any one-point AF," which performs AF on one arbitrary area within the shooting screen, and "multi-point / zone AF," which performs AF on a position arbitrarily selected from multiple points / zones within the shooting screen.

[0030] Furthermore, the camera body 100 described above can perform processing to capture an image in a focused state in line with the photographer's intention, taking into consideration subject information, shooting setting information, and field curvature, which will be described later.

[0031] [Explanation of focus detection process] FIG. 2 is a diagram showing the configuration of a pixel according to this embodiment and the corresponding relationship between the exit pupil of the optical system and the photoelectric conversion unit of the pixel. Each pixel is provided with a microlens 251, a color filter 252, and two photoelectric conversion units, a photoelectric conversion unit 201a and a photoelectric conversion unit 201b. Light passing through a partial region 253a of the exit pupil is incident on the photoelectric conversion unit 201a via the microlens 251 and the color filter 252, and light passing through a partial region 253b of the exit pupil is incident on the photoelectric conversion unit 201b via the microlens 251 and the color filter 252. With this configuration, it is possible to perform so-called image-surface phase-difference focus detection (hereinafter, referred to as "image-surface phase-difference AF") using signals obtained from the photoelectric conversion unit 201a and the photoelectric conversion unit 201b. Also, an imaging signal can be generated by adding the signals obtained from the photoelectric conversion unit 201a and the photoelectric conversion unit 201b for each pixel.

[0032] In this embodiment, by providing the pixels shown in Figure 2 across the entire screen area of ​​the imaging unit 22, it is possible to detect the focus state by image plane phase difference AF for any subject imaged on the entrance surface of the imaging unit 22.

[0033] FIG. 3 is a diagram showing another configuration of a pixel according to the present embodiment and the corresponding relationship between the exit pupil of the optical system and the aperture of the pixel. The pixel shown in FIG. 3 is a pixel dedicated to focus detection, unlike the pixel shown in FIG. 2. Each pixel is provided with a microlens 251, a color filter 252, an aperture 254, and a photoelectric conversion unit 201. The shape of the exit pupil 253 is determined by the aperture 254. Light that has passed through the exit pupil 253 enters the photoelectric conversion unit 201 through the microlens 251, the color filter 252, and the aperture 254. A pixel having this configuration photoelectrically converts only light that has passed through the pupil plane 253, so that a pair of pixels, that is, a pixel in which the aperture 254 of the pixel shown in FIG. 3 is configured on the left side, must be provided separately to obtain a pair of focus detection signals. Furthermore, by distributing the pixel shown in FIG. 3 and its paired pixel discretely throughout the pixels (imaging pixels) that do not have the configuration of an opening 254, it becomes possible to detect the focus state by imaging surface phase difference AF for any subject imaged on the entrance surface of the imaging unit 22.

[0034] Next, imaging surface phase difference AF will be described, assuming that each pixel constituting the imaging unit 22 has the configuration shown in FIG. The following processing is performed on an AF image A formed by collecting focus detection signals output from the photoelectric conversion unit 201a of each pixel, and an AF image B formed by collecting focus detection signals output from the photoelectric conversion unit 201b of each pixel, using a known phase difference calculation method or the like.

[0035] First, the amount of correlation between the AF A image and the AF B image, which are the focus detection signals in this embodiment, is calculated. The correlation amount is found at each position while relatively shifting the positions of the AF A image and the AF B image by the shift amount stored in the non-volatile memory 56. The correlation amount COR(k) is calculated by the following formula (1).

[0036] TIFF2025025064000002.tif6106, where (-kmax ≦ k ≦ kmax)

[0037] In formula (1), k is the relative shift amount between the AF A image and the AF B image during phase difference calculation, and is an integer between -kmax and kmax. The value of k closest to the phase difference between the AF A image and the AF B image, that is, the value of k that minimizes the correlation amount COR(k), is then found. Here, if the value of k is calculated as an integer, the resolution becomes coarse, so an appropriate interpolation process is performed to perform so-called sub-pixel calculation.

[0038] In this embodiment, the difference DCOR between the correlation amount COR(k) and the correlation amount COR(k-1) is calculated, and the shift amount dk at which the sign of the difference changes is detected. The difference DCOR is calculated by the following formula (2).

[0039] DCOR(k) = COR(k)-COR(k-1) …(2) Using this difference amount DCOR, a shift amount dk at which the sign of the difference amount changes is detected. The shift amount dk at which the sign of the difference amount changes is the phase difference. If the value of k just before the sign changes is k1 and the value of k just after the sign changes is k2 (k2=k1+1), the shift amount dk is calculated by the following formula (3).

[0040] dk = k1+ |DCOR(k1)| / |DCOR(k1)-DCOR(k2)| …(3) In this manner, the shift amount dk of the AF A image and the AF B image of less than one pixel is calculated, and the process ends. There are various known methods for calculating the phase difference, and other methods may also be used. Then, the defocus amount Def is calculated from the obtained phase difference. The calculated phase difference is multiplied by the sensitivity stored in advance in the non-volatile memory 56 to convert the phase difference into the defocus amount Def.

[0041] In this embodiment, the imaging plane phase difference AF is described as the focus detection method, but the focus detection method in the present invention is not limited to the imaging plane phase difference method. For example, the imaging unit 22 may not be provided with focus detection pixels as shown in Fig. 2 or Fig. 3, but may be provided with only imaging pixels, and the focus detection (hereinafter, referred to as "contrast AF") may be performed by a known so-called contrast method using the obtained signal. Contrast AF is a method of extracting specific frequency components of an image signal (so-called band-pass filter processing) and performing contrast evaluation to detect focus. Unlike image plane phase difference AF, focus detection is performed using contrast evaluation values ​​of multiple frames.

[0042] [Image processing] Next, the imaging process in the first embodiment will be described with reference to the flowchart in Fig. 4. Note that the process shown in Fig. 4 is mainly realized by the system control unit 50.

[0043] First, in S1, the imaging unit 22 starts to be driven under the control of the system control unit 50, and image data is acquired. Note that the image data acquired in S1 is not for recording, which will be described later, but for detecting various information and displaying images, so an image smaller in size than an image to be recorded is acquired. For example, an image having a sufficient resolution for focus detection, subject detection, or live view display is acquired. Here, a moving image for live view display is shot, and as a driving operation, charge accumulation and readout for a time according to the frame rate of the live view display are performed by shooting using a so-called electronic shutter. The live view display performed here is for the photographer to check the shooting range and shooting conditions, and is generally 30 frames / second (shooting interval 33.3 ms) or 60 frames / second (shooting interval 16.6 ms), but is not limited to these. After starting to drive the imaging unit 22, the process proceeds to S2.

[0044] In S2, the camera body 100 acquires shooting setting information. The shooting setting information includes exposure setting values, lens aperture values, etc., as well as AF method information, AF operation information, and imaging operation information. The AF method information includes an arbitrary one-point AF mode and a multi-point / zone AF mode, which are the arbitrary area setting methods of this embodiment. The AF operation information includes a one-shot AF mode in which focus adjustment is performed once, a continuous focus adjustment mode, and a servo AF mode in which continuous shooting is performed while keeping the focus on the subject. The imaging operation information includes a single shooting mode in which one image is captured, and a continuous imaging mode in which multiple images are captured continuously. After acquiring these shooting setting information, the process proceeds to S3.

[0045] In S3, during operation of the shutter button 61 provided on the digital camera 100, the first shutter switch 62 is turned ON by a so-called half-press (shooting preparation instruction), and it is detected whether or not the first shutter switch signal SW1 is generated. If the first shutter switch signal SW1 is detected, the process proceeds to S4 to perform the focus detection process and focus adjustment process in this embodiment. On the other hand, if the first shutter switch signal SW1 is not detected, the process returns to S2 to obtain the latest shooting setting information while maintaining the shooting standby state.

[0046] In S4, from the AF method information among the shooting setting information acquired in S2, it is determined whether the mode is a single-point AF mode or a multi-point / zone AF mode, and the focus detection area (AF frame) is set according to the determined mode. The single-point AF mode is a mode in which the AF frame is set to one location, and is used when the photographer can clearly identify the subject on which he or she wants to focus with AF. On the other hand, while the multi-point and zone AF modes both have the AF frame set to multiple locations, the zone AF mode differs from the multi-point AF mode in that the AF frame is set to multiple locations within a limited range. Unlike the single-point AF mode, the multi-point / zone AF mode is used when the photographer is not clear on the subject on which he or she wants to focus with AF, or when shooting landscapes where the locations on which he or she wants to focus are roughly the same distance.

[0047] The process of setting the AF frame in accordance with the AF method information, which is performed in S4, will now be described with reference to FIG. Figures 5(a) and (b) show examples in the arbitrary single-point AF mode, where 501 in Figure 5(a) is the shooting screen, 502 is the AF frame in the arbitrary single-point AF mode, and 504 in Figure 5(b) is a person as an example of a subject in the arbitrary single-point AF mode.

[0048] Figures 5(c) and (d) show examples in multi-point / zone AF mode, where 501 in Figure 5(c) is a shooting screen and a dotted rectangle 503 shows multiple AF frames in multi-point / zone AF mode. A solid rectangle 505 in Figure 5(d) shows multiple AF frames selected by the system control unit 50 after the defocus amount on the closest side has been calculated from the multiple AF frames 503 shown in Figure 5(c). A thick rectangle 506 shows the AF frame selected by the system control unit 50 after the defocus amount on the closest side has been calculated from the multiple AF frames 505.

[0049] First, when an arbitrary one-point AF mode, which is an AF mode that allows the photographer to specify a subject in an arbitrary area on which the photographer wants to focus, is selected as the AF method information, the photographer sets an AF frame 502 in an arbitrary area as shown in Fig. 5(a). Note that, although an example in which the AF frame 502 is set in the center of the shooting screen is shown in this embodiment, the AF frame may be set in a position other than the center, such as a peripheral position of the shooting screen. Also, when a person 504 is set as the specific subject of this embodiment as shown in Fig. 5(b), the photographer frames the image so that the AF frame 502 overlaps the face of the person 504.

[0050] On the other hand, if the multi-point / zone AF mode, which is an AF mode in which the system control unit 50 sets the subject to be focused on instead of the photographer, is selected as the AF mode information, multiple AF frames 503 are arranged over substantially the entire shooting screen, as shown in Fig. 5(c). When shooting a landscape, as in Fig. 5(d), the photographer matches the landscape to be shot with the angle of view. After setting the focus detection area in this manner, the process proceeds to S5.

[0051] In S5, the defocus amount for each AF frame set in S4 is calculated. That is, when the arbitrary one-point AF mode is selected as the AF method information, the defocus amount of the person 504 in Fig. 5(b) framed in consideration of the AF frame 502 set by the photographer is calculated. In addition, subject distance information and depth of field information are calculated from the calculated defocus amount using optical information of the lens 103.

[0052] On the other hand, if the multi-point / zone AF mode is selected as the AF method information, the defocus amount is calculated for each of the multiple AF frames 503 shown in Fig. 5(c) for the angle of view adjusted by the photographer. Then, of the calculated defocus amounts for the multiple AF frames 503, the AF frame 505 indicating the closer side than a predetermined distance and the AF frame 506 on the closest side (hereinafter referred to as the "main frame") are displayed on the display unit 28 as shown in Fig. 5(d). After the series of processes is completed, the process proceeds to S6.

[0053] In S6, a process for detecting a specific subject is performed. As a method for detecting a subject, a known learning method using machine learning or a recognition process using image processing can be used. In the learning method using machine learning, the feature amount of each part (face, eyes, whole body) of a subject (person, animal, etc.) is learned in advance, the subject learned in advance is recognized from a captured image, and information on the area (position and size) is obtained.

[0054] For example, there are several types of machine learning: Support Vector Machine ·Convolutional Neural Network Recurrent Neural Network

[0055] As an example of the recognition process, a method is known in which a skin color area is extracted from the gradation color of each pixel represented by image data, and a face is detected based on the degree of matching with a face outline plate prepared in advance. In addition, a method is also known in which face detection is performed by extracting facial feature points such as the eyes, nose, and mouth using a well-known pattern recognition technique. Note that the main area detection method applicable to this embodiment is not limited to these methods, and other methods such as a method using color, brightness value, or contrast value may be used.

[0056] Then, the position and size on the screen of the detected subject are obtained. Note that in this embodiment, in the case of multi-point / zone AF mode, it is possible that the subject cannot be detected. In such a case, it is determined that no subject has been detected. After the subject detection process is performed as described above, the process proceeds to S7.

[0057] In S7, based on the shooting setting information acquired in S2, the defocus amount calculated in S5, and the subject information obtained in S6, it is determined whether or not to perform a defocus amount adjustment process that takes into account the effect of field curvature on the focus state in order to capture an image in a focused state that meets the photographer's intention, taking into account the field curvature.

[0058] Here, an example of a condition for performing the adjustment process in S7 that takes into account the field curvature in this embodiment will be described.

[0059] (1) If it is determined that the subject has not been detected based on the subject information (2) When the AF method information indicates that the camera is in multi-point / zone AF mode (3) When the AF operation information determines that the camera is in One-Shot AF mode (4) When it is determined from the imaging operation information that the imaging mode is single-shot mode (5) When it is determined from the subject information and the defocus amount that the distance to the detected subject is equal to or greater than a predetermined distance threshold value. (6) When it is determined from the subject information that the size of the detected subject is less than a predetermined size threshold. (7) If it is determined from the lens aperture that the depth of field is equal to or greater than a predetermined depth of field threshold.

[0060] <Condition (1)> First, a case in which it is determined whether or not to perform adjustment processing based on subject information will be described with reference to FIG. 6(a) and (b) show cases where a subject is detected in S6. In FIG. 6(a), 601 indicates a subject detection frame, and 602 indicates a person who is the subject. This is an example of a case where a person is detected in portrait photography or the like, and it is a scene where it is considered that the photographer frequently photographs the person who is the subject. Also, FIG. 6(b) shows an example where an airplane 603 flying over a mountain is photographed as the subject, and 601 indicates the subject detection frame, and 603 indicates the airplane who is the subject. Although it is a landscape photograph in the distance, the airplane 603 is detected as a specific subject, and it is a scene where it is considered that the photographer frequently intentionally photographs the airplane 603 or the like as a subject. In this way, when a subject is detected, it is determined that adjustment processing is not to be performed.

[0061] On the other hand, compared to Fig. 6(b), Fig. 6(c) has a similar angle of view, but is a scene in which the airplane 603, the subject of Fig. 6(b), does not exist. In such a case, since the subject is not detected in S6 and it is considered to be a landscape shot in the distance, it is determined that adjustment processing should be performed.

[0062] <Condition (2)> Next, a case where it is determined whether or not to perform adjustment processing from AF method information will be described. As described above, FIG. 5B shows a scene in which the arbitrary one-point AF mode is set and the photographer frames the subject person 504. In this way, when the arbitrary one-point AF mode is set, it is considered that the photographer intends to photograph a specific subject, and therefore would prefer to focus on one arbitrary point rather than focusing on the entire image in consideration of the field curvature. Therefore, it is determined that adjustment processing is not to be performed in such a case.

[0063] On the other hand, in a scene where a landscape is photographed with a distant view as in Fig. 6(c), it is considered that the multi-point / zone AF mode is often set as in Fig. 5(d). Therefore, when the multi-point / zone AF mode is set, it is determined that adjustment processing will be performed.

[0064] Here, a method for displaying the focus adjustment state when the AF method information is the multi-point / zone AF mode will be described with reference to FIG. FIG. 7(a) shows a case where the multi-point / zone AF mode is set in the AF method information during landscape photography, similarly to FIG. 5(d). At this time, when the first shutter switch signal SW1 is detected, for example, as shown in FIG. 5(d), it is considered to superimpose the main frame 506 and the multiple AF frames 505 of the close-side defocus amount on the display unit 28. In contrast, in this embodiment, when performing an adjustment process to photograph the entire image in a focused state in consideration of the field curvature, a frame 701 is displayed on the periphery of the screen of the display unit 28 as shown in FIG. 7(b) to indicate that the entire screen is in focus, instead of the main frame 506 and the multiple AF frames 505 of the close-side defocus amount. Note that, although the case where a frame is displayed on the periphery of the screen of the display unit 28 has been described in this embodiment, a notification method in another form that does not display a specific part of the photographing screen may be used. For example, an icon or the like indicating that the adjustment process has been performed may be displayed.

[0065] <Condition (3)> Next, a case where it is determined whether or not to perform adjustment processing based on AF operation information will be described. The above-mentioned servo AF mode is an AF operation mode that allows continuous shooting while continuing to focus on a subject. Therefore, it is frequently used in scenes where a specific subject or the like is moving. In this case, it is determined that adjustment processing is not performed in the servo AF mode so that processing is performed so that the focus is on the specific subject or the like, rather than on the entire shooting screen.

[0066] On the other hand, the One-Shot AF mode is an AF operation mode that performs focus adjustment processing once. One-Shot AF mode is often used for landscape photography, so in such cases it is considered better to perform adjustment processing. Note that even in the Servo AF mode, the first focus detection may be considered to be the same as the One-Shot AF mode described later, and it may be determined that adjustment processing is to be performed.

[0067] <Condition (4)> Next, a case where the adjustment process is determined based on the imaging operation information will be described. The continuous imaging mode is frequently used in a scene where a specific subject is moving. Therefore, in order to process the image so that the focus is on the specific subject, not on the entire image screen, the adjustment process is determined not to be performed in the continuous imaging mode.

[0068] On the other hand, the single shot mode is an imaging operation in which an image is shot once. In the case of shooting a stationary subject such as a landscape, the single shot mode is considered to be used frequently, and it is determined that adjustment processing is to be performed.

[0069] <Condition (5)> Next, a case will be described where, when it is determined from the subject information that a subject has been detected, whether or not to perform the adjustment process is determined based on the distance to the subject. Based on the defocus amount calculated in S5, it is determined that the adjustment process is performed when the subject is located far away where the effect of the field curvature is noticeable. Since the field curvature increases according to the distance to the subject, it is considered that the adjustment process does not need to be performed when the distance to the subject is close. Therefore, in this embodiment, when the distance to the subject is equal to or greater than a predetermined distance threshold, which is a distance at which the effect of the field curvature is noticeable, it is determined that the adjustment process is performed, and when it is less than the distance threshold, it is determined that the adjustment process is not performed.

[0070] If it is not determined from the subject information that a subject has not been detected, it is determined that adjustment processing is to be performed for the same reason as in condition (1).

[0071] <Condition (6)> Next, a case will be described where, when it is determined from the subject information that a subject has been detected, whether or not to perform adjustment processing is determined based on the size of the subject. Based on the size of the subject obtained in S6, it is determined that adjustment processing is performed when the subject is located far away where the effect of field curvature is noticeable. Since field curvature increases according to the distance to the subject, it is considered that adjustment processing does not need to be performed when the distance to the subject is short. Therefore, in this embodiment, when the size of the subject is less than a predetermined size threshold, which is the size at the distance where the effect of field curvature is noticeable, it is determined that adjustment processing is performed, and when it is equal to or greater than the size threshold, adjustment processing is not performed.

[0072] If it is not determined from the subject information that a subject has not been detected, it is determined that adjustment processing is to be performed for the same reason as in condition (1).

[0073] <Condition (7)> Finally, a case in which it is determined whether or not to perform the adjustment process based on the depth of field will be described.

[0074] From the subject depth information calculated in S5, the adjustment process is performed outside the depth of field range where the influence of the field curvature is noticeable. The depth of field range is calculated from the F-number, focal length information, and subject distance information included in the optical information of the lens 103. If the calculated depth of field range is narrower than the depth of field threshold value representing the depth of field range including the amount of defocus caused by the field curvature determined by the F-number and focal length information, it is considered that the influence of the field curvature is not noticeable and it is determined that the adjustment process is unnecessary. If the calculated depth of field range is wider than the depth of field threshold value representing the depth of field range including the amount of defocus caused by the field curvature, it is considered that the range of the influence of the field curvature is within the depth of field range and is noticeable, and it is determined that the adjustment process is to be performed.

[0075] In the judgment in S7, if it is determined that the adjustment process is to be performed for all of the above-mentioned conditions (1) to (7), it is determined that the adjustment process is to be performed. If it is determined that the adjustment process is not to be performed for any one of the conditions, it is determined that the adjustment process is not to be performed.

[0076] The determination method is not limited to the above, and may be determined based on any one of the conditions, or may be determined by combining some of the conditions (1) to (7). In that case, if it is determined that the adjustment process is not to be performed based on any one of the conditions, it is determined that the adjustment process is not to be performed. Alternatively, a priority order may be set for the conditions (1) to (7), and it may be determined whether or not to perform the adjustment process based on the highest priority condition among the conditions that can be determined based on the obtained information. For example, if the subject detection is turned off by the camera settings, the subject information required for determining the condition (1) cannot be obtained, so it is determined whether or not to perform the adjustment process based on the conditions (2) to (7). In this embodiment, the order of the conditions (1) to (7) is the order of the highest priority, but the order of the priority is not limited to this order.

[0077] If it is determined that the adjustment process is to be performed as a result of the above determination, the process proceeds to S8, and if it is determined that the adjustment process is not to be performed, the process proceeds to S11.

[0078] 5(d), in order to perform focus adjustment using the defocus amount in the main frame 506 on the closest side, the coordinates (x, y) of the main frame 506 in the image are set as the image height position for calculating the adjustment amount (p) of the defocus amount in S8. After the image height position is set in this way, the process proceeds to S9.

[0079] In S9, an adjustment amount (p) is calculated based on the image height position set in S8. Note that, when calculating the adjustment amount (p), the adjustment amount for each coordinate is stored in advance in the non-volatile memory 56, and the adjustment amount (p) at the image height position set in S8 is applied. Furthermore, the adjustment amount for each coordinate stored in the non-volatile memory 56 may be a value for all coordinates, but may be an adjustment amount for discrete coordinates. In this case, the adjustment amount (p) for the image height position may be calculated and applied by linear interpolation, polynomial approximation, or the like, based on the distance from the coordinates (x0, y0) that are the reference position to the set image height position.

[0080] The adjustment amount of the lens 103, which is the field curvature information, is held on the lens side in a lens-interchangeable camera, and is therefore held in the lens system control circuit 4. In a lens-integrated camera, the adjustment amount is held in the non-volatile memory 56. Furthermore, the method of calculating the adjustment amount (p) is not limited to the above method, and any method may be used as long as it is a method for finding the adjustment amount (p) of the lens 103 that takes into account the effect of the curvature of field. Once the adjustment amount (p) is calculated in S9, the process proceeds to S10.

[0081] In S10, the corrected defocus amount (Def+p) is calculated by adding the adjustment amount (p) calculated in S9 to the defocus amount (Def) at the main frame position calculated in S5, and the process proceeds to S13.

[0082] On the other hand, S11 is a process that is performed if it is determined in S7 that the adjustment process is not to be performed. If the adjustment process is not to be performed, the main frame is set at a position where a specific subject is detected in the subject detection, and the process proceeds to S12.

[0083] In S12, the defocus amount (Def) in the main frame calculated in S5 is determined as the defocus amount to be used in the focus adjustment process, and the process proceeds to S13.

[0084] In S13, the driving amount of the focus lens is calculated based on the defocus amount (Def+p) set in S10 or the defocus amount (Def) set in S12, and is sent to the lens system control circuit 4. The lens system control circuit 4 controls the lens driving circuit 3 based on this driving amount to drive the focus lens included in the lens 103, thereby adjusting the focus.

[0085] In S14, the shutter button 61 is fully pressed (photographing instruction) to turn ON, and it is determined whether or not the second shutter switch signal SW2 has been detected. If the second shutter switch signal SW2 is detected, the process proceeds to S15 to acquire a photographed image. If the second shutter switch signal SW2 is not detected, the process proceeds to S16 to determine whether or not to continue the series of photographing processes including focus detection process, focus adjustment process, and image capture process without acquiring a photographed image.

[0086] In S15, imaging processing is performed to obtain a photographed image. The photographed image is stored in the recording medium 200. After the photographed image is stored, the process proceeds to S16.

[0087] In S16, it is determined whether or not an instruction to continue shooting has been given by the photographer. If an instruction to continue shooting has been given, the process proceeds to S4 and a series of shooting processes from S4 to S16 is carried out. If an instruction to continue shooting has not been given, the series of shooting processes from S4 to S14 is ended. The instruction to continue shooting is used to determine whether the continuous focus adjustment mode or the continuous shooting mode is related to the second shutter switch signal SW2.

[0088] As described above, according to this embodiment, it is possible to change whether or not to perform adjustment processing to capture the entire image in focus in consideration of the field curvature. Note that, in this embodiment, the adjustment amount is added to the defocus amount for correction, but after shooting, the difference in spatial frequency between the center and periphery of the image data may be calculated and correction may be performed by refocus processing. In addition, bracket shooting may be performed during shooting based on the field curvature information of the shooting optical system, and the user may select an image from the images captured after shooting.

[0089] <Other embodiments> The present invention may be applied to a system made up of a plurality of devices, or to an apparatus made up of a single device.

[0090] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0091] <Summary> The disclosure of this embodiment includes the following configuration.

[0092] (Item 1) a focus detection means for detecting a focus state in one or more predetermined regions based on image data obtained from the imaging means; a subject detection means for detecting a subject from an image of the image data and outputting subject information; a determination means for determining whether or not one or more predetermined conditions are satisfied based on at least one of the photographing setting information set by a user and the subject information, and determining whether or not to perform an adjustment process for correcting the effect of the curvature of field on the focus state; an adjustment unit that adjusts the focus state detected by the focus detection unit when the determination unit determines that the adjustment process is to be performed; a calculation means for calculating a driving amount of the focus lens based on the focus state adjusted by the adjustment means, or based on the focus state detected by the focus detection means when the focus state has not been adjusted by the adjustment means; A focus adjustment device comprising: (Item 2) The shooting setting information is Information indicating whether a first mode in which a region for detecting a focus state is selected by a user or a second mode in which a region for detecting a focus state is not selected by a user is set; and Information indicating whether a third mode in which the subject is continuously focused or a fourth mode in which a process for detecting the focus state is performed once is set; and Information indicating whether the single shot mode or the continuous shooting mode is set; and information regarding the depth of field; The subject information includes Information indicating whether the subject was detected, When the subject is detected, at least one of the distance to the subject and the size of the subject in the image is included. 2. The focus adjustment device according to item 1, (Item 3) The one or more predetermined conditions include a plurality of conditions, The focus adjustment device according to item 2, characterized in that the determination means determines whether or not all of the plurality of conditions are satisfied, and when it is determined that all of the plurality of conditions are satisfied, determines to perform the adjustment process. (Item 4) The one or more predetermined conditions include a plurality of conditions with priority set, The focus adjustment device according to item 2, characterized in that the determination means determines whether or not the conditions are satisfied among the plurality of conditions in order of priority, and when it is determined that the conditions are satisfied, determines to perform the adjustment process. (Item 5) 5. The focus adjustment device according to any one of items 2 to 4, wherein the one or more predetermined conditions include the second mode being set. (Item 6) 5. The focus adjustment device according to any one of items 2 to 4, wherein the one or more predetermined conditions include the fourth mode being set. (Item 7) 5. The focus adjustment device according to any one of items 2 to 4, wherein the one or more predetermined conditions include the single shooting mode being set. (Item 8) 5. The focus adjustment device according to any one of items 2 to 4, wherein the one or more predetermined conditions include that the depth of field is equal to or greater than a predetermined threshold value. (Item 9) 5. The focus adjustment device according to any one of items 2 to 4, wherein the one or more predetermined conditions include a condition in which a subject is not detected. (Item 10) The focus adjustment device according to any one of items 2 to 4, characterized in that the one or more predetermined conditions include that a subject is detected and the distance to the subject is equal to or greater than a predetermined threshold. (Item 11) 5. The focus adjustment device according to any one of items 2 to 4, wherein the predetermined condition includes that an object is detected and that the size of the object in the image is less than a predetermined threshold. (Item 12) The image data is displayed on a display unit, and a display control unit is configured to superimpose a notification indicating that focus adjustment is being performed based on the image data on the display unit, The focus adjustment device according to any one of items 1 to 11, characterized in that the display control means displays the notification in a first form when the calculation means determines the drive amount based on a focus state adjusted by the adjustment means, and displays the notification in a second form when the calculation means determines the drive amount based on a focus state detected by the focus detection means. (Item 13) An imaging means; A focus adjustment device according to any one of items 1 to 12, 1. An electronic device comprising: (Item 14) a focus detection step in which a focus detection means detects a focus state in one or more predetermined regions based on image data obtained from the imaging means; a subject detection step in which a subject detection means detects a subject from an image of the image data and outputs subject information; a determination step in which a determination means determines whether or not one or more predetermined conditions are satisfied based on at least one of the photographing setting information set by a user and the subject information, and determines whether or not to perform an adjustment process for correcting the effect of the curvature of field on the focus state; an adjustment step in which, when it is determined in the determination step that the adjustment process is to be performed, an adjustment unit adjusts the focus state detected in the focus detection step; a calculation step of calculating a driving amount of the focus lens based on the focus state adjusted in the adjustment step, or based on the focus state detected in the focus detection step if the focus state has not been adjusted in the adjustment step; A focus adjustment method comprising the steps of: (Item 15) A program for causing a computer to function as each of the means of the focus adjustment device according to any one of items 1 to 12. (Item 16) Item 16. A computer-readable storage medium storing the program according to item 15.

[0093] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0094] 3: lens drive circuit, 4: lens system control circuit, 22: imaging section, 28: display section, 50: system control section, 60: mode changeover switch, 70: operation section, 100: camera body, 103: lens, 150: lens unit, 502, 503, 505, 506: AF frame, 602, 603: subject

Claims

1. a focus detection means for detecting a focus state in one or more areas based on image data obtained from the imaging means; a subject detection means for detecting a subject from an image of the image data and outputting subject information; a determination means for determining whether or not one or more predetermined conditions are satisfied based on at least one of photographing setting information set by a user and the subject information, and determining whether or not to perform an adjustment process for correcting the effect of field curvature on the focus state; an adjustment unit that adjusts the focus state detected by the focus detection unit when the determination unit determines that the adjustment process is to be performed; a calculation means for calculating a driving amount of a focus lens based on a focus state adjusted by the adjustment means, or based on a focus state detected by the focus detection means when the focus state has not been adjusted by the adjustment means; A focus adjustment device comprising:

2. The shooting setting information is Information indicating whether a first mode in which the user selects an area for detecting a focus state or a second mode in which the user does not select an area for detecting a focus state is set; and Information indicating whether a third mode in which the subject is continuously focused or a fourth mode in which a process for detecting a focus state is performed once is set; and Information indicating whether the single shot mode or the continuous shooting mode is set; and information regarding the depth of field; The subject information includes Information indicating whether the subject was detected, When the subject is detected, at least one of the distance to the subject and the size of the subject in the image is included.

2. The focus adjustment device according to claim 1.

3. The one or more predetermined conditions include a plurality of conditions, 3. The focus adjustment device according to claim 2, wherein the determination means determines whether or not all of the plurality of conditions are satisfied, and when it is determined that all of the plurality of conditions are satisfied, determines to perform the adjustment process.

4. the one or more predetermined conditions include a plurality of conditions with priority set; 3. The focus adjustment device according to claim 2, wherein the determination means determines whether or not a condition is satisfied among the plurality of conditions in order of priority, and when it is determined that the condition is satisfied, determines that the adjustment process is to be performed.

5. 3. The focus adjustment device according to claim 2, wherein the one or more predetermined conditions include that the second mode is set.

6. 3. The focus adjustment device according to claim 2, wherein the one or more predetermined conditions include that the fourth mode is set.

7. 3. The focus adjustment device according to claim 2, wherein the one or more predetermined conditions include the single-shot mode being set.

8. The focus adjustment device according to claim 2 , wherein the one or more predetermined conditions include that the depth of field is equal to or greater than a predetermined threshold value.

9. 3. The focus adjustment device of claim 2, wherein the one or more predetermined conditions include no subject being detected.

10. 3. The focus adjustment device according to claim 2, wherein the one or more predetermined conditions include a condition in which an object is detected and a distance to the object is equal to or greater than a predetermined threshold.

11. 3. The focus adjustment device according to claim 2, wherein the one or more predetermined conditions include that an object is detected and a size of the object in the image is less than a predetermined threshold.

12. The image data is displayed on a display unit, and a display control unit is configured to superimpose a notification indicating that focus adjustment is being performed based on the image data on the display unit, 2. The focus adjustment device according to claim 1, wherein the display control means causes the notification to be displayed in a first form when the calculation means determines the drive amount based on a focus state adjusted by the adjustment means, and causes the notification to be displayed in a second form when the calculation means determines the drive amount based on a focus state detected by the focus detection means.

13. An imaging means; A focus adjustment device according to any one of claims 1 to 12, 1. An electronic device comprising:

14. a focus detection step in which a focus detection means detects a focus state in one or more areas based on image data obtained from the imaging means; a subject detection step in which a subject detection means detects a subject from an image of the image data and outputs subject information; a determination step in which a determination means determines whether or not one or more predetermined conditions are satisfied based on at least one of the photographing setting information set by a user and the subject information, and determines whether or not to perform an adjustment process for correcting the effect of the curvature of field on the focus state; an adjustment step in which, when it is determined in the determination step that the adjustment process is to be performed, an adjustment unit adjusts the focus state detected in the focus detection step; a calculation step of calculating a driving amount of the focus lens based on the focus state adjusted in the adjustment step, or based on the focus state detected in the focus detection step if the focus state has not been adjusted in the adjustment step; A focus adjustment method comprising the steps of:

15. A program for causing a computer to function as each of the means of the focus adjustment device according to any one of claims 1 to 12.

16. A computer-readable storage medium storing the program according to claim 15.

Images