Electronic apparatus

Abstract

To provide an electronic apparatus that detects eyes from a face of a subject and performs an autofocus control so as to focus on the eyes, capable of improving a focusing accuracy.SOLUTION: An electronic apparatus includes: a detection part configured to detect eyes from an image; a selection part configured to select one eye from the eyes detected by the detection part; an operation part that accepts an operation to make the selection part select an eye different from the previously selected eye; and a display part configured to display a display frame indicating that the eye is the eye selected by the selection part in a superimposed manner on the image, and comprises a display part that displays a display fame indicating that it is the eye selected by the selection part so as to be overlapped to an image. Each defocus amount is calculated for each focus point corresponding to the display frame, and autofocus is performed.SELECTED DRAWING: None

Description

[Technical Field]

[0001] The present invention relates to an electronic device. [Background technology]

[0002] There is known a camera that detects the face of a subject, then detects the eyes of the face, and controls the camera so as to focus on the eyes (see Patent Document 1). There has been a demand for improved focusing accuracy. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2001-215403 Summary of the Invention

[0004] According to a first aspect of the present invention, an electronic device includes a detection unit that detects eyes from an image, a selection unit that selects one eye from the eyes detected by the detection unit, an operation unit that accepts an operation to cause the selection unit to select an eye different from the selected eye, and a display unit that displays a display indicating that the eye selected by the selection unit is superimposed on the image. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 2 is a schematic diagram illustrating a configuration of a main part of the camera system. [Figure 2] 10 is a flowchart illustrating the flow of processing executed by a body-side control unit. [Figure 3] FIG. 2 is a diagram illustrating an example of a photographing range. [Figure 4] FIG. 4(a) is a diagram illustrating an example of a display frame showing detected eyes, and FIG. 4(b) is a diagram illustrating an example of a display frame showing detected faces. [Figure 5] FIG. 10 is a diagram illustrating a case where multiple eyes or faces are detected. [Figure 6] FIG. 2 is a diagram illustrating an example of the exterior surface on the rear side of the camera body when held in a landscape position. [Figure 7] FIG. 6 is a diagram illustrating a case where the eye located on the right side of the face display frame in FIG. 5 is selected. [Figure 8] 8 is a diagram illustrating a case where the eyes of a face located on the right side of the face display frame in FIG. 7 are selected. FIG. [Figure 9] 8 is a diagram illustrating another example in which the eyes of the face located on the right side of the face display frame in FIG. 7 are selected. FIG. [Figure 10] 10(a) to 10(c) are diagrams illustrating the relationship between the size of the eyes or face in an image and the display frame. [Figure 11] 10 is a flowchart illustrating a process flow for determining a display frame when a first setting is selected in the first embodiment. [Figure 12] 10 is a diagram illustrating an example of the exterior surface on the rear side of the camera body when held in a vertical position. FIG. [Figure 13] FIG. 10 is a diagram illustrating a state in which a plurality of faces are lined up in the Y-axis direction (vertical direction). [Figure 14] 10 is a flowchart illustrating a process flow for determining a display frame when a first setting is selected in the second embodiment. [Figure 15] FIG. 10 is a schematic diagram illustrating distance calculation. [Figure 16] 16(a) and 16(b) are diagrams showing an example in which the face faces to the right relative to the camera system, and FIG. 16(c) is a diagram showing an example in which the face faces to the left relative to the camera system. DETAILED DESCRIPTION OF THE INVENTION

[0006] Hereinafter, an embodiment of the invention will be described with reference to the drawings. (First embodiment) 1 is a schematic diagram illustrating the configuration of the main parts of a camera system 1 according to a first embodiment of the invention. The camera system 1 will be described as an example in which a detachable photographic lens (referred to as an interchangeable lens 3) is attached to a camera body 2, but the camera may also be one in which the camera body 2 and the photographic lens are integrated. In FIG. 1, the horizontal direction in a plane intersecting the optical axis O of the interchangeable lens 3 is defined as the X axis, and the vertical direction is defined as the Y axis.

[0007] <Camera body> The camera body 2 has a body side control unit 230, a body side communication unit 240, an attitude sensor 250, an image sensor 260, a signal processing unit 270, an operation member 280, and a display unit 290. The body side control unit 230 is connected to the body side communication unit 240, the attitude sensor 250, the image sensor 260, the signal processing unit 270, the operation member 280, and the display unit 290.

[0008] The body side control unit 230 is composed of a microcomputer and its peripheral circuits, etc. The body side control unit 230 includes a memory unit 235 that stores control programs and the like executed by the body side control unit 230, a focus detection unit 230a that performs focus detection, an eye / face detection unit 230b that detects eyes and faces, a display control unit 230c, a display range selection unit 230d that selects a display range, a focus detection range selection unit 230e that selects a focus detection range, and a selection unit 230f that has the display range selection unit 230d and the focus detection range selection unit 230e. The body side control unit 230 executes control programs stored in the memory unit 235 to control each unit within the camera body 2. The body side control unit 230 controls the recording and reading of data in the memory unit 235.

[0009] The body side communication unit 240 performs predetermined communication with the lens side communication unit 340 of the interchangeable lens 3. The body side communication unit 240 is connected to the body side control unit 230. The attitude sensor 250 detects angle information indicating whether the camera system 1 is held in a horizontal position or a vertical position, and sends a detection signal to the body side control unit 230. The camera system 1 is held in a horizontal position (a camera position where the long side of the imaging surface 260S of the image sensor 260 is horizontal) when shooting a landscape screen that is long in the X axis direction of FIG. 1, and is held in a portrait position (a camera position where the long side of the imaging surface 260S of the image sensor 260 is vertical) when shooting a portrait screen that is long in the Y axis direction.

[0010] Light from the subject that has passed through the interchangeable lens 3 is guided to the imaging surface 260S of the image sensor 260. The image sensor 260 is a solid-state image sensor such as a CMOS image sensor or a CCD image sensor. The image sensor 260 captures an image of the subject formed on the imaging surface 260S in response to a control signal from the body-side control unit 230 and outputs a signal. The image sensor 260 is capable of capturing both video and still images. Video capture not only records video, but also includes capturing so-called through images (also called live view images) for continuously displaying monitor images on the display unit 290.

[0011] The image sensor 260 has a photoelectric conversion unit for image generation and a photoelectric conversion unit for focus detection. Image pickup pixel signals based on charges generated in the photoelectric conversion unit for image generation are used to generate image data by an image signal processing unit 270a of the signal processing unit 270. Furthermore, focus detection pixel signals based on charges generated in the photoelectric conversion unit for focus detection are used in focus detection processing by an AF signal processing unit 270b of the signal processing unit 270 to detect the focus state of the subject image formed by the interchangeable lens 3, in other words, the deviation between the position of the subject image formed by the interchangeable lens 3 and the position of the image pickup surface 260S. The image sensor 260 is connected to the signal processing unit 270 and the body side control unit 230 .

[0012] Signal processing unit 270 includes image signal processing unit 270a and AF signal processing unit 270b. Image signal processing unit 270a generates image data by performing predetermined image processing on imaging pixel signals output from image sensor 260. The generated image data is recorded in a predetermined file format on a storage medium (such as a memory card) not shown, and is used to display a live view image before shooting or a confirmation image after shooting.

[0013] The AF signal processing unit 270b also calculates the defocus amount (the amount of deviation between the imaging position of the interchangeable lens 3 and the imaging surface 260S) by performing focus detection processing using a phase difference detection method using focus detection pixel signals output from the imaging element 260. The focus detection processing may also use a contrast method. The signal processing unit 270 is connected to the body side control unit 230 , the image sensor 260 , and the display unit 290 .

[0014] An operation member 280 including a release button, operation switches, etc. is provided on the exterior surface of the camera body 2. The operation member 280 sends an operation signal to the body-side control unit 230 in response to a user's operation. The user issues instructions for shooting and setting shooting conditions by operating the operation member 280. The user can also use the operation member 280 to issue an instruction to switch the "AF area mode," which will be described later.

[0015] The display unit 290 is configured with an organic EL display panel or a liquid crystal display panel. The display unit 290 displays an image based on image data processed by the image signal processing unit 270a, an operation menu screen, etc., in response to a control signal from the body-side control unit 230. The display unit 290 may also be equipped with a touch panel, which may configure a touch operation member 290A as part of the operation member 280. The display unit 290 is connected to the body-side control unit 230.

[0016] <Interchangeable lenses> The interchangeable lens 3 has a lens side control unit 330, a lens side communication unit 340, a photographing optical system 360, a lens driving unit 370, and an aperture driving unit 380. The lens side control unit 330 is connected to the lens side communication unit 340, the lens driving unit 370, and the aperture driving unit 380.

[0017] The lens side control unit 330 is made up of a microcomputer and its peripheral circuits, etc. The lens side control unit 330 executes a control program stored inside and controls each part of the interchangeable lens 3, such as automatic focus adjustment control.

[0018] The lens side communication unit 340 performs predetermined communication with the body side communication unit 240. Through communication between the lens side communication unit 340 and the body side communication unit 240, instructions to move the photographic optical system 360 (described below) and the like are transmitted from the camera body 2 to the interchangeable lens 3. Furthermore, through communication between the lens side communication unit 340 and the body side communication unit 240, information such as the driving state of the photographic optical system 360 is transmitted from the interchangeable lens 3 to the camera body 2.

[0019] The photographing optical system 360 includes a plurality of lenses and a diaphragm member 362, and guides subject light to the imaging surface 260S of the imaging element 260 of the camera body 2. The plurality of lenses includes a focus lens 361. For example, the focus lens 361 is configured to be movable in the direction of the optical axis O by a lens driving unit 370. By moving the focus lens 361, the position of the subject image formed by the photographing optical system 360 is changed. This causes the AF operation (focusing).

[0020] The diaphragm 362 adjusts the amount of light incident on the imaging element 260. The diaphragm 362 is configured so that the aperture diameter (aperture value) can be changed by driving the diaphragm blades using the diaphragm driving unit 380 or manually.

[0021] Drive instructions such as the direction, amount, and speed of movement of the focus lens 361 are issued by the body-side control unit 230. Alternatively, instructions may be issued from the lens-side control unit 330 based on information from the body-side control unit 230.

[0022] The camera body 2 communicates with the interchangeable lens 3 to send, for example, a focus lens drive signal which is an instruction to drive the focus lens, an aperture drive signal, and information about the camera body 2 (image capture mode information, ISO sensitivity information, etc.) to the interchangeable lens 3. In response to the drive signal from the camera body 2, the interchangeable lens 3 drives the focus lens 361 using a lens drive unit 370.

[0023] Meanwhile, the interchangeable lens 3 communicates with the camera body 2 to transmit to the camera body 2, for example, information on the optical characteristics of the photographing optical system 360, position information of the focus lens 361, and information on the aperture diameter of the diaphragm 362.

[0024] <Example of camera operation> First, an example of the camera operation of the camera system 1 will be described. Fig. 2 is a flowchart illustrating the flow of processing executed by the body-side control unit 230. The body-side control unit 230 starts the processing according to the flowchart in Fig. 2, for example, when starting up by turning on the main switch, or when canceling sleep mode by operating a switch or the like that constitutes the operation member 280. Sleep mode refers to the operation of transitioning to a power-saving mode after a predetermined period of no operation. Cancellation of sleep mode (return to normal mode) can be performed, for example, by half-pressing the release button.

[0025] In step S10, the body side control unit 230 performs exposure settings for a live view image and performs accumulation control (exposure control) on the image sensor 260. The exposure settings for capturing the first image after startup are controlled to a predetermined initial value, and accumulation control is performed on the image sensor 260.

[0026] In step S20, the body side control unit 230 causes the signal processing unit 270 to read out signals from the image sensor 260, and the process proceeds to step S30. The body side control unit 230 calculates each control value by performing exposure calculation in step S30. That is, the body side control unit 230 detects brightness information (Bv value) of the subject based on the imaging pixel signals input to the signal processing unit 270, and determines the aperture value (Av value), shutter speed (Tv value), and sensitivity (Sv value) based on the Bv value and program diagram information stored in the storage unit 235. The calculated control values ​​are stored in the storage unit 235.

[0027] In step S40, the body side control unit 230 sets the aperture 362 to the determined aperture value and performs live view display. The body side control unit 230 causes the image signal processing unit 270a to generate a live view image using the imaging pixel signals output from the image sensor 260, and causes the display unit 290 to display the generated live view image.

[0028] In step S45, the body side control unit 230 causes the AF signal processing unit 270b to perform AF calculation. As a result, the AF signal processing unit 270b calculates the defocus amount using the focus detection pixel signals output from the image sensor 260.

[0029] In step S50, the body side control unit 230 determines whether the release button has been pressed halfway. If the release button has been pressed halfway, the body side control unit 230 makes an affirmative judgment in step S50 and proceeds to step S60, but if the release button has not been pressed halfway, the process returns to step S10.

[0030] In step S60, the body side control unit 230 issues an AF drive instruction. Based on the defocus amount calculated in step S45, the body side control unit 230 moves the focus lens 361 of the interchangeable lens 3 along the direction of the optical axis O. In other words, the lens side control unit 330 issues an instruction to the lens drive unit 370 to drive the focus lens 361, causing it to perform a focus drive operation. The body side control unit 230 then proceeds to step S70.

[0031] In step S70, the body side control unit 230 determines whether the release button has been fully pressed. If the release button has been fully pressed, the body side control unit 230 makes an affirmative judgment in step S70 and proceeds to step S80, but if the release button has not been fully pressed, the body side control unit 230 makes a negative judgment in step S70 and returns to step S10.

[0032] After returning to step S10, the body side control unit 230 again performs the processing from step S10 onwards. The exposure setting for the second frame of the live view image onwards uses the aperture value stored in the storage unit 235, and the shutter speed (Tv value) and sensitivity (Sv value) stored in the storage unit 235.

[0033] In step S80, which is the process that is carried out when the release button is fully pressed (a positive judgment in step S70), the body side control unit 230 performs a storage operation for capturing an image for recording. At that time, the aperture 362 is set to the aperture value (Av value) stored in the storage unit 235, storage control of the image sensor 260 is performed at the shutter speed (Tv value) stored in the storage unit 235, and processing is performed using the sensitivity (Sv value) stored in the storage unit 235.

[0034] In step S90, the body side control unit 230 reads out signals from the image sensor 260 to the signal processing unit 270, and proceeds to step S100. In step S100, the body side control unit 230 causes the image signal processing unit 270a to perform predetermined image processing on the imaging pixel signals output from the image sensor 260, and generate an image for recording. As a result, the image signal processing unit 270a performs the predetermined image processing and generates an image for recording.

[0035] In step S110, the body side control unit 230 displays an image. That is, the body side control unit 230 causes the display unit 290 to display the image processed by the image signal processing unit 270a in step S100 as a confirmation image. In step S120, the body side control unit 230 causes the storage unit 235 to record the file of the image processed by the image signal processing unit 270a.

[0036] In step S130, the body side control unit 230 determines whether or not to end the process. For example, if the main switch is turned off or the camera is transitioning to sleep mode, the body side control unit 230 makes a positive decision in step S130 and ends the process in Fig. 2. If the body side control unit 230 does not want to end the process in Fig. 2 and wants to continue camera operation, it makes a negative decision in step S130 and returns to step S10.

[0037] The display process (S110) and the recording process (S120) may be performed in reverse order, or may be performed in parallel. In addition, in the example of FIG. 2, the defocus amount is calculated in the AF calculation (S45) after the live view display (S40) is performed, but the defocus amount may be calculated in parallel with the exposure calculation (S30), and an instruction to drive the focus lens 361 may be issued in step S60 after a half-press operation is performed in step S50. Furthermore, a configuration may be adopted in which the calculation of the defocus amount and the instruction to drive the focus lens 361 are performed in parallel with the exposure calculation (S30) each time each frame of a live view image is acquired.

[0038] <Focus point> The focus points used in the focus detection process during the AF operation (S60) will now be described. 3 is a diagram illustrating an example of a shooting range 50 captured by the imaging element 260 of the camera body 2. A plurality of (e.g., 135) focus points P are set in advance in the shooting range 50, and the focus points P are shown in FIG. The focus point P indicates a position within the shooting range 50 where focusing is possible, and is also called a focus detection area, focus detection position, or distance measurement point. The number of focus points P and their positions in the photographing range 50 shown in the figure are merely an example, and are not limited to the embodiment shown in FIG.

[0039] The focus detection unit 230a uses a signal from the AF signal processing unit 270b to calculate the defocus amount (amount of focus deviation), which is the amount of deviation between the position of the subject image formed by the light beams that have passed through the interchangeable lens 3 and the position of the imaging surface 260S of the image sensor 260 in the camera body 2. The amount of image deviation (phase difference) between the subject image formed by a pair of light beams that pass through different areas of the interchangeable lens 3 is detected for each focus point P, and the defocus amount can be calculated for each focus point P. The body-side control unit 230 then calculates the drive amount of the focus lens 361 based on the defocus amount calculated for a focus point P selected from among the multiple focus points P, and transmits a lens drive signal for the focus lens 361 to the interchangeable lens 3. By moving the focus lens 361 in accordance with the drive signal, the interchangeable lens 3 can align the position of the subject image, which exists in the area corresponding to the focus point P selected from among the multiple focus points P, with the position of the imaging surface 260S of the image sensor 260.

[0040] The body side control unit 230 can automatically select a focus point P to be used for calculating the defocus amount (in other words, for focusing) from among all focus points P, or can select a focus point P specified by a user operation. The method of selecting a focus point P to be used for calculating the defocus amount from among all focus points P is collectively called the "AF area mode."

[0041] "AF area modes" include, for example, "single-point AF," which calculates the defocus amount using only one focus point P selected by user operation, "wide-area AF," which calculates the defocus amount using multiple focus points P in a wider range than single-point AF, and "auto-area AF," in which the camera detects a subject from the range in which all focus points P are located and calculates the defocus amount using focus points P located within the range of that subject. "AF area modes" can be switched by user operation, for example, from an operation menu screen.

[0042] 3 shows a state in which body side control unit 230, which has been switched to "auto-area AF" in "AF area mode," detects a subject (person) that is close to camera system 1 from the range in which all focus points P are located, and selects nine focus points Pa that are located in the range of the person's head. In this case, AF signal processing unit 270b can calculate the defocus amount for each of the nine focus points Pa.

[0043] When the AF signal processing unit 270b calculates the defocus amount for each of the multiple focus points Pa, the body side control unit 230 can select one of the multiple focus points Pa that is, for example, closest (closest) to the camera system 1 or that has the highest contrast of focus detection pixel signals. The body side control unit 230 then sends a drive signal for the focus lens 361 to the interchangeable lens 3 so as to focus on a part of the person that corresponds to the selected focus point.

[0044] When selecting one focus point, the body side control unit 230 drives the focus lens 361 and displays a display (AF frame) indicating one focused focus point Pa superimposed on the live view image displayed on the display unit 290.

[0045] Note that the body side control unit 230 can also perform AF operation to focus on the entire head (include it in the depth of field) by taking into account the defocus amount of each of the nine focus points Pa located on the head. For example, the body side control unit 230 can calculate the average value of the nine defocus amounts and drive the focus lens 361 (AF operation) using the calculated average defocus amount, or it can perform AF operation by calculating a weighted average so that the nine defocus amounts are included in the depth of field as much as possible.

[0046] When nine focus points Pa are used, the body side control unit 230 displays, during the AF operation (S60), displays (AF frames) indicating each of the nine focus points Pa that are in focus, superimposed on the live view image displayed on the display unit 290.

[0047] <Face and eye detection> In this embodiment, the camera is provided with a function for detecting the face of a subject from image data and a function for detecting the eyes of the detected face. When the camera is switched to "auto-area AF," it is possible to select one of a first setting that enables both the face and eye detection functions, a second setting that enables the face detection function and disables the eye detection function, and a third setting that disables both the face and eye detection functions. The first, second, and third settings are selected, for example, by a user operation from an operation menu screen. A setting that disables the face detection function and enables the eye detection function may also be provided. Although an example of detecting eyes will be described in this embodiment, it is also possible to detect irises and pupils (pupils) in the same way. Furthermore, the eyes to be detected are not limited to human eyes, and may be the eyes of animals such as pets. Furthermore, the eyes are not limited to eyes, and may be the distinctive features of faces or objects.

[0048] The above explanation of the AF operation of "auto-area AF" is for when setting 3 is selected. Therefore, the AF operation when setting 1 and when setting 2 are selected will be explained below.

[0049] <First setting> In the first setting in which both the face and eye detection functions are enabled, the body side control unit 230 displays a display frame 41 (AF frame) indicating the detected eye superimposed on the live view image displayed on the display unit 290, as exemplified in Fig. 4(a). In this embodiment, a frame surrounding the outline of the eye including the white of the eye is exemplified as the display frame 41 indicating the detected eye, but a frame surrounding the iris or pupil (pupil) excluding the white of the eye may also be used. When enlarging and displaying a live view image on the display unit 290, if an eye is detected, the body side control unit 230 performs enlarging the image around the position of the detected eye.

[0050] When the body-side control unit 230 detects both the left and right eyes, it can select, for example, the larger eye based on the sizes of the left and right eyes. FIG. 4(a) shows an example in which a display frame 41 is displayed around the right eye that is determined to be larger. The AF signal processing unit 270b can calculate the defocus amount for each focus point P corresponding to the display frame 41. The body-side control unit 230 can select, from the multiple focus points P, for example, the focus point that is closest (closest) to the camera system 1 or that has the highest contrast. Then, the body-side control unit 230 performs an AF operation to focus on the portion of the display frame 41 that corresponds to the selected focus point P.

[0051] In addition, the body side control unit 230 can also perform AF operation to focus on the entire eye shown in the display frame 41 (including it in the depth of field) by taking into account the defocus amount of each of the multiple focus points P corresponding to the display frame 41. Furthermore, when both the left and right eyes are detected, the body side control unit 230 can also select, for example, one of the predetermined eyes.

[0052] <Second setting> In the second setting, which enables the face detection function and disables the eye detection function, the body-side control unit 230 displays a display frame 42 (AF frame) indicating the detected face superimposed on the live view image displayed on the display unit 290, as illustrated in FIG. 4(b). In this embodiment, the display frame 42 indicating the detected face is illustrated as a frame that surrounds at least a portion of the forehead including the eyebrows, at least a portion of each cheek, and at least a portion below the lips. However, the size of the frame may be changed as appropriate as long as the frame does not include background other than the face. For example, it is preferable to set the frame size to 0.8 times the length of the detected face outline in the X-axis direction (horizontal direction) and 0.8 times the length of the detected face outline in the Y-axis direction (vertical direction), as this makes it less likely that background other than the face will be included within the frame.

[0053] The AF signal processing unit 270b can calculate the defocus amount for each focus point P corresponding to the display frame 42. The body side control unit 230 can select one of the multiple focus points P, for example, the focus point P that is closest (closest) to the camera system 1 or the focus point P with the highest contrast of the focus detection pixel signal. Then, the body side control unit 230 performs AF operation to focus on the part of the display frame 42 that corresponds to the selected focus point P.

[0054] In addition, the body side control unit 230 can also perform AF operation to focus on the entire face shown in the display frame 42 (including it in the depth of field) by taking into account the defocus amount of each of the multiple focus points P corresponding to the display frame 42.

[0055] Furthermore, if a face is detected in the first setting but no eyes are detected in that face, the body side control unit 230, as in the case of the second setting, displays a display frame 42 indicating the detected face superimposed on the live view image displayed on the display unit 290 (FIG. 4(b)). As in the case of the second setting, the AF signal processing unit 270b calculates the defocus amount based on the focus point P corresponding to the display frame 42.

[0056] In the first setting, a face is first detected, and then the eyes of the detected face are detected. If the display frame 42 were displayed when the face was detected, the display frame 41 would be displayed instead of the display frame 42 that was currently being displayed when the eyes were detected, which is expected to impair the user's usability. Therefore, in this embodiment, when a face is detected for the first time, it is determined whether to display the display frame 41 or the display frame 42 depending on whether eyes have been detected in the detected face. With this configuration, if eyes are detected in the detected face, the display frame 41 can be displayed (FIG. 4(a)), and if eyes are not detected in the detected face, the display frame 42 can be displayed (FIG. 4(b)).

[0057] <If multiple eyes or faces are detected> 5 is a diagram illustrating a case where multiple eyes or faces are detected. In the first and second settings, it is assumed that the user may wish to focus on an eye or face other than the eye or face selected by the body-side control unit 230. Therefore, the body-side control unit 230 displays indicators 52 and 53 (triangle marks) superimposed on the live-view image displayed on the display unit 290 to inform the user that an eye or face other than the eye or face corresponding to the display frame 51 (AF frame) displayed on the live-view image has been detected.

[0058] For example, in the case of the first setting, the body side control unit 230 detects three faces 55, 56, and 57 included in the live view image illustrated in FIG. 5, selects the face 55 closest to the center of the screen, detects the left and right eyes of the face 55, selects the eye on the left side (hereinafter referred to as the left eye) that it determines to be larger, and displays a display frame 51 indicating the left eye on the display unit 290.

[0059] When the body-side control unit 230 selects the left eye, it records information indicating that the body-side control unit 230 has selected the left eye in the storage unit 235. The reason for this is that, for example, when a face cannot be detected temporarily in a subsequent live-view image and the eye of a face detected in the subsequent live-view image needs to be selected, the left eye will be selected based on the information (in this example, information indicating that the left eye has been selected) recorded in the storage unit 235. By selecting one of the left or right eyes using the information recorded in the storage unit 235, it is possible to avoid a situation in which the selected eye is switched each time a face is re-detected. The information recorded in the storage unit 235 indicating the eye selected by the body side control unit 230 is held until the main switch is turned off or the "AF area mode" is switched.

[0060] The body-side control unit 230 further displays a display 53 (a display indicating a direction, such as <, hereinafter referred to as a triangle mark) indicating that an eye 54 on the right side (hereinafter referred to as the right eye) has been detected, on the right side of the display frame 51. The display of the triangle mark 53 on the right side of the display frame 51 lets the user know that the right eye 54 can be selected.

[0061] The body-side control unit 230 further displays, on the left side of the display frame 51, an indication 52 (triangle mark) indicating that a face 56 on the left side of the face 55 has been detected. The user can see that the face 56 on the left side can be selected because the triangle mark 52 is displayed on the left side of the display frame 51. In addition, the frame surrounding the right eye 54, the frame surrounding the face 55, the frame surrounding the face 56, and the frame surrounding the face 57 shown by the dashed line in Figure 5 are added for explanation purposes and do not need to be displayed on the display unit 290. Display frames may be displayed around all of the eyes detected from the three faces 55, 56, and 57 included in the live view image shown in Fig. 5, and the form of the display frame for the eye selected by the body-side control unit 230 may be changed from the form of the other display frames. For example, the size, color, or shape of the display frame for the selected eye may be changed. The display 52 is not limited to the < symbol, but may be a triangle, an arrow, or the like.

[0062] <User-selected eyes> 6 is a diagram illustrating an example of the exterior surface on the rear side of the camera body 2 when the camera system 1 is held in a horizontal position. Two of the switches constituting the operation member 280, operation switches 281 and 282, are arranged parallel to a line H indicating the X-axis direction (horizontal direction). For example, by operating operation switches 281 and 282, the user can change the eye or face that is to be focused on horizontally.

[0063] 7 is a diagram illustrating a case where the eye 54 located on the right side of the display frame 51 of the face 55 in FIG. 5 is selected. When the user operates the operation switch 282, the body side control unit 230 selects the eye on the right side of the face 55 and displays a display frame 71 (AF frame) indicating the right eye on the display unit 290. The AF signal processing unit 270b calculates the amount of defocus based on the focus point P corresponding to the display frame 71.

[0064] The body-side control unit 230 further displays a display 73 (triangle mark) on the right side of the display frame 71, indicating that a face 57 on the right side of the face 55 has been detected. The user can see that the face 57 on the right side can be selected because the triangle mark 73 is displayed on the right side of the display frame 71.

[0065] The body-side control unit 230 further displays, on the left side of the display frame 71, a display 72 (triangle mark) indicating that the left eye 74 of the face 55 has been detected. The user can see that the left eye 74 can be selected because the triangle mark 72 is displayed on the left side of the display frame 71. In addition, the frame surrounding the left eye 74, the frame surrounding the face 55, the frame surrounding the face 56, and the frame surrounding the face 57 shown by the dashed dotted line in Figure 7 are added for explanation purposes and do not need to be displayed on the display unit 290.

[0066] When the right eye, which is different from the left eye for which the display frame 51 ( FIG. 5 ) was displayed before the operation, is selected based on the operation of the operation switch 282, the body-side control unit 230 records information indicating that the right eye has been selected based on the user's operation in the storage unit 235. For example, when a face cannot be detected temporarily in a subsequent live-view image and the eye of a face detected in the subsequent live-view image is to be selected, the right eye is selected based on the information recorded in the storage unit 235 (in this example, information indicating that the right eye has been selected). By selecting one of the left or right eyes using the information recorded in the storage unit 235, it is possible to avoid a situation in which the selected eye is switched each time a face is re-detected. The information recorded in the memory unit 235 indicating the eye selected based on the user's operation is retained until the main switch is turned off or the "AF area mode" is switched.

[0067] 8 is a diagram illustrating a case where the eye of face 57 located to the right of display frame 71 of face 55 in Fig. 7 is selected. When the user operates operation switch 282, the body side control unit 230 selects face 57 on the right side of face 55, detects the left and right eyes of face 57, selects the eye on the left side of face 57 as viewed from the front (hereinafter referred to as the left eye), and displays a display frame 81 (AF frame) indicating the left eye on the display unit 290. The AF signal processing unit 270b calculates the amount of defocus based on focus point P corresponding to display frame 81.

[0068] The body-side control unit 230 further displays a display 83 (triangle mark) indicating that an eye 84 on the right side of the face 57 as viewed from the front (hereinafter referred to as the right eye) has been detected, on the right side of the display frame 81. The display of the triangle mark 83 on the left side of the display frame 81 lets the user know that the right eye 84 can be selected.

[0069] The body-side control unit 230 further displays, on the left side of the display frame 81, a display 82 (triangle mark) indicating that a face 55 on the left side of the face 57 has been detected. The user can see that the face 55 on the left side can be selected because the triangle mark 82 is displayed on the left side of the display frame 81. In addition, the frame surrounding the right eye 84, the frame surrounding the face 55, the frame surrounding the face 56, and the frame surrounding the face 57 shown by the dashed dotted line in Figure 8 are added for explanation purposes and do not need to be displayed on the display unit 290. As described above, when the eye of face 57 different from face 55 for which display frame 71 was displayed before the operation is selected based on the operation of operation switch 282, the left eye closer to face 55 is selected, and display frame 81 showing this left eye is displayed.

[0070] Fig. 9 is a diagram illustrating another example of a case where the eye of face 57 located on the right side of display frame 71 of face 55 in Fig. 7 is selected. When the user touches the display position of face 57 on display unit 290 via touch operation member 290A, body-side control unit 230 selects face 57 on the right side of face 55, detects the left and right eyes of face 57, selects the eye on the right side of face 57 (hereinafter referred to as the right eye), and displays display frame 84A indicating the right eye on display unit 290.

[0071] When the eye of a face 57 different from the face 55 on which the display frame 71 was displayed before the touch operation is selected, the body side control unit 230 selects the right eye based on the information recorded in the memory unit 235 (in this example, information indicating that the right eye has been selected), and displays the display frame 84A showing the selected right eye. The AF signal processor 270b calculates the defocus amount based on the focus point P corresponding to the display frame 84A.

[0072] If the touch operation position is the position of the left eye of the face 57, the body side control unit 230 selects the left eye and displays a display frame indicating the selected left eye regardless of the information recorded in the storage unit 235. The AF signal processing unit 270b calculates the defocus amount based on the focus point P corresponding to the display frame indicating the left eye. In this way, the body side control unit 230 selects the eye if the touch operation position is at the eye position, and selects either the left or right eye based on the information recorded in the memory unit 235 if the touch operation position is not at the eye position.

[0073] The body-side control unit 230 further displays, on the left side of the display frame 84A, an indication 85 (triangle mark) indicating that the left eye 81A of the face 57 has been detected. The user can see that the left eye 81A can be selected because the triangle mark 85 is displayed on the left side of the display frame 84A. In addition, the frame surrounding the left eye 81A, the frame surrounding the face 55, the frame surrounding the face 56, and the frame surrounding the face 57 shown by the dashed dotted line in Figure 9 are added for explanation purposes and do not need to be displayed on the display unit 290.

[0074] The display of the triangular mark described with reference to Figures 5 to 9 may be displayed in different ways, such as in color, shape, text, or icon addition, depending on whether the eye in the direction indicated by the triangular mark can be selected or whether a face can be detected in the direction indicated by the triangular mark but the eye cannot be detected, making it possible to select only the face. Furthermore, the display of the display frame (AF frame) described with reference to Figures 5 to 9 may be displayed in different ways, such as in color, shape, text, or icon addition, depending on whether it is displayed on the eye selected by the body-side control unit 230 or on the eye selected by the user's operation.

[0075] <Relationship between eye or face size and display frame> 10(a) to 10(c) are diagrams illustrating the relationship between the size of the eyes or face in the image and the display frame. In the first setting, if the size of the eyes in the live view image is too small, the accuracy of calculating the defocus amount decreases because the eye information included in the focus point P corresponding to the display frame decreases. Therefore, calculating the defocus amount using information about not only the eyes but also the face is effective in preventing a decrease in the accuracy of calculating the defocus amount.

[0076] FIG. 10(a) is a diagram for explaining the case where the eye size in the live view image is sufficiently large. When the eye size is sufficiently large, the accuracy of calculating the defocus amount can be ensured even by using only the eye information. Therefore, the body side control unit 230 for which the first setting is selected detects the left and right eyes of the face 55, and causes the display unit 290 to display, for example, a display frame 51 of the eye size indicating the left eye. The AF signal processing unit 270b calculates the defocus amount based on the focus point P corresponding to the display frame 51.

[0077] FIG. 10(b) is a diagram for explaining the case where the eye size in the live view image is small. When the body side control unit 230 for which the first setting is selected, for example, if the number of pixels A in the X-axis direction (horizontal direction) of the eye is smaller than the Q1 percentage of the number of pixels Th in the horizontal direction of the live view image and larger than the Q2 percentage (Q2 < Q1), the display unit 290 is caused to display a display frame 51 indicating the left eye of the face 55. At this time, in accordance with the eye in FIG. 10(b) which is smaller in size than the eye in FIG. 10(a), a display frame 51 of a small size is displayed.

[0078] The AF signal processing unit 270b shall calculate the defocus amount based on the focus point P corresponding to the frame indicating the face 55 (the same size as the display frame of the face size) regardless of the size of the display frame 51 indicating the left eye of the face 55. By configuring in this way, by using not only the left eye of the face 55 but also the information of the face 55 indicated by the dashed line, a decrease in the accuracy of calculating the defocus amount is prevented.

[0079] The reason for calculating the defocus amount using the information of the face 55 will be further explained. When the eye size in the live view image is small, the eye information included in the focus point P (FIG. 3) decreases. In addition, it is assumed that the focus point P is not arranged at the position corresponding to the detected eye and the eye information is not included in the focus point P. Therefore, it becomes difficult to calculate the defocus amount based on the focus point P corresponding to the display frame 51 indicating the eye. On the other hand, it is considered that the detected eye size becomes small when the distance from the camera system 1 to the eye (face) is long. Such a distant subject may be considered to have no substantial difference between the focus position on the face and the focus position on the eye, unlike a subject close to the camera system 1. That is, the focus positions of the nose and lips may be regarded as equivalent to the focus position of the eye. Therefore, in the present embodiment, a focus point P with a high contrast of the focus detection pixel signal is selected from a plurality of focus points P corresponding to a frame indicating a face 55 larger than the display frame 51, and the defocus amount is calculated. By calculating in this way, the defocus amount can be appropriately calculated based on a large number of focus points P corresponding to the frame indicating the face 55. Note that the frame surrounding the face 55 is shown for explanatory purposes and does not have to be displayed on the display unit 290.

[0080] FIG. 10(c) is a diagram for explaining a case where the eye size is even smaller in the live view image. When the body side control unit 230 for which the first setting is selected has an eye size in the live view image smaller than that in FIG. 10(b), for example, when the number of pixels A in the X-axis direction (horizontal direction) of the eye is smaller than the Q2 percentage (Q2 < Q1) of the number of pixels Th in the horizontal direction of the live view image, instead of the display frame 51 showing the left eye of the face 55, a display frame 59 of the face size showing the face 55 is displayed on the display unit 290.

[0081] The AF signal processing unit 270b is configured to calculate the defocus amount based on the focus point P corresponding to the display frame 59 showing the face 55. By configuring in this way, it is possible to prevent a decrease in the accuracy of calculating the defocus amount by using the information of the face 55. Further, by displaying the display frame 59 having a size larger than the display frame 51 showing the eye on the display unit 290, it is possible to clearly show the user the subject to be focused. 10(a) to 10(c), if the number of pixels A of the eyes in the X-axis direction (horizontal direction) is greater than a predetermined value that is independent of the number of pixels of the live view image, a display frame 51 indicating the eyes may be displayed, and if the number of pixels A of the eyes in the X-axis direction (horizontal direction) is smaller than a threshold value, a display frame 59 indicating the face 55 may be displayed on the display unit 290. Similarly, in Figures 10(a) to 10(c), if the number of pixels A of the eyes in the X-axis direction (horizontal direction) is larger than a predetermined value that does not depend on the number of pixels in the live view image, the defocus amount may be calculated based on focus point P corresponding to a frame (the same size as the display frame for the face size) showing face 55, and if the number of pixels A of the eyes in the X-axis direction (horizontal direction) is smaller than a predetermined value, the defocus amount may be calculated based on focus point P corresponding to a frame (the same size as the display frame for the eye size) showing the eye. In other words, the determination may be made based on a threshold value that is independent of the number of pixels Th in the horizontal direction of the live view image.

[0082] <An example of frame display operation> 11 is a flowchart illustrating the flow of processing for determining a display frame (AF frame) executed by the body side control unit 230 when the first setting is selected. By executing the processing according to the flowchart in FIG. 11, the body side control unit 230 determines the focus point P used in calculating the defocus amount described above.

[0083] In step S310, the body side control unit 230 determines whether a face has been detected based on the live view image. If a face has been detected, the body side control unit 230 makes an affirmative decision in step S310 and proceeds to step S320, but if a face has not been detected, the body side control unit 230 makes a negative decision in step S310 and proceeds to step S360.

[0084] In step S320, the body side control unit 230 determines whether eyes are detected in the face detected in S310 based on the live view image. If the body side control unit 230 detects eyes, it makes an affirmative determination in step S320 and proceeds to step S330. If it does not detect eyes, it makes a negative determination in step S320 and proceeds to step S350.

[0085] In step S330, the body side control unit 230 determines whether the size of the eyes is larger than a predetermined first threshold value. For example, if the number of pixels A in the X-axis direction (horizontal direction) of the eyes is larger than Q2 percent of the number of pixels Th in the horizontal direction of the live view image, the body side control unit 230 makes an affirmative determination in step S330 and proceeds to step S340. If the number of pixels A is less than or equal to Q2 percent of the number of pixels Th, the body side control unit 230 makes a negative determination in step S330 and proceeds to step S350.

[0086] In step S340, the body side control unit 230 causes the display unit 290 to display a display frame 51 for the size of the detected eyes based on the detected eye information. Specifically, as illustrated in FIGS. 10(a) and (b), a display frame 51 indicating the detected eyes is displayed at the detected eye position, and the process proceeds to step S370.

[0087] In step S370, the body side control unit 230 determines whether the size of the eyes is larger than a predetermined second threshold value. For example, if the number of pixels A in the X-axis direction (horizontal direction) of the eyes is larger than Q1 (Q2 < Q1) percent of the number of pixels Th in the horizontal direction of the live view image, the body side control unit 230 makes an affirmative determination in step S370 and proceeds to step S380. If the number of pixels A is less than or equal to Q1 percent of the number of pixels Th, the body side control unit 230 makes a negative determination in step S370 and proceeds to step S390. When the size of the eyes is small, there is a high possibility that the eyes are erroneously detected. In the present embodiment, the reliability of the detected eyes is evaluated by comparing the size of the eyes with the second threshold value.

[0088] In step S380, the body side control unit 230 causes the AF signal processing unit 270b to perform AF calculation based on the eye-sized display frame 51, and ends the processing in Fig. 11. The AF signal processing unit 270b calculates the defocus amount based on the focus point P corresponding to the display frame 51 (Fig. 10(a)).

[0089] In step S350, which is reached after a negative decision is made in step S320 or step S330, the body side control unit 230 causes a display frame 59 of the size of the face based on the information about the detected face to be displayed on the display unit 290. Specifically, as shown in Fig. 10(c) as an example, a display frame 59 showing a face 55 is displayed at the position of the detected face, and the process proceeds to step S390.

[0090] In step S390, the body side control unit 230 causes the AF signal processing unit 270b to perform AF calculation based on the face-size display frame 59, and ends the processing in Fig. 11. If the process proceeds to S390 after S350, the AF signal processing unit 270b calculates the defocus amount based on the focus point P corresponding to the display frame 59 (Fig. 10(c)). Also, if a negative decision is made in S370 and the process proceeds to S390, the AF signal processing unit 270b calculates the defocus amount based on the focus point P corresponding to the frame showing the face 55 (face-size display frame), regardless of the size of the display frame 51 displayed in Fig. 10(b).

[0091] If a negative decision is made in step S310, face detection and eye detection are not performed. In step S360, the body side control unit 230 does not display display frames at the face or eye positions, but displays display frames similar to those when the third setting is selected. That is, a subject closest to the camera system 1 is detected from the range in which all focus points P are located, and a display (AF frame) indicating multiple focus points P located within the range of that subject is displayed on the display unit 290. Then, in step S400, the body side control unit 230 causes the AF signal processing unit 270b to perform a predetermined AF calculation, and ends the processing in Fig. 11. The AF signal processing unit 270b calculates the defocus amount for each of the multiple focus points P located within the range of the subject, in the same way as when the third setting is selected.

[0092] The first threshold used in the determination in step S330 may be set to a different value when the eye size is increasing and when the eye size is decreasing. By setting a different value, it is possible to prevent a decrease in usability for the user due to frequent switching of the display size of the display frame (AF frame) near the first threshold. Furthermore, step S330 may be omitted from the flowchart of Fig. 11. That is, when an eye is detected in step S320, display frame 51 of the eye size may be displayed on display unit 290 based on the information of the detected eye, regardless of the size of the eye.

[0093] <User eye selection in portrait orientation> 12 is a diagram illustrating an example of the exterior surface on the rear side of camera body 2 when camera system 1 is held in a vertical position. Two of the switches constituting operation member 280, operation switches 283 and 284, are arranged approximately parallel to line H indicating the X-axis direction (horizontal direction). For example, by operating operation switches 283 and 284, the user can change the eye or face that is to be focused on horizontally.

[0094] When the camera system 1 is held in the vertical position, the two operation switches 281 and 282, which are arranged parallel to the line H when the camera system 1 is held in the horizontal position (FIG. 6), are not suitable for use in changing the focus target horizontally. Instead of the operation switches 281 and 282, the body-side control unit 230 uses the two operation switches 283 and 284 to change the focus target horizontally, in other words, to select the left or right eye.

[0095] 12, the line H' indicates the longitudinal direction of the exterior surface of the camera body 2. The body-side control unit 230 switches two switches used to select the left or right eye based on the detection signal of the orientation sensor 250. When the camera system 1 held in the horizontal position is changed to the vertical position, if the angle θ between the line H indicating the X-axis direction (horizontal direction) and the line H' exceeds, for example, 75 degrees, the body-side control unit 230 uses the two operation switches 283 and 284 to select the left or right eye.

[0096] Conversely, when the camera system 1 held in the portrait position is changed to the landscape position, if the angle θ becomes smaller than 65 degrees, the body side control unit 230 uses the two operation switches 281 and 282 to select the left or right eye.

[0097] In this way, a difference is set in the threshold value of the angle θ for determining the switching of the two switches used to select the left or right eye when the camera system 1 held in a horizontal position is changed to a vertical position and when the camera system 1 held in a vertical position is changed to a horizontal position, thereby preventing a decrease in the user experience due to frequent switching of the two switches near the threshold value. 12 and 6 may be displayed on the display unit 290, and the user may select the left or right eye by touching the operation icon. For example, if the camera system 1 is configured in a housing like a smartphone, the user can select the left or right eye simply by moving their finger within a predetermined range, which improves usability. 12 and 6, if the camera body 2 has an electronic viewfinder (EVF), the content displayed on the display unit 290 (live view image, display frame, etc.) may be displayed on the EVF. In this case, the user selects the left or right eye by operating the operation switches 281-284 while still observing the EVF. Since the left or right eye can be selected by operating the operation switches 281-284 without taking their eyes off the EVF, usability for the user can be improved.

[0098] <Select the left and right eyes based on the position of your face> 7 to 9 referred to in the above description for selecting the left and right eyes based on the user's operation of operation switch 282 are examples in which a plurality of faces are lined up in the X-axis direction (horizontal direction), so when operation switch 282 is operated with display frame 71 displayed over the right eye of face 55 in Fig. 7, there is no problem in displaying display frame 81 over the left eye of face 57 to the right of face 55 in Fig. 8. In other words, by selecting the right eyes in turn without determining which face's eyes they belong to, the right eye is selected after the left eye of face 55, and the right eye is selected after the left eye of face 57, which is the next to face 55 on the right side.

[0099] However, when multiple faces are lined up in the Y-axis direction (vertical direction) as shown in FIG. 13, the following problem occurs. FIG. 13 is a diagram illustrating a state in which multiple faces are lined up in the Y-axis direction. In the example of FIG. 13, the eye on the left side of face 131 (hereinafter referred to as the left eye) is selected followed by the eye on the left side of face 132 (hereinafter referred to as the left eye), and the eye on the right side of face 131 (hereinafter referred to as the right eye) is selected followed by the left eye of face 132. For this reason, in this embodiment, the eye positions are rearranged in the X-axis direction based on the center position of the faces so that the left and right eyes of the same face are selected in order and then the left and right eyes of another face are selected in order. Specifically, if the center position of face 131 in the X-axis direction (horizontal direction) is C1 and the center position of face 132 in the X-axis direction (horizontal direction) is C2, C2 is located to the right of C1.

[0100] With the display frame 133 displayed around the left eye of the face 131 (FIG. 13), each time the operation switch 282 is operated, the body-side control unit 230 selects the right eye of the face 131, then the left eye of the face 132 after the right eye of the face 131, and then the right eye of the face 132 after the left eye of the face 132. This configuration makes it possible to successively select the left and right eyes of the same face, improving usability for the user.

[0101] <Vertical position> The same applies when the camera system 1 is held in a vertical position, as shown in Fig. 12. In the vertical position, the eye positions are rearranged in the Y-axis direction based on the center position of the face, so that the left and right eyes of the same face are selected in order, and then the left and right eyes of another face are selected in order. Although not shown in the figure, the center position of one face in the Y-axis direction is C3, and the center position of another face in the Y-axis direction is C4. C4 is located to the right of C3.

[0102] With a display frame (AF frame) displayed on the left eye of a certain face, each time the operation switch 284 is operated, the body-side control unit 230 selects the right eye of that face, then the left eye of another face after the right eye of that face, and then the right eye of another face after the left eye of that face. This configuration makes it possible to select the left and right eyes of the same face in succession, improving usability for the user.

[0103] According to the first embodiment described above, the following advantageous effects can be obtained. (1) A camera (camera system 1) as an example of an electronic device includes a body-side control unit 230 that detects faces and eyes from an image, a body-side control unit 230 that selects one eye from the eyes detected by the body-side control unit 230, and a display unit 290 that displays, superimposed on a live-view image, a display frame (AF frame) that indicates the position of the eye selected by the body-side control unit 230, and a triangular mark that indicates that at least one of a face and an eye different from the selected eye has been detected by the body-side control unit 230. This configuration allows the user to visually know which eye has been selected from the faces and eyes detected in the live-view image.

[0104] (2) The display of the above-mentioned triangle mark indicates the direction from the display frame (AF frame) of the position where a face or eye other than the selected eye is detected, so the user can be visually informed of the direction from the display frame (AF frame) of the face or eye other than the selected eye.

[0105] (3) The display unit 290 changes the manner in which the triangle mark is displayed depending on whether the triangle mark is for a face position other than the selected eye or for an eye position other than the selected eye, so that the user can be visually informed whether an eye other than the selected eye has been detected or whether a face other than the selected eye has been detected.

[0106] (4) When multiple eyes are detected on a face, the body-side control unit 230 selects one eye based on the size of the detected eye. Therefore, for example, by focusing on the selected eye, it is possible to obtain the effect of making it easier to focus than when selecting a smaller eye.

[0107] (5) The body-side control unit 230 selects an eye based on an operation by the user, so it is possible to meet the user's request to select an eye other than the eye selected by the body-side control unit 230.

[0108] (6) The image is generated by capturing a subject image formed by the imaging optical system 360. The camera stores information in the storage unit 235 as to whether the eye selected by a user's operation is the right eye or the left eye relative to the face having the eye. The body-side control unit 230 then selects an eye based on the information stored in the storage unit 235 in an image obtained by capturing a frame different from the image for which the information was stored. This configuration makes it possible to avoid a situation in which the selected eye is switched each time a face is re-detected, for example, when a face cannot be detected in a live-view image at one time and an eye is to be selected from a face detected again in a subsequent live-view image.

[0109] (7) The camera includes operation switches 281-284 that the user operates to select information about the direction indicated by the displayed triangle. If the selected eye is located in the first direction determined based on the information about the direction selected by one of the operation switches 281-284, the body-side control unit 230 selects a new eye from the same face as the selected eye, and if the selected eye is not located in the same face as the selected eye, the body-side control unit 230 selects a new face that is closest in the first direction. This configuration allows the user to appropriately select the eye or face located in the first direction they desire.

[0110] (8) The camera includes a touch operation member 290A that is provided on the display surface of the display unit 290 and receives an operation for selecting a portion of the image displayed at the position on the display surface by the user touching the display surface. When two eyes are detected for a face that is part of the displayed image selected via the touch operation member 290A, the body-side control unit 230 selects one of the eyes based on information stored in the storage unit 235. This configuration allows the user to appropriately select the desired eye from the face selected by touch operation.

[0111] (9) The memory unit 235 retains information stored in the memory unit 235 until at least one of the following operations is performed: a power-off operation; and, for example, switching the “AF area mode” to disable face or eye detection by the body-side control unit 230. Therefore, the information recorded in the memory unit 235 (for example, information indicating that the left eye has been selected) can be used appropriately.

[0112] (10) The body side control unit 230 causes the AF signal processing unit 270b to calculate the defocus amount at the focus point P that corresponds to the displayed display frame. This configuration makes it possible to visually inform the user of which part of the image to focus on.

[0113] (11) The body-side control unit 230 determines whether to display an eye-size display frame (AF frame) at the position of the detected eye or a face-size display frame (AF frame) at the position of the detected face, depending on the size of the detected eye. This configuration makes it possible to display a display frame (AF frame) of an appropriate size in the image.

[0114] (12) The body side control unit 230 determines whether to perform AF calculation based on the eye size display frame (S380) or the face size display frame (S390) depending on the detected eye size. With this configuration, when the eye is small, AF calculation is performed using not only eye information but also face information, thereby preventing a decrease in the accuracy of calculation of the defocus amount.

[0115] (Second embodiment) The display frame (AF frame) when the first setting is selected may be displayed in a manner different from that of the first embodiment. Fig. 14 is a flowchart illustrating the flow of processing executed by the body side control unit 230 to determine the display frame when the first setting is selected. The body side control unit 230 determines the focus point P used to calculate the defocus amount described above by executing the processing according to the flowchart in Fig. 14 instead of the processing according to the flowchart in Fig. 11.

[0116] Figure 14 differs from Figure 11 in that steps S330 and S370 of Figure 11 are omitted, and instead steps S345 and S370A of Figure 14 are added, so these differences will be mainly described.

[0117] In step S345, to which the process proceeds after making a positive decision in step S340, the body side control unit 230 calculates the distance R from the position of the detected face to the position of the eye detected in that face. Fig. 15 is a schematic diagram for explaining the calculation of the distance R. In Fig. 15, the position of the detected face 161 is, for example, the center position CA of the range of the detected face 161, and the position of the detected eye 162 is, for example, the center position CB of the range of the detected eye 162. The distance R is the distance between point CA and point CB in Fig. 15.

[0118] In step S370A, which follows step S345, the body side control unit 230 determines whether the distance R is greater than a predetermined threshold. For example, if the distance R is greater than R'×k1, which is the length R' of the eye in the X-axis direction (horizontal direction) multiplied by a coefficient k1, the body side control unit 230 makes a positive decision in step S370A and proceeds to step S380. If the distance R is equal to or less than the threshold R'×k1, the body side control unit 230 makes a negative decision in step S370A and proceeds to step S390. If the distance R is short, there is a high possibility that the eye has been erroneously detected. In this embodiment, the distance R is calculated to evaluate the reliability of the detected eye.

[0119] In the above description, the distance R is the distance from the center position CA of the detected face 161 to the center position CB of the eye 162 detected in that face 161, but it may also be the distance R' (Figure 15) between the center positions of the left and right eyes detected in the detected face 161.

[0120] Furthermore, instead of comparing the distance R (or distance R') described above with the product of the length R' of the eye in the X-axis direction (horizontal direction) and the coefficient k1, the ratio of the distance R (or distance R') to the horizontal direction of the live view image may be compared with a predetermined threshold value X1. Alternatively, the number A of pixels in the X-axis direction of the detected eye may be compared with a predetermined threshold value to make a determination based on the absolute value.

[0121] According to the second embodiment described above, the following advantageous effects can be obtained. The body side control unit 230 calculates the distance R from the center position CA of the face 161 detected in the image to the center position CB of the eye 162 detected in that face 161. Then, depending on the distance R, it determines whether to perform AF calculation based on a display frame (AF frame) of the eye size (S380) or to perform AF calculation based on a display frame of the face size (S390). With this configuration, as in the first embodiment, by performing AF calculation using information not only about the eye but also about the face, it is possible to prevent a decrease in the accuracy of calculation of the defocus amount.

[0122] The following modifications are also within the scope of the present invention, and one or more of the modifications may be combined with the above-described embodiment.

[0123] (Variation 1) In the above description, an example has been described in which the body-side control unit 230 detects the left and right eyes and selects the larger eye based on the size of the eye. If orientation information indicating whether the face is facing right or left with respect to the camera system 1 is available, the body-side control unit 230 may select the eye closer to the camera system 1 based on the face orientation information.

[0124] 16(a) and 16(b) are diagrams showing an example in which the face is facing rightward with respect to the camera system 1. When the eye on the left side (hereinafter referred to as the left eye) is larger than the eye on the right side (hereinafter referred to as the right eye) (FIG. 16(a)), or when the left eye is smaller than the right eye (FIG. 16(b)), the body-side control unit 230 selects one of the left eyes that is closer to the camera system 1 based on the face orientation information, and displays a display frame 51 around the selected left eye.

[0125] Generally, the difference in size between the detected left and right eyes is small, so when trying to select one of the larger eyes, the selected eye may frequently switch between the left and right. By referring to the face direction information, it is possible to prevent a decrease in the user experience due to frequent switching of the selected eye.

[0126] 16(c) is a diagram showing an example in which the face is facing left with respect to the camera system. When the face direction information changes, the body side control unit 230 reselects one of the eyes on the right side that is closer to the camera system 1 based on the new direction information, and displays a display frame 71 around the selected left eye (FIG. 16(c)).

[0127] (Variation 2) As described above, when detecting faces or eyes from live view images, the search range for faces (eyes) may be changed between single shooting and continuous shooting. Single shooting is a shooting mode in which one frame is captured when the release button is fully pressed. Continuous shooting is a shooting mode in which multiple single frames are captured continuously while the release button is fully pressed.

[0128] In general, when performing continuous shooting, it is desirable to start capturing the next frame as soon as possible. On the other hand, when searching for a face (eye) in a live view image, searching for the face (eye) in a narrower range rather than searching for the face (eye) in a wider range shortens the processing time. Therefore, the body side control unit 230 sets the range for detecting faces (eyes) during continuous shooting to, for example, 70% of the range of the live view image. By narrowing the range and shortening the processing time for searching for faces (eyes), it is possible to start capturing the next frame earlier and improve the continuous shooting speed.

[0129] For example, the body side control unit 230 can shorten the processing time required to search for the face (eye) while appropriately tracking the face (eye) detected in the previous frame by setting a range of 70% of the range of the live view image centered on the position of the face (eye) detected in the previous frame. Furthermore, when setting a search range for searching for the eye centered on the position of the eye detected in the previous frame, the range may be set to 50% of the range of the live view image.

[0130] The body side control unit 230 detects faces (eyes) over 100% of the live view image range during single shots, in order to appropriately detect faces (eyes) using a wide range of the live view image.

[0131] (Variation 3) The body side control unit 230 can include information acquired in the AF operation (S60) in the file recorded in the recording process (S110). For example, information indicating the size and position of the detected face, information indicating the size and position of the detected left and right eyes, information indicating the reliability of the detection, information indicating whether the eye for which a display frame has been set has been selected by the judgment of the body side control unit 230 or based on a user operation, information indicating whether the face for which a display frame has been set has been selected by the judgment of the body side control unit 230 or based on a user operation, etc. are recorded in the file together with the data of the image to be recorded.

[0132] (Variation 4) In the above-described embodiment, a phase difference detection method is used as the focus detection method, but a contrast method may also be used. When the contrast method is used, focus detection is performed by calculating an evaluation value that evaluates the contrast of high-frequency components of the image based on imaging pixel signals output from pixels arranged at positions corresponding to the display frame on imaging surface 260S, or imaging pixel signals output from pixels arranged at positions corresponding to the inside and periphery of the display frame. Alternatively, focus detection using the contrast method may be performed when the size of the detected eye is smaller than a predetermined value, and focus detection using the phase difference detection method may be performed when the size of the detected eye is larger than the predetermined value. In this way, when the size of the detected eye is small or when the focus point P is not located at a position corresponding to the detected eye, focus detection using the contrast method can be performed appropriately.

[0133] (Variation 5) In the above-described embodiment, a face is first detected, and then the eyes of the detected face are detected. However, if a face cannot be detected, the shape of the head may be detected, and then the eyes may be detected. Specifically, the face, eyes, and head shapes are detected from multiple images generated consecutively, and their respective positions and sizes are stored for each image. Furthermore, relative information, which is the relative positions and relative sizes of the face, eyes, and head shapes, is stored. If a face cannot be detected, the head shape is first detected, and the position and size of the eyes are detected using the position and size of the detected head shape and the stored relative information. If a head shape cannot be detected in one image, tracking detection may be performed using the position and size of the head shape in a previous image. Furthermore, if a face and head shape cannot be detected, an image pattern including the color when the face was detected may be stored as a feature, and similar areas in the image may be searched for using the feature and the stored relative information to detect the position and size of the eyes. Tracking detection may also be performed using the position and size of similar areas in the image based on the feature of the previous image. This makes it possible to prevent the eye display frame from being displayed differently between an image in which the eyes are detected and an image in which the eyes are not detected by displaying a display frame at a position where the eyes are presumed to be even if the face cannot be detected. In other words, it is possible to prevent the eye frame from being displayed differently between frames of continuously displayed through images.

[0134] (Variation 6) The position, size, and angle of the face detected in the latest frame (an image generated by capturing images at a predetermined time interval is called a frame) can be averaged with the position, size, and angle of the face detected in previous frames (weighted average, with a lower weight for older frames, etc.), and eyes can be detected based on the averaged position, size, and angle of the face. This prevents the selected eye (left or right) from changing due to changes in the position or direction of the face to be selected, which may occur when the position, size, and angle of the detected face vary between frames due to shaking of the imaging device. In addition, if the face or eyes of a person different from the face or eyes detected in the latest frame are selected, if the face or eyes are detected after a predetermined time has passed since the face or eyes were detected in the previous frame, or if the position of the detected face has changed significantly, the position, size, and angle of the face detected in the latest frame may not be averaged with the position, size, and angle of the face detected previously, but may be averaged with frames taken from the new frame onwards. In the above explanation, when a face or eye is detected after a predetermined time has elapsed since the previous face or eye detection, this means that after the face or eye is detected, the face or eye cannot be detected for a predetermined number of frames, and then the face or eye is detected again after the predetermined number of frames.

[0135] (Variation 7) The size and position of the eye detected in the current frame can be averaged with the size and position of the eye detected in previous frames (weighted average, lower weight for older frames, etc.), and a display frame can be displayed. This makes it possible to prevent the size of the display frame from changing or the position from shifting between frames. In addition, if the face or eyes of a person different from the face or eyes currently being detected are selected, if the eyes different from the eyes currently being detected are selected, if the eyes are detected after a predetermined time has passed since the last eye was detected, or if the position of the detected eye has shifted significantly, the position and size of the eye detected in the current frame may not be averaged with the position and size of the eye detected previously, but may be averaged over frames taken from the new frame onwards.

[0136] (Variation 8) Whether to display a frame around the face or the eyes may be changed depending on whether the size of the detected face is larger or smaller than a reference value. The reference value may also be changed when the frame is changed from displaying a frame around the face to displaying a frame around the eyes, or vice versa. For example, if the face size in the captured image data is 50 pixels or more, a frame is displayed around the eyes, and if the face size in the captured image data is smaller than 50 pixels, a frame is displayed around the face. If a display frame was displayed around the face in the previous frame, when the size of the face in the captured image data becomes 55 pixels or more, the display frame is changed from the face display frame to the eye display frame. Conversely, if a display frame was displayed around the eyes in the previous frame, when the size of the face in the captured image data becomes 45 pixels or less, the display frame is changed from the eye display frame to the face display frame. In other words, the face size criteria for changing from the face display frame to the eye display frame may be different from the face size criteria for changing from the eye display frame to the face display frame. This makes it possible to prevent frequent changes in the display of the face frame and the display of the eye frame between images displayed consecutively over time when the size of the eyes has changed somewhat.

[0137] (Variation 9) In the above-described embodiment, an example of detecting a face or eyes and detecting the focus state of the detected face or eyes has been described, but this is not limiting. For example, a configuration may be adopted in which a portion of an image selected by a user from a captured image is registered as a feature, and the registered feature is detected from the image data to detect the focus state. Alternatively, the camera may be configured to set detectable objects other than eyes, such as lips, a soccer ball, food, or a ring. Furthermore, if the camera has shooting modes corresponding to multiple scenes, the object for which the focus state is detected may be changed depending on the shooting mode. Specifically, when the camera's shooting mode is in portrait mode, eyes may be detected and the focus state of the detected eyes may be detected, and when the camera is in sports mode, a ball may be detected and the focus state of the detected ball may be detected. In the case of food, a birthday cake plate, candles, or main dish may be detected.

[0138] The present invention is not limited to the above-described contents, and other embodiments conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention.

[0139] The disclosures of the following priority applications are incorporated herein by reference: Japanese Patent Application No. 2018-248622 (filed December 28, 2018) [Explanation of symbols]

[0140] 1... camera system, 2... camera body, 3... interchangeable lens, 230... body side control unit, 235... memory unit, 40, 41, 51, 59, 71, 81, 84A, 133... display frame, 52, 53, 72, 73, 82, 83, 85... triangle marks, 270... signal processing unit, 280, 281-284... operation members, 290... display unit, 290A... touch operation member, 360... photographing optical system, 370... lens driving unit

Claims

[Claim 1] a detection unit for detecting eyes from an image; a selection unit that selects one eye from the eyes detected by the detection unit; an operation unit that receives an operation to cause a selection unit to select an eye different from the selected eye; a display unit that displays a display indicating that the eye is the eye selected by the selection unit, superimposed on the image; An electronic device comprising:

Images