Electronic device, and program

Abstract

To provide an electronic apparatus that improves focusing accuracy.SOLUTION: A camera system 1 includes a display unit 290 that displays an image, an eye / face detection unit 230b that detects eyes and a face from an image, and a display control unit 230c that controls the display on the display unit, and the display control unit displays the first display so as to be superimposed on the first eye included in the image, and displays a second display having a different shape from the first display so as to be superimposed on the image when the image includes at least one of a second eye different from the first eye and a second face different from a first face having the first eye.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an electronic device , and a program and pertains thereto.

Background Art

[0002] There is known a camera that detects a face of a subject, further detects eyes of the face, and controls focusing on the eyes (see Patent Document 1). Conventionally, improvement in focusing accuracy has been demanded.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] According to a first aspect of the present invention, an electronic device includes a detection unit that detects eyes from an image, a selection unit that selects one eye from the eyes detected by the detection unit, an operation unit that receives an operation of a user for selecting an eye different from the eye selected by the selection unit, and a display unit that displays the image. The operation unit is provided at a position different from the display unit. The display unit performs a display indicating a first direction and a second direction in the image that can receive an instruction by the operation unit together with the image. The operation unit can receive a first operation for instructing the first direction and a second operation different from the first operation for instructing the second direction. According to a second aspect of the present invention, an electronic device includes a display unit that displays an image, a detection unit that detects eyes from the image, a selection unit that selects one eye from the eyes detected by the detection unit, and an operation unit that receives an operation of a user for selecting an eye different from the eye selected by the selection unit. The operation unit receives an operation for instructing a direction in the image, and the operation unit is provided at a position different from the display unit. When the selection unit receives an operation by the operation unit after selecting one eye from the eyes detected by the detection unit as a first candidate, based on the received operation, the selection unit selects an eye different from the first candidate as a second candidate. When a plurality of eyes are detected in the direction received by the operation unit, the selection unit selects another eye included in the same face as the first candidate eye as the second candidate. According to a third aspect of the present invention, a program detects eyes from an image, selects one eye from the detected eyes, and, together with the image, performs a display indicating a first direction and a second direction in the image that can receive an instruction by a user's operation. On the display unit The program performs a first operation for instructing the first direction and a second operation for instructing the second direction, which is different from the first operation, as operations of a user for selecting an eye in the image. , via an operation unit provided at a position different from the display unit This is executed by a computer that receives the operations. According to a fourth aspect of the present invention, a program detects eyes from an image, selects one eye from the detected eyes as a first candidate, Displayed on the display unit receives a user's operation for instructing a direction in the image as an operation for selecting an eye different from the selected eye, , via an operation unit provided at a position different from the display unit and when the program receives a user's operation after selecting the first candidate, based on the received operation, the program selects an eye different from the first candidate as a second candidate. When a plurality of eyes are detected in the received direction, the program selects another eye included in the same face as the first candidate eye as the second candidate. This is executed by a computer.

Brief Description of the Drawings

[0005]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Best Mode for Carrying Out the Invention

[0006] Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings. (First Embodiment) FIG. 1 is a schematic diagram for explaining a main configuration of a camera system 1 according to a first embodiment of the invention. The camera system 1 will be described by taking an example in which a photographing lens (referred to as an interchangeable lens 3) detachable from a camera body 2 is mounted, but it may be a camera in which the camera body 2 and the photographing lens are integrally configured. In FIG. 1, the horizontal direction in the plane intersecting the optical axis O of the interchangeable lens 3 is defined as the X axis, and the vertical direction is defined as the Y axis.

[0007] <Camera Body> The camera body 2 includes a body-side control unit 230, a body-side communication unit 240, an attitude sensor 250, an imaging element 260, a signal processing unit 270, an operation member 280, and a display unit 290. The body-side control unit 230 is connected to the body-side communication unit 240, the attitude sensor 250, the imaging element 260, the signal processing unit 270, the operation member 280, and the display unit 290.

[0008] The body-side control unit 230 is composed of a microcomputer and its peripheral circuits. The body-side control unit 230 includes a storage unit 235 that stores a control program executed by the body-side control unit 230, a focus detection unit 230a that performs focus detection, an eye / face detection unit 230b that detects eyes and faces, a display control unit 230c, a display range selection unit 230d that selects a display range, a focus detection range selection unit 230e that selects a focus detection range, and a selection unit 230f that includes the display range selection unit 230d and the focus detection range selection unit 230e. The body-side control unit 230 executes a control program stored in the storage unit 235 to control each unit in the camera body 2. The storage unit 235 has its data recording and reading controlled by the body-side control unit 230.

[0009] The body-side communication unit 240 performs predetermined communication with the lens-side communication unit 340 of the interchangeable lens 3. The body-side communication unit 240 is connected to the body-side control unit 230. The attitude sensor 250 detects angle information indicating whether the camera system 1 is held in a horizontal position or a vertical position, and sends a detection signal to the body-side control unit 230. The camera system 1 is held in a horizontal position (the position of the camera where the long side of the imaging surface 260S of the imaging device 260 is horizontal) when shooting a horizontally long screen that is long in the X-axis direction in FIG. 1, and is held in a vertical position (the position of the camera where the long side of the imaging surface 260S of the imaging device 260 is vertical) when shooting a vertically long screen that is long in the Y-axis direction.

[0010] Light from the subject that has passed through the interchangeable lens 3 is guided to the imaging surface 260S of the imaging device 260. The imaging device 260 is a solid-state imaging device such as a CMOS image sensor or a CCD image sensor, for example. The imaging device 260 captures the subject image formed on the imaging surface 260S according to a control signal from the body-side control unit 230 and outputs a signal. The imaging device 260 is capable of shooting videos and still images. Shooting videos includes not only recording videos but also shooting so-called through images (also referred to as live view images) for continuously displaying monitor images on the display unit 290.

[0011] The imaging device 260 has a photoelectric conversion unit for image generation and a photoelectric conversion unit for focus detection. The imaging pixel signal based on the charge generated by the photoelectric conversion unit for image generation is used by the image signal processing unit 270a of the signal processing unit 270 to generate image data. Further, the focus detection pixel signal based on the charge generated by the photoelectric conversion unit for focus detection is used by the AF signal processing unit 270b of the signal processing unit 270 for focus detection processing to detect the in-focus state of the subject image formed by the interchangeable lens 3, in other words, the deviation between the position of the subject image formed by the interchangeable lens 3 and the position of the imaging surface 260S. The imaging device 260 is connected to the signal processing unit 270 and the body-side control unit 230.

[0012] The signal processing unit 270 includes an image signal processing unit 270a and an AF signal processing unit 270b. The image signal processing unit 270a performs predetermined image processing on the imaging pixel signals output from the imaging device 260 to generate image data. The generated image data is recorded in a storage medium (such as a memory card) not shown in a predetermined file format, or is used for displaying a live view image before shooting or a confirmation image after shooting.

[0013] Also, the AF signal processing unit 270b uses the focus detection pixel signals output from the imaging device 260 to perform phase difference detection type focus detection processing to calculate a defocus amount (the deviation amount between the imaging position of the interchangeable lens 3 and the imaging surface 260S). The focus detection processing may adopt a contrast method. The signal processing unit 270 is connected to the body side control unit 230, the imaging device 260, and the display unit 290.

[0014] An operation member 280 including a release button, an operation switch, etc. is provided on the outer surface of the camera body 2. The operation member 280 sends an operation signal corresponding to the user's operation to the body side control unit 230. The user gives a shooting instruction, a shooting condition setting instruction, etc. by operating the operation member 280. Also, the user can instruct the switching of the "AF area mode" described later by the operation member 280.

[0015] The display unit 290 is composed of an organic EL display panel or a liquid crystal display panel. The display unit 290 displays an image based on the image data processed by the image signal processing unit 270a, an operation menu screen, etc. according to a control signal from the body side control unit 230. Also, the display unit 290 may be provided with a touch panel to constitute a touch operation member 290A as a part of the operation member 280. The display unit 290 is connected to the body side control unit 230.

[0016] <Interchangeable lens> The interchangeable lens 3 includes a lens-side control unit 330, a lens-side communication unit 340, an imaging optical system 360, a lens drive unit 370, and a diaphragm drive unit 380. The lens-side control unit 330 is connected to the lens-side communication unit 340, the lens drive unit 370, and the diaphragm drive unit 380.

[0017] The lens-side control unit 330 is composed of a microcomputer and its peripheral circuits, etc. The lens-side control unit 330 executes a control program stored therein and controls each part of the interchangeable lens 3, such as automatic focus adjustment control.

[0018] The lens-side communication unit 340 performs predetermined communication with the body-side communication unit 240. Through the communication between the lens-side communication unit 340 and the body-side communication unit 240, instructions to move the imaging optical system 360, etc., which will be described later, are transmitted from the camera body 2 to the interchangeable lens 3. Also, through the communication between the lens-side communication unit 340 and the body-side communication unit 240, information such as the driving state of the imaging optical system 360 is transmitted from the interchangeable lens 3 to the camera body 2.

[0019] The imaging optical system 360 includes a plurality of lenses and a diaphragm member 362, and guides subject light to the imaging surface 260S of the imaging element 260 of the camera body 2. The plurality of lenses includes a focus lens 361. For example, the focus lens 361 is configured to be movable in the direction of the optical axis O by the lens drive unit 370. When the focus lens 361 moves, the position of the subject image formed by the imaging optical system 360 is changed. Thereby, an AF operation (focusing) is performed.

[0020] The diaphragm 362 adjusts the amount of light incident on the imaging element 260. The diaphragm 362 is configured such that the diaphragm blades can be driven by the diaphragm drive unit 380 or manual operation to change the aperture diameter (diaphragm value).

[0021] Drive instructions such as the moving direction, moving amount, and moving speed of the focus lens 361 are given from the body side control unit 230. Alternatively, they may be given from the lens side control unit 330 based on information from the body side control unit 230.

[0022] The camera body 2 sends, for example, a focus lens drive signal, which is a drive instruction for the focus lens, a diaphragm drive signal, information on the camera body 2 (imaging mode information, ISO sensitivity information, etc.) to the interchangeable lens 3 through communication with the interchangeable lens 3. The interchangeable lens 3 drives the focus lens 361 by the lens drive unit 370 in response to the drive signal from the camera body 2.

[0023] On the other hand, the interchangeable lens 3 sends information such as information on the optical characteristics of the photographic optical system 360, position information of the focus lens 361, and information on the aperture diameter of the diaphragm 362 to the camera body 2 through communication with the camera body 2.

[0024] <An example of camera operation> First, an example of camera operation by the camera system 1 will be described. FIG. 2 is a flowchart illustrating the flow of processing executed by the body side control unit 230. The body side control unit 230 starts the processing according to the flowchart of FIG. 2, for example, at startup accompanying the ON operation of the main switch, or when releasing the sleep operation by operating a switch or the like constituting the operation member 280. The sleep operation refers to an operation of shifting to a power saving state when the non-operation state continues for a predetermined time. Release of the sleep operation (return to the normal state) can be performed, for example, by a half-press operation on the release button.

[0025] In step S10, the body side control unit 230 performs exposure setting for the live view image and performs accumulation control (exposure control) on the imaging device 260. The exposure setting for the first imaging after startup is controlled to a predetermined initial value, and accumulation control is performed on the imaging device 260.

[0026] In step S20, the body-side control unit 230 causes the imaging device 260 to read out a signal to the signal processing unit 270 and proceeds to step S30. The body-side control unit 230 calculates each control value by performing an exposure calculation in step S30. That is, the body-side control unit 230 detects the brightness information (Bv value) of the subject based on the imaging pixel signal input to the signal processing unit 270, and based on the Bv value and the information of the program diagram stored in the storage unit 235, determines the aperture value (Av value), shutter speed (Tv value), and sensitivity (Sv value). Each calculated control value is stored in the storage unit 235.

[0027] In step S40, the body-side control unit 230 sets the aperture 362 to the determined aperture value and performs live view display. The body-side control unit 230 causes the image signal processing unit 270a to generate a live view image using the imaging pixel signal output from the imaging device 260, and causes the generated live view image to be displayed on the display unit 290.

[0028] In step S45, the body-side control unit 230 causes the AF signal processing unit 270b to perform an AF operation. As a result, the AF signal processing unit 270b calculates the defocus amount using the focus detection pixel signal output from the imaging device 260.

[0029] In step S50, the body-side control unit 230 determines whether the release button has been half-pressed. The body-side control unit 230 makes an affirmative determination in step S50 and proceeds to step S60 when the release button has been half-pressed, and returns to step S10 when the release button has not been half-pressed.

[0030] In step S60, the body-side control unit 230 issues an AF drive instruction. The body-side control unit 230 moves the focus lens 361 of the interchangeable lens 3 along the direction of the optical axis O based on the defocus amount calculated in step S45. That is, the body-side control unit 230 causes the lens-side control unit 330 to issue a drive instruction for the focus lens 361 to the lens drive unit 370 to perform a focusing drive operation. Thereafter, the body-side control unit 230 proceeds to step S70.

[0031] In step S70, the body side control unit 230 determines whether the release button has been fully pressed. If the release button has been fully pressed, the body side control unit 230 makes an affirmative determination in step S70 and proceeds to step S80. If the release button has not been fully pressed, the body side control unit 230 makes a negative determination in step S70 and returns to step S10.

[0032]

[0031] The body side control unit 230 that has returned to step S10 performs the processing after step S10 again. For the exposure settings from the second frame onward of the live view image, the aperture value stored in the storage unit 235, the shutter speed (Tv value) stored in the storage unit 235, and the sensitivity (Sv value) stored in the storage unit 235 are used.

[0033] In step S80, which is entered when the release button has been fully pressed (affirmative determination in step S70), the body side control unit 230 causes an accumulation operation for capturing an image for recording to be performed. At this time, the aperture 362 is set to the aperture value (Av value) stored in the storage unit 235, the accumulation control of the imaging element 260 is performed at the shutter speed (Tv value) stored in the storage unit 235, and the processing is performed using the sensitivity (Sv value) stored in the storage unit 235.

[0034] In step S90, the body side control unit 230 reads a signal from the imaging element 260 to the signal processing unit 270 and proceeds to step S100. In step S100, the body side control unit 230 causes the image signal processing unit 270a to perform predetermined image processing on the imaging pixel signal output from the imaging element 260 to generate an image for recording. As a result, the image signal processing unit 270a performs predetermined image processing to generate an image for recording.

[0035]

[0032] In step S110, the body side control unit 230 performs image display. That is, the body side control unit 230 causes the image processed by the image signal processing unit 270a in step S100 to be displayed on the display unit 290 as a confirmation image. In step S120, the body side control unit 230 causes the storage unit 235 to record the file of the image processed by the image signal processing unit 270a.

[0036] In step S130, the body side control unit 230 determines whether to end. When, for example, the main switch is turned off or the camera shifts to the sleep operation, the body side control unit 230 makes an affirmative determination in step S130 and ends the processing according to FIG. 2. When the body side control unit 230 does not end the processing according to FIG. 2 and continues the camera operation, the body side control unit 230 makes a negative determination in step S130 and returns to step S10.

[0037] Note that the display process (S110) and the recording process (S120) may be performed in a swapped order or in parallel. Also, in the example of FIG. 2, the defocus amount is calculated in the AF operation (S45) after the live view display (S40). However, the defocus amount may be calculated in parallel with the exposure operation (S30), and after the half-press operation is performed in step S50, a drive instruction for the focus lens 361 may be given in step S60. Furthermore, each frame of the live view image may be configured such that the defocus amount is calculated and a drive instruction for the focus lens 361 is given in parallel with the exposure operation (S30) each time the frame is acquired.

[0038] <Focus point> The focus point used for the focus detection process in the AF operation (S60) will be described. FIG. 3 is a diagram illustrating a shooting range 50 imaged by the imaging device 260 of the camera body 2. A plurality (for example, 135 points) of focus points P are provided in advance in the shooting range 50, and the focus points P are shown together with the captured image in FIG. 3. The focus point P indicates a position where focusing is possible in the shooting range 50, and is also referred to as a focus detection area, a focus detection position, and a ranging point. Note that the number of the illustrated focus points P and their positions in the shooting range 50 are merely examples and are not limited to the mode shown in FIG. 3.

[0039] The focus detection unit 230a calculates the defocus amount (the amount of out-of-focus) which is the deviation amount between the position of the subject image formed by the light beam passing through the interchangeable lens 3 and the position of the imaging surface 260S of the imaging device 260 of the camera body 2, using the signal from the AF signal processing unit 270b. The amount of image deviation (phase difference) of the subject image formed by a pair of light beams passing through different regions of the interchangeable lens 3 can be detected for each focus point P, and the defocus amount can be calculated for each focus point P. Then, the body side control unit 230 calculates the driving amount of the focus lens 361 based on the defocus amount calculated at the focus point P selected from among the plurality of focus points P, and transmits a lens drive signal of the focus lens 361 to the interchangeable lens 3. The interchangeable lens 3 can adjust the position of the subject image existing in the region corresponding to the focus point P selected from among the plurality of focus points P to the position of the imaging surface 260S of the imaging device 260 by moving the focus lens 361 according to the drive signal.

[0040] The body side control unit 230 can automatically select the focus point P used for calculating the defocus amount (in other words, used for focusing) from among all the focus points P, or can also select the focus point P instructed by the user's operation. The general term for the method of selecting the focus point P used for calculating the defocus amount from among all the focus points P is referred to as the "AF area mode".

[0041] The "AF area mode" includes, for example, "single-point AF" that calculates the defocus amount using only one focus point P selected by the user's operation, "wide-area AF" that calculates the defocus amount using a plurality of focus points P in a range wider than single-point AF, and "auto-area AF" that detects a subject by the camera from the range where all the focus points P are arranged and calculates the defocus amount using the focus points P located in the range of the subject. The switching of the "AF area mode" can be selected by the user operation from, for example, the operation menu screen.

[0042] The example of FIG. 3 shows a state in which the body-side control unit 230 switched to "auto-area AF" among the "AF area modes" detects a subject (person) close to the camera system 1 from the range where all the focus points P are arranged and selects nine focus points Pa located in the range of the person's head. In this case, the AF signal processing unit 270b can calculate the respective defocus amounts for each of the nine focus points Pa.

[0043] When the AF signal processing unit 270b calculates the respective defocus amounts for each of the plurality of focus points Pa, the body-side control unit 230 can select one focus point that is, for example, closest (nearest) to the camera system 1 or has the highest contrast of the focus detection pixel signal among the plurality of focus points Pa. Then, the body-side control unit 230 transmits a drive signal of the focus lens 361 to the interchangeable lens 3 so as to focus on the portion corresponding to the selected focus point of the person.

[0044] When selecting one focus point, the body-side control unit 230 causes the focus lens 361 to be driven and displays a display (AF frame) indicating the one focused focus point Pa superimposed on the live view image displayed on the display unit 290.

[0045] Note that the body-side control unit 230 can also perform an AF operation so as to focus the entire head (including it in the depth of field) in consideration of the defocus amount of each of the nine focus points Pa located on the head. For example, the average value of the nine defocus amounts may be calculated, and the focus lens 361 may be driven (AF operation) with the calculated average defocus amount, or a weighted average may be calculated so that the nine defocus amounts are included in the depth of field as much as possible and the AF operation may be performed.

[0046] When using the nine focus points Pa, in the AF operation (S60), the body-side control unit 230 causes the display unit 290 to display a display (AF frame) indicating each of the nine in-focus focus points Pa superimposed on the live view image being displayed.

[0047] <Face Detection and Eye Detection> In the present embodiment, a function of detecting a subject's face from image data and a function of detecting the eyes of the detected face are provided. Then, any one of a first setting for enabling both the face and eye detection functions, a second setting for enabling the face detection function and disabling the eye detection function, and a third setting for disabling both the face and eye detection functions can be selected when switched to "Auto Area AF". The selection of the first setting, the second setting, and the third setting is selected, for example, by a user operation from an operation menu screen. A setting for disabling the face detection function and enabling the eye detection function may be provided. In the present embodiment, an example of detecting eyes is described, but the iris and pupils may be detected in the same way. Further, the eyes to be detected are not limited to human eyes, and may be the eyes of animals such as pets. Furthermore, it is not limited to eyes, and may be the face or characteristic parts of an object.

[0048] The above description exemplified as the AF operation of "Auto Area AF" is the AF operation when the third setting is selected. Therefore, the AF operation when the first setting is selected and the AF operation when the second setting is selected will be described below.

[0049] <First Setting> In the case of the first setting that enables the detection functions for both the face and eyes, as illustrated in FIG. 4(a), the body-side control unit 230 causes the display unit 290 to display a display frame 41 (AF frame) indicating the detected eyes superimposed on the live view image being displayed. In the present embodiment, as the display frame 41 indicating the detected eyes, a frame surrounding the contour of the eyes including the white part is exemplified, but a frame surrounding the iris or pupil (the eye) not including the white part may also be used. When the body-side control unit 230 enlarges the live view image on the display unit 290, if eyes are detected, the enlargement is performed centered on the position of the detected eyes.

[0050] When the body-side control unit 230 detects the left and right eyes, based on the sizes of the left and right eyes, for example, it can select one of the larger eyes. FIG. 4(a) is an example in which the display frame 41 is displayed for the right eye determined to be larger. The AF signal processing unit 270b can calculate the defocus amount for each focus point P corresponding to the display frame 41. The body-side control unit 230 can select one of the plurality of focus points P, for example, the focus point closest to (nearest to) the camera system 1 or the focus point with the highest contrast. Then, the body-side control unit 230 performs an AF operation to focus on the portion corresponding to the selected focus point P within the display frame 41.

[0051] Note that the body-side control unit 230 can also perform an AF operation to focus on the entire eye indicated by the display frame 41 (including it in the depth of field) in consideration of the defocus amounts of the respective focus points P corresponding to the display frame 41. Also, when the body-side control unit 230 detects the left and right eyes, for example, it can select one of the predetermined one eye.

[0052] <Second setting> In the case of the second setting where the face detection function is enabled and the eye detection function is disabled, as illustrated in FIG. 4(b), the body side control unit 230 causes the display frame 42 (AF frame) indicating the detected face to be displayed superimposed on the live view image being displayed on the display unit 290. In the present embodiment, as the display frame 42 indicating the detected face, a frame surrounding at least a part of the forehead including the eyebrows, at least a part of each cheek, and at least a part under the lips is exemplified. However, if the frame does not include a background other than the face, the size of the frame may be changed as appropriate. For example, if the size of the frame is determined by 0.8 times the length in the X-axis direction (horizontal direction) of the contour of the detected face and 0.8 times the length in the Y-axis direction (vertical direction) of the contour of the detected face, it is preferable because it is less likely to include a background other than the face within the frame.

[0053] The AF signal processing unit 270b can calculate the defocus amount for each focus point P corresponding to the display frame 42. The body side control unit 230 can select one focus point P that is, for example, the closest (nearest) to the camera system 1 or has the highest contrast among the plurality of focus points P. Then, the body side control unit 230 performs an AF operation to focus on the portion corresponding to the selected focus point P within the display frame 42.

[0054] Note that the body side control unit 230 can also perform an AF operation so as to focus on the entire face indicated by the display frame 42 (including it in the depth of field) in consideration of the defocus amount of each of the plurality of focus points P corresponding to the display frame 42.

[0055] Also, when a face is detected in the first setting but eyes are not detected in that face, the body side control unit 230 causes the display frame 42 indicating the detected face to be displayed superimposed on the live view image being displayed on the display unit 290 in the same manner as in the case of the second setting (FIG. 4(b)). The AF signal processing unit 270b shall calculate the defocus amount based on the focus point P corresponding to the display frame 42 in the same manner as in the case of the second setting.

[0056] In the first setting, first, a face is detected, and then the eyes of the detected face are detected. If the display frame 42 is displayed when the face is detected, the display frame 41 will be displayed instead of the display frame 42 that is being displayed when the eyes are detected, which may also impair the user experience. Therefore, in the present embodiment, when detecting the first face, whether to display the display frame 41 or the display frame 42 is determined based on whether eyes can be detected for the detected face. By configuring in this way, if eyes are detected for the detected face, the display frame 41 can be displayed (Fig. 4(a)), and if eyes are not detected for the detected face, the display frame 42 can be displayed (Fig. 4(b)).

[0057] <When multiple eyes or faces are detected> Fig. 5 is a diagram for explaining the case where multiple eyes or faces are detected. In the first setting and the second setting, it is assumed that the user desires to focus on eyes or faces other than the eyes or faces selected by the body side control unit 230. Therefore, the body side control unit 230 overlays and displays on the live view image displayed on the display unit 290, displays 52 and 53 (triangle marks) that inform the user that eyes or faces other than the eyes or faces corresponding to the display frame 51 (AF frame) displayed in the live view image have been detected.

[0058] For example, in the case of the first setting, the body side control unit 230 detects three faces 55, 56, and 57 included in the live view image illustrated in Fig. 5, selects the face 55 at the position closest to the center of the screen, detects the left and right eyes of the face 55, and selects the left eye (hereinafter referred to as the left eye) determined to be larger, and assumes that the display frame 51 indicating the left eye is displayed on the display unit 290.

[0059] When the body-side control unit 230 selects the left eye, the body-side control unit 230 records in the storage unit 235 information indicating that the left eye has been selected. The reason for this is, for example, that in a subsequent live view image, the face cannot be detected for a while, and when selecting the eye of the face detected in a subsequent live view image, the left eye is selected based on the information recorded in the storage unit 235 (in this example, information indicating that the left eye has been selected). By using the information recorded in the storage unit 235 to select one of the left and right eyes, it is possible to avoid a situation where the selected eye changes every time the face is redetected. The information indicating the eye on the side selected by the body-side control unit 230 and recorded in the storage unit 235 is retained until the main switch is turned off or the "AF area mode" is switched.

[0060] The body-side control unit 230 further displays, on the right side of the display frame 51, a display 53 (such as < indicating the direction, hereinafter referred to as a triangular mark) indicating that the right eye 54 (hereinafter referred to as the right eye) on the right side has been detected. Since the triangular mark 53 is displayed on the right side of the display frame 51, the user can understand that the right eye 54 can be selected.

[0061] The body-side control unit 230 further displays, on the left side of the display frame 51, a display 52 (triangular mark) indicating that the left face 56 of the face 55 has been detected. Since the triangular mark 52 is displayed on the left side of the display frame 51, the user can understand that the left face 56 can be selected. Note that the frames surrounding the right eye 54, the face 55, the face 56, and the face 57 shown by the dashed-dotted line in FIG. 5 are for explanatory purposes and do not have to be displayed on the display unit 290. Display frames may be displayed for all the eyes detected from the three faces 55, 56, and 57 included in the live view image illustrated in FIG. 5, and the form of the display frame of the eye selected by the body-side control unit 230 may be changed from the forms of the other display frames. For example, the size of the display frame of the selected eye may be changed, the color of the display frame may be changed, or the shape of the display frame may be changed. Note that the display 52 is not limited to < and may be a triangle, an arrow, or the like.

[0062] <Eye selection by the user> FIG. 6 is a diagram illustrating an outer surface on the back side of the camera body 2 when the camera system 1 is set in a horizontal position. Of the switches constituting the operation member 280, two operation switches 281 and 282 are arranged in parallel with a straight line H indicating the X-axis direction (horizontal direction). The user can change the eye or face to be focused in the horizontal direction, for example, by operating the operation switches 281 and 282.

[0063] FIG. 7 is a diagram for explaining a case where the eye 54 located on the right side of the display frame 51 of the face 55 in FIG. 5 is selected. When the operation switch 282 is operated by the user, the body side control unit 230 selects the right eye of the face 55 and causes the display unit 290 to display a display frame 71 (AF frame) indicating the right eye. The AF signal processing unit 270b calculates a defocus amount based on the focus point P corresponding to the display frame 71.

[0064] The body side control unit 230 further displays, on the right side of the display frame 71, a display 73 (triangle mark) indicating that the right face 57 of the face 55 has been detected. Since the triangle mark 73 is displayed on the right side of the display frame 71, the user knows that the right face 57 can be selected.

[0065] The body side control unit 230 further displays, on the left side of the display frame 71, a display 72 (triangle mark) indicating that the left eye 74 of the face 55 has been detected. Since the triangle mark 72 is displayed on the left side of the display frame 71, the user knows that the left eye 74 can be selected. Note that the frames surrounding the left eye 74, the face 55, the face 56, and the face 57, which are indicated by the dashed-dotted line in FIG. 7, are attached for the purpose of explanation and do not need to be displayed on the display unit 290.

[0066] When selecting the right eye, which is different from the left eye where the display frame 51 (Fig. 5) was displayed before the operation, based on the operation of the operation switch 282, the body side control unit 230 records information indicating that the right eye has been selected based on the user's operation in the storage unit 235. For example, when the face cannot be detected in the subsequent live view image and then the eyes of the detected face in the subsequent live view image are selected, this is to select the right eye based on the information recorded in the storage unit 235 (in this example, the information indicating that the right eye has been selected). By using the information recorded in the storage unit 235 to select one of the left and right eyes, it is possible to avoid a situation where the selected eye changes every time the face is redetected. The information indicating the eye on the selected side based on the user's operation, which is recorded in the storage unit 235, is retained until the main switch is turned off or the "AF area mode" is switched.

[0067] Fig. 8 is a diagram for explaining the case where the eyes of the face 57 located on the right side of the display frame 71 of the face 55 in Fig. 7 are selected. When the operation switch 282 is operated by the user, the body side control unit 230 selects the face 57 on the right side of the face 55, detects the left and right eyes of the face 57, selects the left eye (hereinafter referred to as the left eye) facing the face 57, and causes the display unit 290 to display a display frame 81 (AF frame) indicating the left eye. The AF signal processing unit 270b calculates the defocus amount based on the focus point P corresponding to the display frame 81.

[0068] The body side control unit 230 further displays a display 83 (triangle mark) indicating that the right eye (hereinafter referred to as the right eye) 84 facing the face 57 has been detected on the right side of the display frame 81. Since the user can see the triangle mark 83 displayed on the left side of the display frame 81, the user knows that the right eye 84 can be selected.

[0069] The body side control unit 230 further displays a display 82 (triangle mark) indicating that the face 55 on the left side of the face 57 has been detected on the left side of the display frame 81. Since the user can see the triangle mark 82 displayed on the left side of the display frame 81, the user knows that the left face 55 can be selected. Note that the frames surrounding the right eye 84 indicated by the dashed line in FIG. 8, the frame surrounding the face 55, the frame surrounding the face 56, and the frame surrounding the face 57 are for explanatory purposes and do not need to be displayed on the display unit 290. As described above, when an eye of the face 57 different from the face 55 where the display frame 71 was displayed before the operation is selected based on the operation of the operation switch 282, the left eye closer to the face 55 is selected, and the display frame 81 indicating this left eye is displayed.

[0070] FIG. 9 is a diagram for explaining another example when an eye of the face 57 located on the right side of the display frame 71 of the face 55 in FIG. 7 is selected. When the display position of the face 57 on the display unit 290 is touched by the user via the touch operation member 290A, the body side control unit 230 selects the face 57 on the right side of the face 55, detects the left and right eyes of the face 57, and selects the right eye (hereinafter referred to as the right eye) on the right side of the face 57 and causes the display unit 290 to display a display frame 84A indicating the right eye.

[0071] When an eye of the face 57 different from the face 55 where the display frame 71 was displayed before the operation is selected based on the touch operation, the body side control unit 230 selects the right eye based on the information recorded in the storage unit 235 (in this example, the information indicating that the right eye was selected), and displays a display frame 84A indicating the selected right eye. The AF signal processing unit 270b calculates the defocus amount based on the focus point P corresponding to the display frame 84A.

[0072] Note that when the touch operation position is the position of the left eye of the face 57, the body side control unit 230 selects the left eye regardless of the information recorded in the storage unit 235, and displays a display frame indicating the selected left eye. The AF signal processing unit 270b calculates the defocus amount based on the focus point P corresponding to the display frame indicating the left eye. In this way, the body side control unit 230 selects the eye if the touch operation position is the position of the eye, and selects either the left or right eye based on the information recorded in the storage unit 235 if the touch operation position is not the position of the eye.

[0073] The body-side control unit 230 further causes a display 85 (triangle mark) indicating that the left eye 81A on the left side of the face 57 has been detected to be displayed on the left side of the display frame 84A. Since the triangle mark 85 is displayed on the left side of the display frame 84A, the user can understand that the left eye 81A can be selected. Note that the frames surrounding the left eye 81A, the face 55, the face 56, and the face 57, which are indicated by the dashed-dotted line in FIG. 9, are attached for the purpose of explanation and do not have to be displayed on the display unit 290.

[0074] The display of the triangle mark described with reference to FIGS. 5 to 9 may have different display modes, for example, in terms of color, shape, characters, addition of icons, etc., depending on whether the eye in the direction indicated by the triangle mark can be selected or only the face can be selected when a face is detected in the direction indicated by the triangle mark but no eye is detected. Also, the display of the display frame (AF frame) described with reference to FIGS. 5 to 9 may have different display modes, for example, in terms of color, shape, characters, addition of icons, etc., depending on whether it is displayed on the eye selected by the body-side control unit 230 or on the eye selected by the user's operation.

[0075] <Relationship between the size of the eye or face and the display frame> FIGS. 10(a) to 10(c) are diagrams for explaining the relationship between the size of the eye or face in the image and the display frame. In the first setting, if the size of the eye in the live view image is too small, the amount of information of the eye included in the focus point P corresponding to the display frame decreases, resulting in a decrease in the accuracy of calculating the defocus amount. Therefore, calculating the defocus amount using not only the eye but also the face information is effective in preventing a decrease in the accuracy of calculating the defocus amount.

[0076] FIG. 10(a) is a diagram for explaining the case where the eye size is sufficiently large in the live view image. When the eye size is sufficiently large, the accuracy of calculating the defocus amount can be ensured even by using only the eye information. Therefore, the body-side control unit 230 for which the first setting is selected detects the left and right eyes of the face 55, and causes the display unit 290 to display a display frame 51 of the eye size indicating, for example, the left eye. The AF signal processing unit 270b calculates the defocus amount based on the focus point P corresponding to the display frame 51.

[0077] FIG. 10(b) is a diagram for explaining the case where the eye size is small in the live view image. When the body-side control unit 230 for which the first setting is selected determines that, for example, the number of pixels A in the X-axis direction (horizontal direction) of the eye is smaller than the Q1 percentage of the number of pixels Th in the horizontal direction of the live view image and larger than the Q2 percentage (Q2 < Q1), the display unit 290 is caused to display a display frame 51 indicating the left eye of the face 55. At this time, a display frame 51 of a small size is displayed in accordance with the eye in FIG. 10(b) having a smaller size than the eye in FIG. 10(a).

[0078] The AF signal processing unit 270b shall calculate the defocus amount based on the focus point P corresponding to the frame indicating the face 55 (the same size as the display frame of the face size) regardless of the size of the display frame 51 indicating the left eye of the face 55. By configuring in this way, the accuracy degradation of calculating the defocus amount is prevented by using not only the left eye of the face 55 but also the information of the face 55 indicated by the dashed line.

[0079] The reason for calculating the defocus amount using the information of the face 55 will be further explained. When the eye size is small in the live view image, the eye information included in the focus point P (FIG. 3) decreases. Also, it is assumed that the focus point P is not arranged at the position corresponding to the detected eye and the eye information is not included in the focus point P. Therefore, it becomes difficult to calculate the defocus amount based on the focus point P corresponding to the display frame 51 indicating the eye. On the other hand, it is considered that the detected eye size becomes small when the distance from the camera system 1 to the eye (face) is long. Such a distant subject may be considered to have no substantial difference between the focus position on the face and the focus position on the eye, unlike a subject close to the camera system 1. That is, the focus positions of the nose and lips may be regarded as equivalent to the focus position of the eye. Therefore, in the present embodiment, a focus point P with a high contrast of the focus detection pixel signal is selected from a plurality of focus points P corresponding to a frame indicating a face 55 having a size larger than the display frame 51, and the defocus amount is calculated. By calculating in this way, the defocus amount can be appropriately calculated based on a large number of focus points P corresponding to the frame indicating the face 55. Note that the frame surrounding the face 55 is shown for the purpose of explanation and does not have to be displayed on the display unit 290.

[0080] FIG. 10(c) is a diagram for explaining a case where the eye size is even smaller in the live view image. When the body side control unit 230 for which the first setting is selected has an eye size in the live view image that is even smaller than that in FIG. 10(b), for example, when the number of pixels A in the X-axis direction (horizontal direction) of the eye is smaller than the Q2 percentage (Q2 < Q1) of the number of pixels Th in the horizontal direction of the live view image, instead of the display frame 51 showing the left eye of the face 55, a display frame 59 having the size of the face showing the face 55 is displayed on the display unit 290.

[0081] The AF signal processing unit 270b is configured to calculate the defocus amount based on the focus point P corresponding to the display frame 59 showing the face 55. By configuring in this way, a decrease in the accuracy of calculating the defocus amount is prevented by using the information of the face 55. In addition, by displaying the display frame 59 having a size larger than the display frame 51 showing the eye on the display unit 290, the user can be clearly shown the subject to be focused. Also, in FIGS. 10(a) to 10(c), when the number of pixels A in the X-axis direction (horizontal direction) of the eyes is larger than a predetermined value that does not depend on the number of pixels of the live view image, a display frame 51 indicating the eyes may be displayed, and when the number of pixels A in the X-axis direction (horizontal direction) of the eyes is smaller than the threshold value, a display frame 59 indicating the face 55 may be displayed on the display unit 290. Similarly, in FIGS. 10(a) to 10(c), when the number of pixels A in the X-axis direction (horizontal direction) of the eyes is larger than a predetermined value that does not depend on the number of pixels of the live view image, the defocus amount may be calculated based on the focus point P corresponding to a frame (the same size as the display frame of the face size) indicating the face 55, and when the number of pixels A in the X-axis direction (horizontal direction) of the eyes is smaller than the predetermined value, the defocus amount may be calculated based on the focus point P corresponding to a frame (the same size as the display frame of the eye size) indicating the eyes. That is, it may be determined by a threshold value that does not depend on the number of horizontal pixels Th of the live view image.

[0082] <An example of the frame display operation> FIG. 11 is a flowchart exemplifying the flow of processing for determining a display frame (AF frame) executed by the body side control unit 230 when the first setting is selected. The body side control unit 230 determines the focus point P used for calculating the defocus amount described above by executing the processing according to the flowchart of FIG. 11.

[0083] In step S310, the body side control unit 230 determines whether a face has been detected based on the live view image. When the body side control unit 230 detects a face, it makes an affirmative determination in step S310 and proceeds to step S320, and when it does not detect a face, it makes a negative determination in step S310 and proceeds to step S360.

[0084] In step S320, the body side control unit 230 determines whether eyes are detected in the face detected in S310 based on the live view image. When the body side control unit 230 detects eyes, it makes an affirmative determination in step S320 and proceeds to step S330. When the body side control unit 230 does not detect eyes, it makes a negative determination in step S320 and proceeds to step S350.

[0085] In step S330, the body side control unit 230 determines whether the size of the eyes is larger than a predetermined first threshold value. For example, when the number of pixels A in the X-axis direction (horizontal direction) of the eyes is larger than Q2 percent of the number of pixels Th in the horizontal direction of the live view image, the body side control unit 230 makes an affirmative determination in step S330 and proceeds to step S340. When the number of pixels A is less than or equal to Q2 percent of the number of pixels Th, the body side control unit 230 makes a negative determination in step S330 and proceeds to step S350.

[0086] In step S340, the body side control unit 230 causes the display unit 290 to display a display frame 51 for the size of the detected eyes based on the detected eye information. Specifically, as illustrated in FIGS. 10(a) and 10(b), a display frame 51 indicating the eyes is displayed at the detected eye position, and the process proceeds to step S370.

[0087] In step S370, the body side control unit 230 determines whether the size of the eyes is larger than a predetermined second threshold value. For example, when the number of pixels A in the X-axis direction (horizontal direction) of the eyes is larger than Q1 (Q2 < Q1) percent of the number of pixels Th in the horizontal direction of the live view image, the body side control unit 230 makes an affirmative determination in step S370 and proceeds to step S380. When the number of pixels A is less than or equal to Q1 percent of the number of pixels Th, the body side control unit 230 makes a negative determination in step S370 and proceeds to step S390. When the size of the eyes is small, there is a high possibility that the eyes are erroneously detected. In the present embodiment, the reliability of the detected eyes is evaluated by comparing the size of the eyes with the second threshold value.

[0088] In step S380, the body side control unit 230 causes the AF signal processing unit 270b to perform an AF operation based on the display frame 51 of the eye size, and ends the processing according to FIG. 11. The AF signal processing unit 270b calculates a defocus amount based on the focus point P corresponding to the display frame 51 (FIG. 10(a)).

[0089] In step S350, where the determination in step S320 or step S330 is negative and the process proceeds, the body side control unit 230 causes the display unit 290 to display the display frame 59 of the face size based on the detected face information. Specifically, as illustrated in FIG. 10(c), a display frame 59 indicating the face 55 is displayed at the detected face position, and the process proceeds to step S390.

[0090] In step S390, the body side control unit 230 causes the AF signal processing unit 270b to perform an AF operation based on the display frame 59 of the face size, and ends the processing according to FIG. 11. When proceeding from S350 to S390, the AF signal processing unit 270b calculates a defocus amount based on the focus point P corresponding to the display frame 59 (FIG. 10(c)). Also, when the determination in S370 is negative and the process proceeds to S390, the AF signal processing unit 270b calculates a defocus amount based on the focus point P corresponding to the frame indicating the face 55 (the display frame of the face size), regardless of the size of the display frame 51 shown in FIG. 10(b).

[0091] When the determination in step S310 is negative, face detection and eye detection are not performed. In step S360, the body side control unit 230 displays the same display frame as when the third setting is selected without displaying a display frame at the face position or the eye position. That is, a subject closer to the camera system 1 is detected from the range where all the focus points P are arranged, and a display (AF frame) indicating a plurality of focus points P located in the range of the subject is displayed on the display unit 290. Then, in step S400, the body side control unit 230 causes the AF signal processing unit 270b to perform a predetermined AF calculation and ends the process according to FIG. 11. The AF signal processing unit 270b calculates the defocus amount for each of the plurality of focus points P located within the subject range, in the same manner as when the third setting is selected.

[0092] A difference may be provided between the first threshold value used for the determination in step S330 when the eye size is changing in the direction of increasing and when the eye size is changing in the direction of decreasing. By providing a difference, it is possible to prevent a deterioration in the user experience due to frequent switching of the display size of the display frame (AF frame) in the vicinity of the first threshold value. Also, in the flowchart of FIG. 11, step S330 may be omitted. That is, when the eye is detected in step S320, regardless of the size of the eye, the display frame 51 of the eye size may be displayed on the display unit 290 based on the detected eye information.

[0093] <Eye selection by the user in the vertical position> FIG. 12 is a diagram illustrating the outer surface on the back side of the camera body 2 when the camera system 1 is set in the vertical position. Two operation switches 283 and 284 among the switches constituting the operation member 280 are arranged substantially parallel to a straight line H indicating the X-axis direction (horizontal direction). The user can change the eye or face to be focused in the horizontal direction, for example, by operating the operation switches 283 and 284.

[0094] When the camera system 1 is set in the vertical position, the two operation switches 281 and 282 that were arranged parallel to the straight line H in the state where the camera system 1 was set in the horizontal position (FIG. 6) are not suitable for use in an operation to change the subject to be focused in the horizontal direction. Instead of the operation switches 281 and 282, the body side control unit 230 uses the two operation switches 283 and 284 for an operation to change the subject to be focused in the horizontal direction, in other words, an operation to select the left and right eyes.

[0095] In FIG. 12, the straight line H' indicates the longitudinal direction of the outer surface of the camera body 2. The body-side control unit 230 switches two switches used for the operation of selecting the left and right eyes based on the detection signal of the attitude sensor 250. When the camera system 1 that was originally in the horizontal position is changed to the vertical position, if the angle θ between the straight line H indicating the X-axis direction (horizontal direction) and the straight line H' exceeds, for example, 75 degrees, the body-side control unit 230 uses the two operation switches 283 and 284 for the operation of selecting the left and right eyes.

[0096] Conversely, when the camera system 1 that was originally in the vertical position is changed to the horizontal position, if the angle θ becomes less than 65 degrees, the body-side control unit 230 uses the two operation switches 281 and 282 for the operation of selecting the left and right eyes.

[0097] In this way, when the camera system 1 that was originally in the horizontal position is changed to the vertical position and when the camera system 1 that was originally in the vertical position is changed to the horizontal position, a difference is provided in the threshold value of the angle θ for determining the switching of the two switches used for the operation of selecting the left and right eyes, so that it is possible to prevent a decrease in the user experience due to frequent switching of the two switches in the vicinity of the threshold value. Further, an operation member (such as an icon) corresponding to the operation switches 281-284 in FIG. 12 or FIG. 6 may be displayed on the display unit 290, and the user may select the left and right eyes by touching the operation icon. For example, when the camera system 1 is configured with a housing such as a smartphone, the user can select the left and right eyes only by moving a finger within a predetermined range, which can improve the usability. Furthermore, in FIG. 12 and FIG. 6, when the camera body 2 has an electronic viewfinder (EVF), the content (live view image, display frame, etc.) displayed on the display unit 290 may be displayed on the EVF. At this time, the user selects the left and right eyes by operating the operation switches 281-284 while observing the EVF. Since the user can select the left and right eyes by operating the operation switches 281-284 without taking their eyes off the EVF, the usability of the user can be improved.

[0098] <Select the left and right eyes based on the position of the face> In the above description of selecting the left and right eyes based on the operation of the operation switch 282 by the user, FIGS. 7 to 9 referred to were examples in which a plurality of faces were arranged in the X-axis direction (horizontal direction). Therefore, when the operation switch 282 was operated with the display frame 71 displayed on the right eye of the face 55 in FIG. 7, there was no problem in causing the display frame 81 to be displayed on the left eye of the face 57 on the right side of the face 55 in FIG. 8. That is, by sequentially selecting the right eye without determining which face's eye it is, the right eye is selected next to the left eye in the face 55, and the right eye is selected next to the left eye in the next right face 57 of the face 55.

[0099] However, as shown in FIG. 13, when a plurality of faces are arranged in the Y-axis direction (vertical direction), the following problems occur. FIG. 13 is a diagram for explaining a state in which a plurality of faces are arranged in the Y-axis direction. In the example of FIG. 13, the left eye (hereinafter referred to as the left eye) of the face 132 is selected after the left eye (hereinafter referred to as the left eye) facing the left of the face 131, and the right eye (hereinafter referred to as the right eye) facing the right of the face 131 is selected after the left eye of the face 132. Therefore, in the present embodiment, the positions of the eyes are rearranged in the X-axis direction based on the position of the center of the face so that the left and right eyes in the same face are sequentially selected and then the left and right eyes of another face are sequentially selected. Specifically, if the center position in the X-axis direction (horizontal direction) of the face 131 is C1 and the center position in the X-axis direction (horizontal direction) of the face 132 is C2, C2 is located on the right side of C1.

[0100] Each time the operation switch 282 is operated with the display frame 133 displayed on the left eye of the face 131 (FIG. 13), the body side control unit 230 selects the right eye of the face 131, selects the left eye of the face 132 after the right eye of the face 131, and selects the right eye of the face 132 after the left eye of the face 132. With such a configuration, it becomes possible to continuously select the left and right eyes in the same face, and the usability of the user can be improved.

[0101] <When in the vertical position> The same applies when the camera system 1 is set in the vertical position as shown in FIG. 12. In the case of the vertical position, the eye positions are rearranged in the Y-axis direction with reference to the position of the center of the face so that, after sequentially selecting the left and right eyes in the same face, the left and right eyes of another face are sequentially selected. Although not shown, let the center position in the Y-axis direction of a certain face be C3, and the center position in the Y-axis direction of another face be C4. Assume that C4 is located on the right side of C3.

[0102] Each time the operation switch 284 is operated with a display frame (AF frame) displayed on the left eye of a certain face, the body side control unit 230 selects the right eye of the certain face, then selects the left eye of another face after the right eye of the certain face, and then selects the right eye of another face after the left eye of another face. With this configuration, it becomes possible to continuously select the left and right eyes in the same face, improving the user-friendliness.

[0103] According to the first embodiment described above, the following operational effects can be obtained. (1) A camera (camera system 1) as an example of an electronic device includes a body side control unit 230 that detects a face and eyes from an image, a body side control unit 230 that selects one eye from the eyes detected by the body side control unit 230, and a display unit 290 that superimposes and displays a display frame (AF frame) indicating the position of the eye selected by the body side control unit 230 and a triangular mark indicating that at least one of a face and eyes different from the selected eye has been detected by the body side control unit 230 on the live view image. With this configuration, it is possible to visually inform the user which eye among the faces and eyes detected from the live view image has been selected.

[0104] (2) Since the display of the triangular mark indicates the direction from the display frame (AF frame) at the position where a face or an eye different from the selected eye has been detected, it is possible to visually inform the user in which direction the face and eyes other than the selected eye are located from the display frame (AF frame).

[0105] (3) The display unit 290 varies the display mode of the triangular mark depending on whether it is a display of a triangular mark regarding the position of a face different from the selected eye or a display of a triangular mark regarding the position of an eye different from the selected eye, so it is possible to visually inform the user whether an eye other than the selected eye has been detected or a face other than the selected face has been detected.

[0106] (4) When a plurality of eyes are detected on the face, the body side control unit 230 selects one eye based on the size of the detected eyes. For example, by focusing on the selected eye, an effect such as it being easier to focus compared to the case of selecting a small eye can be obtained.

[0107] (5) Since the body side control unit 230 is configured to select an eye based on an operation by the user, it is also possible to meet the user's desire to select an eye other than the eye selected by the body side control unit 230.

[0108] (6) The above image is generated by imaging a subject image formed by the imaging optical system 360. The camera stores in the storage unit 235 information regarding whether the eye selected by the user's operation is the right eye or the left eye with respect to the face having the eye. Then, the body side control unit 230 selects an eye based on the information stored in the storage unit 235 in an image obtained by imaging a frame different from the image storing the information. With such a configuration, for example, in the case of selecting an eye from a face that was not detected in a live view image but was detected again in a subsequent live view image, it is possible to avoid a situation where the selected eye changes every time the face is redetected.

[0109] (7) The above camera includes operation switches 281 - 284 for the user to perform an operation of selecting information regarding the direction indicated by the display of the triangle mark. The body - side control unit 230, when there is an eye of the same face as the selected eye in the first direction determined based on the information regarding the direction selected by any one of the operation switches 281 - 284, newly selects the eye of the same face as the selected eye; when there is no eye of the same face as the selected eye, it newly selects the face closest in distance in the first direction. With this configuration, the user can appropriately select the eye or face located in the first direction desired by the user.

[0110] (8) The above camera includes a touch - operation member 290A provided on the display surface of the display unit 290, and receives an operation for the user to select a part of the image displayed at the position where the display surface is touched. The body - side control unit 230, when two eyes are detected for the face that is a part of the displayed image selected via the touch - operation member 290A, selects one of the eyes based on the information stored in the storage unit 235. With this configuration, the user can appropriately select the eye desired by the user from the face selected by the touch operation.

[0111] (9) The storage unit 235 holds the information in the storage unit 235 until at least one of a power - off operation and an operation to set the body - side control unit 230 not to perform face or eye detection, for example, by switching the "AF area mode", is performed. Thus, the information recorded in the storage unit 235 (for example, information indicating that the left - hand eye has been selected) can be appropriately utilized.

[0112] (10) The body - side control unit 230 causes the AF signal processing unit 270b to calculate the defocus amount at the focus point P corresponding to the displayed display frame. With this configuration, the user can be visually informed of which part of the image is to be focused on.

[0113] (11) The body-side control unit 230 determines whether to display a display frame (AF frame) of the detected eye size at the detected eye position or a display frame (AF frame) of the face size at the detected face position according to the detected eye size. With such a configuration, a display frame (AF frame) of an appropriate size can be displayed in the image.

[0114] (12) The body-side control unit 230 determines whether to perform an AF operation based on the display frame of the eye size (S380) or an AF operation based on the display frame of the face size (S390) according to the detected eye size. With such a configuration, when the eyes are small, by performing the AF operation using not only the eyes but also the face information, it is possible to prevent a decrease in the accuracy of calculating the defocus amount.

[0115] (Second Embodiment) The display of the display frame (AF frame) when the first setting is selected may be performed in a method different from that of the first embodiment. FIG. 14 is a flowchart illustrating the flow of the process of determining the display frame executed by the body-side control unit 230 when the first setting is selected. The body-side control unit 230 determines the focus point P used for calculating the above-described defocus amount by executing the process according to the flowchart of FIG. 14 instead of the process according to the flowchart of FIG. 11.

[0116] FIG. 14 is different in that steps S330 and S370 in FIG. 11 are omitted and steps S345 and S370A in FIG. 14 are added compared to FIG. 11. Therefore, the explanation will be centered on these differences.

[0117] In step S345 that proceeds with an affirmative determination in step S340, the body side control unit 230 calculates a distance R from the detected position of the face to the detected position of the eyes on that face. FIG. 15 is a schematic diagram for explaining the calculation of the distance R. In FIG. 15, the detected position of the face 161 is, for example, the center position CA of the detected range of the face 161, and the detected position of the eyes 162 is, for example, the center position CB of the detected range of the eyes 162. The distance R is the distance between the point CA and the point CB in FIG. 15.

[0118] In step S370A that proceeds next to step S345, the body side control unit 230 determines whether the distance R is greater than a predetermined threshold value. For example, the body side control unit 230 makes an affirmative determination in step S370A and proceeds to step S380 when the distance R is greater than R´×k1 obtained by multiplying the length R´ in the X-axis direction (horizontal direction) of the eyes by a coefficient k1. The body side control unit 230 makes a negative determination in step S370A and proceeds to step S390 when the above distance R is less than or equal to the threshold value R´×k1. When the distance R is short, there is a high possibility that the eyes are erroneously detected. In the present embodiment, the reliability of the detected eyes is evaluated by calculating the distance R.

[0119] In the above description, the distance R is the distance from the center position CA of the detected face 161 to the center position CB of the eyes 162 detected on that face 161, but it may be the distance R´ (FIG. 15) between the center positions of the detected left and right eyes on the detected face 161.

[0120] Also, instead of comparing the distance R (or the distance R´) with the product of the length R´ in the X-axis direction (horizontal direction) of the eyes and the coefficient k1, the ratio of the distance R (or the distance R´) to the horizontal direction of the live view image may be compared with a predetermined threshold value X1. Also, the number of pixels A in the X-axis direction of the detected eyes may be compared with a predetermined threshold value to make a determination based on the absolute value.

[0121] According to the second embodiment described above, the following operational effects can be obtained. The body-side control unit 230 calculates the distance R from the center position CA of the face 161 detected in the image to the center position CB of the eye 162 detected in the face 161. Then, depending on the distance R, it determines whether to perform AF calculation based on the display frame (AF frame) of the eye size (S380) or to perform AF calculation based on the display frame of the face size (S390). With this configuration, similar to the case of the first embodiment, by performing AF calculation using not only the information of the eyes but also the information of the face, it is possible to prevent a decrease in the accuracy of calculating the defocus amount.

[0122] The following modifications are also within the scope of the present invention, and it is also possible to combine one or more of the modification examples with the above-described embodiments.

[0123] (Modification Example 1) In the above description, an example has been described in which the body-side control unit 230 that has detected the left and right eyes selects one of the larger eyes based on the size of the eyes. When orientation information indicating whether the face is facing right or left with respect to the camera system 1 is obtained, the body-side control unit 230 may select one of the eyes closer to the camera system 1 based on the orientation information of the face.

[0124] FIGS. 16(a) and 16(b) are diagrams showing an example in which the face is facing right with respect to the camera system 1. The body-side control unit 230 selects one of the left eyes closer to the camera system 1 based on the orientation information of the face, and causes the display frame 51 to be displayed on the selected left eye, whether the left eye on the left side (hereinafter referred to as the left eye) is larger than the right eye on the right side (hereinafter referred to as the right eye) (FIG. 16(a)) or the left eye is smaller than the right eye (FIG. 16(b)).

[0125] Generally, since the difference in the sizes of the detected left and right eyes is often small, if one tries to select the larger eye, the selected eye may frequently switch between left and right. By referring to the orientation information of the face, it is possible to prevent a decrease in the usability of the user due to the frequent switching of the selected eye.

[0126] FIG. 16(c) is a diagram showing an example in which the face is facing left with respect to the camera system. When the orientation information of the face changes, the body side control unit 230 reselects one of the right eyes closer to the camera system 1 based on the new orientation information, and causes the display frame 71 to be displayed on the selected left eye (FIG. 16(c)).

[0127] (Modification Example 2) As described above, when detecting a face or eyes from a live view image, the search range for the face (eyes) may be changed between single shooting and continuous shooting. Single shooting is a shooting mode in which one frame is captured when the release button is fully pressed. Continuous shooting is a shooting mode in which a plurality of frames are continuously captured while the release button is fully pressed.

[0128] Generally, it is desirable to start capturing the next frame earlier during continuous shooting. On the other hand, when searching for a face (eyes) from a live view image, searching for the face (eyes) in a narrower range rather than in a wider range results in a shorter processing time. Therefore, the body side control unit 230 sets the range for detecting the face (eyes) during continuous shooting to, for example, 70% of the range of the live view image. By shortening the processing time for searching for the face (eyes) by narrowing the range, the capture of the next frame can be started earlier, and the continuous shooting speed can be improved.

[0129] The body side control unit 230 can appropriately track the face (eyes) detected in the previous frame while shortening the processing time for searching for the face (eyes) by setting, for example, a range of 70% of the range of the live view image centered on the position of the face (eyes) detected in the previous frame. Also, when setting the search range for searching for eyes centered on the position of the eyes detected in the previous frame, it may be set to a range of 50% of the range of the live view image.

[0130] Note that the body side control unit 230 sets the range for detecting the face (eyes) during single shooting to 100% of the range of the live view image. This is to appropriately detect the face (eyes) using the wide range of the live view image.

[0131] (Modification Example 3) In the recording process (S110), the body side control unit 230 can include the information acquired in the AF operation (S60) in the file to be recorded. For example, information indicating the size and position of the detected face, information indicating the size and position of the detected left and right eyes, information indicating the reliability at the time of detection, information indicating whether the eye for which the display frame is set was selected by the judgment of the body side control unit 230 or selected based on the user's operation, information indicating whether the face for which the display frame is set was selected by the judgment of the body side control unit 230 or selected based on the user's operation, etc. are recorded in the file together with the data of the image for recording.

[0132] (Modification Example 4) In the above-described embodiment, a method using the phase difference detection method as the focus detection method has been exemplified, but the contrast method may be adopted. When the contrast method is adopted, on the imaging surface 260S, based on the imaging pixel signal output from the pixels arranged at the position corresponding to the display frame, or the imaging pixel signal output from the pixels arranged at the positions corresponding to the inside and periphery of the display frame, focus detection is performed by calculating an evaluation value for evaluating the contrast of the high-frequency component of the image. Also, it is also possible to perform focus detection by the contrast method when the size of the detected eye is smaller than a predetermined value, and perform focus detection by the phase difference detection method when the size of the detected eye is larger than the predetermined value. By doing so, when the size of the detected eye is small, or when the focus point P is not arranged at the position corresponding to the detected eye, appropriate focus detection can be performed by performing focus detection by the contrast method.

[0133] (Modification Example 5) In the above-described embodiment, first the face is detected, and subsequently the eyes of the detected face are detected. However, when the face cannot be detected, the shape of the head may be detected, and subsequently the eyes may be detected. Specifically, in a plurality of continuously generated images, the shapes of the face, eyes, and head are detected, and their respective positions and sizes are stored for each image. Further, relative information, which is the relative position and relative size of each of the face, eyes, and head shape, is stored. When the face cannot be detected, first the shape of the head is detected, and the position and size of the detected head shape and the stored relative information are used to detect the position and size of the eyes. When the shape of the head cannot be detected in one image, tracking detection using the position and size of the head shape in past images may be performed. When neither the face nor the head shape can be detected, an image pattern including the color when the face was detected is stored as a feature amount, the feature amount is used to search for similar regions in the image, and the position and size of the eyes are detected using the stored relative information. Tracking detection using the position and size of similar regions in the image of the feature amount of past images may be performed. Thereby, even when the face cannot be detected, by displaying a display frame at a position where the eyes are presumed to be, it is possible to prevent the display frame of the eyes from being displayed or not being displayed between an image in which the eyes were detected and an image in which the eyes could not be detected. That is, it is possible to prevent the eye frame from being displayed or not being displayed for each frame of the through images continuously displayed.

[0134] (Modification Example 6) Calculate the average value (weighted average, with lower weights for older frames, etc.) of the position, size, and angle of the face orientation detected in the latest frame (an image generated by imaging at a predetermined time interval is called a frame) and the position, size, and angle of the face orientation detected in the previous frame, and perform eye detection based on the averaged face position, size, and angle. It is possible to prevent the selected eyes (left and right) from changing due to the change in the position, size, and angle of face detection between frames due to the shaking of the imaging device, resulting in a change in the selected face position and orientation. In addition, when a face or eyes of a person different from the face or eyes of the person detected in the latest frame are selected, when a face or eyes are detected after a predetermined time has elapsed since the face or eyes were detected in the previous frame, or when the position of the detected face has changed significantly, instead of averaging the position, size, and angle of the face detected in the latest frame and the position, size, and angle of the face detected previously, averaging may be performed on frames captured after the new frame. In the above description, the case where a face or eyes are detected after a predetermined time has elapsed since the face or eyes were detected in the previous frame means the case where the face or eyes could not be detected for a predetermined number of frames after the face or eyes were detected and then the face or eyes could be detected again after the predetermined number of frames.

[0135] (Modification Example 7) A value obtained by averaging the size and position of the eyes detected in the current frame with the size and position of the eyes detected in the previous frame (such as weighted average, with lower weight for older frames) may be calculated, and a display frame may be displayed. This can prevent the size and position of the display frame from changing between frames. In addition, when a face or eyes of a person different from the face or eyes of the person currently being detected are selected, when eyes different from the currently detected eyes are selected, when eyes are detected after a predetermined time has elapsed since the eyes were detected in the previous frame, or when the detected position of the eyes has shifted significantly, instead of averaging the position and size of the eyes detected in the current frame and the position and size of the eyes detected previously, averaging may be performed on frames captured after the new frame.

[0136] (Modification Example 8) Whether to display a display frame on the face or on the eyes may be changed depending on whether the size of the detected face is larger or smaller than a reference value. Also, it may be changed when changing from displaying a display frame on the face to displaying a display frame on the eyes and vice versa. For example, when the size of the face is 50 pixels (picture elements) or more in the size in the captured image data, a display frame is displayed on the eyes, and when the size of the face is less than 50 pixels (picture elements) in the size in the captured image data, a display frame is displayed on the face. If a display frame is being shown on the face in the previous frame, when the size of the face becomes 55 pixels (picture elements) or more in size in the captured image data, change to showing an eye display frame from the face display frame. Conversely, if an eye display frame is being shown in the previous frame, when the size of the face becomes 45 pixels or less in size in the captured image data, change to showing a face display frame from the eye display frame. That is, the criteria for the size of the face when changing from showing a face display frame to showing an eye display frame and the criteria for the size of the face when changing from showing an eye display frame to showing a face display frame may be made different. This can prevent the face display frame and the eye display frame from being frequently changed between images that are continuously displayed over time when the size of the eyes changes somewhat.

[0137] (Modification Example 9) In the above-described embodiments, an example of detecting a face and eyes and detecting the focusing state of the detected face and eyes was shown, but it is not limited to this. For example, a part of an image selected by the user from the captured and acquired image may be registered as a feature object, the registered feature object may be detected from the image data, and a configuration for detecting the focusing state may be used. Alternatively, in addition to the eyes, objects such as lips, soccer balls, food, and rings that can be detected may be made configurable with the camera. Also, when the camera has shooting modes corresponding to a plurality of scenes, the object for detecting the focusing state may be changed according to the shooting mode. Specifically, when the shooting mode of the camera is the portrait mode, the eyes may be detected and the focusing state of the detected eyes may be detected, and when it is the sports mode, the ball may be detected and the focusing state of the detected ball may be detected, and so on. In the case of food, a birthday plate of a cake, candles, or a main dish may be detected.

[0138] The present invention is not limited to the above-described content. Other aspects conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

[0139] The disclosure content of the following priority basis application is incorporated herein by reference. Japanese Patent Application No. 2018-248622 (filed on December 28, 2018)

Description of Reference Signs

[0140] 1…Camera system, 2…Camera body, 3…Interchangeable lens, 230…Body-side control unit, 235…Memory unit, 40, 41, 51, 59, 71, 81, 84A, 133…Display frame, 52, 53, 72, 73, 82, 83, 85…Triangle mark, 270…Signal processing unit, 280, 281 - 284…Operating member, 290…Display unit, 290A…Touch operating member, 360…Photographing optical system, 370…Lens driving unit

Claims

1. A detection unit that detects eyes from an image, A selection unit that selects one eye from the eyes detected by the detection unit, An operation unit that receives an operation by a user to select an eye different from the eye selected by the selection unit, A display unit that displays the image, and The operation unit is provided at a position different from the display unit, The display unit displays, together with the image, a display indicating a first direction and a second direction in the image that can receive an instruction by the operation unit, The operation unit can receive a first operation indicating the first direction and a second operation different from the first operation and indicating the second direction. An electronic device.

2. A display unit that displays an image, A detection unit that detects eyes from the image, A selection unit that selects one eye from the eyes detected by the detection unit, An operation unit that receives an operation by a user to select an eye different from the eye selected by the selection unit, and The operation unit receives an operation indicating a direction in the image, The operation unit is provided at a position different from the display unit, When the selection unit selects one eye from the eyes detected by the detection unit as a first candidate and then receives an operation by the operation unit, based on the received operation, the selection unit selects an eye different from the first candidate as a second candidate, The selection unit selects, as the second candidate, another eye included in the same face as the first candidate's eye when a plurality of eyes are detected in the direction received by the operation unit. An electronic device.

3. The selection unit according to claim 2, wherein when a plurality of eyes are detected in the direction indicated by the operation unit, regardless of the distance in the direction from the first candidate's eye, the selection unit selects another eye included in the same face as the first candidate's eye as the second candidate. The electronic device described.

4. The selection unit according to claim 2 or 3, wherein when there is no other eye included in the same face as the first candidate's eye in the direction indicated by the operation unit, the selection unit selects an eye of a face different from the face having the first candidate's eye as the second candidate. The electronic device described.

5. The selection unit according to claim 2 or 3, wherein when there is no other eye included in the same face as the first candidate's eye in the direction indicated by the operation unit, the selection unit selects an eye of a face with the closest distance to the face having the first candidate's eye as the second candidate. The electronic device described.

6. The selection unit selects, as the second candidate, an eye of a face that is closest in distance in the direction to the face having the eye of the first candidate when there is no other eye included in the same face as the eye of the first candidate in the direction indicated by the operation unit. The electronic device according to claim 2 or 3.

7. The selection unit selects, as the second candidate, an eye included in a face different from the face having the eye of the first candidate and located on the side opposite to the direction indicated by the operation unit. The electronic device according to claim 5 or 6.

8. The operation for indicating the direction in the image is an operation for indicating either side when the image is bisected. The electronic device according to any one of claims 2 to 7.

9. After the selection unit selects one eye from the eyes detected by the detection unit as the first candidate, when the selection unit receives an operation at the operation unit, based on the received operation, the selection unit selects, as the second candidate, one eye located on the side of the direction of the image from the center of the face having the eye of the first candidate. The electronic device according to any one of claims 2 to 8.

10. An imaging element that captures an image by an optical system and outputs a signal for generating the image, A control unit that instructs control to move the optical system so that the image of the subject corresponding to the eye selected by the selection unit is focused on the imaging element. The electronic device according to any one of claims 1 to 9, comprising:

11. The electronic device is a camera. The electronic device according to any one of claims 1 to 10.

12. Detect eyes from an image, Select one eye from the detected eyes, Together with the image, cause a display unit to display a display indicating a first direction and a second direction in the image that can receive an instruction by a user's operation, A program executed by a computer that receives, via an operation unit provided at a position different from the display unit, as a user operation for selecting an eye in the image, a first operation for indicating the first direction and a second operation for indicating the second direction, which is different from the first operation.

13. Detect eyes from an image, Select one eye from the detected eyes as a first candidate, Receive, via an operation unit provided at a position different from the display unit, a user operation for indicating a direction in the image displayed on the display unit as an operation for selecting an eye different from the selected eye. After selecting the first candidate and receiving a user operation, based on the received operation, a different eye from the first candidate is selected as the second candidate. A program executed by a computer that, when a plurality of eyes are detected in the received direction, selects another eye included in the same face as the eye of the first candidate as the second candidate.

14. The electronic device according to any one of claims 1 to 11, wherein the display unit is an electronic viewfinder.

15. The display unit performs a first display on the eye selected by the selection unit. The electronic device according to claim 1, wherein the display unit performs a display indicating the first direction and the second direction together with the first display.

16. The electronic device according to claim 1, wherein an eye of a face different from the face having the eye selected by the selection unit can be selected by an operation of the operation unit.

Images