Imaging device, imaging method, and program

Abstract

The present invention relates to an imaging device, an imaging method and a program that enable the acquisition of an image of a quality that the user desires. This imaging device sets a first region, which includes a first specific part of a subject, as a focusing region for focusing. Moreover, if the first specific part was not detected, the image capturing device sets, as a focusing region, a second region which includes a second specific part of the subject, the second specific part being of the same type as the first specific part. This technology can be applied to imaging devices.

Description

[Technical Field]

[0001] The present technology relates to an imaging device, an imaging method, and a program, and more particularly to an imaging device, an imaging method, and a program that enable a user to obtain an image of a desired quality. [Background technology]

[0002] Patent Document 1 proposes an imaging device that detects a face area, which is the area of ​​a person's face, and if a pupil area, which is the area of ​​the pupil, can be detected from the detected face area, focuses on the pupil area, and if a pupil area cannot be detected, focuses on the face area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2015 / 045911 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when capturing an image of an animal, the imaging device detects the pupil region without detecting the animal's face region. Therefore, if the imaging device cannot detect the animal's pupil region, there is no object to focus on, making it difficult for the user to obtain an image of the quality desired.

[0005] The present technology has been made in view of such circumstances, and enables a user to obtain an image of the quality desired. [Means for solving the problem]

[0006] An imaging device according to one aspect of the present technology includes a setting unit that sets a first region including a first specific part of a subject as a focus region to be focused on, and if the first specific part is not detected, the setting unit sets a second region including a second specific part of the subject, which is a specific part of the same type as the first specific part, as the focus region.

[0007] In one aspect of the present technology, a first region including a first specific part of a subject is set as a focus region to be focused on, and if the first specific part is not detected, a second region including a second specific part of the subject that is the same type as the first specific part is set as the focus region. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating an example of the configuration of an imaging device according to an embodiment of the present technology. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a digital signal processing unit. [Figure 3] 10A and 10B are diagrams showing display examples in an pupil detection mode for each subject; [Figure 4] 10 is a diagram showing some of the settings relating to the pupil detection mode of the imaging device 11. FIG. [Figure 5] FIG. 10 is a diagram illustrating an example of internal processing in an animal pupil detection mode. [Figure 6] 10A and 10B are diagrams illustrating an example of internal processing when a setting for selecting left or right pupil in the person's pupil detection mode is applied. [Figure 7] 10A and 10B are diagrams illustrating an example of internal processing when the left / right pupil selection setting of the present technology is applied in the animal pupil detection mode. [Figure 8] 10 is a flowchart illustrating an image capturing process in a person's pupil detection mode. [Figure 9] 9 is a flowchart illustrating the image capturing process in the person's pupil detection mode, following FIG. 8. [Figure 10] 10 is a flowchart illustrating the image capturing process in the person's pupil detection mode, following FIG. 9. [Figure 11]10 is a flowchart illustrating an imaging process in an animal pupil detection mode. [Figure 12] 12 is a flowchart illustrating the imaging process in the animal pupil detection mode, following FIG. 11. [Figure 13] 13 is a flowchart illustrating the imaging process in the animal pupil detection mode, following FIG. 12. [Figure 14] 10A and 10B are diagrams illustrating an example of selection of pupil regions when left / right pupil selection is set to auto. [Figure 15] 10 is a flowchart illustrating a bird's pupil region selection process. [Figure 16] 16 is a flowchart illustrating the bird's pupil region selection process, following FIG. 15. [Figure 17] 17 is a flowchart illustrating the bird's pupil region selection process, following FIG. 16. [Figure 18] FIG. 10 is a diagram showing an example of a detection target switching setting screen. [Figure 19] FIG. 10 is a diagram showing an example of a display screen when the detection target is switched. [Figure 20] FIG. 10 is a diagram showing a hierarchical structure of object types. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present technology will be described in the following order. 1. Imaging device 2.Eye detection mode for each subject 3. Animal Eye Detection Mode 4. Operation of the imaging device 5.Other

[0010] <1. Imaging device> <Configuration example of imaging device> FIG. 1 is a block diagram showing an example of the configuration of an imaging device according to an embodiment of the present technology.

[0011] 1 has a specific part detection mode for detecting an area in focus on a specific part of a subject such as a person, animal, or bird. The specific part may be a face or an eye. Examples of the detection mode for each specific part of a subject include a human eye detection mode, an animal eye detection mode, and a bird eye detection mode.

[0012] It should be noted that the imaging device 100 can provide detection modes for specific parts other than the pupils of a person or an animal. The user can select and set a desired detection mode from among the detection modes for the subject or the specific part of the subject.

[0013] 1, the imaging device 100 is configured to include a lens 101, an aperture 102, an imaging element 103, an analog signal processing unit 104, an A / D conversion unit 105, and a digital signal processing unit 106. The imaging device 100 is configured to include a lens driver 121, a TG (Timing Generator) 122, a gyro (sensor) 123, and a system controller 131.

[0014] The imaging device 100 is configured to include a display unit 141 , a storage unit 142 , an input unit 143 , an output unit 144 , a communication unit 145 , an operation unit 146 , and a drive 147 .

[0015] The lens 101 adjusts the focus on the subject and collects light from the subject. The diaphragm 102 adjusts the exposure.

[0016] The image sensor 103 captures an image of a subject to obtain a captured image. That is, the image sensor 103 photoelectrically converts light from the subject and outputs the converted image signal to the analog signal processor 104. The image sensor 103 can capture both still images and moving images through such photoelectric conversion.

[0017] An analog signal processing unit 104 performs analog signal processing on the image signal obtained by the image sensor 103. An A / D conversion unit 105 A / D converts the image signal that has undergone analog signal processing, and obtains image data that is a digital signal.

[0018] Digital signal processing unit 106 performs digital signal processing on the image data obtained by A / D conversion unit 105. As part of its digital signal processing, digital signal processing unit 106 performs at least processing to detect the area of ​​the subject or a specific part of the subject from the moving image supplied as image data, and to set a focus area, which is an area to focus on. Hereinafter, the specific part of the subject will be simply referred to as the specific part.

[0019] In addition, the digital signal processing unit 106 also performs processing to set one of the areas as the focus area based on the detection results of the area of ​​the subject or specific part, and to control the display of a focus preview frame that notifies the area that will be set as the focus area before focusing begins, and a focus frame that indicates the focus area after focusing begins.

[0020] The digital signal processing may be of any type, and processing other than that described above may also be performed. For example, the digital signal processing unit 106 may perform color mixing correction, black level correction, white balance adjustment, demosaic processing, matrix processing, gamma correction, YC conversion, etc. as digital signal processing. Furthermore, the digital signal processing unit 106 may perform codec processing, which is processing related to encoding and decoding of image data, as digital signal processing.

[0021] The lens driver 121 drives the lens 101 and the aperture 102 to control the focal length, exposure, etc. The TG 122 generates a synchronization signal and supplies it to the image sensor 103 to drive the image sensor 103 and control imaging. The gyro 123 is a sensor that detects the position and orientation of the image sensor 100. The gyro 123 outputs information indicating the position and orientation of the image sensor 100 to the A / D conversion unit 105.

[0022] The system controller 131 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and executes programs and processes data to control each processing unit of the imaging device 100. The system controller 131 also receives operation input from a user based on a signal supplied from an operation unit 146, and performs processing or control corresponding to the operation input.

[0023] For example, the system controller 131 can control the focal length or exposure based on information indicating the focus area supplied from the digital signal processor 106.

[0024] The display unit 141 is configured, for example, by a liquid crystal display or the like, and displays an image corresponding to image data stored in the memory of the digital signal processing unit 106. For example, the display unit 141 can display an image captured by the image sensor 103, a stored captured image, and the like.

[0025] The storage unit 142 stores image data stored in the memory of the digital signal processing unit 106. At that time, the storage unit 142 stores coded data coded by the digital signal processing unit 106 in order to reduce the amount of data. The coded data stored in the storage unit 142 is read and decoded by the digital signal processing unit 106 and displayed on the display unit 141, for example.

[0026] The input unit 143 has an external input interface such as an external input terminal, and outputs various data (for example, image data and coded data) supplied from outside the imaging device 100 via the external input interface to the digital signal processing unit .

[0027] The output unit 144 has an external output interface such as an external output terminal, and outputs various data supplied via the digital signal processing unit 106 to the outside of the imaging device 100 via the external output interface.

[0028] The communication unit 145 performs predetermined communication with other devices, which is at least one of wired communication and wireless communication, and transmits and receives data to and from other devices via the predetermined communication. For example, the communication unit 145 transmits various data (e.g., image data and encoded data) supplied from the digital signal processing unit 106 to other devices via the predetermined communication. The communication unit 145 also receives various data from other devices via the predetermined communication and outputs the received data to the digital signal processing unit 106.

[0029] The operation unit 146 is configured with any input device such as, for example, keys, buttons, or a touch panel. The operation unit 146 includes, for example, a shutter button. The operation unit 146 receives an operation input by a user and outputs a signal corresponding to the operation input to the system controller 131.

[0030] The drive 147 reads information (programs, data, etc.) stored in a removable recording medium 148, such as a semiconductor memory, attached to the drive 147. The drive 147 supplies the information read from the removable recording medium 148 to the system controller 131. Furthermore, when a writable removable recording medium 148 is attached to the drive 147, the drive 147 can store information (image data, encoded data, etc.) supplied via the system controller 131 in the removable recording medium 148.

[0031] The lens 101, aperture 102, and lens driver 121 described above may be formed as an interchangeable lens 151 that is detachable (interchangeable) from the imaging device 100 in a separate housing from the imaging device 100.

[0032] <Configuration example of digital signal processing unit> FIG. 2 is a block diagram showing an example of the configuration of the digital signal processing unit 106.

[0033] The digital signal processing unit 106 includes a memory 211 , an object detection unit 212 , an object tracking unit 213 , an area setting unit 214 , a display control unit 215 , and a codec processing unit 216 .

[0034] The memory 211 stores image data supplied from the A / D conversion unit 105. The image data is, for example, image data of each frame of a moving image or image data of a still image.

[0035] The subject detection unit 212 detects the area of ​​the subject or specific body part from the image data stored in the memory 211, based on a signal corresponding to a user's operation input supplied from the system controller 131. The subject detection unit 212 outputs the detection result of the area of ​​the subject or specific body part to the area setting unit 214 and the display control unit 215.

[0036] The subject detection unit 212 is made up of a person detection unit 212-1, an animal detection unit 212-2, and a bird detection unit 212-3.

[0037] When the detection mode for each specific part of the subject is set to a person's pupil detection mode, the person detection unit 212-1 detects a person's face area and outputs the detection result of the detected face area to the area setting unit 214 and the display control unit 215. When the user performs an operation to instruct the start of focusing, the person detection unit 212-1 detects the person's pupil area based on the detection result of the face area and outputs the detection result of the pupil area to the area setting unit 214 and the display control unit 215.

[0038] When the detection mode for each specific part of the subject is set to the animal pupil detection mode, the animal detection unit 212-2 detects the animal pupil area and outputs the detection result of the detected animal pupil area to the area setting unit 214 and the display control unit 215.

[0039] To detect the animal's pupil, for example, a method such as deep learning using an animal dictionary is used. There are two types of animal dictionaries: one for frontal orientation and one for sideways orientation. When the user performs an operation to instruct the start of focusing, the animal detection unit 212-2 detects the animal's pupil region according to the focus frame and outputs the detection result of the detected animal's pupil region to the region setting unit 214 and the display control unit 215.

[0040] The focus frame is a frame used by the user to set a rough focus position. When a focus frame is set, area detection processing is performed with priority from the inside of the focus frame.

[0041] Examples of animal types include, but are not limited to, dogs, cats, lions, etc. Note that, for example, different animal detection units may be provided for different types of animals that have similar characteristics to be detected.

[0042] When the detection mode for each specific part of the subject is set to the bird's pupil detection mode, the bird detection unit 212-3 detects the bird's pupil area and outputs the detection result of the detected bird's pupil area to the area setting unit 214 and the display control unit 215.

[0043] The bird's pupil is detected, for example, by a method such as deep learning using a bird dictionary. When a user performs an operation to instruct the start of focusing, the bird detection unit 212-3 detects the bird's pupil region according to the focus frame and outputs the detection result of the detected bird's pupil region to the region setting unit 214 and the display control unit 215.

[0044] Subject tracking section 213 tracks the in-focus area set by area setting section 214. For example, tracking is performed on each area set as the in-focus area by area setting section 214 (a detected or tracked face area, pupil area, or an area specified by a normal AF (autofocus) frame, which will be described later). Subject tracking section 213 outputs information indicating the tracked in-focus area to area setting section 214 and display control section 215 as a tracking result.

[0045] Area setting section 214 sets, as a focus area, an area of ​​a specific part of the subject detected by subject detection section 212 according to the detection mode for each specific part of the subject. If subject detection section 212 does not detect an area of ​​a specific part of the subject, area setting section 214 uses the tracking result of the specific part of the subject tracked by subject tracking section 213 instead.

[0046] If the area of ​​a specific part of the subject is not detected and cannot be tracked, the area setting unit 214 sets the area set as the focus area in normal AF (autofocus) mode as the focus area, rather than in the detection mode for each specific part of the subject. This area can be moved by the user and can be specified using the AF frame in normal AF mode (hereinafter referred to as the normal AF frame). The area setting unit 214 supplies information about the set focus area to the system controller 131.

[0047] The display control unit 215 generates a focus frame in response to a signal corresponding to a user's operation input supplied from the system controller 131, and displays the focus frame on the display unit 141 by superimposing it on the image from the memory 211. Information about the focus frame is output to the subject detection unit 212.

[0048] Furthermore, the display control unit 215 generates a focusing preview frame, an in-focus frame, a focus frame, a normal AF frame, etc. based on a signal corresponding to a user's operation input supplied from the system controller 131. The display control unit 215 causes the generated focusing preview frame, in-focus frame, focus frame, and normal AF frame to be superimposed on an image from the memory 211 and displayed on the display unit 141. Information on the focusing preview frame, in-focus frame, focus frame, and normal AF frame is output to the subject detection unit 212 as necessary.

[0049] The display control unit 215 generates images of a GUI (Graphical User Interface) such as a menu, button, or cursor, and displays them together with the captured image, the photographed image, and the like.

[0050] The codec processing unit 216 performs processing related to encoding and decoding of image data of moving images and still images stored in the memory 211 .

[0051] <2. Eye detection mode for each subject> <Outline of eye detection modes for each subject> FIG. 3 is a diagram showing a display example when the pupil detection mode for each subject is set.

[0052] 3 shows, from top to bottom, examples of screens that are displayed on the display unit 141 when the human pupil detection mode, animal pupil detection mode, and bird pupil detection mode of the imaging device 11 are set.

[0053] 3 are screens that are displayed before the user performs an operation to instruct the start of focusing, and screens B1 to B3 on the right side are screens that are displayed after the user performs an operation to instruct the start of focusing.

[0054] The human eye detection mode, animal eye detection mode, and bird eye detection mode are switched by the user.

[0055] When the person's eye detection mode is set, as shown on screen A1, a face area that overlaps with the focus frame F is detected from the face area that is the area of ​​the person's face, and a focusing preview frame PF is displayed in the detected face area. In the example of screen A1, the focusing preview frame PF is displayed so that it partially overlaps with the focus frame F.

[0056] In this state, if the user performs an operation to instruct the start of focusing, as shown on screen B1, the pupil area is detected within the face area, and the detected pupil area is set as the in-focus area. Then, a focusing frame AF is displayed in place of the focusing preview frame PF within the in-focus area, and focusing is performed within the in-focus area surrounded by the focusing frame AF. Note that, as will be described in more detail later, in human pupil detection mode, if the pupil area is not detected, the detected face area is set as the hand-held area, and a hand-held frame is displayed.

[0057] When the animal pupil detection mode is set, as shown on screen A2, the animal's pupil area is detected both inside and outside the focus frame F, and the focus preview frame PF is displayed in the detected pupil area. Note that at this time, the detection of the pupil area is performed with priority given to the area inside the focus frame F.

[0058] In this state, if the user performs an operation to instruct the start of focusing, the pupil area is set as the in-focus area, and a focusing frame AF is displayed in place of the focusing preview frame PF, as shown on screen B2. Furthermore, focusing is performed on the pupil area surrounded by the focusing frame AF. Note that, as will be described in more detail later, in animal pupil detection mode, if one pupil area is not detected, the other detected pupil area is set as the hand-held area, and a hand-held frame is displayed.

[0059] When bird's eye detection mode is set, as shown on screen A3, bird's eye regions are detected both inside and outside the focus frame F, and a focus preview frame PF is displayed in the detected eye region. Note that eye region detection is performed with priority given to the area inside the focus frame F.

[0060] In this state, if the user performs an operation to instruct the start of focusing, the pupil area is set as the in-focus area, and the in-focus frame AF is displayed in place of the focusing preview frame PF, as shown on screen B3. Furthermore, focusing is performed on the pupil area surrounded by the in-focus frame AF.

[0061] In bird's eye detection mode, if the left and right pupil areas are detected, a focus preview frame PF is displayed in the pupil area closest to the imaging device, which is set as the in-focus area, and an in-focus frame AF is displayed. Note that, as will be described in detail later, in bird's eye detection mode, if the pupil area is not detected, an area preset by the user is set as the hand-held area, and a hand-held frame is displayed.

[0062] Furthermore, for example, the focusing preview frame PF is displayed in a white frame, and the in-focus frame AF is displayed in a green frame, so that the in-focus frame AF is displayed in a different display method from the in-focus preview frame PF.

[0063] As described above, when the human pupil detection mode is set, a face area is detected before the user performs an operation to instruct the start of focusing, and a focusing preview frame PF is displayed in the detected face area.

[0064] On the other hand, when the animal or bird pupil detection mode is set, the pupil area is detected before the user performs an operation to instruct the start of focusing, and the focusing preview frame PF is displayed in the detected pupil area.

[0065] By displaying the focus preview frame before the user performs an operation to instruct the start of focusing, the user can know in advance whether the intended or unintended position has been detected, and can therefore choose not to perform autofocus. This makes it easy to capture an image focusing on a specific part of the subject.

[0066] <Settings related to eye detection mode> FIG. 4 is a diagram showing some of the settings related to the pupil detection mode of the imaging device 11.

[0067] FIG. 4 shows that the detection target in the pupil detection mode can be set to person, animal, bird, or auto.

[0068] When a person is set as the detection target, the person's face or eyes are detected in the eye detection mode. When an animal is set as the detection target, the animal's eyes are detected in the eye detection mode. When a bird is set as the detection target, the bird's eyes are detected in the eye detection mode.

[0069] When auto is set as the detection target, the eyes of the subject are detected in a separately set priority order. For example, if the priority order of people, animals, and birds is set, the eyes of people are detected, and if the eyes of people are not detected, the eyes of animals are detected, and if the eyes of people and animals are not detected, the eyes of birds are detected, and so on.

[0070] FIG. 4 also shows that auto, right pupil, and left pupil can be set as left or right pupil selection in the pupil detection mode.

[0071] When auto is set as the left / right pupil selection in the pupil detection mode, the pupil to be focused on is left to the imaging device 111. When the right pupil is set as the left / right pupil selection in the pupil detection mode, the right pupil is the focus target. When the left pupil is set as the left / right pupil selection in the pupil detection mode, the left pupil is the focus target.

[0072] Even when left / right pupil selection is set in these pupil detection modes, the user can temporarily switch the pupil to be focused on by operating a predetermined button, etc. Note that, hereinafter, the operation by the user to temporarily switch the pupil to be focused on will be referred to as manual designation.

[0073] The above left / right pupil selection is performed in the human pupil detection mode. In the animal pupil detection mode, left / right pupil priority selection is performed. Here, "priority" means that "if both the right and left pupils can be selected, the specified pupil will be the focus target."

[0074] Furthermore, even when left / right pupil (priority) selection is set for these pupil detection modes, it is possible to temporarily switch the pupil to be focused on according to manual selection by the user. Note that in the case of animal pupil detection mode, if both the right pupil and the left pupil can be selected, the specified pupil can be used as the focus target.

[0075] As described above, the operation relating to left / right pupil selection differs between the human pupil detection mode and the animal pupil detection mode.

[0076] The animal eye detection mode will be described in detail below.

[0077] 3. Animal Eye Detection Mode <Internal processing overview> FIG. 5 is a diagram showing an example of internal processing in the animal pupil detection mode.

[0078] In the animal pupil detection mode, the detection process and tracking process of the animal pupil (area) are carried out in parallel.

[0079] 5, from top to bottom, the display screen displayed on the display unit 141, the results of detecting the animal's pupil, and the results of tracking the animal's pupil are shown over time. On each display screen, the animal (fox) that is the subject of pupil detection is displayed facing forward, and the focusing frame AF is displayed when the focusing area is set based on the results of detecting or tracking the fox's pupil at each timing.

[0080] Here, "Detection OK" indicates that the pupil was successfully detected by the detection process, while "Detection NG" indicates that the pupil could not be detected by the detection process due to various reasons such as the size of the subject's movement, blur, insufficient detection performance, etc. "Tracking OK" indicates that the pupil was successfully tracked by the tracking process.

[0081] Furthermore, the frame P displayed for either the detection result or the tracking result indicates whether the focus area is set based on the detection result or the tracking result at that timing, and the focus frame AF is displayed based on that result.

[0082] The following will be explained in chronological order. At timing t1 in Figure 5, an in-focus frame AF is displayed on the left eye of the fox on the display screen. At timing t1, the fox's left eye is successfully detected and tracked. In other words, the in-focus frame AF displayed at timing t1 is displayed with the in-focus area set based on the detection result of the fox's left eye.

[0083] At time t2, an in-focus frame AF is displayed on the fox's left eye on the display screen. At time t2, the fox's left eye was not detected, but was tracked successfully. In other words, the in-focus frame AF displayed at time t2 is displayed with the in-focus area set based on the tracking results of the fox's left eye.

[0084] At time t3, an in-focus frame AF is displayed on the fox's left eye on the display screen. At time t3, the fox's left eye was not detected, but tracking was successful. In other words, the in-focus frame AF displayed at time t3 is displayed with the in-focus area set based on the tracking results of the fox's left eye.

[0085] At time t4, an in-focus frame AF is displayed on the display screen over the fox's left eye. At time t4, the fox's left eye is successfully detected and tracked. In other words, the in-focus frame AF displayed at time t4 is displayed with the in-focus area set based on the detection result of the fox's left eye.

[0086] As described above, in animal pupil detection mode, if the animal's pupil is detected successfully, the focus area is set based on the detection results and the focus frame AF is displayed; if the animal's pupil is not detected but tracking is successful, the focus area is set based on the tracking results and the focus frame AF is displayed.

[0087] That is, although the display screen remains almost unchanged, at times t2 and t3 the in-focus frame AF is displayed with the in-focus area set based on the tracking results, not the detection results of the animal's eye. Note that at these times, the detection accuracy of the eye on the opposite (right) side to the in-focus frame AF is not guaranteed.

[0088] However, whether the in-focus frame AF is displayed based on the detection result or the tracking result is an internal process, and is therefore not known to the user.

[0089] <Internal processing when the left / right pupil selection setting in the human eye detection mode is applied> FIG. 6 shows an example of internal processing when the left / right pupil selection setting in the human pupil detection mode is applied in the animal pupil detection mode.

[0090] 6, similarly to Fig. 5, from top to bottom, the display screen displayed on the display unit 141, the detection results of the animal's pupils, and the tracking results of the animal's pupils are shown over time. Furthermore, in Fig. 6, a setting for selecting the left or right pupil has been added below the tracking results.

[0091] As described above, the pupil to be focused on can be set in advance as left or right pupil selection in the imaging device 11. Furthermore, the pupil to be focused on can also be switched during imaging processing in response to manual designation by the user.

[0092] At timing t11 in Figure 6, an in-focus frame AF is displayed on the left eye of the fox on the display screen. At timing t11, the left eye is selected as the focus target, and the fox's left eye is successfully detected and tracked. In other words, the in-focus frame AF displayed at timing t11 is displayed with the in-focus area set based on the detection result of the fox's left eye.

[0093] Here, the imaging device 11 switches the focus target to the right pupil in response to a manual instruction from the user.

[0094] At times t12 and t13, a focus frame E is displayed near the fox's nose on the display screen. At times t12 and t13, the focus target is switched to the right eye. However, the right eye cannot be detected or tracked. However, the left eye, which continues to be tracked, can be tracked.

[0095] Although the fox's left eye can be tracked, the switched right eye cannot be detected or tracked, so the in-focus area is set to the area that would be set as the in-focus area in normal AF mode, as described above. That is, the in-focus frame E displayed at times t12 and t13 is the area that would be set as the in-focus area in normal AF (the area located near the fox's nose in Figure 6) that has been set as the in-focus area and displayed.

[0096] In other words, if the timing of switching to the right pupil makes it impossible to detect or track the right pupil, the operation will be in normal AF mode, and the operation in pupil detection mode will not be performed.

[0097] Then, at timing t14, an in-focus frame AF is displayed on the display screen over the fox's right eye. At timing t14, the right eye is selected as the focus target, the detection and tracking processes for the right eye return, and the fox's right eye is successfully detected and tracked. In other words, the in-focus frame AF displayed at timing t14 is displayed with the in-focus area set based on the detection results of the right eye.

[0098] As described above, there are many times when the tracking process in animal eye detection mode cannot respond to changes in the pupil. Furthermore, in human eye detection mode, if an eye cannot be detected, the camera is set to focus on the detected face, but in animal eye detection mode, no face is detected, so normal AF operation occurs. Furthermore, when the depth of field is shallow, once the camera has focused on the nose, the pupil becomes blurred, making it difficult to return the focus to the pupil.

[0099] For the above reasons, in the animal pupil detection mode, it is difficult to determine the timing for switching the pupil to be focused on to the opposite pupil.

[0100] Therefore, in the present technology, when left / right pupil selection is performed in the animal pupil detection mode, left / right pupil "priority" selection is executed.

[0101] <Internal processing for left / right pupil priority selection> FIG. 7 shows an example of internal processing when a setting for left / right pupil priority selection is applied in the animal pupil detection mode of the present technology.

[0102] In Figure 7, as in Figure 6, from top to bottom, the display screen displayed on the display unit 141, the detection results of the animal's pupil, the tracking results of the animal's pupil, and the settings for left / right pupil priority selection are shown over time.

[0103] At timing t21 in Figure 7, an in-focus frame AF is displayed on the left eye of the fox on the display screen. At timing t21, the left eye is selected as the focus target, and the fox's left eye is successfully detected and tracked. In other words, the in-focus frame AF displayed at timing t21 is displayed with the in-focus area set based on the detection result of the left eye.

[0104] Here, the imaging device 11 switches the focus target to the right pupil in response to a manual instruction from the user.

[0105] At times t22 and t23, an in-focus frame AF is displayed on the fox's left eye on the display screen. At times t22 and t23, the focus target is switched to the right eye. However, because this is a priority selection, if the timing is not right for switching, for example, because the right eye has not been detected, or if it has been detected but the detection accuracy is poor, the results of detecting and tracking the left eye before the switch are used.

[0106] That is, at times t22 and t23, the fox's right eye cannot be detected and tracked, and the left eye cannot be detected but can be tracked. Therefore, the in-focus frame AF displayed at times t22 and t23 is displayed with the in-focus area set based on the tracking result of the left eye.

[0107] At timing t24, an in-focus frame AF is displayed on the right pupil of the fox on the display screen. At timing t24, the focus target has been switched to the right pupil in response to manual designation by the user.

[0108] That is, at time t24, the fox's right eye is detected and tracked successfully, and the focusing target can be switched. Therefore, the focusing frame AF displayed at time t24 is displayed with the focusing area set based on the detection result of the right eye.

[0109] As described above, in the animal pupil detection mode, when the timing is right to switch to the pupil on the side on which priority selection is set or the side on which manual selection is specified, the pupil on the side on which priority selection is set or the side on which manual selection is specified is used for focusing.

[0110] On the other hand, when the timing is such that it is not possible to switch to the pupil on the side on which priority selection is set or the side on which manual selection is specified, the pupil on the side on which priority selection is not set or the side on which manual selection is not specified is used for focusing.

[0111] This makes it possible to obtain high-quality images while taking into consideration the user's operations.

[0112] <4. Operation of the imaging device> <Image capture processing in human eye detection mode> 8 to 10 are flowcharts illustrating the imaging process of the imaging device 100 in the person's pupil detection mode.

[0113] 8 to 10 is started when the power is turned on by operating the power button, for example. The detection mode for each specific part of the subject is set in advance as the human pupil detection mode from a setting screen or the like.

[0114] 8, the imaging device 11 acquires and displays an image. Specifically, the imaging element 103 acquires an electrical signal for each pixel by photoelectrically converting light from a subject collected via the lens 101 and the aperture 102 on a pixel-by-pixel basis. The image signal consisting of the electrical signals from each pixel is passed through the analog signal processing unit 104 and the A / D conversion unit 105 to generate digital image data, which is then stored in the memory 211 of the digital signal processing unit 106.

[0115] The display control unit 215 causes the display unit 141 to display an image based on the image data stored in the memory 211 as a live view image.

[0116] In step S12, the person detection unit 212-1 detects a face area from the image data stored in the memory 211. The person detection unit 212-1 supplies information on the detected face area to the area setting unit 214 and the display control unit 215.

[0117] In step S13, the person detection unit 212-1 detects the pupil area in the face area within the focus frame under the control of the system controller 131, and outputs the detection result (information on the pupil area) to the area setting unit 214 and the display control unit 215.

[0118] In step S14, the region setting unit 214 determines whether or not the pupil has been detected. If it is determined in step S14 that the pupil has been detected, the process proceeds to step S15.

[0119] In step S15, the region setting unit 214 uses the pupil detection result to perform the subsequent processing for setting the focus region.

[0120] In step S16, the region setting unit 214 determines whether the setting of left / right pupil selection is automatic. If it is determined in step S16 that the setting of left / right pupil selection is automatic, the processing proceeds to step S17.

[0121] In step S17, the region setting unit 214 sets the face region as the focus region. After step S17, the process proceeds to step S18 in FIG.

[0122] In step S18, subject tracking section 213 tracks the in-focus area set by area setting section 214, and outputs the tracking result to area setting section 214 and display control section 215. The tracking process in step S18 is performed on the face area set as the in-focus area.

[0123] The user issues a command to start focusing by pressing a predetermined button or half-pressing the shutter button. The command to start focusing is issued for each image capture unit (frame). The operation unit 146 receives an operation input by the user and outputs a signal corresponding to the operation input to the system controller 131.

[0124] In step S19, the system controller 131 determines whether or not a focus start command has been issued. If it is determined in step S19 that a focus start command has been issued, the process proceeds to step S20.

[0125] In step S20, the system controller 131 controls the lens driver 121 to drive the optical system, such as the lens 101 and the diaphragm 102, so that the focus is on the foreground pupil within the focus area. That is, there are cases where both left and right pupils are detected within the focus area. In this case, in the person pupil detection mode, control is performed so that the focus is on the foreground pupil as seen from the image capture device 100. The area setting unit 214 identifies the foreground pupil.

[0126] In step S21, the region setting unit 214 determines whether or not the foreground pupil has been identified. If it is determined in step S21 that the foreground pupil has been identified, the process proceeds to step S22.

[0127] In step S22, the region setting unit 214 changes the in-focus region to the front pupil. After step S22, the process proceeds to step S35.

[0128] If it is determined in step S21 that the pupil in the foreground could not be identified, the process proceeds to step S35, that is, the in-focus area remains the face area and is not changed.

[0129] If it is determined in step S19 that a focus start command has not been issued, the process proceeds to step S36.

[0130] Returning to FIG. 8, if it is determined in step S16 that the left / right pupil selection setting is not automatic, the process proceeds to step S23.

[0131] In step S23, region setting unit 214 determines whether or not the pupil specified by the user is selectable. Note that, as described above, pupils specified by the user include pupils set by left / right pupil selection and pupils specified manually. For example, if the pupil specified by the user has been detected and the detection accuracy is high, it is determined in step S23 that the pupil specified by the user is selectable, and the processing proceeds to step S24.

[0132] In step S24, the region setting unit 214 sets the pupil region designated by the user as the focus region.

[0133] If it is determined in step S23 that the pupil specified by the user is not selectable, the process proceeds to step S26.

[0134] If it is determined in step S14 that the pupil has not been detected, the process proceeds to step S25.

[0135] In step S25, the area setting unit 214 determines whether or not a face has been detected. If it is determined in step S25 that a face has been detected, the process proceeds to step S .

[0136] In step S26, the region setting unit 214 sets the face region as the focus region.

[0137] If it is determined in step S25 that a face has not been detected, the process proceeds to step S27.

[0138] In step S27, the region setting unit 214 determines whether or not the in-focus region has been tracked. If it is determined in step S27 that the in-focus region has been tracked, the process proceeds to step S .

[0139] In step S28, the region setting unit 214 sets a focus region based on the tracking result.

[0140] After steps S24, S26, and S28, the process proceeds to step S32 in FIG.

[0141] On the other hand, if it is determined in step S27 that the in-focus area has not been tracked, the process proceeds to step S29 in FIG.

[0142] In step S29, the region setting unit 214 determines whether or not AF is set to be performed using a normal AF frame. If it is determined in step S29 that AF is set to be performed using a normal AF frame, the process proceeds to step S30.

[0143] In step S30, the area setting unit 214 sets the area specified by the normal AF frame as the in-focus area.

[0144] In step S31, the area setting unit 214 determines whether or not tracking is set to be performed using a normal AF frame. If it is determined in step S31 that tracking is set to be performed using an area specified by a normal AF frame, the process proceeds to step S32.

[0145] In step S32, subject tracking section 213 tracks the in-focus area and outputs the tracking result to area setting section 214 and display control section 215. Tracking here is performed on the areas set as in-focus areas in steps S24, S26, S28, and S30 (detected or tracked pupil area, face area, area specified by a normal AF frame). After step S32, the process proceeds to step S33.

[0146] If it is determined in step S31 that the setting is not such that tracking is performed using a normal AF frame, the process proceeds to step S33.

[0147] In step S33, the system controller 131 determines whether or not a focus start command has been issued. If it is determined in step S33 that a focus start command has been issued, the process proceeds to step S .

[0148] In step S34, the system controller 131 controls the lens driver 121 to drive the optical system, such as the lens 101 and the diaphragm 102, so that the focus is adjusted to the in-focus area. Then, the process proceeds to step S35.

[0149] In step S35, the display control unit 215 causes the display unit 141 to display, as a live view image, an image based on the image data stored in the memory 211. The display control unit 215 also causes the display unit 141 to display, superimposed on the live view image, an in-focus frame indicating the set in-focus area.

[0150] If it is determined in step S33 that the focus start instruction has not been issued, the process proceeds to step S36.

[0151] In step S36, the display control unit 215 generates a focusing preview frame indicating the area to be set as the in-focus area, and causes the display unit 141 to display the focusing preview frame superimposed on the live view image.

[0152] After steps S35 and S36, the process proceeds to step S37 in FIG.

[0153] Furthermore, if it is determined in step S29 that the setting is not to perform AF using a normal AF frame, the process proceeds to step S37 in Fig. 10. In this case, a frame indicating the in-focus area is not displayed on the live view image.

[0154] In step S37, the system controller 131 determines whether or not the shutter button has been fully pressed, based on a signal corresponding to an operation input from the operation unit 146. If it is determined in step S37 that the shutter button has been fully pressed, the process proceeds to step S38.

[0155] In step S38, the image sensor 103 obtains an electrical signal for each pixel of the image by photoelectrically converting, on a pixel-by-pixel basis, light from the subject that has been collected via an optical system such as the lens 101 and the aperture 102. The image signal, which is the electrical signal for each pixel of the image, is converted into digital image data via the analog signal processing unit 104 and the A / D conversion unit 105, and the digital image data is stored in the memory 211 of the digital signal processing unit 106 as captured image data.

[0156] In step S39, the display control unit 215 causes the display unit 141 to display an image based on the photographed image data stored in the memory 211 as a photographed image.

[0157] In step S40, the codec processing unit 216 encodes the captured image data stored in the memory 211. The codec processing unit 216 supplies the encoded image data to the storage unit 142.

[0158] In step S41, the codec processing unit 216 causes the storage unit 142 to store the encoded captured image data.

[0159] In step S42, the system controller 131 determines whether or not to end the process, for example, whether or not the power button has been operated.

[0160] If it is determined in step S42 that the process is to be ended, the image capturing process is ended.

[0161] If it is determined in step S42 that the process should not be ended, the process returns to step S11 in FIG. 8, and the subsequent processes are repeated.

[0162] As described above, in the person's pupil detection mode, if the pupil area cannot be detected, the face area is set as the in-focus area.

[0163] <Image capture processing in animal pupil detection mode> 11 to 13 are flowcharts illustrating the imaging process of the imaging device 100 in the animal pupil detection mode.

[0164] The imaging process in the animal pupil detection mode shown in Figures 11 to 13 starts when the power is turned on by operating the power button, for example. The detection mode for each specific part of the subject is set in advance as the animal pupil detection mode on a setting screen, etc. Figures 11 to 13 illustrate an example in which the animal pupils are detected by the animal detection unit 212-2 that detects the pupils of a cat or a dog.

[0165] In step S61 of FIG. 11, an image is acquired and displayed in the same manner as in step S11 of FIG.

[0166] In step S62, the animal detection unit 212-2 uses an animal dictionary to detect an pupil region from the image data stored in the memory 211. The animal detection unit 212-2 supplies the detection result (information on the detected pupil region) to the region setting unit 214 and the display control unit 215.

[0167] In step S63, the region setting unit 214 determines whether or not the pupil region has been detected. If it is determined in step S63 that the pupil region has been detected, the process proceeds to step S64.

[0168] In step S64, the region setting unit 214 uses the pupil detection result to perform the subsequent processing for setting the focus region.

[0169] In step S65, the region setting unit 214 determines whether the setting of left / right pupil selection is automatic. If it is determined in step S65 that the setting of left / right pupil selection is automatic, the processing proceeds to step S66.

[0170] In step S66, the region setting unit 214 automatically selects the pupil region and sets it as the in-focus region.

[0171] <Selecting the pupil area when using the auto setting> FIG. 14 is a diagram showing an example of selection of pupil regions when the left / right pupil selection in step S66 of FIG. 11 is set to auto.

[0172] 14 shows images P1 to P7 of foxes as subjects. The foxes in images P1 to P7 have different facial orientations (angles). The solid rectangles indicate the foreground pupil regions detected in each image, and the dashed rectangles indicate the background pupil regions detected in each image.

[0173] Image P1 shows a fox facing left. Image P2 shows a fox facing diagonally forward to the left. Image P3 shows a fox facing slightly diagonally forward to the left. Image P4 shows a fox facing straight ahead. Image P5 shows a fox facing slightly diagonally forward to the right. Image P6 shows a fox facing diagonally forward to the right. Image P7 shows a fox facing right.

[0174] Of these images, images P1, P2, P6, and P7, which show a fox facing diagonally forward or to the side (left and right), each show cases in which only the pupil area located in front of the imaging device 100 was detected.

[0175] In images P3 and P5, which show a fox facing slightly diagonally forward, two pupil regions are detected in each image, and it is easy to determine whether one pupil region is located closer to the image capture device 100 and the other pupil region is located further back.

[0176] In image P4, which shows a fox facing forward, it is difficult to determine which of the left and right pupil regions is in front. In this case, as described above, the pupil region that is in the foreground as seen from image capture device 100 and that is closest to the center of the focus frame is selected, and a focus preview frame or in-focus frame is displayed in the selected pupil region.

[0177] Returning to FIG. 11, if it is determined in step S65 that the left / right pupil selection setting is not automatic, the process proceeds to step S67.

[0178] In step S67, region setting unit 214 determines whether or not the pupil specified by the user is selectable. For example, if the specified pupil region has been detected or the detection accuracy of the specified pupil region is higher than a predetermined threshold, it is determined in step S67 that the manually specified pupil is selectable, and the process proceeds to step S68.

[0179] In step S68, the region setting unit 214 sets one pupil region designated by the user as the focus region.

[0180] If it is determined in step S67 that one pupil region specified by the user is not selectable, the process proceeds to step S69.

[0181] In step S69, the region setting unit 214 sets the other pupil as the in-focus region.

[0182] If it is determined in step S63 that the pupil region has not been detected, the process proceeds to step S70.

[0183] In step S70, the area setting unit 214 determines whether or not the in-focus area has been tracked. If it is determined in step S70 that the in-focus area has been tracked, the process proceeds to step S71.

[0184] In step S71, the area setting unit 214 sets a focus area based on the tracking result.

[0185] After steps S66, S68, S69, and S71, the process proceeds to step S75 in FIG.

[0186] If it is determined in step S70 that the in-focus area has not been tracked, the process proceeds to step S72 in FIG.

[0187] In step S72, the region setting unit 214 determines whether or not AF is set to be performed using a normal AF frame. If it is determined in step S72 that AF is set to be performed using a normal AF frame, the process proceeds to step S73.

[0188] In step S73, the area setting unit 214 sets the area specified by the normal AF frame as the in-focus area.

[0189] In step S74, the area setting unit 214 determines whether or not tracking is set to be performed using a normal AF frame. If it is determined in step S74 that tracking is set to be performed using a normal AF frame, the process proceeds to step S75.

[0190] In step S75, the subject tracking unit 213 tracks the in-focus area. The tracking here is performed on the areas set as in-focus areas in steps S66, S68, S69, S71, and S73 (detected or tracked pupil areas, face areas, and areas specified by normal AF frames). Then, the process proceeds to step S76.

[0191] If it is determined in step S74 that the setting is not set to perform tracking using a normal AF frame, step S75 is skipped and the process proceeds to step S76.

[0192] If it is determined in step S72 that the setting is not to perform AF using the normal AF frame, the process proceeds to step S80 in FIG.

[0193] 12 to step S85 in FIG. 13 are basically the same processes as step S33 in FIG. 9 to step S42 in FIG. 10, and therefore, the description thereof will be omitted to avoid duplication.

[0194] As described above, in the animal pupil detection mode, if one of the selected pupil regions cannot be detected, the other pupil region is set as the focus region.

[0195] <Image capture processing in bird eye detection mode> FIG. 15 is a flowchart illustrating the imaging process of the image capturing device 100 in the bird's eye detection mode.

[0196] The image capture process in the bird's eye detection mode in Fig. 15 is started when the power is turned on by operating the power button, for example. The bird's eye detection mode is set in advance as the detection mode for each specific part of the subject on a setting screen, etc. Fig. 15 illustrates an example in which the bird's eye is detected by bird detection unit 212-3.

[0197] In step S101 of FIG. 15, an image is acquired and displayed, similarly to the processing in step S11 of FIG.

[0198] In step S102, the bird detection unit 212-3 uses a bird dictionary to detect an pupil region from the image data stored in the memory 211. The bird detection unit 212-3 supplies the detection result to the region setting unit 214 and the display control unit 215.

[0199] In step S103, the region setting unit 214 determines whether or not the pupil region has been detected. If it is determined in step S103 that the pupil region has been detected, the process proceeds to step S104.

[0200] In step S104, the region setting unit 214 uses the pupil detection result to perform the subsequent processing for setting the focus region.

[0201] In step S105, the region setting unit 214 automatically selects the pupil region and sets it as the focus region.

[0202] If it is determined in step S103 that the pupil region has not been detected, the process proceeds to step S106.

[0203] In step S106, the region setting unit 214 determines whether or not the in-focus region has been tracked. If it is determined in step S106 that the in-focus region has been tracked, the process proceeds to step S107.

[0204] In step S107, the region setting unit 214 sets a focus region based on the tracking result.

[0205] After steps S105 and S107, the process proceeds to step S111 in FIG.

[0206] If it is determined in step S106 that the in-focus area has not been tracked, the process proceeds to step S108 in FIG.

[0207] In step S108, the region setting unit 214 determines whether or not AF is set to be performed using a normal AF frame. If it is determined in step S108 that AF is set to be performed using a normal AF frame, the process proceeds to step S109.

[0208] In step S109, the area setting unit 214 sets the area specified by the normal AF frame as the in-focus area.

[0209] In step S110, the region setting unit 214 determines whether or not tracking is set to be performed using a normal AF frame. If it is determined in step S110 that tracking is set to be performed using a normal AF frame, the process proceeds to step S111.

[0210] In step S111, the subject tracking unit 213 tracks the in-focus area. The tracking here is performed on the areas set as in-focus areas in steps S105, S107, and S109 (detected or tracked pupil areas, face areas, and areas specified by normal AF frames). Then, the process proceeds to step S112.

[0211] If it is determined in step S110 that the setting is not set to perform tracking using a normal AF frame, step S111 is skipped and the process proceeds to step S112.

[0212] If it is determined in step S108 that the setting is not to perform AF using the normal AF frame, the process proceeds to step S116 in FIG.

[0213] 16 to step S121 in FIG. 17 are basically the same processes as step S33 in FIG. 9 to step S42 in FIG. 10, and therefore, the description thereof will be omitted to avoid duplication.

[0214] As described above, in bird pupil detection mode, if the pupil area cannot be detected, it is set as the in-focus area in normal AF mode, that is, the area specified by the normal AF frame is set as the in-focus area.

[0215] <5.Other> <Detection target switching setting screen> FIG. 18 is a diagram showing an example of a detection target switching setting screen.

[0216] The detection target switching setting screen is a screen for setting a detection mode that can be switched by a predetermined operation among detection modes for a specific part of a subject.

[0217] In FIG. 18, from the left, a person icon and a check box below the person icon, an animal icon and a check box below the animal icon, and a bird icon and a check box below the bird icon are displayed.

[0218] On the detection target switching screen, the icon of each subject can be selected by moving the cursor C. Then, the check box below the icon of the selected subject can be checked or unchecked.

[0219] When the execute button provided at the bottom right of the screen is pressed, the detection mode for which a check mark has been entered is set as a detection mode that can be selected in response to a user operation.

[0220] In addition, subject detection modes that the user does not need can be disabled by unchecking the checkbox. In other words, subject detection modes for which the checkbox is unchecked can be excluded from selection. This allows users to use the modes according to their preferences.

[0221] That is, when a check mark is entered in the check box below the person icon, the person's eye detection mode is enabled and becomes selectable.

[0222] When the checkbox below the animal icon is checked, the animal eye detection mode for detecting the animal's eyes is enabled and becomes selectable.

[0223] When the check box below the bird icon is checked, the bird's eye detection mode for detecting the bird's eye is enabled as the detection mode of the imaging device 11 and becomes selectable.

[0224] In addition, the detection target switching screen may be configured to detect not only the eyes of animals or birds, but also specific parts of the subject, such as the eyes, face, part of the face, neck, head, or the entire body (subject), of any living creature, including fish, reptiles, and amphibians.

[0225] Furthermore, the subject to be detected is not limited to living things, and it may be possible to set the detection of specific parts of the subject, such as the headlights, front emblem, windshield, handlebars, driver, or driver's seat of a vehicle, or the headlights or helmet of a motorcycle.

[0226] <Display screen when the detection target is switched> In Fig. 18, the checkboxes below the person icon, the animal icon, and the bird icon are checked. Therefore, as shown in Fig. 19, the detection mode of the imaging device 11 can be switched to any one of the person pupil detection mode, the animal pupil detection mode, and the bird pupil detection mode in response to a user operation.

[0227] FIG. 19 is a diagram showing an example of a display screen when the detection target is switched.

[0228] In Fig. 19, the left side shows a display screen Mode_H when switched to human pupil detection mode. The center shows a display screen Mode_A when switched to animal pupil detection mode. The right side shows a display screen Mode_B when switched to bird pupil detection mode. Each display screen displays a person, an animal, and a bird.

[0229] In the display screen Mode_H, a focusing preview frame PF, which is displayed before a focus start command is issued, is displayed around the face of a person to be detected.

[0230] In the display screen Mode_A, a focusing preview frame PF, which is displayed before a focus start command is issued, is displayed around the eyes of the animal to be detected.

[0231] In the display screen Mode_B, the detection target is a bird, and a focusing preview frame PF, which is displayed before a focus start command is issued, is displayed around the eye of the bird that is the detection target.

[0232] In the imaging device 11, when the detection mode is a person's pupil detection mode and the display screen Mode_H is displayed, if a user operates a predetermined button, the detection mode is switched to an animal's pupil detection mode and the display screen Mode_H changes to the display screen Mode_A.

[0233] In the imaging device 11, when the detection mode is animal pupil detection mode and the display screen Mode_A is displayed, if the user operates a predetermined button, the detection mode is switched to bird pupil detection mode and the display screen Mode_A is changed to the display screen Mode_B.

[0234] In the imaging device 11, when the detection mode is the bird's eye detection mode and the display screen Mode_B is displayed, if a user operates a predetermined button, the detection mode changes to person The mode is switched to the pupil detection mode, and the display screen Mode_B is changed to the display screen Mode_H.

[0235] <Example of subject type hierarchy> In the above explanation, the human pupil detection mode, animal pupil detection mode, and bird detection mode are managed at the same hierarchical level, but for example, the bird detection mode may be managed at a hierarchical level below the animal pupil detection mode.

[0236] FIG. 20 is a diagram showing a hierarchical structure of object types.

[0237] In the example of Figure 20, people and animals are managed in the same hierarchy, while dogs, cats, small animals, and birds are managed in a hierarchy below animals.

[0238] In this way, it is possible to manage each subject type in a hierarchical structure.

[0239] <Next best method for each detection mode> In each of the above processes, if the pupil area is not detected, the focusing process is performed on the face area, the focusing process is performed on the other pupil area, or the focusing process is performed under other conditions such as the normal AF frame.

[0240] The operation when the pupil region is not detected is not limited to the above process, and may be changed depending on the user's settings, etc. In this case, the operation when the pupil region is not detected is changed to the "next best means" depending on the target subject or specific part that was not detected.

[0241] For example, the location on which a user wants to focus varies depending on the type of subject and the user's preferences. Also, the parts that can be recognized may vary depending on the subject.

[0242] Specifically, for people, the "face," "right pupil," and "left pupil" can be recognized. For animals, the "right pupil," "left pupil," and "nose" (internal) can be recognized. For birds, the "pupil (front pupil)" can be recognized. For cars, the "headlight," "front window," and "driver" can be recognized.

[0243] Furthermore, there are "right pupils" and "left pupils." Even if the imaging device 11 is instructed to focus on the "right pupil," if the "right pupil" cannot be seen (detected), the "next best" method will differ depending on the type of subject.

[0244] For example, if the subject is an animal and the imaging device 11 is instructed to focus on the "right eye," if the "right eye" is not visible, the next best option is to try to focus on the "face," but the focus will end up on the high "nose." However, focusing on the "left eye," which is visible, is preferable to focusing on the "nose."

[0245] When the subject is a person, even if the imaging device 11 is instructed to focus on the "right pupil," if the "right pupil" is not visible, it feels more natural to the user to focus on the "face" because people do not have as high a "nose" as animals. Also, when the subject is a person, if the imaging device 11 focuses on the visible "pupil" as with an animal, the user feels that the imaging device 11 is ignoring the user's instructions. Therefore, when the subject is a person, the "face" is preferable as the second best option.

[0246] Furthermore, there are also subjects such as cars whose constituent parts differ from one car to another.

[0247] There may be cases where there are limitations or differences in characteristics of the recognition means, such as when only the "foreground eye" can be recognized, as in the case of birds. In such cases, the area preset by the user becomes the next best option.

[0248] As described above, by individually controlling the "next best means" according to the subject, it is possible to automatically focus in accordance with the user's intentions. Also, it is possible to automatically select the "next best means" that can be provided to the user according to different characteristics.

[0249] As described above, in the present technology, the area of ​​the first specific part of the subject is set as the focus area in response to the user's operation. Also, if the area of ​​the first specific part is not detected, the area of ​​the second specific part of the subject, which is the same type of specific part as the first specific part, is set as the focus area.

[0250] Furthermore, in the present technology, if the area of ​​the first specific part is not detected, an area according to the type of subject is set as the in-focus area.

[0251] Therefore, the focus can be automatically adjusted in accordance with the user's intention.

[0252] Although the above description has been made on the process of detecting the eyes of animals such as dogs and cats, this technology can be applied to the eyes, face, part of a face, neck, head, or other specific parts of a subject, or the entire body (subject), of all living creatures such as birds, fish, reptiles, amphibians, etc. This technology can also be applied to combinations of specific parts of these subjects or subjects.

[0253] Furthermore, this technology is not limited to living organisms and can also be applied to specific parts of a subject, such as a vehicle's headlights, front emblem, windshield, or driver's seat, or a motorcycle's headlights or helmet.

[0254] In these cases, a detection mode for detecting a specific part of a subject is set in advance and used, and this allows the user to convey to the imaging device the user's intention, such as which of multiple detection results or detection methods to prioritize, or which of multiple subjects to prioritize.

[0255] Unlike people, when focusing on the pupils of animals with long fur, the focus may be on the hair that covers the pupils instead of the pupils. In this case, the focal position can be adjusted further back, or the imaging device can be set in advance to focus on hair as a subject that is easy to focus on, and the imaging device can focus based on this setting, thereby obtaining imaging results that better suit the user's intentions.

[0256] The above-described series of processes can be executed by hardware or software. When the above-described series of processes is executed by software, the programs constituting the software are installed from a network or a recording medium.

[0257] This recording medium is, for example, as shown in Fig. 1, configured by a removable recording medium 148 on which the program is recorded, which is distributed to users separately from the device main body for distribution of the program. This removable recording medium 148 includes magnetic disks (including flexible disks) and optical disks (including CD-ROMs and DVDs). It also includes magneto-optical disks (including MDs (Mini Discs)) and semiconductor memories.

[0258] In this case, the program can be installed in the storage unit 142 by inserting the removable recording medium 148 into the drive 147 .

[0259] The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received via the communication unit 145 and installed in the storage unit 142.

[0260] Alternatively, this program can be installed in advance in the storage unit 142 or a ROM (Read Only Memory) in the system controller 131.

[0261] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0262] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0263] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0264] For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0265] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by multiple devices.

[0266] Furthermore, when one step includes multiple processes, the multiple processes included in one step can be executed by one device or can be shared and executed by multiple devices.

[0267] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0268] <Configuration combination example> The present technology can also be configured as follows. (1) a setting unit that sets a first area including a first specific portion of the subject as a focusing area; When the first specific part is not detected, the setting unit sets a second area including a second specific part of the subject, which is a specific part of the same type as the first specific part, as the focusing area. Imaging device. (2) The first specific portion is one of the left and right pupils of the subject, and the second specific portion is the other pupil. The imaging device according to (1) above. (3) The user can set the first specific part to the left or right pupil of the subject. The imaging device according to (2) above. (4) The subject is an animal. The imaging device according to any one of (1) to (3). (5) The setting unit sets the second region as the focused region when the first specific region is detected and the detection accuracy of the first specific region is lower than a predetermined threshold. The imaging device according to any one of (1) to (4). (6) When the first specific portion is not detected, the setting unit sets an area according to the type of the subject as the focusing area. The imaging device according to (1) above. (7) When the first specific portion is not detected, the setting unit sets, as the focusing area, the second area or a third area including the first specific portion and a third specific portion of the subject that is larger than the first specific portion, depending on the type of the subject. The imaging device according to (6) above. (8) The first specific part is one of the left and right pupils, the second specific part is the other pupil, and the third specific part is the face. The imaging device according to (7) above. (9) When the first specific portion is not detected, the setting unit sets the second region, a third region including the first specific portion and including a third specific portion of the subject that is larger than the first specific portion, or a fourth region specified by a user operation as the focusing region, depending on the type of the subject. The imaging device according to (6) above. (10) The type of subject is a person, an animal, or a bird. The imaging device according to any one of (6) to (9). (11) The type of subject to be detected can be set in advance. The imaging device according to any one of (1) to (10) above. (12) and a region detection unit that detects at least the first region using a dictionary according to the type of the subject. The imaging device according to any one of (1) to (11). (13) Further comprising a tracking unit that tracks at least the first area, The setting unit sets the second area as the focusing area when the first area cannot be tracked. The imaging device according to any one of (1) to (12). (14) a focus instruction unit that instructs the start of focusing for each image capture unit; and a display control unit that controls, when the start of focusing is instructed, display of information indicating the focusing area; The imaging device according to any one of (1) to (13) above, further comprising: (15) an imaging instruction unit that instructs imaging; a focus control unit that controls the imaging unit so that, when the imaging is instructed, the imaging unit focuses on the focus area set by the setting unit and performs the imaging; The imaging device according to any one of (1) to (14) above, further comprising: (16) The imaging device A first area including a first specific portion of the subject is set as a focus area to be focused on; If the first specific part is not detected, a second area including a second specific part of the subject, which is a specific part of the same type as the first specific part, is set as the focusing area. Imaging method. (17) causing the computer to function as a setting unit that sets a first area including a first specific portion of the subject as a focusing area; When the first specific part is not detected, the setting unit sets a second area including a second specific part of the subject, which is a specific part of the same type as the first specific part, as the focusing area. program. [Explanation of symbols]

[0269] 100 imaging device, 101 lens, 102 aperture, 103 imaging element, 104 analog signal processing unit, 105 A / D conversion unit, 106 digital signal processing unit, 121 lens driver, 131 system controller, 141 display unit, 142 storage unit, 146 operation unit, 211 memory, 212 object detection unit, 212-1 person detection unit, 212-2 animal detection unit, 212-3 bird detection unit, 213 object tracking unit, 214 area setting unit, 215 display control unit, 216 codec processing unit

Claims

1. a setting unit that sets a first pupil region including one of the left and right pupils of the subject as a focusing region; When the setting unit cannot select the first pupil region as the focus region according to the detection result of the one pupil, in a first mode according to the type of the subject, the setting unit sets a second pupil region including the other of the left and right pupils as the focus region. Imaging device.

2. In a second mode according to the type of the subject, if the first pupil area cannot be selected as the focus area according to the detection result of the one pupil, the setting unit sets a face area as the focus area. The imaging device according to claim 1 .

3. In a third mode according to the type of the subject, if the first pupil region cannot be selected as the focus region according to the detection result of the one pupil, the setting unit sets the region of a focus frame as the focus region. The imaging device according to claim 2 .

4. The first mode, the second mode, and the third mode can be switched by a user operation. The imaging device according to claim 3 .

5. The first mode is a mode in which the type of subject is an animal; the second mode is a mode in which the type of subject is a person, The third mode is a mode in which the type of subject is a bird. The imaging device according to claim 3 .

6. The setting unit sets the second pupil region as the focus region when the one pupil is detected in the first mode and the detection accuracy is lower than a predetermined threshold. The imaging device according to claim 1 .

7. In the first mode, when the second pupil region cannot be selected as the focusing region according to the detection result of the other pupil, the setting unit sets the region of the focusing frame as the focusing region. The imaging device according to claim 1 .

8. In the second mode, the setting unit first detects the face area and then detects the first pupil area for the detected face area. The imaging device according to claim 2 .

9. further comprising a tracking unit that tracks at least the first pupil region; When a user's operation to set the second pupil region as a focus region is performed but the second pupil region cannot be selected as a focus region, the setting unit sets the first pupil region based on a tracking result as the focus region. The imaging device according to claim 1 .

10. The setting unit changes the focus area from the first pupil area to the second pupil area when the second pupil area changes from an unselectable state to a selectable state while tracking the first pupil area. The imaging device according to claim 9 .

11. a focus instruction unit that instructs the start of focusing for each image capture unit; a display control unit that controls, when the start of focusing is instructed, display of information indicating the focusing area; Further equipped The imaging device according to claim 1 .

12. an imaging instruction unit that instructs imaging; a focus control unit that controls the imaging unit to focus on the focus area set by the setting unit when the imaging is instructed; Further equipped The imaging device according to claim 1 .

13. The imaging device setting a first pupil region including one of the left and right pupils of the subject as a focusing region; When the first pupil region cannot be selected as the focus region according to the detection result of the one pupil, in a first mode according to the type of the subject, a second pupil region including the other of the left and right pupils is set as the focus region. An imaging method comprising:

14. A computer comprising: setting a first pupil region including one of the left and right pupils of the subject as a focusing region; When the first pupil region cannot be selected as the focus region according to the detection result of the one pupil, in a first mode according to the type of the subject, a second pupil region including the other of the left and right pupils is set as the focus region. A program for executing a process including:

Images