Image capturing apparatus, control method, and storage medium

Abstract

To provide an imaging apparatus capable of performing tracking using reliable authentication registration information.SOLUTION: An image pickup apparatus including a main object setting unit configured to set a main object as a tracking target from among detected objects, and a registration unit configured to register feature information of the main object, the image pickup apparatus being capable of switching the main object in a case where there is an object matching the registered feature information among objects different from the main object, the image pickup apparatus comprising: A first determination unit configured to determine whether or not a distance difference between the main object and the other object satisfies a registration condition; and a second determination unit configured to determine whether or not a distance difference between the main object and the other object satisfies the registration condition, when it is determined by the second determination means that the registration condition is not satisfied, registration of new feature information by the registration means is not permitted while tracking to the main subject is continued regardless of determination results by the first determination means and the second determination means thereafter.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an imaging device, a control method, and a program. [Background technology]

[0002] In recent years, digital cameras have seen advances in the development of tracking technology that can detect people in images captured by the image sensor and continue to track them. However, there is a demand for continued tracking of the photographer's intended subject even in situations where tracking is more difficult. For example, scenes involving multiple subjects (people) moving around can be anticipated, such as in team sports competitions. In these situations, tracking can be difficult because the subjects may overlap and temporarily occlude the target subject behind another subject, resulting in "mistracking" of the other subject. Even in such scenes, there is a demand for correct tracking once the target subject no longer overlaps with the foreground subject and can be detected again.

[0003] In response to such needs, Patent Document 1 discloses a technology for registering the facial orientation of a tracking target in each frame, comparing the facial orientation to determine the tracking target, and preventing erroneous tracking. Therefore, a personal authentication technology for identifying individuals has been proposed as a technology for continuously tracking a subject to be detected. A specific embodiment of this personal authentication technology extracts feature information possessed by each detected subject, compares the extracted feature information with pre-registered reference feature information, and performs a personal authentication determination by checking whether the extracted feature information matches.

[0004] Furthermore, as a countermeasure against erroneous tracking using such personal authentication technology, Patent Document 2 discloses a technology in which, when a user specifies a tracking target, feature information is registered and it is determined based on the registered information whether the tracking target is the subject designated by the user as the tracking target.If it is determined that the subject is not the subject designated by the user as the tracking target, a subject that intersects with or separates from the tracking object is reset as the tracking target. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-243730 [Patent Document 2] JP 2020-96262 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology disclosed in Patent Document 1 may fail to correctly select a tracking target when there are multiple people with the same facial orientation, or when the facial orientation of the tracking target changes while it is temporarily blocked by another person, etc. Furthermore, the technology disclosed in Patent Document 2 does not take into consideration the possibility of erroneously registering feature information for an erroneously tracked person in a situation where multiple subjects are moving around and intersecting.

[0007] During personal authentication, features are extracted using facial parts such as the eyes, nose, and mouth. Therefore, if the subject is facing sideways, only some of the subject's features can be detected, reducing the accuracy of personal authentication. Therefore, it is common practice not to register feature information. Furthermore, if the subject is facing backwards, the subject's features cannot be detected, making it impossible to register the feature information. Under these circumstances, there is a risk that a different person (another subject) may be mistakenly tracked while registration is being carried over, resulting in the feature information of the mistakenly tracked person being registered. Therefore, when tracking a person using authentication registration information (feature information), it is necessary to reduce the risk of mistakenly registering the feature information of an unrelated person, even in situations where multiple people are moving around and crossing each other.

[0008] An object of the present invention is to provide an imaging device capable of performing tracking using reliable authentication registration information. [Means for solving the problem]

[0009] In order to achieve the above-mentioned object, an imaging device of one embodiment of the present invention comprises a main subject setting means for setting a main subject to be tracked from among detected subjects, and a registration means for registering feature information of the main subject, and is capable of switching the main subject when a subject other than the main subject matches the registered feature information, and is further equipped with a first judgment means for judging whether the registration conditions for the feature information of the main subject are met, and a second judgment means for judging whether the distance difference between the main subject and another subject meets the registration conditions, and is characterized in that if the first judgment means judges that the registration conditions are met, the registration means is permitted to register the feature information of the main subject, and if the second judgment means judges that the registration conditions are not met, the registration means is not permitted to register new feature information while tracking of the main subject continues, regardless of subsequent judgment results by the first judgment means and the second judgment means. [Effects of the Invention]

[0010] According to the present invention, it is possible to obtain the effect of enabling tracking using reliable authentication registration information. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a side cross-sectional view of an imaging device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram illustrating a control system of the imaging apparatus according to the embodiment of the present invention. [Figure 3] 10 is a flowchart showing a process when a main subject is set. [Figure 4] 10A and 10B are explanatory diagrams of the operation of switching the main subject using the result of personal authentication. [Figure 5] 10 is a flowchart showing an authentication registration process. [Figure 6] FIG. 10 is an explanatory diagram of the operation of the authentication registration process. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited to the configurations described in the embodiments. Furthermore, multiple features of the present embodiments may be combined as desired. Furthermore, the same reference numerals are used to designate the same or similar components in the drawings, and redundant description will be omitted as much as possible.

[0013] (Figure 1: Side cross-sectional view of the imaging device) Fig. 1 is a side cross-sectional view showing the configuration of a digital camera (hereinafter also simply referred to as "camera") 100 as an imaging device according to an embodiment of the present invention. Fig. 2 is a block diagram showing the configuration of a control system for camera 100 in Fig. 1. Digital camera 100 is one form of imaging device, and other imaging devices such as a video camera may also be used.

[0014] 1, camera 100 of this embodiment has an interchangeable lens unit 120 attached to the front side (subject side) of camera body 101, which is detachable from camera body 101. Lens unit 120 includes a focus lens 121 and an aperture 122 from the subject side, and is electrically connected to camera body 101 via mount contacts 123. Lens unit 120 makes it possible to adjust the amount of light taken into camera body 101 and the focal position. In addition, focus lens 121 can also be manually adjusted by the user.

[0015] (Image sensor 104: Processing unit 102) The image sensor 104, located on the rear side inside the camera body 101, is realized by a CCD sensor, a CMOS sensor, etc., and is configured to include an infrared cut filter, a low-pass filter, etc. When capturing an image, the image sensor 104 forms an image on its upper surface that has passed through the imaging optical system of the lens unit 120, performs photoelectric conversion, and transmits a signal for generating the captured image to the arithmetic unit 102 inside the camera body 101.

[0016] (Image storage unit 107: Display unit 105: Shutter 103) The arithmetic unit 102 generates a captured image based on a signal received from the image sensor 104, stores the image in an image storage unit 107, and displays the image on a display unit 105 implemented by an LCD or the like provided on the exterior rear surface of the camera 100. A shutter 103 disposed on the subject side of the image sensor 104 shields the image sensor 104 from light when not capturing an image, but is opened when capturing an image to expose the image sensor 104. As will be described later, an operation unit 106 is provided on the top or other part of the camera body 101 to allow the user to perform appropriate operations.

[0017] (Figure 2: Block diagram of the control system for the imaging device) FIG. 2 is a block diagram of the control system of the camera 100 shown in FIG. 1. Next, the configuration of the control system will be described with reference to FIG. 2. The arithmetic unit 102 includes a multi-core CPU capable of parallel processing of multiple tasks, RAM, and a ROM 206 that is also used to store feature information for personal authentication processing, which will be described later. In addition to these, the arithmetic unit 102 also includes a dedicated circuit (not shown) for executing specific arithmetic processing at high speed. As shown in FIG. 2, the arithmetic unit 102 includes a control unit 201, a subject detection unit 202, a tracking calculation unit 203, a focus calculation unit 204, and an exposure calculation unit 205. The CPU loads programs stored in the ROM 206, etc., into the RAM and executes them, thereby realizing various functions of the arithmetic unit 102, including the control unit 201.

[0018] (Control unit 201) The control unit 201 controls each part of the camera body 101 and the lens unit 120. In other words, 104 to 107 described with reference to Fig. 1 are connected to the control unit 201 so that required information can be communicated. The control unit 201 provides control signals to these parts, and conversely, the control unit 201 can also obtain various information from these parts.

[0019] (Subject detection unit 202: detector 213) The subject detection unit 202 includes a detector 213, a subject recognition unit 214, and a main subject setting unit 215. The detector 213 performs processing to detect "specific regions" such as a human face from an image, and can obtain various information (subject detection results) such as the position, size, tilt, and certainty of the face of a person in the image. In some cases, no "specific regions" are detected, while in other cases, multiple "specific regions" are detected.

[0020] A part detector that detects facial parts such as the eyes, nose, and mouth of a person is included in detector 213. Any known detection method can be used, such as AdaBoost or convolutional neural network (CNN). Specific implementation may be a program running on a CPU, or a combination thereof.

[0021] (Subject recognition unit 214) The object recognition unit 214 receives the object detection result obtained by the detector 213, extracts "feature information" of the detected person's face, and can authenticate whether or not it is a specific individual. When making such an individual authentication judgment, the object detection unit 202 reads out the feature information of the individual pre-stored (registered) in the ROM 206, and compares it with the "feature information" of the person's face extracted by the object recognition unit 214 from the image of the object detection result. This determines whether or not it matches the registered individual.

[0022] (Main subject setting section 215) The main subject setting unit 215 sets a target subject (hereinafter referred to as "main subject") to be used for AF control from among the detected subjects. The main subject is set using a known calculation method based on size, position, reliability of the detection result, etc. Furthermore, when setting the main subject using the determination result of personal authentication, the main subject is set using the determination result of personal authentication by the subject recognition unit 214. If no subject is detected by the detector 213, it is also possible to set a "main subject area" that is an area to be the main subject based on past detection results, feature amounts such as edges of the target frame, defocus information of the subject, etc.

[0023] (Tracking calculation unit 203: Focus calculation unit 204: Exposure calculation unit 205) The tracking calculation unit 203 performs tracking processing of the target subject region based on the detection information of the main subject. A known tracking method, such as template matching, which compares features between frames, may be used. The focus calculation unit 204 acquires defocus information for focusing on the main subject based on the information from the subject detection unit 202 or the tracking calculation unit 203, and calculates a control value for controlling the focus lens 121. The exposure calculation unit 205 calculates control values ​​for controlling the aperture 122, shutter 103, image sensor 104, etc., to achieve appropriate exposure for the main subject.

[0024] The control unit 201 receives control values ​​calculated by the exposure calculation unit 205 and focus calculation unit 204, and controls the focus lens 121, aperture 122, shutter 103, display unit 105, etc. The display unit 105 displays an area within the screen that is the target of AF control (hereinafter referred to as an "AF frame"), and a detected subject included in the AF frame may be set as the main subject. The operation unit 106 shown in FIG. 1 is realized by a release switch, a mode dial, etc., and the control unit 201 receives image capture instructions, mode change instructions, etc. from the user via the operation unit 106. This completes the configuration of the control system of the camera 100.

[0025] (Figure 3: Flowchart showing the process when setting the main subject) Next, a series of processing steps from setting a main subject in the main subject setting unit 215 in this embodiment to performing AE and AF processing by the exposure calculation unit 205 and the focus calculation unit 204 will be described with reference to Fig. 3. The processing steps shown in Fig. 3 are basically performed by the subject detection unit 202.

[0026] (Step S301: Step S302) First, in step S301, the control unit 301 acquires image data generated based on a signal from the image sensor 104, passes it to the object detection unit 202, and proceeds to step S302. In step S302, the detector 213 detects an object based on the image data acquired in step S301, and proceeds to step S303. The detection target may be, for example, a person, an animal, or an object with a noticeable color brightness (hereinafter referred to as "object detection"), and object information (type, position, size, reliability, distance) of these detected objects is acquired.

[0027] (Step S303) In step S303, main subject setting unit 215 sets a main subject to be the target of AF control among the subjects detected in step S302. When setting the main subject, a new main subject may be set based on a user instruction. For example, if the user operates the release switch of operation unit 106 to issue an instruction to perform AF control on a detected subject within the AF frame, the main subject is selected from the detected subjects within the AF frame by referring to the subject detection information acquired in step S302. Alternatively, the main subject may be manually specified by touching the subject displayed on display unit 105. Furthermore, if there is no user instruction, it is determined based on the subject information acquired in step S302 whether to continue tracking the previous main subject or switch to another subject.

[0028] (Step S304: Step S305) In step S304, main subject setting unit 215 checks whether the main subject set in step S303 has changed from the previous main subject, and if it is determined that the main subject has changed (Yes), proceeds to step S305. On the other hand, if it is determined that the main subject remains the same (No), proceeds to step S306. In step S305, subject recognition unit 214 resets registration information for personal authentication in step S308, which will be described later, and proceeds to step S306. That is, feature information and the like are reset (cleared) in step S305.

[0029] (Step S306) In step S306, main subject setting unit 215 refers to the information registered for personal authentication of the main subject in step S308, and checks whether there is a detected subject whose score indicating the degree of match of the feature amounts with detected subjects other than the main subject is equal to or exceeds a threshold. The score is set so that it increases as the degree of match of the feature amounts with detected subjects other than the main subject increases. If it is determined that there is a detected subject whose score is equal to or exceeds the threshold (Yes), the process proceeds to step S307, whereas if it is determined that there is no detected subject whose score is equal to or exceeds the threshold (No), the process proceeds to step S308.

[0030] (Step S307) In step S307, main subject setting unit 215 determines that the main subject set in step S303 was a "mistracked" subject and that the detected subject with a score equal to or greater than the threshold is the correct main subject, resets the main subject, and proceeds to step S308.

[0031] (Fig. 4: Illustration of the operation of switching the main subject using the results of personal authentication: Fig. 4(A)) Here, a case where the main subject is reset using the determination result of personal authentication will be described with reference to Fig. 4. Fig. 4(A) shows a person 401 who is the main subject and a person 402 who is not the main subject. "Feature information" for personal authentication is registered for the person 401 who is the main subject.

[0032] (Figure 4(B):Figure 4(C)) FIG. 4(B) shows a situation in which person 401, the main subject, and person 402, a non-main subject, overlap in close proximity, resulting in erroneous tracking of person 402. When the distance between the subjects (401, 402) becomes closer, there is a risk of erroneously tracking the main subject in this way. FIG. 4(C) is an explanatory diagram of a case in which person 401 is redetected while erroneous tracking of person 402 continues. Here, by extracting feature information about person 401 and calculating an authentication score that indicates the degree of match with the feature information registered in FIG. 4(A), if the authentication score for person 401 exceeds a threshold, person 401 is determined to be the correct main subject. Therefore, by resetting person 401 to be the main subject, it is possible to correctly re-track him / her.

[0033] (Step S308: Step S309) In step S308, the subject recognition unit 214 performs authentication and registration processing on the main subject, which will be described later with reference to Figures 5 and 6, and then proceeds to step S309. In step S309, AE control by the exposure calculation unit 205 and AF control by the focus calculation unit 204 are performed on the main subject, and this processing ends.

[0034] (Fig. 5: Flowchart showing the authentication registration process; Fig. 6: Explanation of the operation of the authentication registration process) Next, the authentication registration process of S308 in Fig. 3 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing the authentication registration process, and Fig. 6 is an explanatory diagram of the operation of the authentication registration process. As mentioned above, the process shown in Fig. 5 is basically executed by the subject detection unit 202.

[0035] (Step S501: Step S502) In step S501, it is checked whether authentication registration processing has yet been performed on the main subject in S507, which will be described later. If it has not been performed (YES), the process proceeds to S502. If it has been performed (No), the authentication registration processing ends. In step S502, the control unit 201 checks whether the main subject is in focus through AF control by the focus calculation unit 204. If it is determined that the main subject is in focus (Yes), the process proceeds to S503. If it is determined that the main subject is not in focus (No), the process ends. Here, if the image is out of focus and blurred, facial features such as the pupils, nose, and mouth for authentication registration cannot be detected. For this reason, authentication registration is not performed in such a case.

[0036] (Step S503: Step S504) In step S503, the detection result of detector 213 is used. That is, it is checked whether the main subject is facing approximately forward, and if it is not facing approximately forward (No), the process proceeds to step S504, whereas if it is facing approximately forward (Yes), the process proceeds to step S506.

[0037] (Figure 6(A)) 6A, facial features can be accurately detected when the person is facing forward, as in person 601, but some or all of the facial features cannot be detected when the person is facing diagonally, sideways, backwards, etc., as in person 602. As a result, feature extraction for authentication and registration cannot be performed sufficiently, and face authentication cannot be performed, or even if it is performed, the accuracy is so poor that authentication and registration is not performed.

[0038] (Step S504) In step S504, it is determined whether or not there is a subject that is getting closer to the main subject than last time, and if it is determined that there is a subject that is getting closer (Yes), the process proceeds to S505. On the other hand, if it is determined that there is no such subject (No), the process ends. Here, the determination of whether or not there is a subject that is getting closer than last time will be described with reference to Figs. 6(B) to 6(D).

[0039] (Figure 6(B): Figure 6(C)) 6(B) to 6(D), person 603 is a person different from the main subject, and person 604 indicates the main subject. Distance 605 is the distance between person 603 and person 604, and threshold 606 is a threshold that is compared with distance 605 between people to determine whether or not person 603 and person 604 are close to each other. If distance 605 between people is equal to or less than threshold 606, it is determined that another person 603 is close to the main subject (604).

[0040] In FIG. 6(B), person 604 has become the main subject, but does not meet the condition of being approximately facing forward. Also, distance 605 is longer than threshold 606, so it is determined that the person is not in close proximity. In FIG. 6(C), the main subject person 604 still does not meet the condition of being approximately facing forward, and distance 605 is equal to or less than threshold 606, so it is determined that the person is in close proximity. Here, when the state changes in chronological order from FIG. 6(B) to FIG. 6(C) (FIG. 6(B) → FIG. 6(C)), the person 603 changes from not being in close proximity to the main subject (604) to being in close proximity. Therefore, it is determined that the condition that "there is a subject that is getting closer" is met compared to the previous time.

[0041] (Step S505: Step S506: Step S507) In step S505, the state transitions to a "registration abandoned state." The "registration abandoned state" is a state in which execution of authentication registration is not permitted, and the "registration abandoned state" will continue until the authentication registration information is reset in S305 of FIG. 3. An example of a state in which execution of authentication registration is not permitted is when new feature information is not registered. In step S506, it is checked whether the "authentication abandoned state" has been set, and if it is the "authentication abandoned state" (Yes), this processing ends. On the other hand, if it is not the "registration abandoned state" (No), the processing proceeds to S507. Then, in step S507, feature information for personal authentication is generated by extracting features from the facial features of the main subject, and the feature information is stored in memory. Note that a flash memory or the like capable of non-volatile storage can be used.

[0042] (Figure 6(D)) In FIG. 6(D), the condition that main subject 604 faces approximately forward is met, and when the state changes chronologically from FIG. 6(B) to FIG. 6(D) (FIG. 6(B) → FIG. 6(D)), the conditions for authentication and registration are met, and authentication and registration are performed. On the other hand, when the state changes chronologically from FIG. 6(B) → FIG. 6(C) → FIG. 6(D), the state transitions to the "registration abandoned state" at the point in FIG. 6(C). For this reason, even if main subject 604 faces approximately forward in FIG. 6(D), authentication and registration are not performed.

[0043] Furthermore, if it takes a long time to authenticate and register main subject 604 after tracking has begun, the risk of erroneous tracking increases even if the condition "there is a subject that is approaching more closely than last time" is not met. For this reason, if a predetermined time has passed since a new main subject was set in step S303 of Fig. 3 and authentication and registration have not been performed, the state may transition to a "registration abandoned state," and this "registration abandoned state" may continue until the authentication and registration information is reset in step S305 of Fig. 3.

[0044] As described above, this embodiment checks multiple conditions when performing authentication and registration, and if the condition "there is a subject approaching" is met, the state transitions to a "registration abandonment state" and authentication and registration is not performed until a new main subject is set. By doing this, when there is a risk of erroneous tracking, authentication and registration can be avoided and tracking processing can be performed accurately using reliable authentication and registration information.

[0045] In addition, if at least one of steps S501, S502, and S503 in FIG. 5 is satisfied, processing for registering feature information of the main subject to be tracked may be executed in step S507.

[0046] <Modification> (1) For example, in step S306 in Fig. 3, if it is determined that there is a subject with an authentication score equal to or greater than a threshold, the main subject is switched to the authenticated subject, but it is also possible to simply switch to the subject with the highest score. Also, if there are multiple subjects with the highest score, it is also possible to switch to the subject whose face is facing most forward. (2) Furthermore, in Figures 6(B) to 6(C), threshold value 606 for distance 605 between person 603 and main subject 604 may not be fixed but may be changeable. For example, control unit 201 may automatically change threshold value 606 depending on the number of close plays in a sports competition within a predetermined time period. Alternatively, a user may set threshold value 606 using operation unit 106 depending on the type of competition. For this purpose, a database may be compiled containing the number of times that distance 605 is equal to or smaller than threshold value 606 in a number of competitions, and threshold value 606 may be determined by referring to this database.

[0047] <Additional Note> The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) An imaging device comprising: a main subject setting means for setting a main subject to be tracked from among detected subjects; and a registration means for registering feature information of the main subject, and capable of switching the main subject when a subject that matches the registered feature information exists among subjects different from the main subject, a first determination means for determining whether a registration condition for feature information of a main subject is satisfied; a second determination means for determining whether or not the difference in distance between the main subject and the other subjects satisfies the registration condition; an imaging device that, when the first determination means determines that the registration conditions are met, allows the registration means to register feature information of the main subject, and, when the second determination means determines that the registration conditions are not met, does not allow the registration means to register new feature information while tracking of the main subject continues, regardless of subsequent determination results by the first determination means and the second determination means. (Configuration 2) When the first determination means determines that the registration condition is met, The imaging device according to configuration 1, characterized in that the imaging device is in one or more of the following states: the face of the main subject is facing substantially forward, the main subject is in focus by a focusing unit that performs focus control on the main subject, and the feature information of the main subject is not registered by the registration means. (Configuration 3) The camera further includes a proximity determination unit for determining whether or not a determination condition that a difference in distance between a main subject and another subject is equal to or less than a threshold value is satisfied, The imaging device according to configuration 1 or 2, characterized in that, when the proximity determination means has determined that the determination condition has been satisfied in the previous determination and that the determination condition has not been satisfied in the current determination, the second determination means determines that the registration condition has not been satisfied. (Configuration 4) 3. The imaging device according to configuration 1 or 2, characterized in that if the registration means is unable to register feature information within a predetermined time after the main subject is set by the main subject setting means, registration of feature information is not permitted while tracking of the main subject continues. (Configuration 5) a switching determination means for determining whether or not a main subject has been switched; 3. The imaging device according to configuration 1 or 2, further comprising a resetting means for resetting the feature information registered in the registering means when the switching determining means determines that the main subject has switched. (Configuration 6) The imaging device according to configuration 1 or 2, further comprising a resetting means for, when a subject with an authentication score equal to or greater than a threshold exists, determining that the subject with the authentication score equal to or greater than the threshold is the correct main subject and resetting the main subject. (method) A control method for an imaging device that includes a main subject setting unit that sets a main subject to be tracked from among detected subjects, and a registration unit that registers feature information of the main subject, and that can switch the main subject when a subject that matches the registered feature information is present among subjects different from the main subject, comprising: a first determination step of determining whether a registration condition for feature information of a main subject is satisfied; a second determination step of determining whether or not a distance difference between the main subject and another subject different from the main subject satisfies the registration condition, A control method for an imaging device, characterized in that if it is determined in the first determination step that the registration conditions are met, registration of feature information of the main subject by the registration means is permitted, and if it is determined in the second determination step that the registration conditions are not met, registration of new feature information by the registration means is not permitted while tracking of the main subject continues, regardless of the determination results of the subsequent first and second determination steps. (program) A program for causing a computer to execute a control method for an imaging device that includes a main subject setting unit that sets a main subject to be tracked from among detected subjects, and a registration unit that registers feature information of the main subject, and that can switch the main subject when a subject that matches the registered feature information is present among subjects different from the main subject, The control method includes: a first determination step of determining whether a registration condition for feature information of a main subject is satisfied; a second determination step of determining whether or not a distance difference between the main subject and another subject different from the main subject satisfies the registration condition, When the first determination step determines that the registration conditions are met, the program allows the registration means to register feature information of the main subject, and when the second determination step determines that the registration conditions are not met, the program does not allow the registration means to register new feature information while tracking of the main subject continues, regardless of the subsequent determination results of the first and second determination steps.

[0048] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications and variations are possible within the scope of the gist of the present invention. For example, the present invention can be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or recording medium, and having a computer processor in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more functions. [Explanation of symbols]

[0049] 100 Digital Camera (Camera) 101 Camera body 102 Arithmetic equipment 103 Shutter 104 Image sensor 105 Display section 106 Operation section 107 Image storage unit 120 Lens Unit 121 Focus Lens 122 aperture 201 Control Unit 202 Subject detection unit 203 Tracking calculation section 204 Focus calculation unit 205 Exposure calculation section 206 ROM 213 detector 214 Subject Recognition Unit 215 Main subject setting section

Claims

1. An imaging device comprising: a main subject setting means for setting a main subject to be tracked from among detected subjects; and a registration means for registering feature information of the main subject, and capable of switching the main subject when a subject that matches the registered feature information exists among subjects different from the main subject, a first determination means for determining whether a registration condition for feature information of a main subject is satisfied; a second determination means for determining whether or not the difference in distance between the main subject and the other subjects satisfies the registration condition; an imaging device characterized in that, when the first determination means determines that the registration conditions are met, the registration means is permitted to register feature information of the main subject, and, when the second determination means determines that the registration conditions are not met, the registration means is not permitted to register new feature information while tracking of the main subject continues, regardless of subsequent determination results by the first determination means and the second determination means.

2. When the first determination means determines that the registration condition is satisfied, The imaging device according to claim 1, characterized in that the imaging device is in one or more of the following cases: the face of the main subject is facing approximately forward, the main subject is in focus by a focusing unit that performs focus control on the main subject, and the characteristic information of the main subject is not registered by the registration means.

3. The camera further includes a proximity determination unit for determining whether or not a determination condition that a difference in distance between a main subject and another subject is equal to or less than a threshold value is satisfied, The imaging device according to claim 1 or 2, characterized in that if the proximity determination means satisfies the determination condition in the previous determination but does not satisfy the determination condition in the current determination, the second determination means determines that the registration condition is not satisfied.

4. 3. The imaging device according to claim 1, wherein if the registration means is unable to register feature information within a predetermined time after the main subject is set by the main subject setting means, registration of feature information is not permitted while tracking of the main subject continues.

5. a switching determination means for determining whether or not a main subject has been switched; 3. The imaging apparatus according to claim 1, further comprising a resetting unit that resets the feature information registered in the registering unit when the switching determining unit determines that the main subject has been switched.

6. 3. The imaging device according to claim 1, further comprising a resetting unit that, when a subject with an authentication score equal to or greater than a threshold exists, determines that the subject with the authentication score equal to or greater than the threshold is the correct main subject and resets the main subject.

7. A control method for an imaging device that includes a main subject setting unit that sets a main subject to be tracked from among detected subjects, and a registration unit that registers feature information of the main subject, and that can switch the main subject when a subject that matches the registered feature information is present among subjects different from the main subject, comprising: a first determination step of determining whether a registration condition for feature information of a main subject is satisfied; a second determination step of determining whether or not a distance difference between the main subject and another subject different from the main subject satisfies the registration condition, a control method for an imaging device, characterized in that if the first judgment step determines that the registration conditions are met, registration of feature information of the main subject by the registration means is permitted, and if the second judgment step determines that the registration conditions are not met, registration of new feature information by the registration means is not permitted while tracking of the main subject continues, regardless of the judgment results of the subsequent first judgment step and second judgment step.

8. A program for causing a computer to execute a control method for an imaging device that includes a main subject setting unit that sets a main subject to be tracked from among detected subjects, and a registration unit that registers feature information of the main subject, and that can switch the main subject when a subject that matches the registered feature information is present among subjects different from the main subject, The control method includes: a first determination step of determining whether a registration condition for feature information of a main subject is satisfied; a second determination step of determining whether or not a distance difference between the main subject and another subject different from the main subject satisfies the registration condition, When the first judgment step determines that the registration conditions are met, the program allows the registration means to register feature information of the main subject, and when the second judgment step determines that the registration conditions are not met, the program does not allow the registration means to register new feature information while tracking of the main subject continues, regardless of the judgment results of the subsequent first and second judgment steps.

Images