Information processing system, information processing method, and program

Abstract

To enable a user of a photographing device to be identified even when information suitable for identifying a user of the photographing device is not available at the time of photographing.SOLUTION: When an image is captured without detecting registrant characteristic information corresponding to the user characteristic information, a photographing device 1 performs recording control to record the captured image obtained by the image capture in a memory in association with the user characteristic information. After that, when registrant characteristic information is acquired, the photographing device 1 determines photographer information to be associated with the captured image associated with the user characteristic information, on the basis of the registrant characteristic information and the user characteristic information recorded in the memory.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a method for personal authentication of a user (photographer) of a photographing device using an eye image captured while the photographer is looking through a viewfinder. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2024-2562 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technique described in Patent Document 1 has a problem in that unless information suitable for identifying (for example, authenticating) the user of the imaging device is collected at the time of imaging, the user of the imaging device cannot be identified.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to make it possible to identify the user of an imaging device even when information suitable for identifying the user of the imaging device is not available at the time of imaging. [Means for solving the problem]

[0006] The information processing system disclosed herein is an information processing system that associates a captured image captured by a capturing means with photographer information regarding the photographer of the captured image, and is characterized by comprising: a first acquisition means that acquires a first characteristic of a person; a second acquisition means that acquires a second characteristic of the person; an identification means that identifies the photographer by a matching process between the first characteristic acquired by the first acquisition means and the second characteristic acquired by the second acquisition means; a recording control means that, when a capture is performed in a state where a first characteristic corresponding to the second characteristic acquired by the second acquisition means is not detected in the matching process, associates the captured image obtained by the capture with the second characteristic and records the associated image in memory; and a determination means that determines the photographer information to be associated with the captured image based on the second characteristic recorded by the recording control means and the first characteristic acquired by the first acquisition means after the capture. [Effects of the Invention]

[0007] According to the present disclosure, even when information suitable for identifying the user of the imaging device is not available at the time of imaging, the user of the imaging device can be identified. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram showing the external configuration of the imaging device. [Figure 2] FIG. 2 is a diagram illustrating a hardware configuration of the imaging device. [Figure 3] FIG. 2 is a diagram illustrating a first example of a functional configuration of the imaging device. [Figure 4] 10 is a flowchart showing a first example of the operation of the imaging device. [Figure 5] FIG. 2 is a diagram illustrating a configuration of a first information acquisition unit. [Figure 6] FIG. 10 is a diagram illustrating a second example of the functional configuration of the imaging device. [Figure 7] 10 is a flowchart showing a second example of the operation of the imaging device. [Figure 8] FIG. 2 is a diagram showing a first example of information displayed on a display panel of the imaging device. [Figure 9] 10 is a flowchart showing a first example of the operation of the holding unit. [Figure 10] FIG. 2 is a diagram showing the configuration of a holding unit. [Figure 11] 10 is a flowchart showing a second example of the operation of the holding unit. [Figure 12] FIG. 10 is a diagram showing a second example of information displayed on the display panel of the imaging device. [Figure 13] FIG. 1 is a diagram illustrating a configuration of an information processing system. [Figure 14] FIG. 2 is a diagram illustrating a hardware configuration of a server. [Figure 15] FIG. 10 is a diagram illustrating a third example of the functional configuration of the imaging device. [Figure 16] FIG. 2 is a diagram illustrating a functional configuration of a server. [Figure 17] 10 is a flowchart illustrating the operation of the server. [Figure 18] 10 is a flowchart showing the operation of the information processing system. [Figure 19] FIG. 10 is a diagram showing a second example of the external configuration of the imaging device. [Figure 20] FIG. 10 is a diagram illustrating a third example of the functional configuration of the imaging device. [Figure 21] FIG. 10 is a diagram showing a third example of information displayed on the display panel of the imaging device. [Figure 22] 10 is a flowchart showing a third example of the operation of the imaging device. [Figure 23] FIG. 10 is a diagram illustrating a fourth example of the functional configuration of the imaging device. [Figure 24] 10 is a flowchart showing a fourth example of the operation of the imaging device. [Figure 25] FIG. 10 is a diagram showing a fourth example of information displayed on the display panel of the imaging device. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the present disclosure is not limited to the following embodiments, and any embodiment that meets the gist of the present disclosure can be applied. (First embodiment) First, a first embodiment will be described. In this embodiment, an example will be described in which an information processing system is included in a photographing device. Note that the information processing system does not have to be included in the photographing device. For example, this embodiment may be realized by using an information processing device that is communicably connected to the photographing device.

[0010] 1A and 1B are diagrams showing an example of the external configuration of an imaging device. Fig. 1A is an example of a front perspective view of the imaging device 1. Fig. 1B is an example of a rear perspective view of the imaging device 1. As shown in FIG. 1(a), this embodiment illustrates an example in which the photographing device 1 includes a photographing lens unit 1a and a camera housing 1b. A release button 111, which is an operating member that accepts photographing operations from a user (a user of the photographing device 1), is disposed on the camera housing 1b. In the following description, the (actual) user of the photographing device 1 will be referred to as the photographer as necessary. As shown in FIG. 1(b), a viewfinder 112 is disposed on the back of the camera housing 1b, through which the photographer looks to view a display device 216 (display panel) (described later) disposed within the camera housing 1b. Also disposed on the back of the camera housing 1b are operation members 113-115 that accept various operations from the user. For example, the operation member 113 has a touch panel that accepts touch operations. The operation member 114 has an operation lever that can be pushed down in each direction. The operation member 115 has a four-way key that can be pressed in each of four directions. The operation member 113 (touch panel) has a display panel (e.g., a liquid crystal panel) and has the function of displaying images on the display panel.

[0011] FIG. 2 is a diagram showing an example of the hardware configuration of the image capturing device 1. As shown in FIG. FIG. 2 illustrates an example in which the photographing lens unit 1a includes a focus adjustment circuit 210 and an aperture control circuit 211. The focus adjustment circuit 210 moves a photographing lens (not shown) provided in the photographing lens unit 1a to change the focus position. The aperture control circuit 211 operates an aperture (not shown) provided in the photographing lens unit 1a to change the aperture value. In this embodiment, the focus adjustment circuit 210 and the aperture control circuit 211 operate based on a signal input from a CPU (Central Processing Unit) 212 provided in the camera housing 1b via a mount contact 226.

[0012] As described above, this embodiment illustrates an example in which camera housing 1b has release button 111, viewfinder 112, and operation members 113 to 115. Also, in Fig. 2, this embodiment illustrates an example in which camera housing 1b has CPU (Central Processing Unit) 212, image sensor 213, eye image sensor 214, light source 215, display device 216, and photography processing circuit 217. Also, this embodiment illustrates an example in which camera housing 1b has eye photography processing circuit 218, signal input circuit 219, light source drive circuit 220, display device drive circuit 221, memory unit 222, storage unit 223, communication circuit 224, and I / F circuit 225.

[0013] The CPU 212 is an example of a processor that controls the entire image capturing apparatus 1 . The storage unit 223 stores various computer programs and various data as information necessary for the CPU 212 to perform processing. The memory unit 222 temporarily stores the computer programs and data that the CPU 212 processes.

[0014] In the photographing device 1 of this embodiment, a case will be exemplified in which one CPU 212 uses one memory (memory unit 222) to execute each process shown in the flowcharts described below. However, this is not necessarily the case. For example, multiple CPUs, storage units, and memory units may work together to execute each process shown in the flowcharts described below. Furthermore, at least some of the processes shown in the flowcharts may be executed by dedicated hardware. Examples of dedicated hardware include an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array). Furthermore, the processor is not limited to a CPU. The processor may also be, for example, a GPU (Graphics Processing Unit).

[0015] The release button 111 is connected to the signal input circuit 219. FIG. 2 illustrates an example in which the release button 111 has a switch SW1 and a switch SW2. The switch SW1 is turned on with the first stroke of the release button 111, and is used to start a shooting preparation operation (e.g., photometry and distance measurement) of the photographing device 1. The switch SW2 is turned on with the second stroke of the release button 111, and is used to start a shooting operation. The ON signals from the switches SW1 and SW2 are input to the signal input circuit 219 and transmitted to the CPU 212.

[0016] Furthermore, operation members 113 to 115 are also connected to the CPU 212. When the photographer operates the operation members 113 to 115, the operation members 113 to 115 output operation signals according to the operation by the photographer to the CPU 212. The CPU 212 performs processing (control) according to the operation signals.

[0017] The display device 216 displays various information on the screen (display surface) of the display device 216. For example, the display device 216 is a liquid crystal panel, and displays a captured image (subject image) on the screen. The display device drive circuit 221 drives the display device 216. The photographer can view the screen of the display device 216 through the viewfinder 112.

[0018] The image sensor 213 is disposed on a planned imaging plane of the photographing lens unit 1a. The photographing processing circuit 217 performs processing including A / D conversion of signals from the image sensor 213 (plurality of pixels) to obtain a photographed image signal, and outputs the signal to the CPU 212. Light source 215 is a light source for illuminating the eyeball of the photographer looking through viewfinder 112. Fig. 2 illustrates an example in which light source 215 is driven by light source drive circuit 220. An optical image of the eyeball illuminated by light source 215 passes through an eyepiece (not shown) provided in camera housing 1b, is reflected by an optical splitter (not shown) provided in camera housing 1b, and is then formed on eye imaging element 214 by a light receiving lens (not shown) provided in camera housing 1b.

[0019] The eye photographing processing circuit 218 performs processing including A / D conversion of the signal from the eye imaging element 214 to obtain an eye image signal, and outputs the signal to the CPU 212. The eye image is a photographed image that (mainly) includes the eyeball.

[0020] The communication circuit 224 is a circuit that performs various processes for communicating with the outside of the photographing device 1. The communication circuit 224 may perform wireless communication with the outside of the photographing device 1, may perform wired communication, or may perform communication via a network. The I / F circuit 225 is a circuit that performs various processes for exchanging data with devices connected to the I / F circuit 225. To the I / F circuit 225, for example, a storage medium is connected.

[0021] The hardware of the image capturing device is not limited to the configuration exemplified in Figures 1 and 2. The hardware of the image capturing device itself may be realized using known technology. For example, this embodiment may be realized by using hardware of a known image capturing device such as a mirrorless camera or a compact digital camera.

[0022] FIG. 3 is a diagram showing an example of the functional configuration of the image capturing device 1 (or the information processing system included therein). FIG. 3 illustrates an example in which the image capturing device 1 includes a first information acquiring unit 101, a data acquiring unit 102, a storage unit 103, a second information acquiring unit 104, a notification unit 105, and a matching unit 106.

[0023] The first information acquisition unit 101 acquires information relating to the characteristics of a registrant who is registered as a legitimate user of the photographing device 1. For example, the first information acquisition unit 101 identifies or limits the user (photographer) of the photographing device 1 as a registrant, and then acquires information relating to the characteristics of the registrant. In the following description, information relating to the characteristics of a registrant will be referred to as registrant characteristic information as necessary.

[0024] The authentication of the photographer is performed by comparing the registrant characteristic information with information on the photographer's characteristics acquired at the time of shooting or the like. In the following description, information on the photographer's characteristics acquired at the time of shooting or the like will be referred to as user characteristic information as necessary. The registrant characteristic information and user characteristic information are information used by the matching unit 106, which will be described later, to authenticate the photographer. The user characteristic information is, for example, information indicating characteristics that can be acquired (calculated) from a living body. The registrant characteristic information is also information to be compared with the user characteristic information. The photographing device 1 determines that the photographer is a registrant when the user characteristic information is identical to or similar to the registrant characteristic information. Therefore, it is preferable that the registrant characteristic information and the user characteristic information be, for example, information indicating characteristics that can be acquired (calculated) from information on the same part of a living body.

[0025] The data acquisition unit 102 acquires an image (photographed image) photographed by the photographing device 1 based on a photographing instruction from the photographer, and stores (stores) it in the storage unit 103. At this time, the photographed image may be stored outside the photographing device 1 by the external data storage unit 1031. The external data storage unit 1031 may store the captured images in, for example, a storage medium that is detachable from the photographing device 1. Examples of such storage media include an SD memory card and a CF (CompactFlash (registered trademark)) card. The external data storage unit 1031 may also store the captured images in a storage medium connected to the photographing device 1 by wire or wirelessly. Examples of such storage media include a USB (Universal Serial Bus) memory. The external data storage unit 1031 may also store the captured images in storage on the cloud.

[0026] The second information acquisition unit 104 acquires user characteristic information. For example, the second information acquisition unit 104 acquires the user characteristic information when or almost simultaneously with the image capture by the data acquisition unit 102. Then, the storage unit 103 stores the user characteristic information in association with (linked to) the captured image corresponding to the user characteristic information. For example, the storage unit 103 stores the user characteristic information in association with the captured image acquired simultaneously or almost simultaneously with the user characteristic information. In this embodiment, for example, an example of a recording control means is realized by using the storage unit 103.

[0027] This association may be achieved, for example, by managing the user characteristic information and the hash value of the captured image together. A hash value is a fixed-length value calculated for data of variable length, and is designed to have a different value if the original data is different. In principle, it is possible for multiple pieces of data to have the same hash value (hash value collision). However, it is safe to assume that hash value collisions will not occur in reality. Furthermore, it is difficult to generate data whose hash value has a specific value. Therefore, it is safe to assume that it is impossible to replace a captured image associated with user characteristic information with another image. Hash values ​​can be calculated using known technology (e.g., SHA256 technology). Therefore, a detailed description of hash values ​​will be omitted here.

[0028] When the first information acquisition unit 101 has acquired registrant characteristic information and the storage unit 103 has stored user characteristic information associated with the captured image, the notification unit 105 notifies the matching unit 106 that the information can be matched. The matching unit 106 matches the registrant characteristic information with the user characteristic information to determine whether the actual user (photographer) of the photographing device 1 and the registrant registered as the user of the photographing device 1 are the same person. For example, the matching unit 106 determines whether the person identified from the registrant characteristic information and the person identified from the user characteristic information match. If the result of this determination is that they are the same person, authentication of the photographer is successful. In this case, the matching unit 106 performs a predetermined operation.

[0029] An example of the predetermined operation is to irreversibly assign photographer information to a photographed image. This makes it possible to guarantee who took the photograph (guarantee authenticity). In the following description, information that guarantees who took the photograph (guarantees authenticity) will be referred to as photographer information as necessary. Note that, in order to irreversibly assign photographer information to a photographed image, for example, electronic signature technology may be used. Electronic signature technology is a technology that can prove that data has not been tampered with since the electronic signature was assigned, and can be realized by known technology. Therefore, a detailed description of electronic signatures will be omitted here. Note that the photographer information only needs to be associated with the photographed image, and does not have to be assigned to the photographed image.

[0030] The above is an example of the general flow of operations of the image capturing device 1 (and the information processing system included therein) of this embodiment. An example of the flow of authentication of a photographer can be summarized as follows. The first information acquisition unit 101 acquires registrant characteristics, which are information relating to the characteristics of registrants. The data acquisition unit 102 takes a photograph based on a photographing instruction from the photographer. The second information acquisition unit 104 acquires user characteristic information relating to the characteristics of the photographer in accordance with the photographing instruction from the photographer (for example, at the time of the photographing instruction or almost simultaneously with the photographing instruction). The storage unit 103 stores the user characteristic information and the photographed image in association with each other. The notification unit 105 confirms that the registrant characteristic information and the user characteristic information are present. Once this confirmation is made, the matching unit 106 matches the registrant characteristic information with the user characteristic information and authenticates the photographer.

[0031] Here, when user characteristic information is acquired by the second information acquisition unit 104, there are cases where registrant characteristic information corresponding to the user information has not yet been acquired by the first information acquisition unit 101. When registrant characteristic information has not yet been acquired, this also includes cases where the first information acquisition unit 101 has failed to acquire registrant characteristic information. In this embodiment, even in such cases, authentication of the photographer can be performed.

[0032] In this embodiment, an example is shown in which, when user characteristic information is acquired but registrant characteristic information has not yet been acquired, the storage unit 103 continues to store the user characteristic information in association with the captured image until the registrant characteristic information is acquired. Also, in this embodiment, an example is shown in which the matching unit 106 matches the user information and registrant characteristic information and authenticates the photographer when (or after) the user characteristic information and registrant characteristic information are obtained. By doing so, even if authentication of the photographer fails (or cannot be performed) because registrant characteristic information has not yet been acquired, the matching unit 106 can perform authentication later.

[0033] If the registrant characteristic information has already been acquired, the photographer information is added to the captured image, for example, each time a photograph is taken. On the other hand, if the registrant characteristic information has not yet been acquired when the user characteristic information is acquired, the photographer information is not added to the captured image at the time of shooting. In this case, the photographer information is added to the captured image all at once after the registrant characteristic information has been acquired.

[0034] A specific example of processing by the photographing device 1 (or the information processing system included therein) will be described below. For ease of understanding, the following description will be given using an example in which "the photographer of the photographed image is person A" as necessary.

[0035] First, an example of the processing of the image capturing device 1 when acquiring registrant characteristic information will be described. The first information acquisition unit 101 identifies the user of the photographing device 1 as "Mr. A" or limits it to "several people including Mr. A," and then acquires the authentication information of the user (registered person characteristic information). For example, the photographer looks through the finder 112 of the photographing device 1. The first information acquisition unit 101 acquires, for example, an eye image obtained by the eye imaging element 214, the light source 215, and the eye photographing processing circuit 218, and acquires the registered person characteristic information based on the eye image. As described above, in this embodiment, the case where the registrant characteristic information is used in the matching unit 106 together with the user characteristic information acquired by the second information acquisition unit 104 will be exemplified.

[0036] The method for acquiring the registrant characteristic information is not limited to the above-mentioned method. For example, information on the characteristics of people including the photographer (Mr. A) may be registered in advance as the registrant characteristic information in the photographing device 1. Furthermore, for example, the first information acquiring unit 101 may acquire the registrant characteristic information by reading information stored in a storage medium (e.g., an ID card) as information on the characteristics of people including the photographer (Mr. A).

[0037] Furthermore, the registrant characteristic information needs to be information (values) that can be matched with the user characteristic information by the matching unit 106. For example, the registrant characteristic information is acquired in advance using the same method as the user characteristic information. An example of a method for acquiring the user characteristic information will be described in detail in the following explanation of shooting.

[0038] Next, an example of the processing performed by the image capturing device 1 during image capturing will be described. The photographer issues a shooting instruction while looking through the viewfinder 112 of the photographing device 1, for example. In this embodiment, a case will be exemplified in which the photographing is performed by the data acquisition unit 102 and the photographed image is stored as an image file by the storage unit 103. The photographed image may be stored inside the photographing device 1 or outside the photographing device 1.

[0039] Although it is considered that most of the photographed images were taken by the registrant identified by the registrant characteristic information acquired by the first information acquisition unit 101, this is not necessarily the case. For example, if someone else temporarily takes over the photography at the shooting location, the photographed images will not have been taken by the registrant. In order to correctly determine that the captured image was taken by person A, it is preferable that the photographing device 1 obtains information about the characteristics of the photographer at the moment of shooting (user characteristic information) and determines that the photographer is person A based on that information.

[0040] Therefore, the second information acquisition unit 104 acquires eye images obtained by the eye imaging element 214, the light source 215, and the eye image processing circuit 218, for example, at or almost simultaneously with the image capture, and acquires user feature information based on the eye images. User feature information is information that can identify an individual person and is also called a feature or a feature vector. A feature has the property that feature values ​​calculated based on any of multiple eye images of the same person have a high degree of similarity to each other. A feature value also has the property that a feature value calculated based on the eye image of a certain person has a low degree of similarity to a feature value calculated based on the eye image of another person. The calculation of the feature value (user feature information) itself can be realized using well-known techniques in fields such as object recognition, face recognition, and iris authentication. Therefore, details of the calculation method of the feature value (user feature information) will be omitted here.

[0041] The user feature information acquired by the second information acquisition unit 104 is associated with a captured image captured simultaneously or almost simultaneously with the user feature information and stored in the storage unit 103. Note that, although an example is given here in which the user feature information is stored in the storage unit 103, this is not necessarily the case. For example, an eye image from which the user feature information can be calculated may be stored in the storage unit 103. Furthermore, the user feature information may be information calculated based on information other than the eye image (e.g., biometric information). For example, at least one of information on the features of a person's face (overall) and information on the features of a person's fingerprint may be included in the user feature information instead of or in addition to information on features based on the eye image. Such user feature information (information on the features of the photographer) or information from which the user feature information can be calculated may be stored in the storage unit 103.

[0042] The notification unit 105 determines whether or not both conditions, that registrant characteristic information has been acquired and that user characteristic information associated with the captured image exists, are met. If the result of this determination shows that registrant characteristic information has been acquired by the first information acquisition unit 101 and that user characteristic information associated with the captured image exists, the notification unit 105 determines that matching and authentication by the matching unit 106 is necessary. On the other hand, if at least one of the cases where registrant characteristic information has not been acquired by the first information acquisition unit 101 and where user characteristic information associated with the captured image does not exist, the notification unit 105 determines that matching and authentication by the matching unit 106 cannot be performed.

[0043] As described above, the registrant characteristic information is acquired, for example, by the first information acquisition unit 101. Furthermore, the user characteristic information is calculated, for example, by the second information acquisition unit 104 based on the eye image at the time of capture. If the calculation of the user characteristic information is successful, the user characteristic information is associated with the captured image and stored by the storage unit 103. However, for example, if the photographer did not place their eye in the finder 112 when capturing the image, or if the photographer placed their eye in the finder 112 but happened to have their eyes closed when capturing the eye image, acquisition of the user characteristic information may fail. In this case, there is no user characteristic information associated with the captured image.

[0044] The registrant characteristic information is information stored in a form that can be referenced by the photographing device 1. On the other hand, the user characteristic information is information that is stored in association with a photographed image each time the photographed image is obtained. Each time user characteristic information is obtained, the notification unit 105 determines whether or not matching and authentication by the matching unit 106 is necessary, and notifies the matching unit 106 of the result of the determination. Note that if the user characteristic information associated with the photographed image has already been matched with the registrant characteristic information by the matching unit 106, re-matching is not necessary. An example of the detailed operation of the notification unit 105 will be described later.

[0045] If the notification unit 105 determines that matching and authentication are necessary, the matching unit 106 compares (matches) the registrant characteristic information with the user characteristic information for each piece of user characteristic information. Then, the matching unit 106 determines whether or not the authentication of the photographer has been successful based on the result of the comparison. For example, the matching unit 106 may determine that the authentication of the photographer has been successful if the similarity between the registrant characteristic information and the user characteristic information is equal to or greater than a predetermined value, and may determine that the authentication has failed if not.

[0046] If there are multiple pieces of registrant characteristic information, the notification unit 105 may, for example, determine whether the similarity between the registrant characteristic information that has the highest similarity to the user characteristic information among the multiple pieces of registrant characteristic information and the user characteristic information is greater than or equal to a predetermined value.

[0047] If the authentication of the photographer is successful, the matching unit 106 performs a predetermined operation. The predetermined operation includes, for example, adding photographer information to the captured image associated with the user characteristic information in a form that cannot be tampered with. The photographer information includes, for example, identification information (name and ID) of the person who is the source of the registrant characteristic information. If this person is person A, it can be said that "this captured image was taken by person A." By the matching unit 106 adding such photographer information to the captured image in a form that cannot be tampered with, the authenticity of the photographer for the captured image can be guaranteed.

[0048] The above is an example of the functions of each unit of the photographing device 1 when the first information acquisition unit 101 acquires registrant characteristic information prior to photographing. However, there are cases where the registrant characteristic information has not been acquired by the first information acquisition unit 101 at the time of shooting. For example, there are cases where a photo opportunity occurs before the registrant characteristic information is acquired by the first information acquisition unit 101. In such cases, it is not possible to add photographer information to the captured image at the time of shooting. Therefore, in this embodiment, an example will be given of such a case where the registrant characteristic information is acquired after shooting.

[0049] In this embodiment, a case is illustrated in which, if registrant characteristic information has not been acquired by the first information acquisition unit 101 at the time of image capture, the notification unit 105 does not issue an instruction to the matching unit 106 to perform matching and authentication. Also, in this embodiment, a case is illustrated in which, when registrant characteristic information is acquired by the first information acquisition unit 101 after image capture, the notification unit 105 instructs the matching unit 106 to perform matching and authentication for each piece of unmatched user characteristic information stored in the storage unit 103. In this case, the matching unit 106 that has received the instruction performs matching and authentication for each piece of unmatched user characteristic information stored in the storage unit 103 as described above.

[0050] The notification unit 105 does not have to determine whether matching and authentication are necessary, and does not have to instruct matching and authentication. In this case, the matching unit 106 may match the registrant characteristic information and the user characteristic information regardless of whether the registrant characteristic information and the user characteristic information are present. If at least one of the registrant characteristic information and the user characteristic information is absent, the matching unit 106 determines that authentication of the photographer has failed.

[0051] Fig. 4 is a flowchart showing an example of the operation of the photographing device 1 of this embodiment. The flowchart in Fig. 4 is executed repeatedly at a predetermined cycle, for example. The flowchart in Fig. 4 may also be started when a predetermined operation is performed on the photographing device 1 (for example, an operation for photographing or an operation for acquiring registrant characteristic information).

[0052] In S401, the first information acquisition unit 101 determines whether or not new registrant characteristic information has been acquired. For example, if the first information acquisition unit 101 acquires the registrant characteristic information immediately after the power of the photographing device 1 is turned on from off, the determination in S401 may be YES. Furthermore, for example, if the first information acquisition unit 101 acquires the registrant characteristic information after a predetermined time has elapsed since the photographing device 1 acquired the registrant characteristic information or after the user has issued a logoff instruction, the determination in S401 may be YES.

[0053] If the result of this determination is that new registrant characteristic information has not been acquired (NO in S401), the process of S402 is performed. In S402, the second information acquisition unit 104 determines whether new user characteristic information has been acquired when capturing an image. For example, if the second information acquisition unit 104 acquires user characteristic information immediately after the power of the image capturing device 1 is turned on from off, the result of the determination in S402 may be YES. Furthermore, if the second information acquisition unit 104 acquires user characteristic information after a predetermined time has elapsed since the image capturing device 1 acquired the user characteristic information or after the user has issued a logoff instruction, the result of the determination in S402 may be YES. Furthermore, the image capturing device 1 may attempt to acquire user characteristic information every time image capturing is performed, and if the user characteristic information is acquired, the result of the determination in S402 may be YES.

[0054] If the result of this determination is that new user characteristic information has not been acquired (NO in S402), the registrant characteristic information and user characteristic information have not been newly acquired. In this case, the processing according to the flowchart in FIG. 4 ends.

[0055] On the other hand, if the user characteristic information has been newly acquired (YES in S402), the process of S403 is performed. In S403, the storage unit 103 stores the user characteristic information in association with the captured image. Here, it is assumed that the captured image is acquired simultaneously (or almost simultaneously) with the acquisition of the user characteristic information.

[0056] Next, in S404, the second information acquisition unit 104 notifies the notification unit 105 that the user characteristic information has been acquired and associated with the captured image. Note that the processing (notification) of S404 may be performed by the storage unit 103.

[0057] Next, in S405, the notification unit 105 determines whether or not registrant characteristic information has been acquired. If registrant characteristic information has been acquired, this is the case when it has been determined that registrant characteristic information has been newly acquired in the processing of S401 in the flowchart of FIG. 4 that has already been performed. This determination can be made, for example, based on whether or not a notification has been made in S410, which will be described later, in the flowchart of FIG. 4 that has already been performed. If the result of this determination is that registrant characteristic information has not been acquired (NO in S405), user characteristic information has been acquired during photography, but registrant characteristic information has not been acquired. In this case, the processing according to the flowchart of FIG. 4 ends.

[0058] On the other hand, if the registrant characteristic information has already been acquired (YES in S405), there is registrant characteristic information and user characteristic information (photography is performed with the registrant characteristic information acquired before photography). In this case, the processing of S406 is performed. In S406, the notification unit 105 instructs the matching unit 106 to perform matching and authentication. Then, in S407, the matching unit 106 matches the registrant characteristic information with the user characteristic information by comparing them.

[0059] Next, in S408, the matching unit 106 determines whether authentication of the photographer has been successful based on the result of the comparison in S407. This determination is made, for example, by determining whether the person identified from the user characteristic information and the person identified from the registrant characteristic information are the same person. If the result of this determination is that authentication has been successful (YES in S408), the processing of S409 is performed. In S409, the matching unit 106 assigns photographer information about the successfully authenticated user in an unalterable form to the captured image associated with the user characteristic information used for matching when it is determined that authentication has been successful. Then, the processing according to the flowchart of FIG. 4 ends. On the other hand, if the result of the determination in S408 is that authentication has failed (NO in S408), the processing according to the flowchart of FIG. 4 ends.

[0060] As a result of the determination in S401 described above, if it is determined that new registrant characteristic information has been acquired (YES in S401), the process of S410 is performed. In S410, the first information acquisition unit 101 notifies the notification unit 105 that the registrant characteristic information has been acquired.

[0061] Next, in S411, the second information acquisition unit 104 determines whether or not new user characteristic information has been acquired. If the result of this determination is that new user characteristic information has been acquired (YES in S411), the process of S412 is performed. As described in the process of S402, this illustrates a case where a captured image is acquired simultaneously (or almost simultaneously) with the acquisition of the user characteristic information. In S412, the storage unit 103 stores the user characteristic information in association with the captured image.

[0062] Next, in S413, the second information acquisition unit 104 notifies the notification unit 105 that the user characteristic information has been acquired and associated with the captured image. Note that the processing (notification) of S413 may be performed by the storage unit 103. When the processing of S413 is performed, the registrant characteristic information and the user characteristic information are available (the registrant characteristic information and the user characteristic information were acquired at the time of capturing the image). In this case, the processing of S406 and S407 described above is performed.

[0063] If the result of the determination in S411 described above is that the user characteristic information has not been acquired (NO in S411), the process of S414 is performed. In S414, the notification unit 105 determines whether or not the user characteristic information has already been acquired. If the user characteristic information has already been acquired, this is the case when it has been determined that the user characteristic information has been newly acquired in the processes of S402 and S411 in the flowchart of FIG. 4 that have already been performed. This determination is made, for example, based on whether or not the notification in S404 in the flowchart of FIG. 4 has already been performed. If the result of this determination is that the user characteristic information has not been acquired (NO in S411), the user characteristic information has not been acquired. In this case, the process in the flowchart of FIG. 4 ends.

[0064] On the other hand, if user characteristic information has already been acquired (YES in S414), there is registrant characteristic information and user characteristic information (user characteristic information was acquired at the time of shooting, and registrant characteristic information was acquired before shooting). In this case, the process of S415 is performed. In S415, the notification unit 105 collects user characteristic information that has already been notified of being associated with the captured image in S404 in the flowchart of FIG. 4. Note that the user characteristic information collected in S415 does not need to include user characteristic information that has already been used in the matching process by the matching unit 106. Then, the processes of S406 and S407 described above are performed. In this case, in S406, the notification unit 105 instructs the matching unit 106 to individually match all of the user characteristic information collected in S415 with the registrant characteristic information.

[0065] As described above, in this embodiment, the photographing device 1 (or the information processing system included therein) acquires registrant characteristic information as an example of a first characteristic of a person. The photographing device 1 also acquires user characteristic information as an example of a second characteristic of a person. The photographing device 1 identifies the photographer by matching the registrant characteristic information with the user characteristic information. When photographing is performed without detecting registrant characteristic information corresponding to the user characteristic information, the photographing device 1 performs recording control to associate the photographed image obtained by photographing with the user characteristic information and record them in memory. The memories that record the photographed image and the user characteristic information that are associated with each other may be the same storage medium or different storage media. After that, when registrant characteristic information is acquired, the photographing device 1 determines photographer information to be associated with the photographed image associated with the user characteristic information, based on the registrant characteristic information and the user characteristic information recorded in memory.

[0066] Therefore, even if information for identifying the photographer is not available at the time of shooting, information about the photographer (photographer information) can be identified after shooting. Therefore, for example, even if registrant characteristic information is registered after shooting, the authenticity of the photographer can be guaranteed for the photographed image.

[0067] (Second embodiment) Next, a second embodiment will be described. In this embodiment, a case will be described in which the photographer (for example, the person looking through the viewfinder 112) is authenticated before registrant characteristic information is acquired. In this way, it is possible to prevent a third party from being registered as a registrant. Furthermore, since sufficient time can be taken for the authentication at this time, highly accurate authentication can be performed. As described above, the main difference between this embodiment and the first embodiment is the configuration and processing for acquiring registrant characteristic information. Therefore, in the description of this embodiment, the same parts as those in the first embodiment will be assigned the same reference numerals as those assigned to FIGS. 1 to 4, and detailed description thereof will be omitted.

[0068] Fig. 5 is a diagram showing an example of the configuration of first information acquisition unit 101. Fig. 5 illustrates an example in which first information acquisition unit 101 includes preliminary authentication information acquisition unit 1011, preliminary collation unit 1012, preliminary authentication information holding unit 1013, authentication information holding unit 1014, and authentication information acquisition unit 1015.

[0069] Furthermore, in this embodiment, when a photographer looks through the viewfinder 112 to give a shooting instruction, authentication (identification of the photographer) is performed using an image of the photographer's eye placed in the viewfinder 112. However, the method of authenticating a photographer in this embodiment is not limited to this example. For example, when a photographer gives a shooting instruction while viewing an image displayed in live view on a display panel provided with the operation unit 113 without looking through the viewfinder 112, a captured image of the face of a person viewing the live view screen may be used for authentication. In this case, the captured area may be, for example, an area in front of the live view screen that includes an area at a predetermined distance from the operation unit 113. For example, when a distance sensor provided in the photographing device 1 determines that a person's face is within a predetermined distance from the photographing device 1, the photographing device 1 may perform authentication based on an image of the face. Furthermore, when a person's face in a photographed image is equal to or larger than a predetermined size, the photographing device 1 may perform authentication based on an image of the face.

[0070] The preliminary authentication information acquisition unit 1011 acquires an eye image of the photographer (Mr. A) looking through the viewfinder 112. As described in the first embodiment, the eye image is obtained, for example, by using the eye imaging element 214 and the eye photographing processing circuit 218. Then, the preliminary authentication information acquisition unit 1011 calculates (extracts) user preliminary authentication information from the eye image.

[0071] The user preliminary authentication information is information that can identify an individual person, and is also called a feature or a feature vector. The feature (user preliminary authentication information) has the same properties as the user feature information described in the first embodiment. As described in the first embodiment, the calculation of the feature (user preliminary authentication information) itself can be realized by using, for example, known techniques in the fields of object recognition, face recognition, iris recognition, etc.

[0072] The preliminary verification unit 1012 compares the user preliminary authentication information acquired by the preliminary authentication information acquisition unit 1011 with the registrant preliminary authentication information stored in the preliminary authentication information storage unit 1013 . The preliminary authentication information storage unit 1013 stores registrant preliminary authentication information of people including Mr. A. The registrant preliminary authentication information of Mr. A is information representing feature amounts extracted in advance from an eye image of Mr. A, and is information on one or more people who are assumed to be users of the photographing device 1.

[0073] The preliminary verification unit 1012 compares (verifies) the user preliminary authentication information with the registrant preliminary authentication information. If the comparison shows that there is registrant preliminary authentication information whose similarity with the user preliminary authentication information is equal to or greater than a predetermined value, the preliminary verification unit 1012 determines that the photographer identified from the user preliminary authentication information is the registrant identified by the registrant preliminary authentication information. If the person identified from the registrant preliminary authentication information is Mr. A, the person (user) looking through the finder 112 is determined to be Mr. A (the registrant).

[0074] If the preliminary collation unit 1012 determines that the photographer is a registrant, the authentication information acquisition unit 1015 acquires the authentication information of the photographer. The authentication information holding unit 1014 stores the authentication information as registrant characteristic information. The authentication information acquisition unit 1015 may be different from or the same as the second information acquisition unit 104 in Fig. 1. The authentication information acquisition unit 1015 may acquire, as the registrant characteristic information, authentication information suitable for "comparison" in the matching unit 106. Furthermore, when authenticating during photography, if an eye image of the photographer looking through the viewfinder 112 is used, as in the preliminary authentication by the preliminary matching unit 1012, the second information acquisition unit 104 may acquire user characteristic information based on the eye image acquired by the preliminary authentication information acquisition unit 1011.

[0075] Furthermore, if the user characteristic information is acquired in advance and stored in the preliminary authentication information holding unit 1013 in association with the registrant preliminary authentication information, the authentication information acquisition unit 1015 may be omitted.

[0076] As described above, in this embodiment, the photographing device 1 (or the information processing system included therein) authenticates the photographer (e.g., the person looking through the finder 112), and if the photographer is a registrant, stores the authentication information of the photographer as registrant characteristic information. Therefore, the reliability of the registrant characteristic information can be further improved.

[0077] (Third embodiment) Next, a third embodiment will be described. Photographer information is usually assigned at the time of shooting. However, there are cases where photographer information is not assigned at the time of shooting. That is, there are cases where verification and authentication are not performed at the time of shooting. In such cases, verification and authentication are performed after shooting. In this embodiment, a case where the photographer is notified that photographer information will not be assigned and the photographer is prompted to confirm whether or not to assign photographer information is illustrated. As described above, this embodiment differs from the first and second embodiments mainly in the configuration and processing for notifying when photographer information is not assigned at the time of shooting. Therefore, in the description of this embodiment, the same parts as those in the first and second embodiments are denoted by the same reference numerals as those in FIGS. 1 to 5, and detailed description thereof will be omitted.

[0078] 6 is a diagram showing an example of the functional configuration of (an information processing system included in) the image capturing device 1 of this embodiment. In FIG. 6, the image capturing device 1 further includes a display unit 107 in addition to a first information acquiring unit 101, a data acquiring unit 102, a storage unit 103, a second information acquiring unit 104, a notification unit 105, and a matching unit 106. 7 is a flowchart showing an example of the operation of the image capturing device 1 of this embodiment. The flowchart in FIG. 7 is obtained by adding S701 to S703 to the flowchart in FIG.

[0079] As described in the first embodiment, if the result of the determination in S405 is that registrant characteristic information has not been acquired (NO in S405), the user characteristic information has been acquired but the registrant characteristic information has not, so photographer information cannot be added to the captured image. In this case, in the first embodiment (FIG. 4), the processing according to the flowchart in FIG. 4 ends. In contrast, in this embodiment (FIG. 7), the processing of S701 is performed.

[0080] In S701, the display unit 107 displays information indicating that photographer information cannot be added to the captured image on the display panel provided in the operation member 113. An example of the information displayed in S701 is shown in FIG. 8(a). Note that the information displayed in S701 may be displayed on a display device other than the display panel provided in the operation member 113. For example, the information displayed in S701 may be displayed on a display device connected to the image capturing device 1 so as to be able to communicate with the image capturing device 1. Note that the same applies to other information displayed on the display panel provided in the operation member 113.

[0081] Furthermore, as described in the first embodiment, if the result of the determination in S414 is that user characteristic information has already been acquired (YES in S414), it is possible to add photographer information to the captured image because registrant characteristic information has been acquired after shooting. In this case, in the first embodiment (FIG. 4), the process of S415 is performed. In contrast, in the present embodiment (FIG. 7), the processes of S415, S406, and S407 are performed after the processes of S702 and S703 are performed.

[0082] In S702, the display unit 107 displays information on the display panel provided on the operation member 113 to prompt the photographer to confirm whether or not it is OK to add photographer information to the captured image. An example of the information displayed in S702 is shown in FIG. 8(b). FIG. 8(b) illustrates an example in which the photographer selects the OK button 811 when the photographer wishes to add photographer information to the captured image. FIG. 8(b) also illustrates an example in which the photographer selects the cancel button 812 when the photographer does not wish to add photographer information to the captured image. The OK button 811 and the cancel button 812 may be selected by operating the operation members 114 and 115, for example, or by pressing the operation member 113 (touch panel).

[0083] Next, in S703, the notification unit 105 determines whether or not to assign photographer information to the captured image. In the example shown in FIG. 8(b), when the OK button 811 is selected, the notification unit 105 determines that photographer information is to be assigned to the captured image. On the other hand, when the cancel button 812 is selected, the notification unit 105 determines that photographer information is not to be assigned to the captured image. If the result of this determination is that photographer information is to be assigned to the captured image (YES in S703), the processing of S415 (collection of user characteristic information) described in the first embodiment is performed. On the other hand, if the result of the determination in S703 is that photographer information is not to be assigned to the captured image (NO in S703), the processing according to the flowchart in FIG. 7 ends.

[0084] If it is not necessary to check whether the photographer authentication information is attached to the captured image, the process (determination) of S703 may not be performed. In this case, the process of S415 may be performed after S702. Furthermore, processing similar to S702 and S703 may be performed between S411 and S412. In this case, for example, if the photographer selects to add photographer information to the captured image as a result of performing processing similar to S702 and S703, the processing of S412 is performed. On the other hand, if the photographer selects not to add photographer information to the captured image, for example, the processing of S412 is not performed and the processing according to the flowchart of FIG. 7 ends.

[0085] As described above, in this embodiment, the photographing device 1 (or the information processing system included therein) notifies the photographer that photographer information cannot be added to the photographed image. Also, in this embodiment, the photographing device 1 adds the photographer information to the photographed image after having the photographer confirm whether or not to add the photographer information to the photographed image. Therefore, the photographer can know that photographer information cannot be added to the photographed image. Also, the photographer can decide whether or not to add photographer information to the photographed image at their own discretion. This makes it possible to prevent the photographed image file from being inadvertently altered.

[0086] In the first to third embodiments, the photographing device 1 acquires user characteristic information based on an eye image at the time of photographing. Also, in the first to third embodiments, the photographing device 1 adds photographer information to the photographed image in a tamper-proof form after authenticating the photographer. However, the present disclosure is not limited to this example, and any implementation form is possible as long as the gist of the present disclosure is implemented.

[0087] (Fourth embodiment) Next, a fourth embodiment will be described. In the first to third embodiments, a case has been illustrated in which captured images are stored in the holding unit 103 (or the external data storage unit 1031) and the storage medium storing the captured images is not removed from the photographing device 1. However, there are cases in which the storage medium storing the captured images is removed from the main body of the photographing device 1 and then reattached. Therefore, in this embodiment, a case in which captured images are stored in a storage medium that is detachable (removable and attachable) to the photographing device 1 will be described. As described above, this embodiment differs from the first to third embodiments mainly in the configuration and processing due to the fact that the storage medium storing the captured images is detachable from the photographing device 1. Therefore, in the description of this embodiment, the same parts as those in the first to third embodiments will be denoted by the same reference numerals as those in FIGS. 1 to 8, and detailed description thereof will be omitted.

[0088] In this embodiment, a case will be described in which registrant characteristic information is not acquired at the time of shooting, and the external data storage unit 1031 is removed from the photographing device 1 before acquiring the registrant characteristic information after shooting, and then reattached to the photographing device 1. In the following description, a storage medium that is detachable (can be removed and attached) to the photographing device 1 will be referred to as an external storage medium as necessary.

[0089] FIG. 9 is a flowchart showing an example of the operation of the holding unit 103 when the external data storage unit 1031 is attached to the photographing device 1.

[0090] When an external storage medium that has been removed from the main body of the photographing device 1 is attached to the main body of the photographing device 1, the holding unit 103 (external data storage unit 1031) detects this in S901.

[0091] Next, in S902, the external data storage unit 1031 collects all user feature information associated with the captured images stored in the external storage medium. Note that the user feature information collected in S903 does not necessarily include user feature information that has already been used in the matching process by the matching unit 106.

[0092] Next, in S903, the external data storage unit 1031 notifies the notification unit 105 of the user characteristic information collected in S902 one by one. An example of the operation of the notification unit 105 upon receiving this notification is the same as that described in the first embodiment. For example, the processing of S405 in FIG. 4 or FIG. 7 may be performed.

[0093] As described above, in this embodiment, even if the storage medium on which the photographed image is stored is removed from the photographing device 1 and then reattached to the photographing device 1 without photographer information being added to the photographed image, the authenticity of the photographer for the photographed image can be guaranteed.

[0094] The user characteristic information may be stored in a storage medium that cannot be removed from the image capturing device 1, or the user characteristic information may be stored in an external storage medium together with the captured image.

[0095] (Fifth embodiment) Next, a fifth embodiment will be described. User characteristic information is sensitive information that can identify an individual. Furthermore, when association with a captured image is performed using a hash value, it is preferable that the hash value not be tampered with. Therefore, in this embodiment, a case where processing including encryption is performed on user characteristic information is illustrated. As described above, this embodiment differs from the first to fourth embodiments mainly in the configuration and processing due to different processing on user characteristic information. Therefore, in the description of this embodiment, the same parts as those in the first to fourth embodiments are denoted by the same reference numerals as those in FIGS. 1 to 9, and detailed description thereof will be omitted.

[0096] 10 is a diagram showing an example of the configuration of the holding unit 103. In FIG. 10, the holding unit includes an external data storage unit 1031, an encryption / decryption unit 1032, and a signature processing unit 1033. The encryption / decryption unit 1032 encrypts and decrypts the user characteristic information. The signature processing unit 1033 digitally signs and verifies the signature on the user characteristic information. The storage unit 103 (encryption / decryption unit 1032, signature processing unit 1033) may encrypt and digitally sign the user characteristic information when storing the user characteristic information. The storage unit 103 (encryption / decryption unit 1032, signature processing unit 1033) may also perform signature verification and decryption on the user characteristic information when reading the user characteristic information. Note that the encryption, decryption, digital signature, and signature verification themselves can be realized using known technology. Therefore, detailed descriptions of the encryption, decryption, digital signature, and signature verification will be omitted here.

[0097] As described above, in this embodiment, the photographing device 1 (or the information processing system included therein) stores the user characteristic information after encrypting and signing it. Therefore, even if the user characteristic information is stored in a storage medium detachable from the photographing device 1, it is possible to prevent the user characteristic information from being used improperly. For example, if the user characteristic information is stored in an external storage medium, there is a high risk that the user characteristic information will be used improperly. For example, it is effective to apply the method of this embodiment to such a case (e.g., the fourth embodiment). Furthermore, the method of this embodiment may be applied to a storage medium other than a storage medium detachable from the photographing device 1. For example, the method of this embodiment may be applied to user characteristic information stored in a storage medium external to the photographing device 1 other than a storage medium detachable from the photographing device 1. Furthermore, this embodiment makes it possible to prevent photographer information from being assigned to images photographed by other people.

[0098] (Sixth embodiment) Next, a sixth embodiment will be described. In the fourth embodiment, a case where an external storage medium storing captured images without photographer information is removed from the photographing device 1 is exemplified. In this embodiment, a case where the photographer is notified of this and the user is asked whether or not to remove the external data storage unit 1031 is exemplified. As described above, this embodiment and the fourth embodiment differ mainly in the configuration and processing for notifying the user when the external data storage unit 1031 is removed. Therefore, in the description of this embodiment, the same parts as those in the first to fifth embodiments are denoted by the same reference numerals as those in FIGS. 1 to 10, and detailed description thereof will be omitted.

[0099] FIG. 11 is a flowchart showing an example of the operation of the holding unit 103 of this embodiment. In S1101, the holding unit 103 waits until a request to remove the external storage medium is received. The request to remove the external storage medium is generated, for example, when the photographer performs a predetermined operation on the operation members 113 to 115.

[0100] When a request to remove the external storage medium is made, the process of S1102 is performed. In S1102, the external data storage unit 1031 collects all user feature information associated with the captured images stored in the external storage medium. Note that the user feature information collected in S1102 does not necessarily include user feature information that has already been used in the matching process by the matching unit 106.

[0101] Next, in S1103, the external data storage unit 1031 determines whether there is any user characteristic information for which the matching process has not been performed. If the result of this determination is that there is no user characteristic information for which the matching process has not been performed (NO in S1103), the processes of S1104 to S1105 are omitted and the process of S1106 is performed. In S1106, the external data storage unit 1031 removes the external storage medium from the imaging device 1.

[0102] If there is a mechanism for electrically removing the external storage medium, the external storage medium may be removed from the photographing device 1 by the operation of that mechanism. Alternatively, if the external storage medium is manually removed from the photographing device 1, the display unit 107 may display information indicating that the external storage medium may be removed on a display panel or the like provided on the operation member 113. An example of the information thus displayed in S1106 is shown in Fig. 12(a). When the processing of S1106 ends, the processing according to the flowchart of Fig. 11 ends.

[0103] If the result of the determination in S1103 described above indicates that there is user characteristic information for which the matching process has not been performed (YES in S1103), a captured image to which photographer information has not been assigned is stored in the external storage medium. In this case, the process of S1104 is performed. In S1104, the display unit 107 displays, on a display panel or the like provided on the operation member 113, information indicating that a captured image to which photographer information has not been assigned is stored in the external storage medium, and information for prompting the user to confirm whether or not to remove the external storage medium. An example of the information displayed in S1104 is shown in FIG. 12(b). FIG. 12(b) illustrates an example in which the photographer selects the OK button 1211 if the photographer wishes to remove the external storage medium. FIG. 12(b) also illustrates an example in which the photographer selects the cancel button 1212 if the photographer does not wish to remove the external storage medium. The OK button 1211 and the cancel button 1212 may be selected by operating the operation members 114 to 115, or by pressing the operation member 113 (touch panel), for example.

[0104] Next, in S1105, the external data storage unit 1031 determines whether or not to remove the external storage medium. In the example shown in FIG. 12(b), if the OK button 1211 is selected, the external data storage unit 1031 determines that the external storage medium is to be removed. On the other hand, if the cancel button 1212 is selected, the external data storage unit 1031 determines that the external storage medium is not to be removed. If the result of this determination is that the external storage medium is not to be removed (NO in S1105), the processing according to the flowchart in FIG. 11 ends. On the other hand, if the external storage medium is to be removed (YES in S1105), the processing of S1106 described above is performed. In this case, the external storage medium is removed from the image capturing device 1.

[0105] As described above, in this embodiment, the photographing device 1 (or the information processing system included therein) notifies the photographer that photographed images without photographer information are stored in a storage medium before the storage medium on which the photographed images without photographer information are stored is removed from the photographing device 1. Furthermore, in this embodiment, the photographing device 1 performs processing for removing the storage medium after having the photographer confirm whether or not to remove the storage medium on which the photographed images without photographer information are stored. Therefore, the photographer can know that the storage medium to be removed contains photographed images without photographer information. Furthermore, the photographer can decide whether or not to remove the storage medium at their own discretion. Therefore, it is possible to more accurately prevent the user characteristic information from being used improperly. Furthermore, in this embodiment, it is possible to more accurately prevent the photographer information from being assigned to photographed images taken by another person. The notification may be performed after the storage medium on which the photographed images without photographer information are stored is removed from the photographing device 1.

[0106] (Seventh embodiment) Next, a seventh embodiment will be described. In the fourth and sixth embodiments, a case has been exemplified in which a photographed image without photographer information attached thereto is taken out of the photographing device 1 using a storage medium detachable from the photographing device 1, and then reattached to the photographing device 1. In addition, in the first to sixth embodiments, a case has been exemplified in which photographer information is attached to a photographed image within the photographing device 1.

[0107] However, the captured image does not necessarily have to be stored in a storage medium that can be removed from the photographing device 1, and the addition of photographer information does not necessarily have to be performed within the photographing device. Therefore, in this embodiment, a case is illustrated in which the photographed image is transmitted from the photographing device 1 to an external device, and the external device adds photographer information to the photographed image. As described above, this embodiment differs from the first to sixth embodiments mainly in the configuration and processing due to the different devices that store the photographed image and add photographer information. Therefore, in the description of this embodiment, the same parts as those of the first to sixth embodiments are denoted by the same reference numerals as those used in FIGS. 1 to 12, and detailed description thereof will be omitted.

[0108] Fig. 13 is a diagram showing an example of the configuration of an information processing system. Fig. 13 illustrates a case where an image capturing device 1 and an external server 2 are connected to be able to communicate with each other via a network 3. In this case, the image capturing device 1 can transmit captured images to the server 2 as well as to a removable storage medium for the image capturing device 1. In addition, the server 2 can also add photographer information.

[0109] FIG. 14 is a diagram illustrating an example of a hardware configuration of the server 2. As shown in FIG. The CPU 1411 controls the entire server 2 using computer programs and various data, thereby realizing each function of the server 2. The ROM (Read Only Memory) 1412 is a non-volatile memory that stores various computer programs and various data as information required for the CPU 1411 to perform processing. The RAM (Random Access Memory) 1413 is a volatile memory that temporarily stores computer programs and data processed by the CPU 1411. The RAM 1413 temporarily stores, for example, data processed by the CPU 1411 and program control variables. The RAM 1413 is, for example, a DRAM (Dynamic Random Access Memory). The auxiliary storage device 1414 is a non-volatile memory that stores various data and the like that may be changed. The auxiliary storage device 1414 may store computer programs that may be changed, such as the control program of the CPU 1411. The auxiliary storage device 1414 is, for example, a hard disk drive (HDD), a solid state drive (SSD), or an embedded multi media card (eMMC). The auxiliary storage device 1414 may include multiple storage devices (for example, an HDD and an eMMC).

[0110] In the server 2 of this embodiment, a case is exemplified in which one CPU 1411 executes each process shown in the flowcharts described below using one memory (RAM 1413). However, this is not necessarily the case. For example, multiple CPUs, RAMs, ROMs, and auxiliary storage devices may work together to execute each process shown in the flowcharts described below. Furthermore, at least some of the processes shown in the flowcharts may be executed by dedicated hardware. Examples of dedicated hardware include ASICs and FPGAs. Furthermore, the processor is not limited to a CPU. The processor may also be, for example, a GPU.

[0111] The operation device 1415 accepts operations by a user and outputs various instructions to the CPU 1411. The operation device 1415 includes, for example, a keyboard, a pointing device (e.g., a mouse), and a touch panel. The display device 1416 displays various information. The display device 1416 displays, for example, a GUI (Graphical User Interface) for a user to operate the server 2. The display device 1416 includes, for example, a computer display such as a liquid crystal display and light-emitting elements such as LEDs (Light Emitting Diodes). The communication I / F 1417 is an interface through which the server 2 communicates with an external device. When the server 2 is connected to the external device via a wired connection, a communication cable is connected to the communication I / F 1417. When the server 2 has a function for wireless communication with the external device, the communication I / F 1417 includes an antenna. The data bus 1418 connects the various components of the server 2. The various components of the server 2 transmit information via the data bus 1418.

[0112] The hardware of the server is not limited to the configuration exemplified in Fig. 14. The hardware of the server itself may be realized by known technology.

[0113] 15 is a diagram showing an example of the functional configuration of the image capturing device 1 (or the information processing system included therein) of this embodiment. FIG. 15 illustrates an example in which the image capturing device 1 further includes a communication unit 108 in addition to a first information acquisition unit 101, a data acquisition unit 102, a storage unit 103, a second information acquisition unit 104, a notification unit 105, and a matching unit 106. The image capturing device 1 may also include a display unit 107.

[0114] 16 is a diagram illustrating an example of the functional configuration of the server 2 of this embodiment. In FIG. 16, the server 2 includes a second communication unit 201, a second storage unit 202, and a second verification unit 203. The communication unit 108 of the image capturing device 1 and the second communication unit 201 of the server 2 are connected to each other so as to be able to communicate with each other via a network 3. The network 3 may be a wired network or a wireless network. The network 3 may also be, for example, a local area network (LAN), a wide area network (WAN), the Internet, or a combination of these networks.

[0115] First, photographed images to which photographer information has not been added are stored in the storage unit 103 of the photographing device 1. The photographed images to which photographer information has not been added are transmitted from the communication unit 108 of the photographing device 1. The second communication unit 201 of the server 2 receives the photographed images to which photographer information has not been added. The second storage unit 202 stores the photographed images to which photographer information has not been added that have been received by the second communication unit 201.

[0116] Furthermore, the user characteristic information associated with the photographed image is also stored in the second storage unit 202 in the same way as the photographed image to which no photographer information is attached. Furthermore, at least one of the captured image and the user characteristic information may be transmitted to the server 2 via the network 3 in this manner, or may be transmitted to the server 2 by connecting a removable storage medium (external data storage unit 1031) to the server 2 relative to the photographing device 1.

[0117] Next, the first information acquisition unit 101 of the photographing device 1 acquires registrant characteristic information. In this embodiment, in this manner, in a state in which the photographing device 1 has registrant characteristic information and the photographed image to which photographer information has not been assigned and the user characteristic information associated with the photographed image are stored in the server 2, the server 2 assigns photographer information to the photographed image stored in the server 2.

[0118] FIG. 17 is a flowchart showing an example of the operation of the server 2 of this embodiment. First, in S1701, the second communication unit 201 receives the registrant characteristic information transmitted from the communication unit 108 of the image capturing device 1. Next, in S1702, the second matching unit 203 collects all user feature information associated with the captured image. Note that the user feature information collected in S1702 does not necessarily include user feature information that has already been used in the matching process by the second matching unit 203. Then, the following processes (S1703 to S1705) are performed for each piece of user characteristic information collected in S1702.

[0119] First, in S1703, the second matching unit 203 matches the registrant characteristic information with the user characteristic information by comparing them. Next, in S1704, the second matching unit 203 determines whether or not the authentication of the photographer has been successful based on the result of the comparison. This determination is made, for example, by determining whether or not the person identified from the user characteristic information and the person identified from the registrant characteristic information are the same person. Note that this processing is similar to the processing of the matching unit 106 already described.

[0120] If the result of this determination is that the authentication is successful (YES in S1704), the processing of S1705 is performed. In S1705, the second matching unit 203 instructs the second storage unit 202 to add, in a tamper-proof form, photographer information about the successfully authenticated user to the captured image associated with the user characteristic information used for matching when it is determined that the authentication is successful. Based on the instruction, the second storage unit 202 adds the photographer information to the captured image in a tamper-proof form. Then, the processing according to the flowchart of FIG. 17 ends. On the other hand, if the result of the determination in S1704 is that the authentication is unsuccessful (NO in S1704), the processing according to the flowchart of FIG. 17 ends.

[0121] As described above, in this embodiment, the photographing device 1 transmits the photographed image, user characteristic information, and registrant characteristic information to the server 2. The server 2 compares the registrant characteristic information with the user characteristic information and assigns photographer information to the photographed image based on the comparison result. Therefore, for photographed images that do not have photographer information assigned when they are extracted from the photographing device 1, photographer information can be assigned to the photographed image outside the photographing device 1. Note that the photographer information assigned to the photographed image by the photographing device 1 and the photographer information assigned to the photographed image by the server 2 may be the same or different. For example, information regarding the type of device to which the photographer information is assigned may be included in the photographer information.

[0122] (Eighth embodiment) Next, an eighth embodiment will be described. In the seventh embodiment, a case where the photographing device 1 transmits registrant characteristic information to the server 2, and the server 2 performs the matching and authentication processing, is exemplified. In contrast, in this embodiment, a case where the photographing device 1 performs the matching and authentication processing is exemplified. As described above, the present embodiment and the seventh embodiment differ mainly in the configuration and processing due to the different entities performing the matching and authentication processing. Therefore, in the description of this embodiment, the same parts as those of the first to seventh embodiments will be assigned the same reference numerals as those assigned to FIGS. 1 to 17, and detailed description thereof will be omitted.

[0123] FIG. 18 is a flowchart showing an example of the operation of the information processing system (the image capturing device 1 and the server 2) of this embodiment. First, in S1801, the second matching unit 203 collects all user feature information associated with the captured image. Note that the user feature information collected in S1801 does not necessarily include user feature information that has already been used in the matching process by the second matching unit 203. Then, the following processes (S1802 to S1811) are performed for each piece of user characteristic information collected in S1801.

[0124] First, in S1802, the second matching unit 203 selects one unselected piece of user characteristic information from the user characteristic information collected in S1801. Next, in S1803, the second matching unit 203 requests the image capturing device 1 through the second communication unit 201 to match and authenticate the user characteristic information selected in S1802. Next, in S1804, upon receiving a request for collation and authentication of user characteristic information via the communication unit 108, the collation unit 106 compares the registrant characteristic information with the user characteristic information to collate the two.

[0125] Next, in S1805, the matching unit 106 determines whether or not the authentication of the photographer has been successful based on the result of the comparison in S1804. The processing in S1804 and S1805 is the same as the verification and authentication processing already explained (see S407 and S408, etc.).

[0126] If the result of this determination is that the authentication is successful (YES in S1805), the processing of S1806 is performed. In S1806, the communication unit 108 transmits authentication result information indicating that the authentication is successful and the registrant information used for the comparison when it is determined that the authentication is successful to the server 2. In this way, the imaging device 1 of this embodiment transmits registrant information (e.g., identification information of the registrant) rather than transmitting registrant characteristic information regarding the characteristics of the registrant to the server 2. This has the advantage of allowing the registrant characteristic information to be handled more securely.

[0127] Next, in S1807, the second matching unit 203 acquires the authentication result information and registrant information transmitted in S1806 via the second communication unit 201. Also in S1807, the second matching unit 203 determines whether or not photographer information should be added to the captured image based on the authentication result information. In the description of this flowchart, since authentication was successful in S1805, the second matching unit 203 determines that photographer information should be added to the captured image and proceeds to S1808. Next, in S1808, the second matching unit 203 instructs the second storage unit 202 to add, in an unalterable form, photographer information about the successfully authenticated user to the captured image associated with the user characteristic information selected in S1802. Based on the instruction, the second storage unit 202 adds, in an unalterable form, the photographer information to the captured image. Then, the processing of S1811, which will be described later, is performed.

[0128] If the result of the determination in S1805 described above is that the authentication has failed (NO in S1805), the process of S1809 is performed. In S1809, the communication unit 108 transmits authentication result information indicating that the authentication has failed to the server 2.

[0129] Next, in S1810, the second matching unit 203 acquires the authentication result information transmitted in S1809 via the second communication unit 201. Also, in S1810, the second matching unit 203 determines whether or not photographer information should be added to the captured image based on the authentication result information. In the description of this flowchart, since authentication failed in S1805, the second matching unit 203 determines that photographer information should not be added to the captured image and proceeds to S1811. In this case, photographer information is not added to the captured image. Then, the processing of S1811 is performed.

[0130] In S1811, the second matching unit 203 determines whether or not all of the user characteristic information collected in S1801 has been selected. If the result of this determination is that all of the user characteristic information collected in S1801 has not been selected (NO in S1811), the processing of S1802 described above is performed again. The processing of S1802 to S1811 is repeated until it is determined in S1811 that all of the user characteristic information collected in S1801 has been selected. If it is determined in S1811 that all of the user characteristic information collected in S1801 has been selected (YES in S1811), the processing according to the flowchart of FIG. 18 ends.

[0131] For example, in S1803, the server 2 (second matching unit 203) may request the image capturing device 1 to match and authenticate all of the user characteristic information collected in S1801 all at once. In this case, the processes of S1802 and S1811 may not be performed. Furthermore, the processes of S1804 and S1805 are performed individually for all of the user characteristic information collected in S1801. Then, based on the result of the determination in S1805 (the result of the determination as to whether or not authentication was successful), the following processes may be performed.

[0132] For example, the communication unit 108 may transmit all of the sets of registrant information and user characteristic information used for matching when it is determined that the authentication was successful, or all of the information that can identify the user characteristic information, to the server 2. Furthermore, the communication unit 108 may transmit all of the user characteristic information used for matching when it is determined that the authentication was unsuccessful, or all of the information that can identify the user characteristic information, to the server 2.

[0133] As described above, in this embodiment, the server 2 requests the photographing device 1 to authenticate the photographer. The photographing device 1 performs matching and authentication processing based on the request. If the photographing device 1 successfully authenticates the photographer, it transmits to the server 2 the registrant information used for matching when it determines that authentication was successful. Upon receiving the registrant information, the server 2 assigns the photographer information about the photographer who requested authentication to the photographed image. In this embodiment, the server 2 assigns the registrant information received from the photographing device 1 to the photographed image as photographer information. In other words, the server 2 can assign photographer information to a photographed image outside of the photographing device 1, even if the photographed image did not have photographer information assigned to it when it was extracted from the photographing device 1. Furthermore, in this embodiment, registrant characteristic information regarding the characteristics of the registrant is not communicated over the network, but rather registrant information (e.g., identification information of the registrant) is communicated. This has the advantage of allowing the registrant characteristic information to be handled more securely.

[0134] (Ninth embodiment) Next, a ninth embodiment will be described. In the first to eighth embodiments, the case where one type of user characteristic information is acquired in the photographing device has been described. In this embodiment, a case where multiple types (two or more types) of user characteristic information can be acquired in the photographing device will be illustrated.

[0135] In this embodiment, for example, when multiple types of user characteristic information are acquired during image capture and registrant characteristic information is acquired after image capture, a type of registrant characteristic information selected from the multiple types is acquired. By doing so, for example, it is possible to acquire registrant characteristic information that is easier to authenticate. Therefore, authentication can be performed with higher accuracy. For example, assume that an eye image and a fingerprint are acquired as user characteristic information during image capture. In this case, if the eye image is captured under strong lighting, the brightness of the eye image may be high. In this case, the eye image may be an image that is not suitable for authentication. On the other hand, a fingerprint can be acquired as information suitable for authentication regardless of lighting conditions. Therefore, acquiring a fingerprint as registrant characteristic information can improve authentication accuracy. Furthermore, for example, if the photographer's fingers have rough skin, the fingerprint may be information that is not suitable for authentication. In such a case, the eye image may be information that is more suitable for authentication than the fingerprint.

[0136] Here, eye images and fingerprints have been used as examples. However, which type of feature information is suitable for authentication varies depending on the circumstances at the time of capture. Therefore, it is effective to acquire multiple types of user feature information. As described above, the present embodiment differs from the first to eighth embodiments mainly in the configuration and processing for acquiring registrant feature information. Therefore, in the description of this embodiment, the same parts as those in the first to eighth embodiments will be assigned the same reference numerals as those assigned in Figures 1 to 18, and detailed description will be omitted. Note that, in this embodiment, when the term "type of feature information" is used, it refers to the type of feature that is the basis of the feature information, unless there is any misunderstanding. Hereinafter, in this embodiment, the case where eye images, face images, fingerprints, and voice are acquired as user feature information will be described as an example.

[0137] Fig. 19 is a diagram showing an example of the external configuration of the imaging device of this embodiment. Fig. 19(a) is an example of a front perspective view of the imaging device 1900. Fig. 19(b) is an example of a rear perspective view of the imaging device 1900. 19(a), the image capturing device 1900 differs from the image capturing device 1 shown in FIG. 1 in the following respects. In the image capturing device 1900 of this embodiment, the release button 111 is equipped with a fingerprint sensor. Furthermore, the image capturing device 1900 of this embodiment has components not included in the image capturing device 1 shown in FIG. 1, including a microphone 116 for capturing voice uttered by the photographer and a face authentication camera 117 for capturing a facial image of the photographer.

[0138] In the photographing device 1900 of this embodiment, similarly to the photographing device 1 shown in FIG. 1, the photographer looks through the finder 112 to obtain data of an image of the photographer's eyes. Furthermore, in the photographing device 1900 of this embodiment, facial image data of the photographer is acquired by having the facial authentication camera 117 capture a facial image of the photographer who faces the camera toward the facial authentication camera 117. Note that, similar to the captured image 1 shown in Fig. 1, facial image data of the photographer who is looking at an image displayed in live view on the display panel provided on the operation member 113 may also be acquired.

[0139] Furthermore, in the photographing device 1900 of this embodiment, when a photographer touches the release button 111, a fingerprint sensor provided in the release button 111 reads the fingerprint of the photographer's finger, thereby acquiring fingerprint data of the photographer. Furthermore, in the photographing device 1900 of this embodiment, the voice of the photographer uttered around the photographing device 1900 is collected by the microphone 116, thereby obtaining data of the voice uttered by the photographer.

[0140] FIG. 20 is a diagram showing an example of the functional configuration of the image capturing device 1900 (or the information processing system included therein). Figure 20 illustrates an example in which the imaging device 1900 has a first information acquisition unit 101, a data acquisition unit 102, a storage unit 103, a second information acquisition unit 104, a notification unit 105, a matching unit 106, a display unit 107, and a feature selection unit 2001.

[0141] The first information acquisition unit 101 acquires registrant characteristic information. The first information acquisition unit 101 acquires, as registrant characteristic information, information on the type of characteristic selected by the feature selection unit 2001 based on the type of user characteristic information already acquired by the second information acquisition unit 104. If multiple types of user characteristic information for the same person have previously been acquired by the second information acquisition unit 104, the first information acquisition unit 101 acquires, as registrant characteristic information, information on the type of characteristic selected by the feature selection unit 2001 based on the multiple types of user characteristic information. The type of characteristic refers to the type of data that is the basis for calculating the user characteristic information. For example, if eye image, face image, fingerprint, and voice data are acquired, the number of feature types is four.

[0142] The second information acquisition unit 104 acquires user characteristic information. The second information acquisition unit 104 can also acquire multiple types of user characteristic information for the same person. The user characteristic information is calculated, for example, by performing a predetermined conversion on data acquired by the microphone 116, the face authentication camera 117, the release button 111 (fingerprint sensor), and the eye image sensor 214 as described above. The predetermined conversion itself is realized by using a known method, and therefore a detailed description thereof will be omitted here.

[0143] When user characteristic information is associated with a captured image stored in the holding unit 103 and registrant characteristic information has not yet been acquired by the first information acquisition unit 101, the characteristic selection unit 2001 selects the type of registrant characteristic information to be acquired by the first information acquisition unit 101. The characteristic selection unit 2001 may select the type of registrant characteristic information to be acquired by the first information acquisition unit 101 based on an operation by a user (a registrant registered as a user of the photographing device 1900). The operation by the user (registrant) is an operation to specify one or more types of user characteristic information from the types of user characteristic information acquired by the photographing device 1900.

[0144] For example, suppose that eye images, face images, fingerprints, and voices are acquired as user characteristic information, and the eye images, fingerprints, and voices are designated by an operation of the user (registrant). In this case, the feature selection unit 2001 selects the eye images, fingerprints, and voices as the types of registrant characteristic information to be acquired by the first information acquisition unit 101. The first information acquisition unit 101 acquires the types of registrant characteristic information (eye images, fingerprints, and voices) selected by the feature selection unit 2001 in this manner. The matching unit 106 authenticates the photographer by matching the same types of registrant characteristic information and user characteristic information. Here, if multiple types of user characteristic information are acquired by the second information acquisition unit 104 and stored in the storage unit 103, the feature selection unit 2001 may select all types of features or one or more types of features regardless of the operation of the user (registrant). In this embodiment, a case where the feature selection unit 2001 selects the type of feature based on the operation of the user (registrant) will be described as an example. In this case, the display panel provided on the operation member 113 may display information indicating the type of feature acquired by the second information acquisition unit 104. In this case, the user (registrant) uses the operation members 113 to 115 to perform an operation to specify the type of feature to be acquired as registrant characteristic information. Note that if no type of feature is selected by the feature selection unit 2001, the first information acquisition unit 101 does not need to acquire registrant characteristic information.

[0145] When multiple types of features are selected by the feature selection unit 2001, the notification unit 105 may notify the matching unit 106 that matching between the registrant feature information and the user feature information is possible when all of the selected types of registrant features have been acquired. Furthermore, the notification unit 105 may notify the matching unit 106 that matching between the registrant feature information and the user feature information is possible when any one or more types of registrant features have been acquired from the types of features selected by the feature selection unit 2001.

[0146] When the feature selection unit 2001 selects one type of feature, the matching unit 106 performs matching using the information of that type as the registrant feature information and the user feature information. In this embodiment, a case is illustrated in which the first information acquisition unit 101 does not acquire registrant feature information when no feature type is selected by the feature selection unit 2001. In this case, the matching unit 106 does not perform matching between the registrant feature information and the user feature information. Note that the matching unit 106 may also perform matching when no feature type is selected by the feature selection unit 2001. For example, when no feature type is selected by the feature selection unit 2001, the matching unit 106 may perform matching using one or more types of information preset as the registrant feature information and the user feature information.

[0147] Furthermore, when multiple types of features are selected by feature selection unit 2001, matching unit 106 may determine the final matching result (authentication result) based on the matching results for each of the multiple types.

[0148] For example, the matching unit 106 matches the registrant characteristic information with the user characteristic information for each type individually, and when the matching results for all types show the same person, it can determine that authentication of the photographer has been successful and perform a predetermined operation. Note that the matching results showing the same person indicate that the actual user (photographer) of the photographing device 1900 and the registrant registered as the user of the photographing device 1900 are the same person. In the following description, when the actual user (photographer) of the photographing device 1900 and the registrant registered as the user of the photographing device 1900 are the same person, this will be simply referred to as the same person, as necessary.

[0149] Furthermore, if there are features that are determined to be of the same person and features that are determined not to be of the same person during matching, the matching unit 106 may combine these matching results to determine the final matching result (authentication result).

[0150] For example, the matching unit 106 may determine the final matching result by majority voting of the matching results or by weighted majority voting. Note that weighted majority voting is a type of majority voting in which the number of successes or failures of a single matching result may be a value other than 1, depending on the importance of the matching result, etc. Alternatively, the matching unit 106 may calculate a score representing the degree of authentication based on the registrant characteristic information and the user characteristic information, and determine the final matching result by using a weighted average of the scores. For example, the matching unit 106 may represent the registrant characteristic information and the user characteristic information as vectors, and use the vectors to calculate cosine similarity as an example of the score. However, the score is not limited to cosine similarity, and may be calculated using other known methods.

[0151] As long as it is possible to uniquely determine whether or not the two persons are the same person using the matching results of each type of feature (registrant characteristic information and user characteristic information), the method of integrating the matching results is not limited to the method described above. Furthermore, if the matching results include features that are determined to be the same person and features that are determined not to be the same person, the matching unit 106 may suspend the determination of the success or failure of authentication. In this case, the photographing device 1900 may prompt the user (registrant) to acquire registrant characteristic information again. In this case, the matching unit 106 performs the predetermined operation described above if authentication of the photographer is successful based on the result of matching the reacquired registrant characteristic information with the same type of user characteristic as the registrant characteristic information.

[0152] The display unit 107 displays information that the user (registrant) refers to in order to specify the type of feature. FIG. 21 is a diagram showing an example of information that the user (registrant) refers to in order to specify the type of feature. FIG. 21(a) illustrates an example in which check boxes for selecting an eye image, a face image, a fingerprint, and a voice are displayed on the display panel of the operation unit 113 under the phrase "Please select the type of feature to acquire." Furthermore, if the user (registrant) is allowed to specify multiple types of features, the display unit 107 may further display an OK button and a Cancel button, as illustrated in FIG. 21(b). In this case, the user (registrant) designates one or multiple types of features to acquire as registrant feature information, and then presses the OK button. The information that the user (registrant) refers to in order to specify the type of feature is displayed, for example, when authenticity information (photographer information) is added during or after photography. Here, an example is shown in which the information that the user (registrant) refers to in order to specify the type of feature is displayed on the display panel of the operation unit 113. However, the display medium of the information that the user (registrant) refers to in order to specify the type of feature is not limited. Also, the information that the user (registrant) refers to in order to specify the type of feature may be audio or the like.

[0153] 22 is a flowchart showing an example of the operation of the image capturing apparatus 1900 of this embodiment. The flowchart in Fig. 22 is obtained by adding S2101 and S2102 to the flowchart in Fig. 7.

[0154] As described in the third embodiment, if the result of the determination in S405 is that registrant characteristic information has not been acquired (NO in S405), the process of S701 is performed. In S701, the display unit 107 displays information indicating that photographer information cannot be added to the captured image on the display panel provided in the operation member 113. In the third embodiment (FIG. 7), when the process of S701 ends, the process according to the flowchart in FIG. 7 ends. In contrast, in the present embodiment (FIG. 21), the process of S2101 is performed after S701.

[0155] In S2101, the display unit 107 displays the type of user characteristic information acquired by the second information acquisition unit 104 on the display panel provided in the operation member 113. An example of the information displayed in S2101 is the information shown in FIG.

[0156] Next, in S2202, the feature selection unit 2001 selects the type of registrant characteristic information to be acquired by the first information acquisition unit 101. At this time, the user (registrant) can use the operation members 113 to 115 to specify the type of registrant characteristic information. When authenticity is verified with the type of registrant characteristic information specified in S2202, the type of registrant characteristic information selected in S2202 is acquired. The timing for specifying the type of registrant characteristic information is not limited to the timing of S2202. For example, if it is cumbersome to specify the type of registrant characteristic information when capturing images, the type of registrant characteristic information may be specified when a series of images is completed.

[0157] In S407, matching is performed using the registrant characteristic information and user characteristic information of the type selected as described above.

[0158] As described above, in this embodiment, when no registrant characteristic information is present at the time of image capture, the imaging device 1900 (or the information processing system included therein) selects the type of characteristic to acquire as registrant characteristic information from among multiple types of user characteristic information. Therefore, for example, it is possible to acquire a type of registrant characteristic information that is easier to authenticate. This allows for more accurate authentication.

[0159] [Modification] In the present embodiment, an example has been described in which biometric information such as eye images, face images, fingerprints, and voice is acquired as the registrant characteristic information and user characteristic information. However, the registrant characteristic information and user characteristic information do not need to be biometric information. For example, by having the photographer input information other than biometric information, such as a password or a PIN (Personal Identification Number) code, the information may be used as the user characteristic information and the registrant characteristic information. Furthermore, by enabling pairing between the image capturing device 1900 and a terminal device and a terminal device owned by the photographer using Bluetooth (registered trademark) or the like, information from the terminal device may be used as the user characteristic information. Note that the terminal device may be, for example, a smartphone or a tablet.

[0160] (Tenth embodiment) Next, a tenth embodiment will be described. In the ninth embodiment, a case was described in which the first information acquisition unit acquires the type of registrant registration information selected by the feature selection unit based on the type of user characteristic information acquired by the second information acquisition unit. Here, the type of registrant characteristic information can also be selected based on predetermined conditions related to the user characteristic information or predetermined conditions related to the information processing device used to acquire the user characteristic information. Therefore, this embodiment illustrates a case in which the type of registrant characteristic information is selected based on predetermined conditions related to the user characteristic information or the information processing device. Here, the predetermined conditions are, for example, conditions based on a quantitative evaluation of the acquired user characteristic information or conditions based on the state of the information processing device. For example, if the acquired user characteristic is an eye image, the predetermined condition may be a condition related to the image quality of the eye image. Alternatively, the predetermined condition may be that the information processing device has been powered off. Thus, this embodiment and the ninth embodiment differ primarily in the configuration and processing for acquiring registrant characteristic information. Therefore, in the description of this embodiment, the same parts as those in the first to ninth embodiments will be denoted by the same reference numerals as those in FIGS. 1 to 22, and detailed description thereof will be omitted. In this embodiment, when the term "type of characteristic information (registrant characteristic information and user characteristic information)" is used, it refers to the type of characteristic that is the basis of the characteristic information, unless there is any misunderstanding.

[0161] FIG. 23 is a diagram showing an example of the functional configuration of an image capturing device 2300 (including an information processing system) of this embodiment. 23 illustrates an example in which the photographing device 2300 includes a first information acquisition unit 101, a data acquisition unit 102, a storage unit 103, a second information acquisition unit 104, a notification unit 105, a matching unit 106, a display unit 107, and a feature selection unit 2001. Also, FIG. 23 illustrates an example in which the photographing device 2300 further includes a feature condition determination unit 2301 and a device state determination unit 2302.

[0162] The feature selection unit 2001 selects the type of registrant feature information to be acquired by the first information acquisition unit 101 when user feature information is associated with the captured image stored in the holding unit 103 and registrant feature information has not yet been acquired by the first information acquisition unit 101. In this embodiment, the feature selection unit 2001 selects the type of registrant feature to be acquired by the first information acquisition unit 101 based on the determination results of the feature condition determination unit 2301 and the device state determination unit 2302 (described later) and the type of acquired user feature. At this time, the user (the registrant who is registered as the user of the photographing device 2300) designates the type of registrant feature information by referring to information displayed on the display panel of the display unit 107. The feature selection unit 2001 selects the type designated by the user (the registrant) as the type of registrant feature to be acquired by the first information acquisition unit 101. An example of a method for selecting the type of registrant characteristics to be acquired by the first information acquisition unit 101 will be described in detail after the description of the characteristic condition determination unit 2301 and the device state determination unit 2302.

[0163] The feature condition determination unit 2301 determines whether or not to retain one or more types of user feature information acquired by the second information acquisition unit 104 as options (usable user feature information). The feature condition determination unit 2301 makes this determination based on whether or not a predetermined condition defined using the user feature information is satisfied. In the following description, the predetermined condition defined using the user feature information will be referred to as a feature condition as necessary. The feature condition may be a condition for evaluating the accuracy of matching (authentication) when using the user feature information. In this case, user feature information of a type that satisfies the feature condition can be excluded from the options as it is likely to cause erroneous authentication during authentication. The feature condition may be, for example, a condition using a value of an evaluation index for evaluating the accuracy of matching (authentication) when using the user feature information acquired by the second information acquisition unit 104. By using the evaluation index, it is possible to evaluate, for example, whether or not the user feature information acquired by the second information acquisition unit 104 should be used for matching by the matching unit 106.

[0164] The evaluation index may be, for example, a matching match likelihood, which indicates a higher value as the likelihood of correct matching increases when matching is performed. In this case, the feature condition may be that the matching match likelihood does not exceed a predetermined value. Furthermore, when the user feature information is sorted in descending order from the highest matching match likelihood, user feature information that is lower than a predetermined number of user feature information from the top may be considered to satisfy the feature condition. Note that, in accordance with the device conditions described below (to exclude user feature information that satisfies the conditions from the options), a case is illustrated here in which user feature information of a type that satisfies the feature condition is excluded from the options as being likely to cause erroneous authentication during authentication. However, the feature condition may be defined so that user feature information of a type that does not satisfy the feature condition is excluded from the options as being likely to cause erroneous authentication during authentication. In other words, the feature condition may be defined so that user feature information of a type that does not satisfy the feature condition is excluded from the options as being likely to cause erroneous authentication during authentication.

[0165] When the user feature information is an eye image, the matching suitability may be calculated using, for example, a trained model that has been machine-learned using an eye image as input so that the more appropriate the eye position, the higher the score. When the user feature information is a fingerprint, the matching suitability may be calculated using, for example, a trained model that has been machine-learned using a fingerprint image as input so that the more appropriate the fingerprint position, the higher the score. Note that the method for calculating the matching suitability is not limited to these, and any method may be used as long as it can calculate a quantitative value as the value of the matching suitability. Furthermore, the matching suitability is calculated during the period from when the user feature information is acquired by the second information acquisition unit 104 to when the feature condition determination unit 2301 determines whether the user feature information satisfies the feature condition. The calculation of the matching suitability may be performed by the feature condition determination unit 2301 or another unit (e.g., the second information acquisition unit 104).

[0166] Like the feature condition determination unit 2301, the device status determination unit 2302 determines whether or not to keep one or more types of user feature information acquired by the second information acquisition unit 104 as options (usable user feature information). However, the device status determination unit 2302 makes this determination based on whether or not a predetermined condition determined using the status of the information processing device is satisfied. In the following description, the predetermined condition determined using the status of the information processing device will be referred to as a device condition as necessary. The device condition may be a condition for evaluating the possibility of spoofing (the possibility that the photographer who provided the user feature information is spoofing a registrant). In this case, the device condition may be a condition using the value of an evaluation index for evaluating the possibility of spoofing. In this case, user feature information of a type that satisfies the device condition can be user feature information with a high possibility of spoofing. Note that the device condition may be defined so that user feature information of a type that does not satisfy the device condition is excluded from the options as user feature information with a high possibility of spoofing.

[0167] The device condition may be, for example, at least one of whether a predetermined time has passed since the acquisition of the user characteristic information, whether a predetermined number of new data items have been acquired, whether the holding unit 103 has been removed, and whether the power has been turned on or off. In these cases, the device condition is met when a predetermined time has passed since the acquisition of the user characteristic information, when a predetermined number of new data items have been acquired, when the holding unit 103 has been removed, or when the power has been turned on or off. User characteristic information that satisfies the device condition is excluded from the selection list as having a high possibility of spoofing (satisfying the device condition). Here, the information processing device is a device used to acquire user characteristic information and may be the image capture device 2300 or a device other than the image capture device 2300 (e.g., a device communicably connected to the image capture device 2300). The device condition is not limited to the above-mentioned conditions. For example, if the information processing device is the image capture device 2300, the device condition may be at least one of whether the lens settings have been changed and whether the image capture mode has been switched.

[0168] The following is a specific example of a method for determining which user characteristic information to leave as an option based on whether the device conditions are satisfied. If the power is turned on or off when an eye image and a password are acquired as user characteristic information, the characteristic condition determination unit 2301 determines that the password satisfies the device conditions. In this case, the characteristic condition determination unit 2301 deletes the password from the options for user characteristic information in the characteristic selection unit 2001. This is because a password can be determined to have a higher possibility of being impersonated by another person than an eye image.

[0169] Furthermore, the user feature information to be left as an option (usable user feature information) may be determined based on both whether the feature conditions are satisfied and whether the device conditions are satisfied. Specific examples of a method for determining the user feature information to be left as an option based on both whether the feature conditions are satisfied and whether the device conditions are satisfied include the following method. The feature condition determination unit 2301 determines, for each type of acquired user feature information, whether to leave the acquired user feature information as an option, based on the matching suitability likelihood calculated for each type of acquired user feature information. Furthermore, the device status determination unit 2302 determines, for each type, whether to leave the acquired user feature information as an option, based on the status of the information processing device.

[0170] The feature condition determination unit 2301 or the device state determination unit 2302 integrates these determination results and makes a final determination as to whether or not to leave the acquired user feature information as an option. Specifically, the device state determination unit 2302 leaves as an option (usable user feature information) user feature information for which the elapsed time since acquisition by the second information acquisition unit 104 is within a predetermined time, regardless of the value of the matching suitability likelihood calculated using the user feature information. Then, the feature condition determination unit 2301 leaves as an option user feature information for which the elapsed time since acquisition by the second information acquisition unit 104 is not within the predetermined time, and for which the matching suitability likelihood is equal to or greater than a predetermined value. Furthermore, the feature condition determination unit 2301 excludes from the option user feature information for which the elapsed time since acquisition by the second information acquisition unit 104 is not within the predetermined time, and for which the matching suitability likelihood is not equal to or greater than a predetermined value.

[0171] Fig. 24 is a flowchart showing an example of the operation of the feature condition determination unit 2301 and the device state determination unit 2302 of the image capturing device 2300 of this embodiment. The processing according to the flowchart in Fig. 24 is executed repeatedly at a predetermined cycle, for example. The processing according to the flowchart in Fig. 24 may be started when a predetermined operation (for example, an operation for capturing an image or an operation for acquiring registrant feature information) is performed on the image capturing device 2300. The processing according to the flowchart in Fig. 24 may be started in conjunction with a specific situation, for example. The latest result of the processing of the flowchart in Fig. 24 is used, for example, in the processing (S2201, S2202) of the flowchart in Fig. 22.

[0172] In S2401, the feature condition determination unit 2301 or the device status determination unit 2302 determines for each type whether or not to leave the user feature information as an option (usable used feature information). This determination is made by, for example, determining whether or not the feature condition is satisfied, determining whether or not the device condition is satisfied, or determining whether or not a determination condition using both the feature condition and the device condition is satisfied. If, as a result of this determination, the user feature information is to be left as an option for all types (YES in S2401), the process of S2402 is performed. On the other hand, if the user feature information is not to be left as an option for at least one type (NO in S2402), the process of S2403 is performed.

[0173] In S2402, the display unit 107 displays information indicating the types of registrant characteristic information that can be acquired when the determination result in S2401 is true on the display panel provided in the operation unit 113. The information displayed in S2402 may be, for example, the information shown in FIG. 21. Also, in S2403, information indicating the types of registrant characteristic information that can be acquired when the determination result in S2401 is false is displayed on the display panel provided in the operation unit 113. The information displayed at this time may be the name of the characteristic (registrant characteristic information) or information indicating the action required to acquire the characteristic (registrant characteristic information). Also, in the information shown in FIG. 21, the user (registrant) may not be able to specify the type of characteristic (registrant characteristic information) that is excluded from the options.

[0174] FIG. 25 is a diagram illustrating an example of information that a user (registrant) refers to in order to specify a feature type when a feature type has been excluded from the options. FIG. 25 illustrates a case in which face images and voices are excluded from the options. In FIG. 25, the displayed feature types (types of registrant feature information) are the same as those shown in FIG. 21. However, when the feature condition determination unit 2301 or the device status determination unit 2302 has excluded a type of user feature information from the options (usable types), the display unit 107 shades the name of the feature (registrant feature information) of that type. This prevents the user (registrant) from specifying the feature (registrant feature information) of that type. For example, the display unit 107 may not accept a specification in the feature selection field (check box in FIG. 25), or may disable selection in the feature selection field. Note that the method for displaying a feature (registrant feature information) so that it cannot be specified is not limited, and the name of the feature may be grayed out, a strikethrough may be displayed, or the name of the feature may not be displayed.

[0175] For example, if user characteristic information is acquired during photography and registrant characteristic information is acquired after photography and used for authentication, there may be a higher possibility of impersonation by a third party compared to general authentication methods. In this embodiment, the type of registrant characteristic information to be selected by the characteristic selection unit 2001 is determined based on predetermined conditions (characteristic conditions and device conditions) related to the user characteristic information and the state of the information processing device. This further improves security against impersonation. Furthermore, by determining the type of registrant characteristic information to be selected based on the determination result of whether the user characteristic information satisfies the characteristic conditions, it is possible to avoid matching using user characteristic information, which may cause a decrease in authentication accuracy. This therefore improves authentication accuracy. Furthermore, by determining the type of registrant characteristic information to be selected based on the determination result of whether the user characteristic information satisfies the device conditions, it is possible to differentiate between the types of registrant characteristic information to be acquired. This therefore improves the convenience and security of the imaging device 2300. Note that security here includes security in the sense that third parties cannot misuse the registrant characteristic information or user characteristic information for authentication.

[0176] [Modification] In the present embodiment, the case where the selection target (option) of the type of user characteristic information is switched and the registrant characteristic information of the selected type is acquired is exemplified. However, this is not necessarily required.

[0177] For example, the photographing device 2300 may perform feature deletion processing to control whether or not to delete user feature information stored in the storage unit 103, based on the determination result of the feature condition determination unit 2301 or the device status determination unit 2302. For example, the photographing device 2300 may delete user information of a type determined by the device status determination unit 2302 to satisfy a device condition, from among the user feature information stored in the storage unit 103. In this way, by using the determination result of the feature condition determination unit 2301 or the device status determination unit 2302, it is possible to avoid using user feature information of a type whose reliability decreases by satisfying a predetermined condition for matching with registrant feature information. This makes it possible to reduce the probability of erroneous authentication.

[0178] Furthermore, for example, the imaging device 2300 can also control whether or not to perform matching in the matching unit 106, based on the determination result of the feature condition determination unit 2301 or the device state determination unit 2302. For example, the matching unit 106 can refrain from performing matching when the result of determination by the feature condition determination unit 2301 is that the matching suitability likelihood calculated using all types of acquired user feature information is less than a predetermined value. By refraining from performing matching when only user feature information unsuitable for matching has been obtained, the probability of erroneous authentication can be reduced.

[0179] (Other embodiments) The present disclosure can be embodied, for example, as a system, device, method, program, or storage medium. Specifically, the present disclosure may be applied to a system consisting of multiple devices, or to a device consisting of a single device. The present disclosure can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or storage medium, and having one or more processors in a computer of the system or device read and execute the program. The present disclosure can also be realized by a circuit (e.g., ASIC or FPGA) that realizes one or more functions. Furthermore, the above-described embodiments are merely examples of specific embodiments for carrying out the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by these embodiments. In other words, the present disclosure can be carried out in various forms without departing from its technical idea or main features.

[0180] The disclosure of the present embodiment also includes, for example, the following configurations, methods, and programs. (Configuration 1) An information processing system that associates a photographed image taken by a photographing means with photographer information relating to a photographer of the photographed image, a first acquiring means for acquiring a first characteristic of a person; a second acquiring means for acquiring a second characteristic of the person; an identification means for identifying a photographer by a comparison process between the first feature acquired by the first acquisition means and the second feature acquired by the second acquisition means; a recording control means for, when photographing is performed in a state in which a first feature corresponding to the second feature acquired by the second acquisition means is not detected in the matching process, associating the photographed image obtained by photographing with the second feature and recording the associated image in a memory; a determination means for determining the photographer information to be associated with the photographed image based on the second feature recorded by the recording control means and the first feature acquired by the first acquisition means after the photographing; An information processing system comprising: (Configuration 2) the identification means performs the matching process when the first feature has been acquired by the first acquisition means, and does not perform the matching process when the first feature has not been acquired by the first acquisition means; The information processing system according to configuration 1, wherein the recording control means, when the matching process is not performed, associates the captured image obtained by the photographing with the second feature and records them in a memory. (Configuration 3) The information processing system according to configuration 1 or 2, characterized in that the determination means reads from the memory second features recorded by the recording control means that have not been subjected to the matching process with the first features, and determines the photographer information to be associated with the captured image based on the read second features and the first features acquired by the first acquisition means after the image is captured. (Configuration 4) 4. The information processing system according to any one of configurations 1 to 3, wherein the first feature and the second feature include features based on biometric information. (Configuration 5) The information processing system described in any one of configurations 1 to 4, characterized in that the first acquisition means performs authentication processing based on a third characteristic of the person, and acquires a first characteristic of the person if the authentication processing is successful. (Configuration 6) 6. The information processing system according to any one of configurations 1 to 5, further comprising an associating unit that associates the photographer information determined by the determining unit with the photographed image. (Configuration 7) a receiving means for receiving an instruction as to whether or not to associate the photographer information with the photographed image; The information processing system according to configuration 6, wherein the associating means associates the photographer information determined by the determining means with the photographed image when the accepting means accepts that the photographer information be associated with the photographed image. (Configuration 8) 8. The information processing system according to any one of configurations 1 to 7, further comprising a notification means for notifying that a first feature corresponding to the second feature has not been detected in the matching process. (Configuration 9) The information processing system described in any one of configurations 1 to 8, characterized in that the recording control means records the captured image in association with the second feature on a storage medium that is detachable from the device equipped with the determination means. (Configuration 10) a receiving means for receiving an instruction as to whether to remove the storage medium when the storage medium, on which the photographed image not associated with the photographer information is recorded, is instructed to be removed from the device including the determining means; 10. The information processing system according to configuration 9, further comprising: an ejection unit that performs processing to eject the storage medium when the ejection of the storage medium is accepted by the acceptance unit. (Configuration 11) The information processing system according to any one of configurations 1 to 10, wherein the recording control means performs processing, including encryption, on data indicating a second characteristic of the person, and then associates the captured image obtained by the photographing with the second characteristic and records the image in memory. (Configuration 12) an associating means for associating the photographer information determined by the determining means with the photographed image, An information processing system according to any one of configurations 1 to 11, characterized in that a device having the first acquisition means and the second acquisition means and a device having the association means are devices connected to each other so that they can communicate with each other. (Configuration 13) The information processing system according to configuration 12, wherein the device having the first acquisition means and the second acquisition means and the device having the determination means and the association means are devices connected to each other so that they can communicate with each other. (Configuration 14) the second acquisition means is capable of acquiring a plurality of types of the second characteristics; The information processing system according to any one of configurations 1 to 13, further comprising a feature selection means for selecting the type of the first feature to be acquired by the first acquisition means based on the type of the second feature recorded by the recording control means. (Configuration 15) The information processing system according to configuration 14, wherein the feature selection means selects a plurality of types of the first feature to be acquired by the first acquisition means from the types of the second feature acquired by the second acquisition means. (Configuration 16) The information processing system of configuration 14 or 15, wherein the determination means determines the photographer information to be associated with the captured image based on the first feature and the second feature of the type selected by the feature selection means. (Configuration 17) a condition determination unit that determines whether the second characteristic recorded by the recording control unit satisfies a predetermined condition, The information processing system according to any one of configurations 14 to 16, wherein the feature selection means selects the type of the first feature to be acquired by the first acquisition means based on the result of the judgment by the condition judgment means. (Configuration 18) The information processing system according to configuration 17, characterized in that the predetermined conditions include conditions determined using at least one of the second characteristics recorded by the recording control means and the state of a device used to acquire the second characteristics. (Configuration 19) 19. The information processing system according to configuration 17 or 18, wherein the predetermined condition includes a condition determined using at least one of an evaluation index for evaluating the accuracy of the matching process when the second feature recorded by the recording control means is used, and an evaluation index for evaluating the possibility that a person who provided the second feature is impersonating a person having the first feature. (Configuration 20) An information processing system according to any one of configurations 17 to 19, characterized in that it has a feature deletion means for deleting from the memory, among the second features acquired by the second acquisition means, those second features of a type that are excluded from selection targets in the feature selection means, based on the result of the judgment by the condition judgment means. (Method 1) 1. An information processing method for associating a photographed image taken by a photographing means with photographer information relating to a photographer of the photographed image, comprising: a first acquisition step of acquiring a first characteristic of a person; a second acquisition step of acquiring a second feature of the person; an identification step of identifying the photographer by a comparison process between the first feature acquired in the first acquisition step and the second feature acquired in the second acquisition step; a recording control step of, when photographing is performed in a state in which a first feature corresponding to the second feature acquired in the second acquisition step is not detected in the matching process, associating the photographed image obtained by photographing with the second feature and recording the associated image in a memory; a determination step of determining the photographer information to be associated with the captured image based on the second feature recorded in the recording control step and the first feature acquired in the first acquisition step after the image capturing; An information processing method comprising: (Program 1) 21. A program for causing a computer to function as each means of the information processing system according to any one of configurations 1 to 20. [Explanation of symbols]

[0181] 1: imaging device, 101: first information acquisition unit, 103: storage unit, 104: second information acquisition unit, 105: notification unit, 106: matching unit

Claims

1. An information processing system that associates a photographed image taken by a photographing means with photographer information relating to a photographer of the photographed image, a first acquiring means for acquiring a first characteristic of a person; a second acquiring means for acquiring a second characteristic of the person; an identification unit that identifies a photographer by comparing the first feature acquired by the first acquisition unit with the second feature acquired by the second acquisition unit; a recording control means for, when photographing is performed in a state in which a first feature corresponding to the second feature acquired by the second acquisition means is not detected in the matching process, associating the photographed image obtained by photographing with the second feature and recording the associated image in a memory; a determination means for determining the photographer information to be associated with the photographed image based on the second feature recorded by the recording control means and the first feature acquired by the first acquisition means after the photographing; An information processing system comprising:

2. the identification means performs the matching process when the first feature has been acquired by the first acquisition means, and does not perform the matching process when the first feature has not been acquired by the first acquisition means; The information processing system according to claim 1, characterized in that, when the matching process is not performed, the recording control means associates the captured image obtained by the photographing with the second feature and records them in memory.

3. The information processing system according to claim 1 or 2, characterized in that the determination means reads from the memory, among the second features recorded by the recording control means, second features that have not been subjected to the matching process with the first features, and determines the photographer information to be associated with the captured image based on the read second features and the first features acquired by the first acquisition means after the image is captured.

4. 3. The information processing system according to claim 1, wherein the first feature and the second feature include features based on biometric information.

5. The information processing system according to claim 1 or 2, characterized in that the first acquisition means performs an authentication process based on a third characteristic of the person, and if the authentication process is successful, acquires a first characteristic of the person.

6. 3. The information processing system according to claim 1, further comprising an associating unit that associates the photographer information determined by the determining unit with the photographed image.

7. a receiving means for receiving an instruction as to whether or not to associate the photographer information with the photographed image; The information processing system according to claim 6, characterized in that when the accepting means accepts that the photographer information be associated with the photographed image, the associating means associates the photographer information determined by the determining means with the photographed image.

8. 3. The information processing system according to claim 1, further comprising a notification unit that notifies that a first feature corresponding to the second feature has not been detected in the matching process.

9. The information processing system according to claim 1 or 2, characterized in that the recording control means records the captured image in association with the second feature on a storage medium that is detachable from the device equipped with the determination means.

10. a receiving means for receiving an instruction as to whether to remove the storage medium when the storage medium, on which the photographed image not associated with the photographer information is recorded, is instructed to be removed from the device including the determining means; 10. The information processing system according to claim 9, further comprising: an ejection unit that performs processing for ejecting the storage medium when the ejection of the storage medium is accepted by the acceptance unit.

11. The information processing system according to claim 1 or 2, characterized in that the recording control means performs processing, including encryption, on data indicating a second characteristic of the person, and then associates the captured image obtained by the photograph with the second characteristic and records it in memory.

12. an associating means for associating the photographer information determined by the determining means with the photographed image, 3. The information processing system according to claim 1, wherein the device including the first acquisition means and the second acquisition means and the device including the association means are connected to each other so as to be able to communicate with each other.

13. 13. The information processing system according to claim 12, wherein a device including the first acquisition means and the second acquisition means and a device including the determination means and the association means are devices connected to each other so as to be able to communicate with each other.

14. the second acquisition means is capable of acquiring a plurality of types of the second characteristics; 3. The information processing system according to claim 1, further comprising a feature selection means for selecting the type of the first feature to be acquired by the first acquisition means based on the type of the second feature recorded by the recording control means.

15. 15. The information processing system according to claim 14, wherein the feature selection means selects a plurality of types of the first feature to be acquired by the first acquisition means from the types of the second feature acquired by the second acquisition means.

16. The information processing system according to claim 14, wherein the determining means determines the photographer information to be associated with the captured image based on the first feature and the second feature of the type selected by the feature selecting means.

17. a condition determining means for determining whether the second characteristic recorded by the recording control means satisfies a predetermined condition, 15. The information processing system according to claim 14, wherein the feature selection means selects the type of the first feature to be acquired by the first acquisition means based on the result of the determination by the condition determination means.

18. 18. The information processing system according to claim 17, wherein the predetermined conditions include conditions determined using at least one of the second characteristics recorded by the recording control means and the state of a device used to acquire the second characteristics.

19. 18. The information processing system according to claim 17, wherein the predetermined condition includes a condition determined using at least one of an evaluation index for evaluating accuracy of the matching process when the second feature recorded by the recording control means is used, and an evaluation index for evaluating the possibility that a person who provided the second feature is impersonating a person having the first feature.

20. 18. The information processing system according to claim 17, further comprising a feature deletion means for deleting from the memory, based on the result of the judgment by the condition judgment means, the second features acquired by the second acquisition means, which are of a type that is excluded from selection by the feature selection means.

21. 1. An information processing method for associating a photographed image taken by a photographing means with photographer information relating to a photographer of the photographed image, comprising: a first acquiring step of acquiring a first feature of a person; a second acquiring step of acquiring a second feature of the person; an identification step of identifying the photographer by a comparison process between the first feature acquired in the first acquisition step and the second feature acquired in the second acquisition step; a recording control step of, when photographing is performed in a state in which a first feature corresponding to the second feature acquired in the second acquisition step is not detected in the matching process, associating the photographed image obtained by the photographing with the second feature and recording the associated image in a memory; a determination step of determining the photographer information to be associated with the photographed image based on the second feature recorded in the recording control step and the first feature acquired in the first acquisition step after the photographing; An information processing method comprising:

22. 3. A program for causing a computer to function as each of the means of the information processing system according to claim 1 or 2.

Images