Information processing apparatus, information processing method, and program

Abstract

To provide an information processing apparatus that can reduce a burden on a user who obtains consent to use a biological identifier.SOLUTION: An information processing apparatus has: acquisition means that acquires person information that is information on a person included in a picked-up image; communication means that communicates with a server that holds information on an approver who consents to the use of a biological identifier; determination means that determines whether a person corresponding to the person information acquired by the acquisition means consents to the use of the biological identifier, on the basis of the information received by the communication means from the server; and processing means that executes processing of using the biological identifier of the person according to a result of determination made by the determination means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the use of biometric identifiers. [Background technology]

[0002] Authentication technologies such as facial recognition extract a biometric identifier from an image to identify an individual and authenticate the individual by comparing it with a database of biometric identifiers.

[0003] On the other hand, because it is difficult to change biometric identifiers, restrictions on their use are being put in place, and the laws restricting them prohibit the use of biometric identifiers without the consent of the individual.

[0004] Therefore, when consent to the use of a biometric identifier from the subject himself / herself has not been obtained, it is desirable to obtain explicit consent or to be able to restrict the extraction and use of a biometric identifier from an image in which a person without consent appears. Patent Document 1 discloses explicitly obtaining consent from the subject himself / herself to the use of a biometric identifier. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2022-119549 Summary of the Invention [Problem to be solved by the invention]

[0006] In Patent Document 1, consent must be explicitly obtained from the subject each time personal authentication or the like is performed. Therefore, even when friends take photos of each other, the users may be required to obtain consent for the use of biometric identifiers each time a photo is taken. This places a burden on the users who take the photos.

[0007] The present invention aims to reduce the burden on users who need to obtain consent for the use of biometric identifiers. [Means for solving the problem]

[0008] In order to solve this problem, for example, an information processing device of the present invention has the following configuration: an acquisition means for acquiring person information that is information about a person included in a captured image; A communication means for communicating with a server that stores information on consenting parties who have consented to the use of biometric identifiers; a determination means for determining whether or not a person corresponding to the personal information acquired by the acquisition means has consented to the use of a biometric identifier based on information received by the communication means from the server; and processing means for executing a process using the biometric identifier of the person in accordance with the result of the determination by the determination means. [Effects of the Invention]

[0009] According to the present invention, it is possible to reduce the burden on the user of obtaining consent for the use of a biometric identifier. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating the overall configuration of a biometric identifier management system according to an embodiment. [Figure 2] FIG. 10 is a diagram showing the flow of consent information determination processing executed by the camera device according to the embodiment. [Figure 3] 6A to 6C are diagrams showing an example of changes in a captured image processed in the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of detected person information. [Figure 5] FIG. 10 is a diagram showing an example of a person's ID input screen. [Figure 6] FIG. 10 is a diagram showing the flow of consent information transmission processing executed by the consent management server according to the embodiment. [Figure 7] FIG. 4 is a diagram showing an example of consenting person information according to the embodiment. [Figure 8] 1 is a diagram illustrating the overall configuration of a biometric identifier management system according to an embodiment. [Figure 9] FIG. 10 is a diagram showing the flow of consent information determination processing executed by the camera device according to the embodiment. [Figure 10] FIG. 4 is a diagram showing an example of detected person information according to the embodiment. [Figure 11] FIG. 4 is a diagram showing an example of consenting person information according to the embodiment. [Figure 12] 1 is a diagram illustrating the overall configuration of a biometric identifier management system according to an embodiment. [Figure 13] FIG. 10 is a diagram showing the flow of consent information determination processing executed by the camera device according to the embodiment. [Figure 14] FIG. 10 is a diagram showing the flow of consent information transmission processing executed by the consent management server according to the embodiment. [Figure 15] 1 is a diagram illustrating the overall configuration of a biometric identifier management system according to an embodiment. [Figure 16] FIG. 10 is a diagram showing the flow of consent information determination processing executed by the camera device according to the embodiment. [Figure 17] FIG. 10 is a diagram showing the flow of consent information transmission processing executed by the consent management server according to the embodiment. [Figure 18] FIG. 4 is a diagram showing an example of consenting person information according to the embodiment. [Figure 19] FIG. 1 is a block diagram showing a hardware configuration of an information processing apparatus according to an embodiment. [Figure 20] FIG. 2 is a sequence diagram showing a processing flow of the biometric identifier management system according to the embodiment. [Figure 21] FIG. 2 is a sequence diagram showing a processing flow of the biometric identifier management system according to the embodiment. [Figure 22] FIG. 2 is a sequence diagram showing a processing flow of the biometric identifier management system according to the embodiment. [Figure 23] FIG. 2 is a sequence diagram showing a processing flow of the biometric identifier management system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0012] First Embodiment First, an example of the first embodiment will be described. Fig. 1 is a diagram showing the overall configuration of a biometric identifier management system according to the first embodiment.

[0013] 1, the biometric identifier management system of the first embodiment includes a camera device 11 and a consent management server 12. The camera device 11 and the consent management server 12 are connected to each other via a network 13 so as to be able to send and receive information to and from each other. The network 13 may include either wireless communication, wired communication, or both.

[0014] Camera device 11 captures an image of a subject such as a person, and extracts facial features of the person as a biometric identifier depending on whether or not the person has consented to the use of the biometric identifier. Camera device 11 includes an imaging unit 111, a face detection unit 112, an ID input unit 113, a feature extraction unit 114, a first communication unit 115, a comparison unit 116, a display control unit 117, and a display unit 118.

[0015] The imaging unit 111 receives light from a subject such as a person via a lens unit or the like, photoelectrically converts the light to generate image (captured image) data, and outputs the data to the face detection unit 112 or the like. The imaging unit 111 has image sensors such as a CMOS (Complementary Metal-Oxide-Semiconductor) sensor and a CCD (Charge Coupled Device) sensor. In the following description, the term "image" may include moving images, still images, video images, and data thereof.

[0016] The face detection unit 112 detects a person's face from an image acquired from the imaging unit 111, and outputs information about the detected face together with the captured image. The information about the face is, for example, the coordinates of the four vertices of a rectangular face frame that surrounds the face in the captured image. The face detection unit 112 registers and updates the information about the face in the detected person information. As will be described in detail later, the detected person information is information that associates the person's ID with a face frame, whether consent has been given, the name, etc.

[0017] The ID input unit 113 acquires an ID, which is information about a person included in a captured image. The ID input unit 113 is an example of an acquisition unit. An ID is a unique value assigned to each person to identify each person. The ID is an example of person information and an example of identification information. The ID input unit 113 acquires, as an ID, information input by a user using an input device such as a touch panel. The ID input unit 113 associates the ID with the person's face. The ID input unit 113 registers the acquired ID in detected person information. Note that if the image includes faces of multiple people, the ID input unit 113 may acquire the ID of each person.

[0018] The feature extraction unit 114 extracts facial features as a biometric identifier from an image of a person's face acquired from the face detection unit 112. In this embodiment, facial features are used as an example of a biometric identifier; however, the biometric identifier is not limited to this. The biometric identifier may be, for example, features extracted from a fingerprint, an iris pattern, or the like. The biometric identifier may also be a combination of features extracted from a face, a fingerprint, and an iris pattern. In this embodiment, the feature extraction unit 114 extracts facial features as a biometric identifier from a person based on the determination result of the comparison unit 116, which will be described later. Specifically, if the determination result of the comparison unit 116 indicates consent to the use of the biometric identifier, the feature extraction unit 114 extracts facial features from the person. On the other hand, if the determination result indicates no consent, the feature extraction unit 114 does not extract facial features from the person. The feature extraction unit 114 stores the extracted facial features of the person as a biometric identifier in a storage device. The feature extraction unit 114 is an example of an extraction unit and a processing unit. The storage device will be described later with reference to FIG. 19 .

[0019] The first communication unit 115 communicates with external devices including the consent management server 12 via the network 13 to send and receive information. The first communication unit 115 is an example of a communication means and a processing means. The first communication unit 115 receives consenting person information including an ID that is identification information of a consenting person who has consented to the use of a biometric identifier from the consent management server 12. Depending on the determination result of the comparison unit 116, the first communication unit 115 may output the feature amount of the consenting person as a biometric identifier to another device. The other device is, for example, an external reinforcement learning device.

[0020] The comparison unit 116 determines whether a person in the captured image has consented to the use of the biometric identifier. The comparison unit 116 is an example of a determination unit. For example, the comparison unit 116 determines whether the person in the captured image has consented to the use of the biometric identifier based on whether the ID acquired by the ID input unit 113 is included among multiple IDs included in the consenting person information received from the consent management server 12. If the comparison unit 116 determines that the consenting person information includes the ID of the person in the captured image, it determines that the person with the ID has consented to the use of the biometric identifier. In this case, the comparison unit 116 extracts the name of the person associated with the ID from the consenting person information and determines that the person has consented to the use of the information associated with the ID and the name and the biometric identifier. On the other hand, if the comparison unit 116 determines that the consenting person information does not include the ID of the person in the captured image, it determines that the person has not consented to the use of the biometric identifier. The comparison unit 116 updates the detected person information based on the determination result.

[0021] If the determination result obtained from the comparison unit 116 indicates that the person whose ID was accepted by the ID input unit 113 has consented to the use of the biometric identifier, the display control unit 117 generates an image including the name of the person near the face image of the person. The display control unit 117 causes the generated image to be displayed on the display unit 118.

[0022] Display unit 118 displays an image based on an instruction from display control unit 117. For example, display unit 118 displays a person's name or the like superimposed on an image captured by imaging unit 111. Display unit 118 may be, for example, a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display.

[0023] The consent management server 12 stores and manages information on persons who have consented to the use of biometric identifiers as consenting person information. The consent management server 12 includes a second communication unit 121, a consent extraction unit 122, and a consent information storage unit 123.

[0024] The second communication unit 121 transmits and receives information to and from the first communication unit 115 of the camera device 11 via the network 13. For example, the second communication unit 121 acquires consenting person information from the consent extraction unit 122 and transmits it to the first communication unit 115.

[0025] The consent extraction unit 122 extracts and acquires the consenting person information stored in the consent information storage unit 123. The consent extraction unit 122 outputs the extracted consenting person information to the second communication unit 121 to transmit to the camera device 11. Note that the timing at which the consent extraction unit 122 acquires the consenting person information is not particularly limited, and may be, for example, when the consenting person information is updated, when a request is received from the camera device 11, etc.

[0026] The consent information holding unit 123 holds consenting person information. The consent information holding unit 123 reads out and passes the consenting person information based on an instruction from the consent extraction unit 122.

[0027] 19 is a block diagram showing the hardware configuration of an information processing device 190 included in the camera device 11. The information processing device 190 is an example of a computer. The information processing device 190 of the camera device 11 includes a processor 191, a memory 192, a storage 193, a communication IF 194, an input IF 195, an output IF 196, and a bus 197. The processor 191, the memory 192, the storage 193, the communication IF 194, the input IF 195, and the output IF 196 are connected via the bus 197 so as to be able to transmit and receive information to and from each other.

[0028] The processor 191 is an arithmetic processing device, such as a CPU (Central Processing Unit). The information processing device 190 may include other processors, such as an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), or a QPU (Quantum Processing Unit), instead of or in addition to the CPU. The processor 191 reads out programs stored in the storage 193 and loads them into the memory 192, thereby implementing various functions. For example, the processor 191 reads out programs to implement some or all of the functions of the face detection unit 112, the ID input unit 113, the feature extraction unit 114, the first communication unit 115, the comparison unit 116, and the display control unit 117. Some or all of the functions of the face detection unit 112, the ID input unit 113, the feature extraction unit 114, the first communication unit 115, the comparison unit 116, and the display control unit 117 may be implemented by one or more circuits, such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array).

[0029] The memory 192 is a high-speed readable / writable storage device such as a RAM (Random Access Memory). The memory 192 functions as a work area when the processor 191 executes a program. The memory 192 temporarily stores the program and parameters necessary for executing the program.

[0030] The storage 193 is a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The storage 193 retains programs, parameters required for executing the programs, and results of executing the programs even when power is not supplied. The storage 193 stores, for example, facial features of people extracted by the feature extraction unit 114.

[0031] The communication IF 194 is an interface for realizing communication with an external device such as the consent management server 12 via a wired or wireless network. The first communication unit 115 of this embodiment communicates with the external device via the communication IF 194.

[0032] The input IF 195 is an interface for receiving information input from an input device. The input device is, for example, a mouse, a keyboard, or a touch panel. The ID input unit 113 of this embodiment acquires the ID input via the input IF 195. The face detection unit 112 of this embodiment detects a person's face from a captured image acquired from the imaging unit 111 via the input IF 195. Note that the input IF 195 for acquiring the ID and the input IF 195 for acquiring the captured image may be physically different entities, but for convenience of explanation, they are numbered the same.

[0033] The output IF 196 is an interface for outputting information to an external device. The external device is, for example, a display device such as a monitor. The display unit 118 of this embodiment displays an image based on the information output from the output IF 196.

[0034] The consent management server 12 also has an information processing device 190 with the same configuration as above. In the consent management server 12, the processor of the information processing device 190 reads a program to realize some or all of the functions of the second communication unit 121, the consent extraction unit 122, and the consent information storage unit 123.

[0035] Fig. 2 is a diagram showing the flow of consent information determination processing executed by the camera device 11 of the first embodiment. In the camera device 11, for example, the processor 191 executes the consent information determination processing by reading a program. Fig. 3 is a diagram showing an example of changes in a captured image processed in the first embodiment. Fig. 4 is a diagram showing an example of detected person information. Fig. 5 is a person's ID input screen.

[0036] In S21, the imaging unit 111 of the camera device 11, which serves as a biometric identifier utilization device, captures an image of a person and generates a captured image J21. FIG. 3(a) is an example of the captured image J21. At this stage, the display control unit 117 may cause the captured image J21 to be displayed on the display unit 118. Two people appear in the captured image J21. The person on the right side of the captured image J21 in FIG. 3(a) is designated as person P31, and the person on the left side is designated as person P32.

[0037] In S22, the face detection unit 112 executes face detection processing to detect a person's face in the captured image J21. The face detection processing is processing to detect a face in the image and calculate the coordinates of a face frame surrounding the face. Note that the face detection unit 112 may detect the positions of facial features such as the eyes, nose, and mouth in the face detection processing. For example, technologies such as MTCNN (Multi-task Cascaded Convolutional Networks) and RetinaFace can be applied as the face detection processing.

[0038] In S22, the face detection unit 112 generates or updates detected person information J22 shown in FIG. 4(a) as a result of the face detection process. The detected person information J22 includes information on ID, face frame, consent, and name. Information in the ID column indicates the ID of the person whose face was detected. Information in the face frame column indicates the coordinates of the four vertices of the rectangular face frame that surrounds the detected face. Information in the consent column indicates whether the person has consented to the use and extraction of the biometric identifier. Information in the name column indicates the name of the person. When the face detection unit 112 detects a face, the information on ID, consent, and name is unknown. Note that the detection result in the first row of the detected person information J22 is information about person P31, and the detection result in the second row is information about person P32.

[0039] In S23, the ID input unit 113 identifies the ID of the person whose face has been detected. The ID input unit 113 determines the ID item of the detected person information J22 based on the information input by the user.

[0040] Here, an example of identifying the ID of person P31 will be described. As a method of identifying the ID, an example will be described in which the display control unit 117 displays an ID input screen 50 on the display unit 118 of the camera device 11, and the ID input unit 113 accepts input of the ID.

[0041] First, the display control unit 117 displays a screen on the display unit 118 (for example, a viewfinder) in which a face frame is displayed superimposed with a dashed line as shown in Fig. 3(b). The face frame can be identified based on the face frame information in the detected person information J22.

[0042] The user designates person P31 in the captured image J21. As an example of a designation method, the user may designate the person using various operation units. The operation unit is, for example, an input device such as a joystick, physical buttons for up, down, left, and right, or a touch panel (not shown) integrated with the display unit 118. Note that the operation unit is not particularly limited.

[0043] For example, when a user selects person P31 on the touch panel, the display control unit 117 switches the screen displayed on the display unit 118 to an ID input screen 50 for person P31 shown in FIG. 5. The ID input screen 50 includes numeric buttons for inputting an ID, a "Delete One Character" button, a "Confirm" button, and an ID display field for displaying the ID being input. The user inputs the ID by selecting multiple numbers on the touch panel and finally pressing the Confirm button. This causes the ID input unit 113 to acquire the ID of person P31. Note that the method for inputting the ID is not particularly limited. For example, the ID input unit 113 may acquire the ID by directly inputting characters using physical keys and soft keys, or by selecting numbers on the screen using a joystick or up / down / left / right buttons. The ID input unit 113 may also acquire ID information by reading it from a membership card bearing an RFID tag or a two-dimensional barcode.

[0044] As a result, the ID input unit 113 acquires and identifies the detected face, that is, the ID of the person in each row of the detected person information J22 in FIG. 4(a).

[0045] The ID input unit 113 updates the detected person information J22 based on the identified ID. It is assumed here that the ID input unit 113 was able to identify the ID for person P31 but was unable to identify the ID for person P32. Therefore, as shown in FIG. 4(b), the ID input unit 113 updates the ID field for person P31 in the first row of the detected person information J22 to "12345," but leaves the ID field for person P32 in the second row as unknown.

[0046] At this stage, the ID of person P31 who appears in captured image J21 has been identified, but it is unknown whether this person P31 has given consent to the use of their biometric identifier. Therefore, the camera device 11 obtains consenting person information, including information on whether consent has been given, from the consent management server 12, which functions as a biometric identifier management server.

[0047] The camera device 11 functioning as the biometric identifier utilization device of the first embodiment receives consenting person information including whether or not the person has consented to the use of the biometric identifier sent from the consent management server 12 along with the person's ID, thereby eliminating the need to explicitly obtain consent from the user each time authentication is performed.

[0048] 6 is a diagram showing the flow of the consent information transmission process executed by the consent management server 12 of the first embodiment. In the consent management server 12, for example, the processor 191 shown in FIG. 19 reads a program to execute the consent information transmission process.

[0049] The consenting person information J61 is information about a person who has consented to the use of a biometric identifier, and is stored in the consent information storage unit 123 in Fig. 1. Therefore, consent regarding the use of a biometric identifier typically needs to be obtained only once when the biometric identifier is added to the consent information storage unit 123.

[0050] FIG. 7 is an example of consenting person information in the first embodiment. The consenting person information J61 shown in FIG. 7 holds IDs and corresponding names for four people. Here, the ID is a required field, but the name may be changed as appropriate depending on the application. In this embodiment, the consenting person information J61 holds names so that the name of the person is displayed near the detected face. The ID is a unique identifier that identifies a person. In other words, the same ID is not associated with multiple people. Note that there are no particular limitations on the values ​​that the consenting person information J61 holds. For example, the consenting person information J61 may include various information as needed, such as gender, address, and telephone number.

[0051] Returning to FIG. 6, in S61, the consent extraction unit 122 extracts and acquires consenting person information J61 from the consent information storage unit 123.

[0052] Then, in S62, the second communication unit 121 sends the consenting person information acquired by the consent extraction unit 122 to the first communication unit 115 of the camera device 11 via the network 13.

[0053] Note that, although an example is shown here in which the first communication unit 115 and the second communication unit 121 exchange information via the network 13, it is not necessary to exchange information via the network 13. For example, the camera device 11 and the consent management server 12 may exchange data using a storage medium such as an SD card as an information transmission medium. However, if an SD card or the like is used as an information transmission medium, it takes time for the camera device 11 to confirm whether or not consent has been given. The method of exchanging data is similar in the following embodiments.

[0054] 2, in S24, the first communication unit 115 receives the consenting person information J23 sent from the second communication unit 121. For example, in S24, the camera device 11 may obtain the consenting person information from the consent management server 12 by sending a request for consenting person information to the consent management server 12. The content of the consenting person information J23 received by the first communication unit 115 is the same as that in FIG.

[0055] In S25, the comparison unit 116 updates the detected person information J22. Specifically, the comparison unit 116 checks whether the ID shown in the detected person information J22 in FIG. 4(b), in this case the ID "12345" of person P31 identified in S23, is included in the consenting person information J23. If the ID is included in the consenting person information J23, the comparison unit 116 determines that the person with the ID has consented to the use of the biometric identifier, extracts a name from the consenting person information J23, and updates the detected person information J22. For example, as shown in FIG. 7, because the ID "12345" is included in the consenting person information J23, the comparison unit 116 extracts the name "Johnny Depp" associated with the ID "12345." 4(c), the comparison unit 116 updates the detected person information J22 by changing the consent status of the ID "12345" in the detected person information J22 from "unknown" to "consented" and changing the name to "Johnny Depp." Note that if the ID is not included in the consented person information J23, the comparison unit 116 may change the consent status of the detected person information J22 from "unknown" to "not consented."

[0056] In S26, the display control unit 117 generates an image by superimposing a face frame surrounding the face of the person with ID "12345" and the name "Johnny Depp" near the person on the captured image J21. FIG. 3(c) is an example of the captured image J21 generated by the display control unit 117 in S26. Note that in the example shown in FIG. 3(c), the face frame is displayed with a dashed line and the name is displayed near the face, but this is a specification of the application, and the position and display format may be changed as appropriate by applying known technology.

[0057] In S27, the display control unit 117 causes the display unit 118 to display the captured image J21 with the face frame and name superimposed thereon.

[0058] In S28, the feature extraction unit 114 extracts features from the facial image of the person who has consented to the use of their biometric identifier. Here, in S25, the comparison unit 116 confirmed that consent to the use of the biometric identifier has been obtained for person P31 with ID "12345." Therefore, the feature extraction unit 114 extracts a facial image by cutting out the captured image J21 using the facial frame of person P31 with ID "12345" in the detected person information J22, and then extracts features from the extracted facial image using a feature extractor. Technology such as FaceNet may be applied to the facial image feature extractor.

[0059] The feature amount extracted from the face image by the feature extraction unit 114 is a biometric identifier that can identify an individual and can be used for personal authentication, etc. Generally, the feature amount is often expressed as a real number vector with several tens to several thousands of dimensions.

[0060] Reinforcement learning is an example of a method for using the extracted facial features (biometric identifier) ​​of the person with ID "12345." For example, in S29, the first communication unit 115 may transmit the features extracted by the feature extraction unit 114 to an additional learning device (not shown) that uses facial features via the network 13 to perform reinforcement learning. By performing reinforcement learning, it is possible to generate a feature extractor that generally has improved ability to identify the person with ID "12345." Furthermore, the camera device 11 may perform additional learning, for example, using the facial features as a biometric identifier using the feature extraction unit 114 within the device. The camera device 11 may also identify a person through face authentication using the facial features as a biometric identifier.

[0061] 20 is a sequence diagram illustrating the flow of processing in the biometric identifier management system of the first embodiment. Explanations of parts that overlap with the above explanations will be simplified.

[0062] As shown in FIG. 20, in S2001, the imaging unit 111 of the camera device 11 captures an image of a subject including a person or the like to generate a captured image.

[0063] In S2002, the face detection unit 112 detects a human face from the captured image.

[0064] In S2003, the ID input unit 113 acquires the ID of the person input by the user etc. In S20031, the first communication unit 115 requests the consent management server 12 for consenting person information.

[0065] In S2010, the consent extraction unit 122 of the consent management server 12 extracts consenting person information held in the consent information holding unit 123.

[0066] In S2011, the second communication unit 121 transmits the consenting person information to the first communication unit 115 of the camera device 11.

[0067] In S2004, the first communication unit 115 of the camera device 11 receives the consenting person information from the consent management server 12.

[0068] In S2005, the comparison unit 116 compares the ID of the detected person information with the ID of the consenting person information, determines whether the ID of the detected person information is included in the ID of the consenting person information, and updates the detected person information based on the determination result. For example, if the ID of the detected person information is included in the ID of the consenting person information, the comparison unit 116 changes the consent status of the person in the detected person information to "yes" and changes the name of the person from "unknown." If the ID of the detected person information is not included in the ID of the consenting person information, the comparison unit 116 may change the consent status of the person in the detected person information to "no."

[0069] In S2006, the display control unit 117 superimposes a face frame, a person's name, and the like on the captured image to generate a new image.

[0070] In S2007, the display control unit 117 causes the display unit 118 to display an image with a face frame and a name superimposed thereon.

[0071] In S2008, the feature extraction unit 114 extracts the feature amount of the person whose consent status in the detected person information is "yes."

[0072] In S2009, the camera device 11 performs processing using the feature amount. The processing using the feature amount may include, for example, outputting the biometric identifier to another device, storing the biometric identifier in a storage means, performing additional learning using the biometric identifier, and identifying a person using the biometric identifier. More specifically, the first communication unit 115 may transmit the feature amount to an external device and use it in reinforcement learning. Furthermore, the feature extraction unit 114 may perform additional learning using the feature amount. Note that S2008 and S2009 may be performed after S2005.

[0073] As described above, in the biometric identifier management system of the first embodiment, the consent management server 12 holds information regarding consent to the use of feature amounts, and this information is available to other devices (camera device 11). This has the effect of reducing the burden on the user, as there is no need to obtain consent each time an image is captured. In addition, the consent management server 12 centrally manages consent to the use of feature amounts, which has the effect of facilitating the management of information regarding consent.

[0074] Furthermore, there is an effect that the camera device 11 functioning as the biometric identifier utilization device does not need to send the biometric identifier to the consent management server 12 functioning as the biometric identifier management server.

[0075] Second Embodiment In the first embodiment, an example was shown in which facial features are used as a biometric identifier only when consent is given to the use of the biometric identifier. In the second embodiment, an example will be described in which an ID does not need to be specified for each person.

[0076] Fig. 8 is a diagram showing the overall configuration of a biometric identifier management system according to the second embodiment. As shown in Fig. 8, the biometric identifier management system according to the second embodiment differs from the first embodiment shown in Fig. 1 in that it does not include an ID input unit 113. In the second embodiment, a feature extraction unit 114 extracts facial features of a person extracted from a captured image as personal information. The feature extraction unit 114 is an example of an acquisition unit.

[0077] FIG. 9 is a diagram showing the flow of consent information determination processing executed by the camera device 11 of the second embodiment. FIG. 10 is a diagram showing an example of detected person information J92 of the second embodiment. FIG. 11 is a diagram showing an example of consenting person information of the second embodiment. Note that although items with the same numbering may have different inputs and outputs, they generally perform the same operation. This also applies to the following embodiments. Note that in the second embodiment, the consenting person information J61 will be described as consenting person information J93.

[0078] The following description will focus on the parts that operate differently from the first embodiment shown in FIGS.

[0079] In S22, the face detection unit 112 performs face detection processing to detect a person's face in the captured image J21, and generates and outputs detected person information J92. Figure 10(a) is an example of the detected person information J92 generated by the face detection unit 112. Compared to the detected person information J22 in Figure 4(a), the detected person information J92 in Figure 10(a) has an additional feature column.

[0080] In S93, the feature extraction unit 114 extracts facial features of the person. The feature extraction unit 114 updates the detected person information J92 by adding the extracted features to the detected person information J92. FIG. 10(b) shows an example of the updated detected person information J92. The method of extracting features in S93 is the same as in S28 of FIG. 2.

[0081] As shown in Fig. 6, in S61, the consent extraction unit 122 acquires consenting person information J93 from the consent information storage unit 123. As shown in Fig. 11, the consenting person information J93 stored in the consent information storage unit 123 of the second embodiment includes the feature amount of each person in addition to the consenting person information J61 of Fig. 7. The feature amount of each person may be an average feature amount.

[0082] Here, the feature amounts of the detected person information J92 shown in FIG. 10(b) are feature amounts calculated by the feature extraction unit 114 from the face included in the image obtained by cutting out the face frame portion from the captured image J21.

[0083] On the other hand, the feature amounts of the consenting person information J93 shown in FIG. 11 are those of a specific person (e.g., ID "12345") that has been extracted and registered in advance. A personal identification method using face recognition compares the feature amounts of a registered person (individual) with feature amounts acquired from an image captured by a camera, and identifies those with similar values ​​as the same person. Note that the method for calculating the feature amounts in FIG. 11 can be achieved by applying known technology, and is a widely known technology in the field of face recognition, so details will be omitted.

[0084] One example is a personal identification method that calculates features from various facial images of a specific person in advance and then averages them. Since feature values ​​for the same person are calculated to be similar, the feature values ​​calculated from individual images in Figure 10(b) are likely to be similar to the average feature values ​​shown in Figure 11 if they are calculated from the same person.

[0085] In S62, the second communication unit 121 transmits the consenting person information J93 acquired by the consent extraction unit 122 to the first communication unit 115 of the camera device 11 via the network 13.

[0086] Here, the feature, which is the biometric identifier, is transferred from the consent management server 12 to the camera device 11, but this does not pose a problem because the person registered in the consent information storage unit 123 has already agreed to the use of the biometric identifier.

[0087] 9 , in S94, the first communication unit 115 receives consenting person information J93 including feature amounts of a person who has consented to the use of the biometric identifier from the second communication unit 121 of the consent management server 12 functioning as the biometric identifier management server. The camera device 11 may acquire consenting person information from the consent management server 12, for example, by transmitting a request for consenting person information to the consent management server 12 in S94.

[0088] In S95, the comparison unit 116 performs a matching process based on the features contained in each row of the consenting person information J93 received from the consent management server 12 shown in Figure 11 and the features of each person acquired by the feature extraction unit 114 and contained in the detected person information J92 shown in Figure 10(b), thereby determining whether the person has consented to the use of the biometric identifier.

[0089] In the feature matching process, the comparison unit 116 calculates the similarity between the features to determine the similarity. A method for calculating the similarity may be selected that is appropriate for the feature extraction algorithm. For example, cosine similarity, Euclidean distance, etc. may be used to calculate the similarity.

[0090] In the example of the second embodiment, the similarity between Feature 1 of the detected person information J92 in FIG. 10(b) and Feature B of the consenting person information J93 in FIG. 11 is high, and the similarity between other combinations is low.

[0091] In this case, in S95, the comparison unit 116 determines that the person in Feature 1 in the first row of the detected person information J92 is the person in Feature B in the second row of the consenting person information J93. As a result, the comparison unit 116 determines that the person has consented to the use of biometric authentication, and updates the first row of the detected person information J92.

[0092] 10(c) shows the updated detected person information J92. Based on the information in Feature B of the consenting person information J93, the comparison unit 116 updates the person's ID in the first row of the detected person information J92 to "12345," the consent status to "yes," and the name to "Johnny Depp."

[0093] On the other hand, if the feature of the person in the detected person information J92 is not similar to any of the feature in any of the rows of the consenting person information J93, the comparison unit 116 may determine that the person has not consented to the use of biometric authentication, and may update the detected person information J92 based on the result of this determination, as shown in the second row of Figure 10(c). Specifically, the comparison unit 116 updates the consent status of the person to "No," deletes the feature, and updates it to "Unknown."

[0094] The processes of S26, S27, and S29 are the same as those in the first embodiment, and an image in which a face frame and a name are superimposed on the captured image J21 is displayed on the display unit 118 as shown in FIG. 3(c).

[0095] In this embodiment, in S93, the feature extraction unit 114 calculates feature amounts even for persons whose consent to the use of their biometric identifier is unknown. However, for persons whose consent cannot be confirmed, the camera device 11 only uses the feature amounts to confirm consent, and does not display the name (use the feature amounts) or transfer the feature amounts to an external device. Furthermore, the camera device 11 performs the next image capture in S21 while the detected person information J92 including the feature amounts is held in memory, and thus does not actually store the feature amounts.

[0096] The handling of feature amounts (biometric identifiers) is currently being legislated in each country and region, and this embodiment can be adopted on the condition that it does not violate legal regulations.

[0097] 21 is a sequence diagram illustrating the flow of processing in the biometric identifier management system of the second embodiment. Explanations of parts that overlap with the above explanations will be simplified or partially omitted.

[0098] 21, after the camera device 11 executes S2001 and S2002, the feature extraction unit 114 extracts features from the person's face in S2101. Also, in S20031, the first communication unit 115 requests consenting person information from the consent management server 12.

[0099] In S2010, the consent extraction unit 122 of the consent management server 12 extracts consenting person information stored in the consent information storage unit 123. The consenting person information here includes feature amounts, as shown in FIG.

[0100] In S2011, the second communication unit 121 transmits the consenting person information to the first communication unit 115 of the camera device 11.

[0101] In S2102, the first communication unit 115 of the camera device 11 receives consenting person information including the feature amount.

[0102] In S2103, the comparison unit 116 compares the features of the person in the detected person information J92 with the features of each row of the consenting person information J93 to determine whether there is a person in the consenting person information that matches the person in the detected person information, and updates the detected person information based on the determination result.

[0103] Thereafter, the camera device 11 executes S2006, S2007, and S2009. S2009 may be executed before S2006 and S2007, or may be executed in parallel with each other.

[0104] As described above, in the second embodiment, facial features of a person captured in a captured image are extracted as a biometric identifier, and the features of the person who has given consent to the use of the biometric identifier are acquired from the consent management server 12 external to the camera device 11. Then, the camera device 11 of the second embodiment determines whether consent has been given based on the acquired features of the person, using the biometric identifier extracted within the device.

[0105] According to the second embodiment, there is no need to specify an ID for each person in a captured image in order to identify the person, which has the effect of reducing the burden on the user.

[0106] <Third embodiment> In the third embodiment, an example will be described in which the camera device 11 acquires consenting person information of a person captured in a captured image J21 from the consent management server 12.

[0107] More specifically, the camera device 11 of the third embodiment sends a query including the ID acquired by the ID input unit 113 to the consent management server 12. The consent management server 12 sends a response indicating whether or not the person corresponding to the ID included in the query, which is information included in the consenting person information, has consented to the use of the biometric identifier to the camera device 11 as a reply to the query. The camera device 11 receives the response, determines whether or not consent has been given, and updates the detected person information. In this way, it is possible to reduce an increase in network traffic and an increase in the processing load of S25 for ID comparison within the camera device 11.

[0108] 12 is a diagram showing the overall configuration of a biometric identifier management system according to the third embodiment. In the third embodiment, an ID input unit 113 is connected to a first communication unit 115.

[0109] Fig. 13 is a diagram showing the flow of consent information determination processing executed by the camera device 11 of the third embodiment. Fig. 14 is a diagram showing the flow of consent information transmission processing executed by the consent management server 12 of the third embodiment.

[0110] 13, in the camera device 11 of the third embodiment, the first communication unit 115 transmits an ID as a query in S131 after S23. As shown in Fig. 14, the second communication unit 121 of the consent management server 12 of the third embodiment receives the ID transmitted by the first communication unit 115 in S141, and the consent extraction unit 122 extracts consenting person information of the person corresponding to the ID in S142.

[0111] The following description will focus on steps S131, S133, and S141 to S143 that are different from the first embodiment.

[0112] In S23, the ID input unit 113 of the camera device 11 identifies the ID of the target person. Here, the detected person information J22 is as shown in FIG. 4(b), and the ID identified by the ID input unit 113 is only "12345."

[0113] In S131, the first communication unit 115 transmits a query including the identified ID to the second communication unit 121 of the consent management server 12 via the network 13. Here, the first communication unit 115 transmits the identified ID "12345" to the consent management server 12.

[0114] In S141, the second communication unit 121 of the consent management server 12 receives the ID “12345” sent from the first communication unit 115.

[0115] In S142, the consent extraction unit 122 checks whether the consenting person information J61 stored in the consent information storage unit 123 contains an ID that matches the received ID "12345." Referring to FIG. 7, a row with the ID "12345" and the name "Johnny Depp" exists as a row that matches the ID "12345." Therefore, the consent extraction unit 122 extracts the information of that row from the consenting person information J61.

[0116] In S143, the second communication unit 121 transmits the information of the row of the consenting person information J61 extracted by the consent extraction unit 122 to the first communication unit 115 of the camera device 11 as a response indicating consent to the use of the biometric identifier.

[0117] In the above example, in S131, the camera device 11 transmits a single ID "12345." However, the camera device 11 may transmit a query including multiple IDs to the consent management server 12. When transmitting multiple IDs, the first communication unit 115 may transmit the multiple IDs simultaneously or sequentially. In this case, the consent extraction unit 122 performs the processes from S141 to S143 for all transmitted IDs. Furthermore, if the consent extraction unit 122 cannot find a corresponding ID from the consenting person information J61, the second communication unit 121 may transmit a message indicating that the ID was not found.

[0118] 13, in S132, the first communication unit 115 of the camera device 11 receives the consenting person information J132. Here, the first communication unit 115 receives the consenting person information J132 of the row corresponding to the transmitted ID.

[0119] In S133, the comparison unit 116 of the camera device 11 compares the ID included in the received consenting person information J132 with the ID of each row of the detected person information J22, and updates the information of the corresponding row of the detected person information J22.

[0120] Thereafter, the camera device 11 performs the processes of S26, S27, S28, and S29.

[0121] 22 is a sequence diagram illustrating the flow of processing in the biometric identifier management system of the third embodiment. Descriptions of processing that overlap with the above description will be simplified or partially omitted.

[0122] As shown in FIG. 22, after the camera device 11 executes S2001 to S2003, in S2201, the first communication unit 115 transmits a query including the ID of the person in the captured image acquired by the ID input unit 113 to the consent management server 12.

[0123] In S2210, the second communication unit 121 of the consent management server 12 receives the query sent from the camera device 11.

[0124] In S2211, the consent extraction unit 122 extracts information about the person corresponding to the received ID from the consenting person information.

[0125] In S2212, the second communication unit 121 transmits the information extracted by the consent extraction unit 122 from the consenting person information (information linked to the ID) to the first communication unit 115 of the camera device 11.

[0126] In S2202, the first communication unit 115 receives the information transmitted by the second communication unit 121.

[0127] In S2203, the comparison unit 116 compares the ID included in the consenting person information received from the consent management server 12 with the ID of each row of the detected person information, and updates the information of the corresponding row of the detected person information.

[0128] After that, the camera device 11 executes steps S2006 to S2009.

[0129] As described above, the camera device 11 of the third embodiment transmits the IDs of people who appear in a captured image to the consent management server 12, and acquires the corresponding consenting person information from the consent management server 12. According to the third embodiment, even if there are a large number of people who have consented to the use of biometric identifiers, it is possible to reduce the amount of data sent from the consent management server 12 to the camera device 11. Therefore, the third embodiment has the effect of preventing an increase in network traffic, reducing the processing load within the camera device 11, and shortening the processing time.

[0130] <Fourth embodiment> In the fourth embodiment, a form will be described in which input of an ID is not required and an increase in network traffic is prevented.

[0131] More specifically, in the fourth embodiment, information that cannot identify a person but can list candidate persons is sent from the camera device 11 to the consent management server 12. The consent management server 12 extracts information about candidate persons from the consenting person information based on the received information and sends it back to the camera device 11.

[0132] Fig. 15 is a diagram showing the overall configuration of a biometric identifier management system according to the fourth embodiment. As shown in Fig. 15, the difference from the second embodiment in Fig. 8 is that in the fourth embodiment, the camera device 11 has a reduction unit 151, and the consent management server 12 has a reduction unit 124. The reduction unit 151 and the reduction unit 124 reduce information on the person's feature amounts, i.e., the amount of data, to generate reduced information.

[0133] Fig. 16 is a diagram showing the flow of consent information determination processing executed by the camera device 11 of the fourth embodiment. Fig. 17 is a diagram showing the flow of consent information transmission processing executed by the consent management server 12 of the fourth embodiment. Fig. 18 is a diagram showing an example of consenting person information of the fourth embodiment.

[0134] 16, the camera device 11 of the fourth embodiment executes S161 and S162 between S93 and S133. The following mainly describes the parts that are different from the above-described embodiments.

[0135] In S93 , the feature extraction unit 114 of the camera device 11 extracts feature amounts for each face image included in the captured image and passes them to the reduction unit 151 .

[0136] In S161, the reduction unit 151 performs processing to reduce information (amount of data) of the feature amounts extracted by the feature extraction unit 114. Hereinafter, the information obtained by reducing the information of the feature amounts will be referred to as reduced information.

[0137] Generally, features have the ability to identify an individual and are equivalent to biometric identifiers. The purpose of feature reduction in S161 is to convert features that act as biometric identifiers capable of identifying an individual into information that does not have the ability to identify an individual but has the ability to identify multiple candidate individuals, including the individual in question, i.e., information that is not a biometric identifier.

[0138] For example, in the case of a feature that has the ability to identify 10 million individuals, a feature calculated from a facial image of Person A can identify one Person A out of 10 million people. However, the reduced information based on this feature only has the ability to list (for example) 100 candidates, including Person A, out of 10 million people, and is not able to identify an individual.

[0139] The feature reduction method is not particularly limited in this embodiment. For example, the reduction unit 151 may obtain reduction information by performing a predetermined calculation on specific dimensions of the feature represented by a vector, or by deleting specific dimensions from the feature. Specifically, if the feature is a real number of a 1024-dimensional vector, the reduction unit 151 may reduce the feature by leaving it as a 1024-dimensional vector and calculating the average value of adjacent dimension values ​​as each value, or by using a Gaussian kernel to calculate a weighted average value using values ​​of surrounding dimensions as each value. Alternatively, the reduction unit 151 may remove specific 512 dimensions from the 1024 dimensions and obtain the remaining 512-dimensional vector as reduction information.

[0140] In the following description, the reduced information is assumed to be 512-dimensional.

[0141] As a result, the reduced information reduced by the reduction unit 151 in S161 loses its ability to identify individuals and does not become a biometric identifier.

[0142] In S162, the first communication unit 115 sends a query including the reduction information obtained by the reduction unit 151 to the second communication unit 121 of the consent management server 12 via the network 13.

[0143] As shown in FIG. 17, in S171, the second communication unit 121 of the consent management server 12 receives the reduction information as received reduction information J171.

[0144] The consent management server 12 then repeats the processes from S172 to S174 described below for each of the individual reception reduction information J1711 included in the received reception reduction information J171.

[0145] In S172, the reduction unit 124 acquires the consenting person information J93 shown in FIG.

[0146] The reduction unit 124 and consent extraction unit 122 repeat the processes of S173 to S174 for the feature amount of each person in the consenting person information J93, and extract and list people who are candidates for the individual reception reduction information J1711.

[0147] 11 is a 1024-dimensional vector, and therefore cannot be directly compared with the 512-dimensional vector of the individual reception reduced information J1711. Therefore, in S173, the reduction unit 124 reduces the feature amounts of the consenting person information J93 to generate 512-dimensional consenting person reduced information J172.

[0148] 16 executed by the reduction unit 151 of the camera device 11 and S173 of Fig. 17 executed by the reduction unit 124 of the consent management server 12 execute the same reduction process on the feature amounts. This enables the consent extraction unit 122 to compare the individual reception reduction information J1711 with the consenting person reduction information J172.

[0149] In S174, the consent extraction unit 122 calculates the similarity between the individual reception reduction information J1711 and the consent person reduction information J172. The method for calculating the similarity has already been described, so a detailed description will be omitted.

[0150] In S174, if the agreement extraction unit 122 determines that the similarity is equal to or greater than a predetermined value, it determines that there is a high possibility that the two are the same person and selects the person as a candidate. On the other hand, if the agreement extraction unit 122 determines that the similarity is less than the predetermined value, it determines that there is a low possibility that the two are the same person. In either case, the process returns to S173, and the reduction unit 124 reduces the feature information of the next person, and in S174, the agreement extraction unit 122 compares the new reduced information with the reduced information and determines the similarity.

[0151] The reduction unit 124 and the consent extraction unit 122 execute the processes of S173 and S174 for all persons in the consenting person information J93 to generate a list of candidate persons for the individual reception reduction information J1711. The list of candidate persons is also a list indicating whether or not the person corresponding to the feature acquired by the feature extraction unit 114 has consented to the use of the biometric identifier. When there is a plurality of pieces of individual reception reduction information J1711, the reduction unit 124 and the consent extraction unit 122 repeat the processes from S172 onward to generate a list of candidate persons for all pieces of individual reception reduction information J1711. In this case, the consent extraction unit 122 generates a table linking each piece of individual reception reduction information J1711 with the list of candidate persons as a similarity determination result.

[0152] In S175, the second communication unit 121 transmits a response including a list of candidate persons extracted from the consenting person information to the first communication unit 115 via the network 13 as a reply to the query.

[0153] For example, suppose that the reduced information of Feature 1 among the features shown in FIG. 10(b) is sent to the consent management server 12, and the consent extraction unit 122 determines that the reduced information of Feature 1 has a high similarity to the reduced information of two features, Feature A and Feature B, in FIG. 11. In this case, the consent extraction unit 122 selects ID "11012" and ID "12345" as candidate persons, and extracts information about the persons in the first and second rows of FIG. 11 from the consenting person information J93. Therefore, the second communication unit 121 transmits information about the two persons (names, whether or not they consent, and features) to the camera device 11 as a response.

[0154] On the other hand, suppose that the reduced information of Feature 2 among the features shown in Fig. 10(b) is sent to the consent management server 12, and the consent extraction unit 122 determines that the reduced information has a low similarity to the reduced information of any of the features shown in Fig. 11. In this case, the second communication unit 121 sends a response to the camera device 11 indicating that the person is not included.

[0155] Returning to FIG. 16, in S132, the first communication unit 115 receives the list of candidates transmitted by the second communication unit 121 as consenting person information J132.

[0156] In S133, for example, the comparison unit 116 compares Feature 1 in the first row of the detected person information J92 shown in Figure 10(b) with Feature A and Feature B, which are the features of the candidate person, and determines that the person is the person in Feature A, i.e., the person with ID "12345".

[0157] Furthermore, the comparison unit 116 determines that the person in the second row of the detected person information J92 in FIG. 10(b) is not included in the candidate persons, that is, is not a person who has consented to the use of the biometric identifier.

[0158] Based on the result of the determination, the comparison unit 116 updates the detected person information J92, resulting in the information shown in FIG. 10(c).

[0159] The subsequent processes (S26, S27, S29) are the same as those in FIG.

[0160] In this embodiment, the consent management server 12 performs information reduction in S173 each time to calculate the consenting person reduction information J172, but the reduction information may also be generated at the same time as generating the consenting person information J93. An example of the consenting person information J93 in this case is shown in Fig. 18. As shown in Fig. 18, the consenting person information J93 includes reduction information of feature amounts linked to the ID of each person.

[0161] By doing so, it is not necessary to perform the process of S173 every time, and in S174, it is sufficient to compare with the reduction information in Fig. 18. As a result, the processing load on the consent management server 12 can be reduced.

[0162] 23 is a sequence diagram illustrating the flow of processing in the biometric identifier management system of the fourth embodiment. Explanations of parts that overlap with the above explanations will be simplified or partially omitted.

[0163] As shown in FIG. 23, after the camera device 11 executes S2001, S2002, and S2101, in S2301 the reduction unit 151 reduces the information of the extracted feature amounts to generate reduced information.

[0164] In S2302, the first communication unit 115 transmits a query including the reduction information to the second communication unit 121 of the consent management server 12.

[0165] In S2310, the second communication unit 121 of the consent management server 12 receives the query sent by the first communication unit 115.

[0166] In S2311, the reduction unit 124 acquires consenting person information from the consent information storage unit 123.

[0167] In S2312, the reduction unit 124 reduces the information on the feature quantities of the consenting person information by the same method as in S2301, to generate consenting person reduced information.

[0168] In S2313, the consent extraction unit 122 compares the reduction information (reception reduction information) received in S2310 with the consenting person reduction information obtained by reducing the amount of data for the feature amounts of each row of the consenting person information, and extracts rows of consenting person information of candidate persons that are similar to the reception reduction information. Note that if the reception reduction information includes individual reception reduction information that is information about multiple people, the consent extraction unit 122 makes a similarity determination for each of the individual reception reduction information.

[0169] In S2314, the second communication unit 121 transmits information on each line of the consenting person information of the candidate person extracted by the consent extraction unit 122 to the camera device 11.

[0170] In S2102, the first communication unit 115 of the camera device 11 receives the information of each line of the consenting person information transmitted by the second communication unit 121.

[0171] The camera device 11 executes the processes of S2103, S2006, S2007, and S2009.

[0172] As described above, in the fourth embodiment, the camera device 11 calculates the feature amounts of people captured in a captured image, converts the calculated feature amounts into reduced information that does not have the ability to identify individuals, and transmits the reduced information to the consent management server 12. Then, the consent management server 12 extracts candidates for people who have given consent to the use of the biometric identifier based on the reduced information, and sends a list of the candidates to the camera device 11.

[0173] By doing so, in the fourth embodiment, it is no longer necessary to specify an ID for each person each time to identify a person in a captured image, and furthermore, even if there are a large number of people who have agreed to the use of biometric identifiers, the consent management server 12 can extract candidate persons, thereby reducing the amount of data sent back from the consent management server 12 to the camera device 11. Therefore, the fourth embodiment can achieve at least one of the following effects: reducing the burden on the user, preventing an increase in network traffic, reducing the processing load within the camera device 11, and shortening the processing time.

[0174] Note that the camera device 11 in each of the above-described embodiments may include, for example, a device with a built-in imaging unit 111, such as a digital camera, a smartphone, or a tablet PC, as well as a configuration in which the imaging unit 111 and the information processing device 190 exist as separate devices.

[0175] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0176] In the above-described embodiment, the processes that utilize a person's biometric identifier depending on the result of the determination by the comparison unit 116 include outputting the biometric identifier to another device, storing the biometric identifier in a storage means, performing additional learning using the biometric identifier, and identifying the person using the biometric identifier, but the processes that are utilized are not limited to these and may be changed as appropriate.

[0177] The disclosure of this specification includes the following information processing device, information processing method, and program. (Item 1) an acquisition means for acquiring person information that is information about a person included in a captured image; A communication means for communicating with a server that stores information on consenting parties who have consented to the use of biometric identifiers; a determination means for determining whether or not a person corresponding to the personal information acquired by the acquisition means has consented to the use of a biometric identifier based on information received by the communication means from the server; and processing means for executing a process using the biometric identifier of the person in accordance with the result of the determination by the determination means. (Item 2) 2. The information processing device according to item 1, wherein the personal information is identification information assigned to each person. (Item 3) The communication means receives consenting person information including identification information of the consenting person from the server, The information processing device described in item 2 is characterized in that the determination means determines that the person has agreed to the use of the biometric identifier if the identification information contained in the consenting person information received from the server includes the identification information acquired by the acquisition means. (Item 4) The communication means is sending a query including the identification information acquired by the acquisition means to the server; 3. The information processing device according to item 2, wherein a response indicating whether the person corresponding to the identification information has consented to the use of a biometric identifier is received from the server in response to the query. (Item 5) 2. The information processing device according to item 1, wherein the person information is a feature amount of a person extracted from a captured image. (Item 6) The communication means receives consenting person information including the characteristic amount of the consenting person from the server, The information processing device described in item 5 is characterized in that the determination means determines whether the person has consented to the use of the biometric identifier by performing a matching process based on the features included in the consenting person information received from the server and the features acquired by the acquisition means. (Item 7) a reduction unit that reduces the amount of data of the feature amount acquired by the acquisition unit to obtain reduced information; The communication means is sending a query including the reduction information obtained by the reduction means to the server; Item 6. The information processing device according to item 5, characterized in that a response indicating whether the person corresponding to the feature acquired by the acquisition means has agreed to the use of the biometric identifier is received from the server in response to the query. (Item 8) 8. The information processing device according to item 7, wherein the reduction means performs a predetermined operation on specific dimensions of the feature values ​​represented by vectors, or obtains reduced information by deleting specific dimensions from the feature values. (Item 9) 9. The information processing device according to any one of items 1 to 8, wherein the biometric identifier is a feature extracted from at least one of a face, a fingerprint, and an iris pattern. (Item 10) The processing means performs the process of using the biometric identifier of the person, outputting the biometric identifier to another device; storing the biometric identifier in a storage means; performing additional learning using the biometric identifier; and 10. The information processing device according to any one of items 1 to 9, wherein the information processing device performs at least one of the following: identifying a person using the biometric identifier. (Item 11) 5. The information processing device according to any one of items 2 to 4, further comprising an extraction unit that extracts a biometric identifier of the person when the result of the determination by the determination unit indicates consent. (Item 12) 9. The information processing device according to any one of items 5 to 8, wherein the determination means deletes the feature amount of the person when the result of the determination indicates non-consent. (Item 13) an acquisition step of acquiring person information that is information about a person included in a captured image; a communication step of communicating with a server that stores information of consenting parties who have consented to the use of biometric identifiers; a determination step of determining whether or not the person corresponding to the personal information acquired in the acquisition step has consented to the use of a biometric identifier based on information received from the server in the communication step; and a processing step of executing a process that uses the biometric identifier of the person depending on the result of the determination in the determination step. (Item 14) 13. A program for causing a computer to function as each means of the information processing device according to any one of items 1 to 12.

[0178] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0179] 11···Camera device, 12···Consent management server, 112···Face detection unit, 113···ID input unit, 114···Feature extraction unit, 115···First communication unit, 116···Comparator, 190···Information processing device, 151···Reduction unit, 124···Reduction unit.

Claims

1. an acquisition means for acquiring person information that is information about a person included in a captured image; A communication means for communicating with a server that stores information on consenting parties who have consented to the use of biometric identifiers; a determination means for determining whether or not a person corresponding to the personal information acquired by the acquisition means has consented to the use of a biometric identifier based on information received by the communication means from the server; and processing means for executing a process using the biometric identifier of the person in accordance with the result of the determination by the determination means.

2. 2. The information processing apparatus according to claim 1, wherein the personal information is identification information given to each person.

3. The communication means receives consenting person information including identification information of the consenting person from the server, The information processing device according to claim 2, characterized in that the determination means determines that the person has agreed to the use of the biometric identifier if the identification information contained in the consenting person information received from the server includes the identification information acquired by the acquisition means.

4. The communication means is sending a query including the identification information acquired by the acquisition means to the server; 3. The information processing apparatus according to claim 2, wherein a response indicating whether or not the person corresponding to the identification information has consented to the use of a biometric identifier is received from the server as a reply to the query.

5. 2. The information processing apparatus according to claim 1, wherein the person information is a feature amount of a person extracted from a captured image.

6. The communication means receives consenting person information including the characteristic amount of the consenting person from the server, The information processing device according to claim 5, characterized in that the determination means determines whether the person has agreed to the use of the biometric identifier by performing a matching process based on the features contained in the consenting person information received from the server and the features acquired by the acquisition means.

7. a reduction unit that reduces the amount of data of the feature amount acquired by the acquisition unit to obtain reduced information; The communication means is sending a query including the reduction information obtained by the reduction means to the server; The information processing device according to claim 5, further comprising: receiving, from the server in response to the query, a response indicating whether or not the person corresponding to the feature acquired by the acquisition means has consented to the use of a biometric identifier.

8. The information processing device according to claim 7, wherein the reduction means obtains reduced information by performing a predetermined operation on specific dimensions of the feature amounts represented by vectors, or by deleting specific dimensions from the feature amounts.

9. 9. The information processing apparatus according to claim 1, wherein the biometric identifier is a feature extracted from at least one of a face, a fingerprint, and an iris pattern.

10. The processing means performs the process of using the biometric identifier of the person, outputting the biometric identifier to another device; storing the biometric identifier in a storage means; performing additional learning using the biometric identifier; and 9. The information processing apparatus according to claim 1, wherein the information processing apparatus performs at least one of the following operations: identifying a person using the biometric identifier;

11. 3. The information processing apparatus according to claim 2, further comprising: an extracting unit that extracts a biometric identifier of the person when the result of the determination by the determining unit indicates consent.

12. 6. The information processing apparatus according to claim 5, wherein the determining means deletes the feature amount of the person when the result of the determination indicates non-consent.

13. an acquisition step of acquiring person information that is information about a person included in a captured image; a communication step of communicating with a server that stores information of consenting parties who have consented to the use of biometric identifiers; a determination step of determining whether or not the person corresponding to the personal information acquired in the acquisition step has consented to the use of a biometric identifier based on information received from the server in the communication step; and a processing step of executing a process that uses the biometric identifier of the person depending on the result of the determination in the determination step.

14. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 8.

Images