Information processing system, information processing method, and recording medium

Abstract

This information processing system (10) comprises: a storage means (110) for storing, in association with each other, face information pertaining to a user and re-identification information used for re-identifying the user; an acquisition means (120) for acquiring face information and re-identification information pertaining to a subject user; an assessment means (130) for assessing whether the face information and re-identification information pertaining to the subject user are a pair of information items stored in association with each other; and an output means (140) for performing output that corresponds to the assessment result. According to this information processing system, it is possible to suitably detect impersonation of the subject user.

Description

[Technical field]

[0001] The present disclosure relates to the technical fields of an information processing system, an information processing method, and a recording medium for processing information such as face information. [Background technology]

[0002] As this type of system, there is known a system that performs face authentication (i.e., matching of face information) using face information. For example, Patent Document 1 discloses that face information used for face matching is linked to identification information such as a person's ID and registered, and used for matching. Patent Document 2 discloses that before performing face recognition processing, features other than the face are determined in stages to narrow down the face images to be matched with the face image to be authenticated.

[0003] As other related techniques, Patent Document 3 discloses a technique for detecting human-like movements and determining whether or not a person is impersonating another person. Patent Document 4 discloses a technique for detecting impersonation by recognizing changes in the color of a person's clothing, etc. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 110012 [Patent Document 2] JP 2020-187543 A [Patent Document 3] JP 2015-041307 A [Patent Document 4] JP 2010-154134 A Summary of the Invention [Problem to be solved by the invention]

[0005] This disclosure is intended to improve upon the techniques disclosed in the prior art documents. [Means for solving the problem]

[0006] One aspect of the information processing system disclosed herein includes a storage means for linking and storing a user's facial information and re-identification information used to re-identify the user, an acquisition means for acquiring the facial information and re-identification information of a target user, a determination means for determining whether the facial information and the re-identification information of the target user are a pair that are linked to each other and stored, and an output means for producing an output according to the result of the determination.

[0007] One aspect of the information processing method disclosed herein is to store a user's facial information and re-identification information used to re-identify the user in association with each other, obtain the facial information and re-identification information of a target user, determine whether the facial information and the re-identification information of the target user are a pair that are stored in association with each other, and provide output according to the result of the determination.

[0008] One aspect of the recording medium of this disclosure is a computer program recorded on a computer to execute an information processing method that links and stores a user's facial information and re-identification information used to re-identify the user, acquires the facial information and re-identification information of a target user, determines whether the facial information and the re-identification information of the target user are a pair that are stored in linkage with each other, and provides output according to the result of the determination. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a hardware configuration of an information processing system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a functional configuration of an information processing system according to a first embodiment. [Diagram 3] 4 is a flowchart showing the flow of operations of the information processing system according to the first embodiment. [Figure 4] 10 is a flowchart showing the flow of operations of an information processing system according to a second embodiment. [Diagram 5]FIG. 13 is a block diagram showing a functional configuration of an information processing system according to a third embodiment. [Figure 6] 13 is a flowchart showing the flow of operations of an information processing system according to a third embodiment. [Figure 7] 13 is a table showing specific examples of re-identification information used by the information processing system according to the fourth embodiment. [Figure 8] FIG. 13 is a block diagram showing a functional configuration of an information processing system according to a fifth embodiment. [Figure 9] FIG. 13 is a block diagram showing a functional configuration of an information processing system according to a sixth embodiment. [Figure 10] 13 is a flowchart showing the flow of operations of an information processing system according to a sixth embodiment. [Figure 11] 20 is a flowchart showing the flow of operations of the information processing system according to the seventh embodiment. [Figure 12] 20 is a flowchart showing the flow of operations of an information processing system according to the eighth embodiment. [Figure 13] FIG. 13 is a conceptual diagram (part 1) showing a display example when acquiring information of a target user. [Figure 14] FIG. 2 is a conceptual diagram (part 2) showing a display example when acquiring information of a target user. [Figure 15] FIG. 11 is a conceptual diagram (part 3) showing a display example when acquiring information of a target user. [Figure 16] FIG. 4 is a conceptual diagram showing a display example when acquiring information of a target user. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of an information processing system, an information processing method, and a recording medium will be described with reference to the drawings.

[0011] First Embodiment An information processing system according to a first embodiment will be described with reference to FIGS. 1 to 3. FIG.

[0012] (Hardware configuration) First, the hardware configuration of an information processing system 10 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the information processing system according to the first embodiment.

[0013] 1, an information processing system 10 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing system 10 may further include an input device 15, an output device 16, and a camera 20. The processor 11, the RAM 12, the ROM 13, the storage device 14, the input device 15, the output device 16, and the camera 20 are connected via a data bus 17.

[0014] The processor 11 reads a computer program. For example, the processor 11 is configured to read a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may read a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may obtain (i.e., read) a computer program from a device (not shown) disposed outside the information processing system 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the computer program that the processor 11 reads. In particular, in this embodiment, when the processor 11 executes the computer program that the processor 11 reads, a functional block for executing a process using face information is realized in the processor 11.

[0015] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a demand-side platform (DSP), or an application specific integrated circuit (ASIC). The processor 11 may be configured as one of these, or may be configured to use a plurality of these in parallel.

[0016] The RAM 12 temporarily stores a computer program executed by the processor 11. The RAM 12 temporarily stores data that is temporarily used by the processor 11 when the processor 11 is executing the computer program. The RAM 12 may be, for example, a D-RAM (Dynamic RAM).

[0017] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store other fixed data. The ROM 13 may be, for example, a programmable ROM (P-ROM).

[0018] The storage device 14 stores data that is to be stored long-term by the information processing system 10. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0019] The input device 15 is a device that receives an input instruction from a user of the information processing system 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel.

[0020] The output device 16 is a device that outputs information related to the information processing system 10 to the outside. For example, the output device 16 may be a display device (for example, a display) that can display information related to the information processing system 10.

[0021] Camera 20 is a camera installed at a location capable of capturing an image of the user (particularly, an image including the user's face). Camera 20 may be a camera that captures still images or a camera that captures moving images. Camera 20 may be configured as a visible light camera or a near-infrared camera. Multiple cameras 20 may be installed, and each may be a camera with different specifications. When multiple cameras 20 are installed, they may be disposed in different locations.

[0022] (Functional configuration) Next, the functional configuration of the information processing system 10 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0023] As shown in Fig. 2, the information processing system 10 according to the first embodiment is configured to include a user information database 110, a target user information acquisition unit 120, a pair determination unit 130, and an output unit 140 as processing blocks for realizing the functions thereof. The user information database 110 may be configured to include, for example, the above-mentioned storage device 14 (see Fig. 1). The target user information acquisition unit 120, the pair determination unit 130, and the output unit 140 may be realized by, for example, the above-mentioned processor 11 (see Fig. 1). The output unit 140 may be configured to be capable of outputting its output by the output device 16 (see Fig. 1).

[0024] The user information database 110 is configured to be able to store information on a plurality of users acquired in advance. The user information database 110 is configured to be able to store face information of a user and re-identification information used for re-identification of the user in association with each other (i.e., as a pair). For example, as shown in the figure, face information and re-identification information of user A may be stored as a pair of face information A and re-identification information A. Similarly, face information and re-identification information of user B may be stored as a pair of face information B and re-identification information B. Face information and re-identification information of user C may be stored as a pair of face information C and re-identification information C. Note that the "face information" here is information on the face of a user, and may be, for example, a face image itself, or may be a feature amount indicating a face feature extracted from a face image. In addition, the "re-identification information" is information that can determine whether or not the user is the same person, and is information different from face information (for example, information acquired from something other than a face). Specific examples of re-identification information will be described in detail in other embodiments described later.

[0025] The target user information acquisition unit 120 is configured to be able to acquire face information and re-identification information from a target user who is a target of judgment. The target user information acquisition unit 120 may be configured to be able to acquire face information and re-identification information from an image of the target user captured by the camera 20, for example. In this case, the target user information acquisition unit 120 may have a function of extracting face information and re-identification information by performing various analysis processes on the image of the target user. The face information and re-identification information may be acquired from one common image, or may be acquired from separate images. However, the target user information acquisition unit 120 may be configured to be able to acquire face information and re-identification information using a means other than the camera 20.

[0026] The pair determination unit 130 is configured to be able to determine whether or not the facial information and re-identification information of the target user are a pair stored in association with each other. Specifically, the pair determination unit 130 determines whether or not each of the facial information and re-identification information acquired by the target user information acquisition unit 120 matches the facial information and re-identification information (i.e., paired facial information and re-identification information) stored in association with each other in the user information database 110. Note that whether or not the facial information and the re-identification information match may be determined using, for example, a preset threshold value. Specifically, the pair determination unit 130 may determine that the facial information and the re-identification information match when the degree of match (similarity) of the facial information and the re-identification information exceeds a threshold value, and may determine that the facial information and the re-identification information do not match when the degree of match (similarity) does not exceed the threshold value.

[0027] The output unit 140 performs output according to the determination result of the pair determination unit 130. The output unit 140 may directly output the determination result of the pair determination unit 130, or may output the result of another process executed according to the determination result of the pair determination unit 130. The output mode of the output unit 140 is not particularly limited, but the output unit 140 may perform image output using a display, for example. Alternatively, the output unit 140 may perform audio output using a speaker.

[0028] (Operation flow) Next, the flow of operations of the information processing system 10 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operations of the information processing system according to the first embodiment.

[0029] 3, when the information processing system 10 according to the first embodiment operates, the target user information acquisition unit 120 first acquires face information and re-identification information from the target user (step S101). The target user information acquisition unit 120 may acquire the re-identification information after acquiring the face information, or may acquire the face information after acquiring the re-identification information. Alternatively, the target user information acquisition unit 120 may acquire the face information and the re-identification information in parallel at the same time.

[0030] Next, the pair determination unit 130 determines whether or not the face information and re-identification information acquired by the target user information acquisition unit 120 are a pair (hereinafter, appropriately referred to as a "correct pair") linked to and stored in the user information database 110 (step S102). Note that a specific flow of this determination process will be described in detail in another embodiment described later. Thereafter, the output unit 140 performs output according to the determination result of the pair determination unit 130 (step S103).

[0031] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the first embodiment will be described.

[0032] 1 to 3, in the information processing system 10 according to the first embodiment, it is determined whether the face information and re-identification information of the target user are a correct pair (i.e., a pair stored in advance), and an output is performed according to the determination result. In this way, it is possible to detect impersonation of the target user.

[0033] For example, if a malicious user attempts to break through the authentication process by spoofing, the user must somehow (for example, by presenting a terminal displaying a face photo or wearing a 3D mask) to have the system recognize the face of another person. In such a case, the facial information of the target user changes before and after the spoofing, but the re-identification information obtained from sources other than the face does not change. Therefore, the pair of facial information and re-identification information stored before the spoofing will no longer match the pair of facial information and re-identification information after the spoofing. Therefore, if the facial information and re-identification information of the target user are not a correct pair, it can be determined that there is a high possibility that the target user is spoofing.

[0034] <Second embodiment> An information processing system 10 according to the second embodiment will be described with reference to Fig. 4. The second embodiment differs from the first embodiment described above only in some operations, and other parts may be the same as the first embodiment. Therefore, hereinafter, parts that differ from the first embodiment already described will be described in detail, and explanations of overlapping parts will be omitted as appropriate.

[0035] (Operation flow) First, the flow of operations of the information processing system 10 according to the second embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of operations of the information processing system according to the second embodiment. In Fig. 4, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0036] 4, when the information processing system 10 according to the second embodiment operates, the target user information acquisition unit 120 first acquires face information and re-identification information from the target user (step S101). Then, the pair determination unit 130 determines whether the face information and re-identification information acquired by the target user information acquisition unit 120 are a pair stored in association with the user information database 110 (step S102).

[0037] Next, when the pair determination unit 130 determines that the face information and re-identification information of the target user are a correct pair (step S201: YES), it outputs that the target user is not spoofing (step S202). On the other hand, when the pair determination unit 130 determines that the face information and re-identification information of the target user are not a correct pair (step S201: NO), it outputs that the target user is spoofing (step S203).

[0038] The output here may be output as an alert indicating that the target user is spoofing (i.e., fraud is occurring). Specifically, a warning display may be displayed or a warning sound may be output indicating the occurrence of spoofing. The output of the output unit 140 may not be output to the target user, but may be output only to a system administrator, a security guard, or the like.

[0039] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the second embodiment will be described.

[0040] 4, in the information processing system 10 according to the second embodiment, when it is determined that the face information and re-identification information of the target user are not a correct pair, information notifying that impersonation is occurring is output. In this way, it is possible to appropriately determine whether the target user is impersonating someone, and promptly notify the person who should be notified of the occurrence of impersonation.

[0041] <Third embodiment> An information processing system 10 according to the third embodiment will be described with reference to Figures 5 and 6. The third embodiment differs from the first and second embodiments described above only in some configurations and operations, and other parts may be the same as the first and second embodiments. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0042] (Functional configuration) First, the functional configuration of the information processing system 10 according to the third embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the functional configuration of the information processing system according to the third embodiment. In Fig. 5, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0043] 5, the information processing system 10 according to the third embodiment is configured to include, as processing blocks for realizing its functions, a user information database 110, a target user information acquisition unit 120, a pair determination unit 130, an output unit 140, and a face matching unit 150. That is, the information processing system 10 according to the third embodiment further includes a face matching unit 150 in addition to the configuration of the first embodiment (see FIG. 2).

[0044] The face matching unit 150 is configured to be able to perform face matching (in other words, face authentication) of the target user using the face information acquired by the target user information acquisition unit 120. Note that a specific method of face matching can be appropriately adopted from existing technologies, so detailed description will be omitted here. The face matching unit 150 according to this embodiment is particularly configured to perform face matching when the pair determination unit 130 determines that the face information and re-identification information of the target user are a correct pair. The matching result of the face matching unit 150 is configured to be able to be output by the output unit 140.

[0045] The face matching described above may be performed to authenticate whether the target user is the registered user. For example, a user whose face matching is successful may be permitted to pass through a gate through which only registered users can pass. Alternatively, a user whose face matching is successful may be permitted to perform a payment process using a payment method associated with the registered user.

[0046] (Operation flow) Next, the flow of operations of the information processing system 10 according to the third embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of operations of the information processing system according to the third embodiment. In Fig. 6, the same processes as those shown in Figs. 3 and 4 are denoted by the same reference numerals.

[0047] 6, when the information processing system 10 according to the third embodiment operates, the target user information acquisition unit 120 first acquires face information and re-identification information from the target user (step S101). Then, the pair determination unit 130 determines whether the face information and re-identification information acquired by the target user information acquisition unit 120 are a correct pair associated with and stored in the user information database 110 (step S102).

[0048] Next, when the pair determination unit 130 determines that the facial information and re-identification information of the target user are not a correct pair (step S201: NO), it outputs that the target user is spoofing (step S203). On the other hand, when the pair determination unit 130 determines that the facial information and re-identification information of the target user are a correct pair (step S201: YES), it executes face matching of the target user (step S301). Then, the output unit 140 outputs the result of face matching by the face matching unit 150 (step S302). Note that the output unit 140 may output, in addition to the result of face matching, that the target user is not spoofing (second embodiment: see FIG. 4).

[0049] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the third embodiment will be described.

[0050] As described in Fig. 5 and Fig. 6, in the information processing system 10 according to the third embodiment, when it is determined that the face information and re-identification information of the target user are a correct pair, face matching is performed and the result is output. In this way, face matching is performed when it is determined that the target user is not spoofing, so face matching is not performed on the target user who is spoofing. Therefore, it is possible to prevent the target user who is spoofing from fraudulently bypassing face matching.

[0051] <Fourth embodiment> The information processing system 10 according to the fourth embodiment will be described with reference to Fig. 7. The fourth embodiment describes a specific example of re-identification information, and the configuration and operation may be the same as those of the first to third embodiments. Therefore, in the following, the description of the parts that overlap with the embodiments already described will be omitted as appropriate.

[0052] (Specific examples of re-identification information) The re-identification information used in the information processing system 10 according to the fourth embodiment will be described with reference to Fig. 7. Fig. 7 is a table showing specific examples of the re-identification information used in the information processing system according to the fourth embodiment.

[0053] 7, the re-identification information used by the information processing system 10 according to the fourth embodiment may be the user's body type, clothing, gait, belongings, accompanying persons, voice, and terminal information (i.e., information that can be acquired from things owned by the user). Note that the re-identification information may include only one of these types of information, or may include multiple types.

[0054] When the re-identification information includes the user's body type, clothing, gait, possessions, or accompanying persons, the information may be acquired, for example, from an image of the user. Note that, as a method for acquiring information on the body type, clothing, gait, possessions, or accompanying persons from an image of the user, it is possible to appropriately adopt existing technology.

[0055] Furthermore, when the re-identification information includes the user's voice, the information may be acquired from a microphone. When the re-identification information is terminal information, the information may be acquired by wireless communication with the user's terminal, etc. Note that it is preferable that the terminal information is not acquired when the user holds the terminal over a predetermined position, but is automatically acquired without user operation.

[0056] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the fourth embodiment will be described.

[0057] 7, in the information processing system 10 according to the fourth embodiment, at least one of the user's body type, clothing, gait, belongings, accompanying persons, voice, and information obtainable from an object held by the user is used as re-identification information. In this way, it is possible to appropriately detect impersonation of the target user by determining whether the face information and the re-identification information are a correct pair.

[0058] <Fifth embodiment> An information processing system 10 according to the fifth embodiment will be described with reference to Fig. 8. The fifth embodiment differs from the first to fourth embodiments in some configurations, and other parts may be the same as the first to fourth embodiments. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate.

[0059] (Functional configuration) First, the functional configuration of the information processing system 10 according to the fifth embodiment will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the functional configuration of the information processing system according to the fifth embodiment. In Fig. 8, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0060] As shown in FIG. 8, the information processing system 10 according to the fifth embodiment is configured to include a user information database 110, a target user information acquisition unit 120, a pair determination unit 130, and an output unit 140 as processing blocks for realizing the functions. The user information database 110 according to the fifth embodiment is particularly configured to be able to acquire and store face information and re-identification information of a user from images captured by a plurality of cameras 20 (surveillance cameras 20a, 20b, and 20c). Note that, for convenience of explanation, three surveillance cameras 20a, 20b, and 20c are illustrated here, but the face information and re-identification information of a user may be acquired using more surveillance cameras.

[0061] The surveillance cameras 20a, 20b, and 20c are cameras installed at different locations. Therefore, the user information database 110 can acquire and store face information and re-identification information at a plurality of locations. In addition, the surveillance cameras 20a, 20b, and 20c can capture images of the user from different angles. Therefore, by using the surveillance cameras 20a, 20b, and 20c, face information and re-identification information can be acquired from a user image captured from an appropriate angle. In addition, the surveillance cameras 20a, 20b, and 20c can capture images of the user at different timings. Therefore, by using the surveillance cameras 20a, 20b, and 20c, face information and re-identification information can be acquired from a user image captured at an appropriate timing. Note that the surveillance cameras 20a, 20b, and 20c may periodically capture images at a predetermined interval, for example, or may capture images when the angle of the face that can be acquired changes or when the re-identification information changes. For example, when the re-identification information is clothing, the timing when the user takes off the jacket he or she was wearing may be an example of the timing when the re-identification information is a companion. Also, when the re-identification information is a companion, the timing when the companion changes due to a change (for example, when a pair of mother and child A and a pair of father and child B become a pair of mother and child B and a pair of father and child A, etc.).

[0062] In addition, each of the face information and the re-identification information acquired at a plurality of locations may be stored as separate information. That is, a plurality of pairs of face information and re-identification information may be stored for one common user. Specifically, for one user, for example, a pair of re-identification information of an outfit in which a jacket is worn and face information at that time, and a pair of re-identification information of an outfit in which a jacket is removed and face information at that time may be stored as separate patterns. In addition, for a common user, newly acquired face information and re-identification information may be stored so as to be overwritten. That is, the information may be updated every time new information is acquired.

[0063] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the fifth embodiment will be described.

[0064] As described in FIG. 8, in the information processing system 10 according to the fifth embodiment, face information and re-identification information are acquired and stored at multiple locations (e.g., cameras installed at multiple locations). In this way, even if face information and re-identification information cannot be acquired at only one location, face information and re-identification information can be acquired at other locations. For example, face information can be acquired at one location, and re-identification information can be acquired at another location. Alternatively, if face information and re-identification information acquired at one location is not appropriate (e.g., if the face or body is hidden and the acquired face information and re-identification information are insufficient), face information and re-identification information can be acquired again at another location.

[0065] Sixth embodiment An information processing system 10 according to the sixth embodiment will be described with reference to Fig. 9 and Fig. 10. The sixth embodiment differs from the first to fifth embodiments in some configurations and operations, and other parts may be the same as the first to fifth embodiments. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0066] (Functional configuration) First, the functional configuration of the information processing system 10 according to the sixth embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the functional configuration of the information processing system according to the sixth embodiment. In Fig. 9, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0067] 9, the information processing system 10 according to the sixth embodiment is configured to include, as processing blocks for realizing its functions, a user information database 110, a target user information acquisition unit 120, a pair determination unit 130, and an output unit 140. The pair determination unit 130 according to the sixth embodiment particularly includes a re-identification information comparison unit 131 and a face information comparison unit 132.

[0068] The re-identification information comparison unit 131 is configured to be able to extract, from the multiple re-identification information stored, information that matches the re-identification information of the target user by comparing the re-identification information of the target user acquired by the target user information acquisition unit 120 with the re-identification information of multiple users stored in the user information database 110. Note that the re-identification information comparison unit 131 may extract only one piece of re-identification information (for example, only the re-identification information with the highest degree of match), or may extract multiple pieces of re-identification information (for example, all pieces of re-identification information with degrees of match exceeding a predetermined threshold).

[0069] The face information comparison unit 132 is configured to be able to read out face information stored in association with the re-identification information extracted by the re-identification information comparison unit 131 (i.e., information that matches the re-identification information of the target user) from the user information database 110. The face information comparison unit is also configured to be able to determine whether the face information of the target user matches the read-out face information by comparing the face information of the target user with the read-out face information. The face information determined by the face information comparison unit 132 is already determined by the re-identification information comparison unit 131 to match the re-identification information, so when it is determined that the face information matches, it can be determined that the face information and the re-identification information pair is a correct pair.

[0070] (Operation flow) Next, the flow of operations of the information processing system 10 according to the sixth embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the flow of operations of the information processing system according to the sixth embodiment. In Fig. 10, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0071] As shown in FIG. 10, when the information processing system 10 according to the sixth embodiment operates, first, the target user information acquisition unit 120 acquires face information and re-identification information from the target user (step S101).

[0072] Next, the re-identification information comparison unit 131 extracts re-identification information that matches the re-identification information of the target user from the re-identification information stored in the user information database 110 (step S601).

[0073] Next, the face information comparison unit 132 reads out face information stored in association with the re-identification information extracted by the re-identification information comparison unit 131 (step S602). Then, the face information comparison unit 132 compares the face information of the target user with the read face information to determine whether the pair of the face information and the re-identification information is a correct pair (step S603).

[0074] Thereafter, the output unit 140 performs output according to the determination result of the pair determination unit 130 (that is, the re-identification information comparison unit 131 and the face information comparison unit 132) (step S103).

[0075] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the sixth embodiment will be described.

[0076] 9 and 10, in the information processing system 10 according to the sixth embodiment, first, it is determined whether the re-identification information matches, and then it is determined whether the re-identification information matches the face information linked to the matching re-identification information. In this way, it is possible to appropriately determine whether the face information and the re-identification information of the target user are a correct pair.

[0077] Seventh embodiment An information processing system 10 according to the seventh embodiment will be described with reference to Fig. 11. The seventh embodiment differs from the sixth embodiment described above only in some operations, and other parts may be the same as the sixth embodiment. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and other overlapping parts will be omitted as appropriate.

[0078] (Operation flow) First, the flow of operations of the information processing system 10 according to the seventh embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of operations of the information processing system according to the seventh embodiment. In Fig. 11, the same processes as those shown in Fig. 10 are denoted by the same reference numerals.

[0079] As shown in FIG. 11, when the information processing system 10 according to the seventh embodiment operates, first, the target user information acquisition unit 120 acquires face information and re-identification information from the target user (step S101).

[0080] Next, the re-identification information comparison unit 131 extracts re-identification information that matches the re-identification information of the target user from the re-identification information stored in the user information database 110 (step S601). Note that, here, the description will proceed assuming that the re-identification information comparison unit 131 extracts a plurality of pieces of re-identification information.

[0081] Next, the face information comparison unit 132 reads out face information stored in association with the re-identification information extracted by the re-identification information comparison unit 131 (step S602). That is, it reads out a plurality of pieces of face information stored in association with each of the extracted plurality of pieces of re-identification information. Then, the face information comparison unit 132 determines whether the face information of the target user matches any of the plurality of pieces of face information read out (step S701).

[0082] Next, if the facial information of the target user matches any of the plurality of pieces of facial information read out (step S701: YES), the facial information comparison unit 132 determines that the facial information of the target user and the re-identification information are a correct pair (step S702). On the other hand, if the facial information of the target user does not match any of the plurality of pieces of facial information read out (step S701: NO), the facial information comparison unit 132 determines that the facial information of the target user and the re-identification information are not a correct pair (step S703).

[0083] Thereafter, the output unit 140 performs output according to the determination result of the pair determination unit 130 (i.e., the re-identification information comparison unit 131 and the face information comparison unit 132) (step S103). That is, the output unit 140 performs output according to whether or not it is determined that the face information and the re-identification information of the subject are a correct pair.

[0084] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the seventh embodiment will be described.

[0085] 11, in the information processing system 10 according to the seventh embodiment, when there are multiple users with matching re-identification information, if the facial information matches any of the facial information of the users, it is determined that the facial information and the re-identification information are a correct pair. In this way, even when there are multiple users with matching re-identification information (for example, when there are multiple users with similar re-identification information), it is possible to appropriately determine whether the facial information and the re-identification information of the target user are a correct pair.

[0086] Eighth embodiment The information processing system 10 according to the eighth embodiment will be described with reference to Fig. 12. The information processing system 10 according to the eighth embodiment differs from the sixth embodiment described above only in some operations, and other parts may be the same as those of the sixth embodiment. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0087] (Operation flow) First, the flow of operations of the information processing system 10 according to the eighth embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the flow of operations of the information processing system according to the eighth embodiment. In Fig. 12, the same processes as those shown in Fig. 10 are denoted by the same reference numerals.

[0088] As shown in Fig. 12, when the information processing system 10 according to the eighth embodiment operates, the target user information acquisition unit 120 first acquires face information and re-identification information from the target user (step S101). In this embodiment, the re-identification information includes multiple types of information. For example, the re-identification information may include two or more pieces of information among the various pieces of information shown in the fourth embodiment (see Fig. 7).

[0089] Next, the re-identification information comparison unit 131 compares the re-identification information of the target user with the re-identification information stored in the user information database 110. Specifically, the re-identification information comparison unit 131 determines whether or not each of the multiple types of information included in the re-identification information matches (step S801).

[0090] Next, the face information comparison unit 132 reads out face information associated with the re-identification information in which a predetermined number or more pieces of information match as a result of the comparison in the re-identification information comparison unit 131 (step S803). The "predetermined number" here is a number set to determine whether the re-identification information matches as a whole, and an appropriate value is set in advance. For example, when the predetermined number is set to three, the re-identification information in which three or more pieces of multiple types of information match is determined to match the re-identification information of the target user, and the face information associated therewith is read out. On the other hand, the re-identification information in which two or less pieces of multiple types of information match is determined to not match the re-identification information of the target user, and the face information associated therewith is not read out.

[0091] The predetermined number may be a fixed value, or may be a value that changes depending on the type of information. For example, a priority may be set for each type of information, and the predetermined number may be varied depending on the priority of the matching information. Specifically, the condition for determining that the re-identification information matches may be that two or more pieces of high-priority information match, three or more pieces of medium-priority information match, and four or more pieces of low-priority information match.

[0092] Thereafter, the face information comparison unit 132 compares the face information of the target user with the read face information to determine whether the pair of face information and re-identification information is a correct pair (step S603).Then, the output unit 140 performs output according to the determination result of the pair determination unit 130 (i.e., the re-identification information comparison unit 131 and the face information comparison unit 132) (step S103).

[0093] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the eighth embodiment will be described.

[0094] 12, in the information processing system 10 according to the eighth embodiment, when a predetermined number or more pieces of information match among a plurality of types of information included in the re-identification information, the re-identification information is determined to match. In this way, even if there is another user whose re-identification information matches in part (for example, there are a plurality of users who have similar re-identification information), it is possible to appropriately determine whether the face information and the re-identification information of the target user are a correct pair.

[0095] <Display example when acquiring target user information> Next, examples of displays presented to the target user when acquiring information of the target user (more specifically, when the target user information acquisition unit 120 captures an image for acquiring face information and re-identification information of the target user) will be specifically described with reference to Figs. 13 to 16. Fig. 13 is a conceptual diagram (part 1) showing an example of a display when acquiring information of the target user. Fig. 14 is a conceptual diagram (part 2) showing an example of a display when acquiring information of the target user. Fig. 15 is a conceptual diagram (part 3) showing an example of a display when acquiring information of the target user. Fig. 16 is a conceptual diagram (part 4) showing an example of a display when acquiring information of the target user.

[0096] As shown in FIG. 13, when acquiring information of the target user, a display may be made to guide the target user to a position so that the user's body type, clothes, gait, and belongings are included in the image. For example, in the example shown in FIG. 13, the target user is too close to the camera 20, and the lower side of the target user's shoulders is not captured. Therefore, it is difficult to detect the target user's body type, clothes, gait, and belongings from the image. In such a case, a message such as "Please step back a little more" may be displayed to the target user. Note that, although an example of guiding the target user in the forward and backward directions with respect to the camera is given here, guidance in the left and right directions or rotational directions may also be performed. Specifically, if the re-identification information is an earring in the right ear, a message such as "Please look left" may be displayed so that the user's earrings are included in the image. In order to appropriately perform such guidance, the re-identification information of the user and its position information may be stored in association with each other. For example, if the re-identification information is a possession, it may be stored that the backpack is located on the user's back. If the re-identification information is clothing, it may be stored that the user has a black down jacket positioned on the upper body and blue jeans positioned on the lower body.

[0097] As shown in Fig. 14, when acquiring information of the target user, a display may be made to align the user's clothing so that it matches the clothing at the time when the user's clothing was stored. For example, in the example shown in Fig. 14, the target user takes off his / her jacket midway through the process, resulting in a mismatch between the stored re-identification information (i.e., the initially acquired clothing information) and the current re-identification information (i.e., the current clothing information). In such a case, a message such as "Please wear the clothing you were wearing at the entrance" may be displayed to the target user.

[0098] As shown in FIG. 15, when acquiring information of the target user, for example, a display may be made instructing the target user to walk in order to acquire the user's gait as re-identification information. For example, in the example shown in FIG. 15, a message is displayed to the target user saying, "Please walk a few steps in front of the camera." Note that, when the camera is installed near a walk-through gate (i.e., a place through which the target user walks), it is also possible to omit displaying such a message. An example of a system equipped with a walk-through gate is one in which, for example, a space for users to walk is installed in front of the camera, and information is acquired from users walking through the space.

[0099] As shown in Fig. 16, when acquiring information of the target user, for example, a display may be displayed instructing the target user to speak in order to acquire the user's voice as re-identification information. For example, in the example shown in Fig. 16, a message is displayed to the target user saying "Please say something into the microphone." Note that, if the target user has already spoken (for example, when talking to a companion), the voice may be automatically acquired, and such a display may be omitted.

[0100] The above-mentioned display example is merely an example, and other displays may be made according to the situation in order to more appropriately acquire information on the target user. Also, instead of or in addition to such displays, a similar notification may be made by voice.

[0101] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-mentioned embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. That is, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, not only the recording medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment.

[0102] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. In addition, the scope of each embodiment does not include those that execute processing by themselves recorded on the recording media, but also those that execute processing by operating on an OS in cooperation with other software and functions of an extension board.

[0103] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing systems, information processing methods, and recording media incorporating such modifications are also included in the technical idea of ​​this disclosure.

[0104] <Additional Notes> The above-described embodiment may be further described as follows, but is not limited to the following.

[0105] (Appendix 1) The information processing system described in Appendix 1 is an information processing system comprising: a storage means for linking and storing a user's facial information and re-identification information used to re-identify the user; an acquisition means for acquiring the facial information and re-identification information of a target user; a determination means for determining whether the facial information and the re-identification information of the target user are a pair that are linked to each other and stored; and an output means for performing output according to the result of the determination.

[0106] (Appendix 2) The information processing system described in Appendix 2 is the information processing system described in Appendix 1, in which the output means outputs information indicating that the target user is engaging in impersonation when it is determined that the face information and the re-identification information of the target user are not a pair stored in association with each other.

[0107] (Appendix 3) The information processing system described in Appendix 3 is the information processing system described in Appendix 1 or 2, further comprising a face matching means that performs face matching of the target user when it is determined that the face information and the re-identification information of the target user are a pair that are linked to each other and stored, and the output means outputs the result of the face matching.

[0108] (Appendix 4) The information processing system described in Appendix 4 is an information processing system described in any one of Appendixes 1 to 3, wherein the re-identification information includes at least one of the user's body type, clothing, gait, possessions, accompanying persons, voice, and information obtainable from items possessed by the user.

[0109] (Appendix 5) The information processing system described in Appendix 5 is the information processing system described in any one of Appendixes 1 to 4, wherein the acquisition means is provided in multiple locations, and the storage means acquires and stores the face information and the re-identification information from the acquisition means provided in multiple locations.

[0110] (Appendix 6) The information processing system described in Supplementary Note 6 is an information processing system described in any one of Supplements 1 to 5, in which the determination means performs the determination by extracting from the stored re-identification information that matches the re-identification information of the target user, and determining whether the facial information linked to the extracted re-identification information matches the facial information of the target user.

[0111] (Appendix 7) The information processing system described in Appendix 7 is the information processing system described in Appendix 6, wherein the determination means determines that the facial information and the re-identification information of the target user are a pair stored in association with each other when the re-identification information of the target user matches multiple pieces of stored re-identification information and when the facial information of the target user matches any of multiple pieces of facial information stored in association with the multiple pieces of re-identification information.

[0112] (Appendix 8) The information processing system described in Appendix 8 is the information processing system described in Appendix 6 or 7, wherein the re-identification information includes multiple types of information, and the determination means extracts the re-identification information that matches a predetermined number or more of the multiple types of information as matching the re-identification information of the target user.

[0113] (Appendix 9) The information processing method described in Supplementary Note 9 is an information processing method that links and stores a user's facial information and re-identification information used to re-identify the user, acquires the facial information and re-identification information of a target user, determines whether the facial information and the re-identification information of the target user are a pair that are linked to each other and stored, and provides output according to the result of the determination.

[0114] (Appendix 10) The recording medium described in Appendix 10 is a recording medium having recorded thereon a computer program that causes a computer to execute an information processing method that links and stores a user's facial information and re-identification information used to re-identify the user, acquires the facial information and re-identification information of a target user, determines whether the facial information and the re-identification information of the target user are a pair that are linked and stored with each other, and provides output according to the result of the determination.

[0115] (Appendix 11) The computer program described in Appendix 11 is a computer program that causes a computer to execute an information processing method that links and stores a user's facial information and re-identification information used to re-identify the user, acquires the facial information and re-identification information of a target user, determines whether the facial information and the re-identification information of the target user are a pair that are linked and stored with each other, and provides output according to the result of the determination. [Explanation of symbols]

[0116] 10. Information Processing Systems 11 Processors 14 Storage device 16 Output Devices 20 Camera 110 User Information Database 120 Target user information acquisition unit 130 Pair Judgment Section 131 Re-identification information comparison unit 132 Face Information Comparison Unit 140 Output section 150 Face matching unit

Claims

1. A storage means for storing face information of a user and re-identification information used for re-identifying the user in association with each other; An acquisition means for acquiring the face information and the re-identification information of a target user; A determination means for determining whether or not the face information and the re-identification information of the target user are a pair that are associated with each other and stored; a face matching means for performing face matching of the target user when it is determined that the face information and the re-identification information of the target user are a pair that are associated with each other and stored; an output means for outputting a result of the face matching when the face matching is executed, and for outputting information indicating that the target user is spoofing an identity when it is determined that the face information and the re-identification information of the target user are not a pair that are associated with each other and stored; An information processing system comprising:

2. The information processing system according to claim 1 , wherein the re-identification information includes at least one of information obtainable from the user's body type, clothing, gait, possessions, accompanying persons, voice, and items possessed by the user.

3. The acquiring means is provided in plurality, 3. The information processing system according to claim 1, wherein the storage unit acquires and stores the face information and the re-identification information from the acquisition unit provided at a plurality of locations.

4. The information processing system of any one of claims 1 to 3, wherein the determination means performs the determination by extracting from the stored re-identification information that matches the re-identification information of the target user, and determining whether the facial information linked to the extracted re-identification information matches the facial information of the target user.

5. The information processing system of claim 4, wherein the determination means determines that the facial information of the target user and the re-identification information are a pair stored in association with each other when the re-identification information of the target user matches multiple pieces of stored re-identification information and when the facial information of the target user matches any of multiple pieces of facial information stored in association with the multiple pieces of re-identification information.

6. The re-identification information includes a plurality of types of information, The determination means extracts the re-identification information in which a predetermined number or more of the plurality of types of information match as the re-identification information of the target user.

6. The information processing system according to claim 4 or 5.

7. At least one computer The face information of the user is associated with re-identification information used for re-identifying the user and stored in the storage device; Obtaining the face information and the re-identification information of a target user; determining whether the face information and the re-identification information of the target user are a pair that are associated with each other and stored; When it is determined that the face information of the target user and the re-identification information are a pair stored in association with each other, performing face matching of the target user; When the face matching is executed, a result of the face matching is output, and when it is determined that the face information of the target user and the re-identification information are not a pair that are associated with each other and stored, information indicating that the target user is spoofing is output. Information processing methods.

8. On the computer, The face information of the user is associated with re-identification information used for re-identifying the user and stored in the storage device; Obtaining the face information and the re-identification information of a target user; determining whether the face information and the re-identification information of the target user are a pair that are associated with each other and stored; When it is determined that the face information of the target user and the re-identification information are a pair stored in association with each other, performing face matching of the target user; When the face matching is executed, a result of the face matching is output, and when it is determined that the face information of the target user and the re-identification information are not a pair that are associated with each other and stored, information indicating that the target user is spoofing is output. A computer program for executing an information processing method.

Images