Information processing system, information processing method, and computer program

Abstract

An information processing system S1 is provided with: a registration unit 11 that registers, in association with one another, living body identification information allowing a living body to be identified, living body information of a prescribed part of the living body, and wearing article identification information which is presented by a wearing article worn by the living body and which can be captured in an image; an acquisition unit 12 that acquires a photographed image in which the living body who wears the wearing article is photographed; a comparison unit 13 that compares the living body by using the wearing article identification information included in the photographed image; and an impersonation determination unit 14 that determines, when a prescribed condition is satisfied, whether a body other than the living body is impersonating the living body by using the image of the prescribed part of the living body included in the photographed image.

Description

Information processing system, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing system, an information processing method, and a recording medium.

[0002]

[0005] Patent Literature 1 describes a technology in which a first authentication means performs authentication based on a first authentication method, a recording means associates the authenticated user with a location where the user is located and records the association in a storage means if authentication using the first authentication method is successful, a second authentication means performs authentication based on a second authentication method, and a restriction means restricts authentication using the second authentication method from being successful if a user to be authenticated in the second authentication method and a location where the user is located are not associated in the storage means. Patent Literature 2 describes a technology in which, in a tournament where a game is played via a game device, users register to participate in the tournament, manages facial images of the users used to identify the users and tournament information which is information used to identify the tournament, performs facial authentication of the users using the facial images of the users, and progresses the tournament based on the results of the facial authentication. Patent Document 3 describes a technology in which a license containing transaction information that defines the usage conditions of the object for the user is printed as a two-dimensional code on a ticket, and a usage management device photographs the two-dimensional code printed on the ticket, reads the license from the photographed data, verifies the signature attached to the license using a public key provided in advance by the provider of the object, and if the signature verification is successful, refers to the transaction information included in the license, and permits the user to use the object if the usage conditions specified by the transaction information are satisfied. Patent Document 4 describes an authentication device that includes an image storage unit that can store an image of a code and a person, a code authentication unit that authenticates the code in the image, a person authentication unit that authenticates the person in the image, and an output unit that outputs an authentication result that is information indicating authentication permission when the authentication result of the code authentication unit is information indicating authentication permission and the authentication result of the person authentication unit also indicates authentication permission.

[0003] International Publication No. 2021 / 131060 JP 2021-164578 A JP 2021-135552 A JP 2016-018406 A

[0004] An object of this disclosure is to provide an information processing system, an information processing method, and a recording medium that aim to improve upon the techniques described in prior art documents.

[0005] One aspect of the information processing system includes a registration means for registering biometric identification information capable of identifying a living organism, biometric information of a specified part of the living organism, and clothing identification information presented by clothing worn by the living organism and capable of appearing in an image, in association with each other; an acquisition means for acquiring a captured image of the living organism wearing the clothing; a matching means for matching the living organism using the clothing identification information included in the captured image; and an impersonation determination means for determining, if specified conditions are met, whether something other than the living organism is impersonating the living organism using an image of a specified part of the living organism included in the captured image.

[0006] One aspect of the information processing method involves registering biometric identification information capable of identifying a living organism, biometric information of a specified part of the living organism, and clothing identification information presented by clothing worn by the living organism and capable of appearing in an image, in association with each other, obtaining a captured image of the living organism wearing the clothing, matching the living organism using the clothing identification information contained in the captured image, and, if specified conditions are met, determining whether something other than the living organism is impersonating the living organism using an image of the specified part of the living organism contained in the captured image.

[0007] One aspect of the recording medium has recorded on it a computer program for causing a computer to execute an information processing method that registers, in association with each other, biometric identification information capable of identifying a living organism, biometric information of a specified part of the living organism, and clothing identification information presented by clothing worn by the living organism and capable of appearing in an image, obtains a captured image of the living organism wearing the clothing, matches the living organism using the clothing identification information contained in the captured image, and, if specified conditions are met, determines whether something other than the living organism is impersonating the living organism using an image of the specified part of the living organism contained in the captured image.

[0008] FIG. 1 is a block diagram showing the configuration of an information processing system according to a first embodiment. FIG. 2 is a block diagram showing the configuration of an information processing system according to a second embodiment. FIG. 3 is a diagram showing an example of a scene in the information processing system according to the second embodiment. FIG. 4 is a flowchart showing the flow of a registration operation in the information processing system according to the second embodiment. FIG. 5 is a diagram showing an example of a scene in the information processing system according to the second embodiment. FIG. 6 is a flowchart showing the flow of a verification operation in the information processing system according to the second embodiment. FIG. 7 is a diagram showing an example of a scene in the information processing system according to the third embodiment. FIG. 8 is a block diagram showing the configuration of an information processing system according to a sixth embodiment. FIG. 9 is a flowchart showing the flow of a verification operation in the information processing system according to the sixth embodiment. FIG. 10 is a diagram showing an example of a scene in the information processing system according to a seventh embodiment. FIG. 11 is a block diagram showing the configuration of an information processing system according to the seventh embodiment. FIG. 12 is a flowchart showing the flow of a verification operation in the information processing system according to the seventh embodiment.

[0009] Hereinafter, an information processing system, an information processing method, and a recording medium according to an embodiment will be described with reference to the drawings. [1: First Embodiment]

[0010] A first embodiment of an information processing system, an information processing method, and a recording medium will be described below. Hereinafter, the first embodiment of the information processing system, the information processing method, and the recording medium will be described using an information processing system S1 to which the first embodiment of the information processing system, the information processing method, and the recording medium is applied. [1-1: Configuration of Information Processing System S1]

[0011] Fig. 1 is a block diagram showing the configuration of an information processing system S1 according to the first embodiment. As shown in Fig. 1, the information processing system S1 includes a registration unit 11, an acquisition unit 12, a matching unit 13, and a spoofing determination unit 14.

[0012] The registration unit 11 registers biometric identification information capable of identifying a living organism, biometric information of a predetermined part of the living organism, and clothing identification information presented by clothing worn by the living organism in association with each other. The clothing identification information is information that can be captured in an image. The acquisition unit 12 acquires a captured image of a living organism wearing the clothing. The matching unit 13 matches the living organism using the clothing identification information included in the captured image. If predetermined conditions are met, the impersonation determination unit 14 determines whether or not something other than the living organism is impersonating the living organism using an image of a predetermined part of the living organism included in the captured image. [1-2: Technical Effects of Information Processing System S1]

[0013] The information processing system S1 in the first embodiment performs biometric verification using clothing identification information, and when a predetermined condition is met, performs spoofing determination using the biometric information, thereby suppressing spoofing. Even if a person is spoofed, the spoofing can be detected. [2: Second embodiment]

[0014] Next, a second embodiment of the information processing system, the information processing method, and the recording medium will be described. Hereinafter, the second embodiment of the information processing system, the information processing method, and the recording medium will be described using an information processing system S2 to which the second embodiment of the information processing system, the information processing method, and the recording medium is applied.

[0015] 2 is a block diagram of an information processing system S2 according to the second embodiment. As shown in FIG. 2, the information processing system S2 according to the second embodiment may include a plurality of imaging devices C (i.e., imaging device C1, imaging device C2, ..., and imaging device CN) and an information processing device 2. [2-1: Configuration of Information Processing Device 2]

[0016] 2 , the information processing device 2 includes a calculation device 21 and a storage device 22. The information processing device 2 may further include a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not necessarily have to include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0017] The arithmetic device 21 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (e.g., an input device 24 described later) not shown in the drawings that is included in the information processing device 2. The arithmetic device 21 may acquire (i.e., download or read) the computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or another communication device). The arithmetic device 21 executes the read computer program. As a result, logical functional blocks for executing the operations to be performed by the information processing device 2 are realized within the arithmetic device 21. In other words, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing the operations (in other words, processing) to be performed by the information processing device 2.

[0018] 2 shows an example of logical functional blocks implemented within the computing device 21 to execute information processing operations. As shown in FIG. 2, the computing device 21 includes a registration unit 211, which is a specific example of a "registration means" described in the appendix, an acquisition unit 212, which is a specific example of an "acquisition means" described in the appendix, a matching unit 213, which is a specific example of a "matching means" described in the appendix, a spoofing determination unit 214, which is a specific example of a "spoofing determination means" described in the appendix, a determination unit 215, which is a specific example of a "determination means" described in the appendix, a correspondence unit 216, and an authentication unit 217. However, the computing device 21 does not necessarily have to include at least one of the determination unit 215, the correspondence unit 216, and the authentication unit 217. Details of the operation of each of the registration unit 211, acquisition unit 212, matching unit 213, impersonation determination unit 214, determination unit 215, correspondence unit 216, and authentication unit 217 will be described in detail later with reference to Figures 3 to 6, Table 1, and Table 2.

[0019] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that the arithmetic device 21 temporarily uses when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the information processing device 2 stores for a long period of time. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-transitory recording medium. The storage device 22 may include a personal information database 221. In the second embodiment, the personal information database 221 may be a database that associates biometric identification information capable of identifying at least a living organism, biometric information of a predetermined part of the living organism, and clothing identification information presented by clothing worn by the living organism and capable of appearing in an image. The personal information database 221 will be described with reference to Table 1. However, the storage device 22 may not include the personal information database 221. In this case, the personal information database 221 may be implemented in another device. The storage device 22 may also include an imaging device information database 222. In the second embodiment, the imaging device information database 222 may be a database that associates the imaging device C with information indicating whether an image captured by the imaging device C is to be used in the spoofing determination operation. The imaging device information database 222 will be described with reference to Table 2. However, the storage device 22 may not include the imaging device information database 222. In this case, the imaging device information database 222 may be implemented in another device.

[0020] The communication device 23 is capable of communicating with devices external to the information processing device 2 via a communication network (not shown). The information processing device 2 may transmit and receive signals to and from each of the multiple imaging devices C via the communication device 23.

[0021] The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operation device (e.g., at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the information processing device 2.

[0022] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (a so-called display) that can display an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (a so-called speaker) that can output sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (a so-called printer) that can print desired information on paper. [2-2: Registration Operation Performed by Information Processing System S2]

[0023] Next, the registration operation performed by the information processing system S2 in the second embodiment will be described with reference to Fig. 3, Fig. 4, and Table 1. Fig. 3 is a conceptual diagram of the information processing system S2 during the registration operation in the second embodiment. Fig. 4 is a flowchart showing the flow of the registration operation performed by the information processing system S2 in the second embodiment.

[0024] As shown in FIG. 3 , in the second embodiment, the living body may be a person. The person who is the target of the registration operation is also referred to as the subject P. In the second embodiment, a face area F may be adopted as a predetermined part of the living body for identifying the subject P. Furthermore, the article of clothing may be a jacket worn by the subject P. A code image W may be added to the jacket as the article of clothing identification information. The code image W may be printed on the jacket as the article of clothing identification information.

[0025] The information processing system S2 may perform a registration operation, for example, when the subject P is present at an entrance G of the specific area SA. An imaging device C for capturing an image of the subject P may be provided near the entrance G. For example, the entrance G may be an entrance to a building. In the second embodiment, at the entrance G, matching of the subject P using a code image W (hereinafter, sometimes referred to as "code matching") and biometric authentication may be used in combination, and the code image W of the subject P may be registered. In the second embodiment, the registration operation may be a temporary association operation, such as from entry to exit of the specific area SA in one day. The registration operation may be an operation performed when the subject P puts on a jacket. For example, the subject P may wear a jacket with a code image W attached thereto near the entrance G.

[0026] 4, the acquisition unit 212 acquires a captured image of the subject P wearing the jacket as an article of clothing (step S20). The acquisition unit 212 may acquire the image captured by the imaging device C via the communication device 23.

[0027] The associating unit 216 detects a face area F included in the captured image (step S21). The authenticating unit 217 performs face authentication to identify an ID as biometric identification information of the subject P (step S22). For example, the ID as biometric identification information and characteristic information of the face area F as biometric information may be associated and registered in advance in the storage device 22. The authenticating unit 217 may perform face authentication using the pre-registered information.

[0028] The association unit 216 detects the code image W attached to the jacket worn by the subject P included in the captured image (step S23). The association unit 216 makes a determination using the position of the detected face area F and the position of the detected code image W (step S24). For example, the association unit 216 may determine whether the position of the face area F and the position of the code image W have a natural positional relationship. The association unit 216 may determine, based on the position of the face area F and the position of the code image W, that the subject P whose face has been authenticated and the subject P wearing the jacket to which the code image W has been attached are the same person.

[0029] The registration unit 211 associates and registers the biometric identification information that can identify the subject P, characteristic facial information (also referred to as "facial feature amount") as biometric information of a predetermined part of the subject P, and information indicated by the code image W presented by the jacket worn by the subject P (step S25). The registration unit 211 may register the associated information in a personal information database 221 exemplified in Table 1 below. [Table 1: Personal Information Database 221]

[0030]

[0031] For example, the registration unit 211 may register an ID "A," a facial feature "AA" of the subject P with ID "A," and information "aa" indicated by the code image W presented by the jacket worn by the subject P, in association with each other. The registration unit 211 may also register an ID "B," a facial feature "BB" of the subject P with ID "B," and information "bb" indicated by the code image W presented by the jacket worn by the subject P, in association with each other. The registration unit 211 may also register an ID "C," a facial feature "CC" of the subject P with ID "C," and information "cc" indicated by the code image W presented by the jacket worn by the subject P, in association with each other. [2-3: Verification and Impersonation Determination Operation Performed by Information Processing System S2]

[0032] Next, the verification operation and spoofing determination operation performed by the information processing system S2 in the second embodiment will be described with reference to Fig. 5, Fig. 6, and Table 2. Fig. 5 is a conceptual diagram of the information processing system S2 during the verification operation and spoofing determination operation in the second embodiment. Fig. 6 is a flowchart showing the flow of the verification operation and spoofing determination operation performed by the information processing system S2 in the second embodiment.

[0033] FIG. 5 illustrates a case where imaging devices C1, C2, C3, and C4 (referred to as imaging device C when there is no need to distinguish between them) are provided in a specific area SA. Each imaging device C may be installed so as to image a predetermined space in order to image the subject P. Each imaging device C may be installed at a height of, for example, 2.5 meters. If the imaging device C is installed at a height higher than the average person's height, the image captured by the imaging device C is likely to include not only the subject P's facial area and upper body area, but also the subject P's entire body area. Therefore, the imaging device C is likely to capture an image including the subject P's facial area F and code image W. The imaging ranges of each imaging device C may be different. Each imaging device C may be installed so as to have a different imaging range. 5, the imaging range of the imaging device C1 includes the subject PA, and the image captured by the imaging device C1 may include a facial area FA of the subject PA and a code image WA. The imaging range of the imaging device C3 includes the subject PB, and the image captured by the imaging device C3 may include a facial area FB of the subject PB and a code image WB.

[0034] The determination unit 215 determines, for each of the multiple imaging devices C, whether or not the spoofing determination unit 214 will perform spoofing determination using the captured image captured by the corresponding imaging device C. The determination unit 215 may store the determination result for each imaging device C in the imaging device information database 222, an example of which is shown in Table 2 below. [Table 2: Imaging device information database 222]

[0035]

[0036] As exemplified in Table 2 above, when the determining unit 215 determines that the imaging device C is the imaging device for which spoofing determination is to be performed using captured images, a check mark may be placed in the column for the imaging device C. That is, in the case exemplified in Table 2, the determining unit 215 determines that spoofing determination is to be performed using captured images captured by imaging devices C3 and C4. Information indicating that a check mark has been placed on the captured images captured by the imaging device C may be added.

[0037] The acquisition unit 212 acquires captured images from each of the multiple imaging devices C, but the operations from "start" to "end" shown in Fig. 6 may be operations related to captured images acquired from any of the imaging devices C. Furthermore, the operations from "start" to "end" shown in Fig. 6 may be operations related to a single image. For example, if the imaging device C is capturing a video, the operations from "start" to "end" shown in Fig. 6 may be operations for each frame.

[0038] 6, the acquisition unit 212 acquires a captured image of a subject P wearing a jacket with a code image W attached thereto (step S26). The association unit 216 detects the code image W included in the captured image (step S27). The matching unit 213 matches the subject P using the code image W and identifies the ID of the subject P (step S28).

[0039] The spoofing determination unit 214 determines whether the captured image has a check mark (step S29). A checked captured image may be an image captured by a checked imaging device C. In other words, the spoofing determination unit 214 determines whether the captured image is an image to be used for spoofing determination. If the captured image is an image to be used for spoofing determination, this may be one example of a case where a predetermined condition is satisfied. Alternatively, a case where a predetermined condition is satisfied may be a case where the subject P appears in an image captured by a predetermined imaging device C.

[0040] The associating unit 216 detects a face area F included in the captured image (step S30). The authenticating unit 217 performs face authentication to identify the ID of the subject P as biometric identification information (step S31).

[0041] The spoofing determination unit 214 determines whether the ID identified in step S28 matches the ID identified in step S31 (step S32). When the captured image includes multiple subjects P, the spoofing determination unit 214 may determine whether the ID identified from the code image W and the face area F, which the association unit 216 has determined to be the same person based on the positions of the code image W and the face area F, matches the ID identified from the face area F.

[0042] If the IDs match (step S32: Yes), the spoofing determination unit 214 determines that someone other than the subject P is not impersonating the subject P (step S33). If the IDs do not match (step S32: No), the spoofing determination unit 214 determines that someone other than the subject P is impersonating the subject P (step S34). That is, the spoofing determination unit 214 determines whether someone other than the subject P is impersonating the subject P by using the face area F of the subject P identified by matching the code image W.

[0043] Alternatively, when the IDs do not match (step S32: No), this may mean, for example, that the ID can be identified from the face area F, but not that the ID cannot be identified from the code image W. When the ID cannot be identified from the code image W, this may be, for example, when the code image W is upside down and cannot appear in the image. In this case, the information processing device 2 may notify the relevant subject P of information that the code image W will be made visible. For example, the information processing device 2 may transmit information to the mobile information terminal carried by the subject P that the code image W cannot appear in the image.

[0044] In the above embodiment, characteristic facial information was used as an example of biometric information, but the biometric information may also be characteristic iris information, and iris authentication may be used to identify individuals and determine whether they are impersonating someone.

[0045] The information processing system S2 may also be applied to sites where biometric authentication such as facial recognition cannot be applied, such as when protective clothing is required. [2-4: Technical Effects of the Information Processing System S2]

[0046] Even if the image used for code matching has lower image quality than the image used for biometric authentication such as face recognition, it can still be used for the intended purpose. Therefore, even an overhead image captured from a distance, rear, or side by a surveillance camera or the like can be used for code matching using a code such as the above-mentioned code image W. In other words, even an image in which the facial area F is not clear can be used for code matching. Furthermore, code matching requires a relatively small amount of computer processing compared to biometric authentication using biometric information, and the processing load is small, so an individual can be identified in a short time. Because code matching allows for high-speed individual identification, a person can also be tracked at a high frame rate, improving the accuracy of tracking the person's behavior.

[0047] On the other hand, in the code matching in the second embodiment, if the clothing is lent, lost, stolen, duplicated, etc., another person can impersonate the person. In contrast, the information processing system S2 in the second embodiment uses biometric information to determine whether someone other than the person is impersonating the person when predetermined conditions are met, and therefore can perform authentication that is as reliable as biometric authentication.

[0048] The computer processing load for biometric information matching is greater than the computer processing load for code matching. Therefore, constantly performing spoofing determinations would result in a very large computer processing load. In contrast, the information processing system S2 performs spoofing determinations only when predetermined conditions are met, thereby reducing the processing load. Particularly in the second embodiment, the information processing system S2 determines, for each imaging device C, whether or not to perform spoofing determinations using images captured by the imaging device C, thereby reducing the computer processing load, the equipment costs associated with computer equipment, the power consumed by computers, and carbon dioxide emissions associated with global warming.

[0049] Furthermore, for example, if the information processing system S2 is applied within a company, it is expected that it is unlikely that an employee, who is the subject P, will impersonate another person by wearing someone else's clothing. Therefore, even if impersonation determination is performed only when necessary, there is little impact on the reliability of identifying an individual through code matching. [3: Third Embodiment]

[0050] Next, a third embodiment of an information processing system, an information processing method, and a recording medium will be described. Hereinafter, the third embodiment of the information processing system, the information processing method, and the recording medium will be described using an information processing system S3 to which the third embodiment of the information processing system, the information processing method, and the recording medium is applied. [3-1: Configuration of Information Processing System S3]

[0051] The configuration of an information processing system S3 in the third embodiment will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the configuration of the information processing system S3 in the third embodiment. As shown in Fig. 7, the information processing system S3 in the third embodiment may include a plurality of imaging devices C and an information processing device 3, similar to the information processing system S2 in the second embodiment. The information processing device in the third embodiment may have the same configuration as the information processing device 2 in the second embodiment described above.

[0052] In the third embodiment, each of the multiple imaging devices C includes a notification unit CA that notifies the subject P whether or not the image captured by the imaging device C itself will be used for spoofing determination, regardless of the determination by the determination unit 215. The notification unit CA may notify the person to be authenticated as if spoofing determination is being performed. In other words, the notification unit CA may notify even when spoofing determination is not being performed.

[0053] In the example shown in Fig. 7, the captured image of the imaging device C1 is not used for spoofing determination, and the notification unit CA1 does not notify that the captured image will be used for spoofing determination. The captured image of the imaging device C2 is not used for spoofing determination, but the notification unit CA2 notifies that the captured image will be used for spoofing determination. The captured image of the imaging device C3 is used for spoofing determination, and the notification unit CA3 notifies that the captured image will be used for spoofing determination. The captured image of the imaging device C4 is used for spoofing determination, but the notification unit CA4 does not notify that the captured image will be used for spoofing determination. [3-2: Technical Effects of Information Processing System S3]

[0054] If it becomes known that there are cases where spoofing determination is not performed, a malicious person may target a place or time when spoofing determination is not performed and commit fraud by spoofing. In contrast, the imaging device C in the third embodiment is equipped with a notification unit CA that issues a notification as if spoofing determination is being performed, so the subject P cannot accurately recognize which imaging device C is using the captured image to perform the spoofing determination. The information processing system S3 can prevent a malicious person from committing fraud by spoofing. [4: Fourth Embodiment]

[0055] Next, a fourth embodiment of an information processing system, an information processing method, and a recording medium will be described. Hereinafter, the fourth embodiment of the information processing system, the information processing method, and the recording medium will be described using an information processing system to which the fourth embodiment of the information processing system, the information processing method, and the recording medium is applied.

[0056] The information processing system in the fourth embodiment may have the same configuration as the information processing system S2 in the second embodiment or the information processing system S3 in the third embodiment. The information processing system in the fourth embodiment differs from the information processing system S2 in the second embodiment and the information processing system S3 in the third embodiment in the determination operation by the determination unit 215. [4-1: Information Processing Operation Performed by the Information Processing System]

[0057] The determination unit 215 dynamically changes the determination of whether or not to perform spoofing determination using the captured image captured by the imaging device C. The determination unit 215 changes the setting of whether or not to perform spoofing determination while the matching operation and the spoofing determination operation are being performed. For example, the determination unit 215 may change the setting from the setting contents exemplified in Table 3 below to the setting contents exemplified in Table 4 below. [Table 3: Imaging device information database 222 before change]

[0058] Table 4: Image capture device information database 222 after change

[0059]

[0060] That is, in the cases exemplified in Tables 3 and 4 above, the determination unit 215 removed the check mark from the image capture device C3 and added a check mark to the image capture device C2 at 10:15. That is, the determination unit 215 changed the image capture device so that the image capture device C3 was not used for the spoofing determination, but the image capture device C2 was used for the spoofing determination. [4-2: Technical Effects of the Information Processing System]

[0061] If the image captured by which imaging device C is used to determine spoofing is fixed, there is a possibility that a malicious person's attempt at spoofing may be overlooked. In contrast, the information system of the fourth embodiment dynamically changes the decision as to whether or not to use the image captured by which imaging device C to determine spoofing, thereby reducing the likelihood of spoofing being overlooked. [5: Fifth Embodiment]

[0062] A fifth embodiment of an information processing system, an information processing method, and a recording medium will be described below. Hereinafter, the fifth embodiment of the information processing system, the information processing method, and the recording medium will be described using an information processing system S5 to which the fifth embodiment of the information processing system, the information processing method, and the recording medium is applied. [5-1: Configuration of Information Processing System S5]

[0063] The configuration of an information processing system S5 in the fifth embodiment will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the configuration of the information processing system S5 in the fifth embodiment. As shown in Fig. 8, the information processing system S5 in the fifth embodiment may include a plurality of imaging devices C and an information processing device 5, similar to at least one of the information processing systems S2 in the second embodiment to the information processing system S2 in the fourth embodiment.

[0064] As shown in FIG. 8 , the information processing device 5 of the fifth embodiment includes a calculation device 21 and a storage device 22, similar to the information processing devices 2 of the second embodiment to the fourth embodiment. Furthermore, the information processing device 5 of the fifth embodiment may include a communication device 23, an input device 24, and an output device 25, similar to the information processing devices 2 of the second embodiment to the fourth embodiment. However, the information processing device 5 does not necessarily include at least one of the communication device 23, the input device 24, and the output device 25. The information processing device 5 of the fifth embodiment differs from the information processing devices 2 of the second embodiment to the information processing device of the fourth embodiment in that the calculation device 21 further includes an impersonation target determination unit 518. Other features of the information processing device 5 may be the same as at least one other feature of the information processing device 2 of the second embodiment to the information processing device of the fourth embodiment. Therefore, the following will describe in detail the differences from the previously described embodiments, and will omit appropriate descriptions of other overlapping features. [5-2: Information Processing Operation Performed by Information Processing System S5]

[0065] An information processing operation performed by the information processing system S5 in the fifth embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of the information processing operation performed by the information processing system S5 in the fifth embodiment. As shown in Fig. 9, in step S28, the operation after the ID as biometric identification information of the subject P is identified by code matching using the code image W differs from the information processing operation in the second embodiment shown in the flowchart of Fig. 6.

[0066] The spoofing target determination unit 518 determines whether or not the subject P as a living body is a target of spoofing determination by the spoofing determination unit 214 (step S50). The spoofing target determination unit 518 may refer to the person information database 221 shown in Table 5 below to determine whether or not the subject P is a target of spoofing determination by the spoofing determination unit 214. [Table 5: Person information database 221 before update]

[0067]

[0068] In the fifth embodiment, the spoofing target determination unit 518 determines that the target P is a target for spoofing determination by the spoofing determination unit 214 when a predetermined time has passed since the target P was determined by the spoofing determination unit 214. For example, as illustrated in Table 5, when the information indicated by the code image WA is "aa," the ID of the target PA is identified as "A." As illustrated in Table 5, the target PA with ID "A" was last determined to be spoofed at 10:10. For example, when the predetermined time is 5 minutes and the current time is 10:16, this corresponds to a case where a predetermined time has passed since the target PA was determined to be spoofed by the spoofing determination unit 214.

[0069] If a predetermined time has passed since the subject P was identified by the impersonation identification unit 214 (step S50: Yes), the association unit 216 detects a face area F according to the position of the code image W (step S51).

[0070] The authentication unit 217 performs facial authentication to identify the ID of the subject P as biometric identification information (step S52). The impersonation determination unit 214 determines whether the ID identified in step S28 matches the ID identified in step S62 (step S53). If the IDs match (step S53: Yes), the impersonation determination unit 214 determines that someone other than the subject P is impersonating the subject P (step S54). If the IDs do not match (step S53: No), the impersonation determination unit 214 determines that someone other than the subject P is impersonating the subject P (step S55).

[0071] The spoofing target determination unit 518 updates the time in the person information database 221 (step S56). For example, if it determines that the target person PA with ID "A" has spoofed, the spoofing determination time in the registration unit 211 may be updated from "10:10" as shown in Table 5 above to "10:16" as shown in Table 6 below. [Table 6: Updated person information database 221]

[0072]

[0073] In the fifth embodiment, the case where a predetermined condition is satisfied may be the case where a predetermined subject P appears in the captured image.

[0074] In addition, in the fifth embodiment, a case in which a specified condition is met may be a case in which the subject P appears in an image captured by a specified imaging device C, as in the second embodiment, and the subject P is subject to impersonation judgment.

[0075] Furthermore, in addition to the fact that no impersonation judgment has been performed for a predetermined time or more, impersonation judgment may be performed on the subject P depending on conditions such as the number of subjects P appearing in the captured image. For example, if there are a large number of subjects P to be subjected to impersonation judgment, the number of imaging devices C that capture images used for the impersonation judgment may be reduced. Conversely, if there are a small number of subjects P to be subjected to impersonation judgment, the number of imaging devices C that capture images used for the impersonation judgment may be increased. The number of subjects P to be subjected to impersonation judgment and the number of imaging devices C that capture images used for the impersonation judgment may be adjusted depending on the processing volume of the impersonation judgment. [5-3: Technical Effects of Information Processing System S5]

[0076] The information processing system S5 in the fifth embodiment can determine whether or not each subject P is subject to spoofing judgment, thereby reducing unnecessary spoofing judgments and reducing the processing load. In particular, if a predetermined time has passed since the spoofing judgment, the subject P is determined to be subject to spoofing judgment, so necessary spoofing judgments can be made. If a predetermined time has passed since the spoofing judgment, the possibility of spoofing increases, so spoofing can be suitably prevented by targeting the subject P for whom a predetermined time has passed since the spoofing judgment. [6: Sixth Embodiment]

[0077] An information processing system, an information processing method, and a recording medium according to a sixth embodiment will be described below. Hereinafter, the sixth embodiment will be described using an information processing system S6 to which the sixth embodiment of the information processing system, the information processing method, and the recording medium is applied. [6-1: Configuration of Information Processing System S6]

[0078] The configuration of an information processing system S6 in the sixth embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the configuration of the information processing system S6 in the sixth embodiment. As shown in Fig. 10, the information processing system S6 in the sixth embodiment may include a plurality of imaging devices C and an information processing device 6, similar to at least one of the information processing systems S2 in the second embodiment to S5 in the fifth embodiment.

[0079] As shown in FIG. 11 , the information processing device 6 of the sixth embodiment includes a calculation device 21 and a storage device 22, similar to the information processing device 2 of the second embodiment to the information processing device 5 of the fifth embodiment. Furthermore, the information processing device 6 of the sixth embodiment may include a communication device 23, an input device 24, and an output device 25, similar to the information processing device 2 of the second embodiment to the information processing device 5 of the fifth embodiment. However, the information processing device 6 does not necessarily include at least one of the communication device 23, the input device 24, and the output device 25. The information processing device 6 of the sixth embodiment differs from the information processing device 2 of the second embodiment to the information processing device 5 of the fifth embodiment in that the calculation device 21 further includes a position detection unit 619 and an estimation unit 620. Other features of the information processing device 6 may be the same as at least one other feature of the information processing device 2 of the second embodiment to the information processing device 5 of the fifth embodiment. Therefore, the following will describe in detail the differences from the already-described embodiments, and will omit appropriate descriptions of other overlapping features. [6-2: Information Processing Operation Performed by Information Processing System S6]

[0080] The information processing operation performed by the information processing system S6 in the sixth embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the flow of the information processing operation performed by the information processing system S6 in the sixth embodiment. As shown in Fig. 12, the operation after the ID is identified in step S28 and the operation when the IDs do not match in step S32 differ from the information processing operation in the second embodiment shown in the flowchart in Fig. 6.

[0081] After the ID is identified in step S28, the position detection unit 619 detects the location of the subject P wearing the jacket to which the code image W is attached based on the matching result by the matching unit 213. The position detection unit 619 may also identify the location of the subject P using the code image W. Because the code image W has known dimensions and shape, the position detection unit 619 can measure the three-dimensional depth and orientation of the subject P using the code image W. Furthermore, the position detection unit 619 may analyze the behavior and posture of the subject P from the three-dimensional depth and angle. The position detection unit 619 registers information indicating the position of the subject P during code matching in the personal information database 221 (step S60). In the sixth embodiment, the personal information database 221 may include time and location information of the subject P, as exemplified in Table 7 below. The time and location information of the subject P registered in the personal information database 221 may be accumulated information. The time and location information of the subject P registered in the person information database 221 may be information that is not deleted until a specific condition, such as the passage of a predetermined period of time, is met. [Table 7: Person Information Database 221]

[0082]

[0083] In step S32, if the IDs do not match (step S32: No), the impersonation determination unit 214 determines that someone other than the person is impersonating the person. When the impersonation determination unit 214 determines that someone other than the person is impersonating the person, the estimation unit 620 refers to information registered in the person information database 221 to estimate the location at which the person other than the person is impersonating the person (step S61). According to the information exemplified in Table 7, the estimation unit 620 can refer to the fact that the target person PA was in front of the cabinet at least at 10:17 from 10:16, the time of the previous impersonation determination of the target person PA, to the present.

[0084] The estimation unit 620 can know the location of the subject P during the period from the previous impersonation determination to the current impersonation determination from the information stored in the person information database 221. In other words, if the impersonation determination result indicates that impersonation has occurred, the location where the impersonation occurred can be estimated by tracing the authentication history. [6-3: Technical Effects of the Information Processing System S6]

[0085] The information processing system S6 in the sixth embodiment can estimate where someone other than the person in question has impersonated the person in question, and can therefore recognize where impersonation judgment should be performed.

[0086] Furthermore, the information processing system S6 can track the subject P and understand his / her behavior using the information stored in the personal information database 221. The information processing system S6 can leave a legal trail of the behavior record with the reliability of biometric authentication by combining the matching of the code image W and face authentication. This can be applied to obtaining a legal audit trail, such as evidence that a qualified person performed the work. [7: Seventh Embodiment]

[0087] Next, a seventh embodiment of an information processing system, an information processing method, and a recording medium will be described. Hereinafter, the seventh embodiment of the information processing system, the information processing method, and the recording medium will be described using an information processing system to which the seventh embodiment of the information processing system, the information processing method, and the recording medium is applied.

[0088] The information processing system in the seventh embodiment may have the same configuration as at least one of the information processing system S2 in the second embodiment to the information processing system S6 in the sixth embodiment. The information processing system in the seventh embodiment differs from the information processing system S2 in the second embodiment to the information processing system S6 in the sixth embodiment in the determination operation by the determination unit 215. [7-1: Information Processing Operation by Information Processing System]

[0089] The determination unit 215 determines to perform spoofing determination using an image captured by an imaging device C that captures an image of a location where spoofing is highly likely to occur. When there are multiple imaging devices C, the determination unit 215 selects and sets the imaging device C that will perform spoofing determination depending on the importance of personal authentication and tracking. For example, in the sixth embodiment, the imaging device C that captures an image of a location estimated by the estimation unit 620 can be estimated to be the imaging device C that captures an image of a location where spoofing is highly likely to occur. The determination unit 215 may determine to perform spoofing determination using an image captured by the imaging device C. [7-2: Technical Effects of the Information Processing System]

[0090] The information processing system according to the seventh embodiment performs biometric authentication in key locations, such as changing rooms and restrooms, where there is a high possibility of personnel changes, thereby preventing impersonation, such as lending a code to another person. [8: Supplementary Note]

[0091] The following supplementary notes are further disclosed with respect to the above-described embodiments. [Supplementary Note 1] An information processing system comprising: a registration means for registering biometric identification information capable of identifying a living organism, biometric information of a predetermined part of the living organism, and clothing identification information presented by clothing worn by the living organism and capable of appearing in an image, in association with each other; an acquisition means for acquiring a captured image of the living organism wearing the clothing; a matching means for matching the living organism using the clothing identification information included in the captured image; and a spoofing determination means for determining, when a predetermined condition is satisfied, whether something other than the living organism is impersonating the living organism using an image of the predetermined part of the living organism included in the captured image. [Supplementary Note 2] The information processing system according to Supplementary Note 1, wherein the acquisition means acquires captured images from each of a plurality of imaging means, and the spoofing determination means further comprises, for each of the plurality of imaging means, a determination means for determining whether to make an spoofing determination using the captured image captured by the corresponding imaging means. [Supplementary Note 3] The information processing system according to Supplementary Note 2, wherein each of the plurality of imaging means comprises a notifying means for notifying the living body whether or not an image captured by the imaging means will be used for spoofing determination, regardless of the decision of the determining means. [Supplementary Note 4] The information processing system according to Supplementary Note 2 or 3, wherein the determining means dynamically changes the decision of whether or not to perform spoofing determination using an image captured by the imaging means. [Supplementary Note 5] The information processing system according to any one of Supplements 2 to 4, wherein the determining means determines to perform spoofing determination using an image captured by an imaging means that captures an image of a location where spoofing is likely to occur. [Supplementary Note 6] The information processing system according to any one of Supplementary Notes 1 to 5, further comprising an spoofing target determining means for determining whether or not the living body is a target of spoofing determination by the spoofing determination means. [Supplementary Note 7] The information processing system described in Supplementary Note 6, wherein the spoofing target determination means determines that the biometric entity is a target for spoofing determination by the spoofing determination means if a predetermined time has passed since the biometric entity was determined by the spoofing determination means.[Supplementary Note 8] The information processing system according to any one of Supplementary Notes 1 to 7, further comprising an estimation means for, when the spoofing determination means determines that a non-living being is spoofing the living being, detecting a location where the living being wearing the clothing was present based on the matching result by the matching means, and estimating at which location the non-living being spoofed the living being. [Supplementary Note 9] An information processing method comprising: registering, in association with each other, biometric identification information capable of identifying a living being, biometric information of a predetermined part of the living being, and clothing identification information presented by clothing worn by the living being and that can be captured in an image; obtaining a captured image of the living being wearing the clothing; matching the living being using the clothing identification information included in the captured image; and, if a predetermined condition is satisfied, determining whether or not the non-living being is spoofing the living being using an image of the predetermined part of the living being included in the captured image. [Supplementary Note 10] A recording medium having recorded thereon a computer program for causing a computer to execute an information processing method of: registering, in association with each other, biometric identification information capable of identifying a living organism, biometric information of a predetermined part of the living organism, and clothing identification information presented by clothing worn by the living organism and capable of appearing in an image; acquiring a captured image of the living organism wearing the clothing; verifying the living organism using the clothing identification information included in the captured image; and, if predetermined conditions are met, determining whether or not something other than the living organism is impersonating the living organism using an image of the predetermined part of the living organism included in the captured image.

[0092] At least some of the constituent elements of each of the above-described embodiments can be appropriately combined with at least some of the other constituent elements of each of the above-described embodiments. Some of the constituent elements of each of the above-described embodiments may not be used. Furthermore, to the extent permitted by law, the disclosures of all documents (e.g., published patent applications) cited in this disclosure are incorporated by reference as part of the description of this disclosure.

[0093] This disclosure may be modified as appropriate within the scope of the claims and the technical idea that can be read from the entire specification. Information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0094] S1, S2, S3, S5, S6 Information processing system 2, 3, 5, 6 Information processing device 11, 211 Registration unit 12, 212 Acquisition unit 13, 213 Matching unit 14, 214 Spoofing determination unit 215 Determination unit 216 Correspondence unit 217 Authentication unit 518 Spoofing target determination unit 619 Position detection unit 620 Estimation unit 221 Person information database 222 Imaging device information database C Imaging device CA Notification unit F Face area W Code image

Claims

1. a registration means for registering biometric identification information capable of identifying a living body, biometric information of a predetermined part of the living body, and clothing identification information presented by an article of clothing worn by the living body, the clothing identification information being capable of appearing in an image, in association with each other; An acquisition means for acquiring an image of the living body wearing the garment; A verification means for verifying the living body by using the clothing identification information included in the captured image; a spoofing determination means for determining whether or not a non-living body is masquerading as the living body by using an image of a predetermined part of the living body included in the captured image when a predetermined condition is satisfied; An information processing system comprising:

2. the acquiring means acquires captured images from each of a plurality of imaging means; The imaging device further includes a determination unit for determining whether or not the spoofing determination unit performs spoofing determination using an image captured by the imaging device. The information processing system according to claim 1 .

3. Each of the plurality of image capturing means includes a notification means for notifying the living body whether or not the captured image captured by the image capturing means is used for spoofing determination, regardless of the determination by the determination means. The information processing system according to claim 2 .

4. The determining means dynamically changes a decision as to whether or not to perform spoofing determination using an image captured by the imaging means. The information processing system according to claim 2 .

5. The determining means determines that the spoofing judgment is to be performed using an image captured by an imaging means for capturing an image of a place where the spoofing is highly likely to occur. The information processing system according to claim 2 .

6. The biometric information processing device further includes a spoofing target determination unit that determines whether the biometric information is a target for spoofing determination by the spoofing determination unit.

3. The information processing system according to claim 1 or 2.

7. The spoofing target determination means determines that the biometric information is a target of spoofing determination by the spoofing determination means when a predetermined time has elapsed since the biometric information was determined by the spoofing determination means.

7. The information processing system according to claim 6.

8. The apparatus further includes an estimation means for detecting a location where a living body wearing the article was present based on a result of the matching by the matching means when the spoofing determination means determines that a non-living body is spoofing the living body, and estimating at which location the non-living body is spoofing the living body. The information processing system according to claim 1 .

9. registering biometric identification information capable of identifying a living body, biometric information of a predetermined part of the living body, and clothing identification information presented by an article of clothing worn by the living body, the clothing identification information being capable of appearing in an image, in association with each other; Acquire a captured image of the living body wearing the garment; Matching the living body using the clothing identification information included in the captured image; If a predetermined condition is satisfied, an image of a predetermined part of the living body included in the captured image is used to determine whether or not a non-living body is masquerading as the living body. A computer implemented information processing method.

10. On the computer, registering biometric identification information capable of identifying a living body, biometric information of a predetermined part of the living body, and clothing identification information presented by an article of clothing worn by the living body, the clothing identification information being capable of appearing in an image, in association with each other; Acquire a captured image of the living body wearing the garment; Matching the living body using the clothing identification information included in the captured image; If a predetermined condition is satisfied, an image of a predetermined part of the living body included in the captured image is used to determine whether or not a non-living body is masquerading as the living body. A computer program for executing an information processing method.