Information processing system, information processing method, and information processing program

Abstract

To authenticate a user and execute predetermined processing with a non-contact and simple constitution.SOLUTION: An information processing system 10 comprises: a storage unit for storing face image data of a person using a predetermined facility; a first imaging unit for imaging the face of a person having visited the predetermined facility; a second imaging unit for imaging a presented two-dimensional code; a face determination unit for, on the basis of feature points of the face image data imaged by the first imaging unit and feature points of the face image data of the person stored in the storage unit, determining whether both of the persons are identical; a processing execution unit for, if the face determination unit determines that they are identical, executing predetermined processing on the basis of information included in the two-dimensional code imaged by the second imaging unit; and an output unit for outputting a result of the predetermined processing.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a technique for identifying a person using facial image data of a captured image of the person and executing a predetermined process on the identified person. [Background technology]

[0002] Conventionally, there is known an information processing system that executes a predetermined process after authenticating a user using biometric information such as a fingerprint. For example, Patent Document 1 discloses a terminal device of a time attendance management system for managing time attendance using fingerprint information.

[0003] Conventionally, there is known a method for identifying a person based on face image data captured by an imaging device such as a camera. For example, Patent Document 2 discloses a system for identifying the same person by collating face images of the person captured by multiple imaging devices in a facility. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-251013 A [Patent Document 2] International Publication No. 2018 / 180588 Brochure Summary of the Invention [Problem to be solved by the invention]

[0005] Meanwhile, in these times of the COVID-19 pandemic, there is a demand for contactless authentication of users and execution of specified processing.

[0006] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide an information processing system, an information processing method, and an information processing program that can authenticate a user and execute specified processing in a non-contact and simple configuration. [Means for solving the problem]

[0007] In order to achieve the above object, an information processing system according to the present invention comprises, as one aspect thereof: A storage unit that stores face image data of a person using a predetermined facility; A first imaging unit that captures an image of a face of a person who visits the predetermined facility; a second imaging unit that captures an image of the presented two-dimensional code; a face determination unit that determines whether or not the face image data captured by the first imaging unit and the face image data of the person stored in the storage unit are the same person based on feature points of the face image data; a processing execution unit that executes a predetermined process based on information included in the two-dimensional code captured by the second imaging unit when the face determination unit determines that the two-dimensional code is the same person; an output unit that outputs a result of the predetermined processing; The gist of the invention is to provide the following:

[0008] In addition, one aspect of the information processing method according to the present invention is An information processing method using an AI camera system that stores facial image data of a person using a specified facility, A first imaging step of imaging a face of a person who visits the predetermined facility; a face determination step of determining whether or not the face image data captured in the first imaging step and the face image data of the person stored in the AI ​​camera system are the same person based on feature points of the face image data; a message output step of outputting a message prompting the user to present a two-dimensional code to a display unit of the AI ​​camera system when the user is determined to be the same person in the face determination step; a second imaging step of imaging the presented two-dimensional code; a processing execution step of executing a predetermined process based on information included in the two-dimensional code captured in the second imaging step; a result output step of outputting a result of the predetermined processing to the display unit; The gist of the invention is to provide the following:

[0009] In addition, an information processing program according to the present invention includes, as one aspect thereof, An information processing program using a computer that stores face image data of a person using a specified facility, A first imaging step of imaging a face of a person who visits the predetermined facility; a face determination step of determining whether or not the face image data captured in the first imaging step and the face image data of the person stored in the computer are the same person based on feature points of the face image data; a message output step of outputting a message prompting the user to present a two-dimensional code when the face determination step determines that the two-dimensional code is the same person; a second imaging step of imaging the presented two-dimensional code; a processing execution step of executing a predetermined process based on information included in the two-dimensional code captured in the second imaging step; a result output step of outputting a result of the predetermined processing; The gist of the present invention is to have the computer execute the above steps. Effect of the Invention

[0010] According to the present invention, it is possible to authenticate a user and execute a predetermined process in a contactless manner with a simple configuration. [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic configuration diagram of an information processing system according to an embodiment of the present invention. [Diagram 2] 1 is a diagram showing an example of a physical configuration of an AI camera system according to an embodiment of the present invention. [Diagram 3] 1 is a diagram showing an example of the appearance of an AI camera system according to an embodiment of the present invention. FIG. [Figure 4]FIG. 11 is a sequence diagram showing the operation of an information processing system according to an embodiment of the present invention, in which an AI camera system authenticates a user using a facility based on facial information, and a server executes a predetermined process based on a two-dimensional code presented by the user. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] <Configuration of information processing system> 1 is a schematic diagram of an information processing system 10 according to an embodiment of the present invention. The information processing system 10 is an information processing system in which an AI camera system 1 authenticates a person (hereinafter also referred to as a user) who visits a facility based on face information, and if the authentication is successful, a server 2 executes a predetermined process based on a two-dimensional code presented by the user. The information processing system 10 can execute a predetermined process based only on the face information of the user and the two-dimensional code presented by the user without touching any of the devices that constitute the information processing system 10, so that the information processing system can execute a desired process while preventing infection during the COVID-19 pandemic.

[0014] Here, the facility may be either a public facility or a private facility, and one or more AI camera systems 1 are installed in one facility. The two-dimensional code presented by the user is, for example, a two-dimensional code stored in advance in an information processing terminal such as a smartphone carried by the user. The two-dimensional code is preferably, for example, a matrix-type QR code (registered trademark), but is not limited to a QR code (registered trademark). Other two-dimensional codes may also be applied, and other types of matrix-type two-dimensional codes or stack-type two-dimensional codes may also be used. The predetermined process includes, for example, a ticket confirmation process at a concert venue (hereinafter abbreviated as "ticket confirmation process"), attendance management process in an attendance system (hereinafter abbreviated as "attendance management process"), and other processes.

[0015] As shown in Fig. 1, the information processing system 10 includes an AI camera system 1 installed in a facility, and a server 2 installed in the facility. The AI ​​camera system 1 and the server 2 are configured to be able to communicate with each other via a communication network 3, which is a wired or wireless in-house communication network. Note that, in the present embodiment, the AI ​​camera system 1 and the server 2 are described as being installed in the same facility, but are not limited to this. When the AI ​​camera system 1 and the server 2 are installed in different facilities, they may be configured to be able to communicate with each other via a communication network 3 consisting of, for example, the Internet network, a public network, or the like.

[0016] The AI ​​camera system 1 is a computer system that uses facial image data to manage entry into a facility, and is equipped with a camera equipped with AI (artificial intelligence) functions. The AI ​​camera system 1 is composed of an electronic device having a central processing unit (CPU) with at least calculation and control functions, and a main storage device (memory) consisting of a RAM or the like with a function for storing programs and data. In addition to the main storage device, the AI ​​camera system 1 may also be equipped with an auxiliary storage device such as a hard disk.

[0017] 2 is a diagram showing an example of a specific physical configuration of the AI ​​camera system 1. The AI ​​camera system 1 may include, for example, a camera 101 having an AI function and capable of extracting in real time feature points of captured facial image data of a person and facial image data stored in the AI ​​camera system 1, an edge computer 102 that analyzes the feature points of the facial image data output from the camera 101 and identifies the person based on the analyzed feature points, and an external storage device 103 that stores facial image data of users who can use the facility and image data captured by the camera 101.

[0018] Here, the feature points of the facial image data refer to facial feature points such as pupils, nostrils, corners of the mouth, eyebrows, eyes, etc., and a method for extracting the feature points of the facial image data is based on a known technique. Also, a method for identifying a person based on the extracted feature points of the facial image data is based on a known technique.

[0019] The AI ​​camera system 1 may be configured as a system in which multiple devices as described above are network-connected, or may be configured as a single physical device, all installed in the same location within the facility, or may be installed in multiple locations within the facility.

[0020] Figure 3 is an external view of an example of the AI ​​camera system 1. The AI ​​camera system 1 shown in Figure 3 is installed on a stand to capture a facial image of a person in a standing position, and shows how the facial image captured by the camera 101 is displayed on the display of the AI ​​camera system 1.

[0021] The AI ​​camera system 1 has a function of comparing the captured facial image data of a person with facial image data of a person stored in the AI ​​camera system 1, determining whether or not they are the same person, and outputting a message prompting the user to present a two-dimensional code if it is determined that they are the same person (hereinafter also referred to as a case where user authentication is successful). Furthermore, when a two-dimensional code is presented, the AI ​​camera system 1 has a function of transmitting a request for a predetermined process to the server 2 together with information contained in the captured two-dimensional code, and a function of outputting the result of the predetermined process received from the server 2.

[0022] Functionally, the AI ​​camera system 1 includes an imaging unit 11, a storage unit 12, a communication unit 13, an output unit 14, and a control unit 15, as shown in FIG.

[0023] The imaging unit 11 has a function of capturing facial image data of a user and a function of capturing an image of a two-dimensional code presented by the user. Here, the two-dimensional code is assumed to be stored in advance in an information processing terminal carried by the user. For example, when the predetermined process is a ticket confirmation process, the two-dimensional code contains ticket information (including a ticket ID that can uniquely identify the ticket) issued to the user for a concert held at the facility. Also, for example, when the predetermined process is a attendance management process, the two-dimensional code contains information indicating the type of attendance (for example, normal attendance, mid-day attendance, normal leaving work, mid-day leaving work, etc.) (hereinafter referred to as attendance type information) together with an ID that can uniquely identify a user who can use the facility. The type of attendance can be selected by the user himself based on the variety of working patterns, and the user selects and presents a two-dimensional code corresponding to the type of attendance that applies on the day.

[0024] The storage unit 12 stores an application program 16 to be executed by the AI ​​camera system 1 in the information processing system 10. The storage unit 12 also stores in advance face image data 17 of people who are allowed to use the facility. For example, if the predetermined process is a ticket confirmation process, face image data of people who have purchased tickets for a concert to be held at the facility is stored in advance in the storage unit 12. Also, for example, if the predetermined process is an attendance management process, face image data of people who are allowed to enter the facility is stored in advance in the storage unit 12.

[0025] The communication unit 13 exchanges various information with the server 2 via the communication network 3. In detail, the communication unit 13 transmits a request for a predetermined process to the server 2 together with information contained in the two-dimensional code presented to the user, and receives a result of the predetermined process from the server 2.

[0026] The output unit 14 is composed of, for example, an image display unit such as a display, an audio output unit such as a speaker, etc. For example, if the user authentication based on the face image fails, a message to that effect is output, and if the user authentication based on the face image is successful, a message prompting the user to present a two-dimensional code is output, and the result of a predetermined process is output.

[0027] The control unit 15 performs user authentication using face image data by executing the application program 16. Specifically, if the feature points of the face image data of a person captured by the imaging unit 11 match the feature points of the face image data stored in the storage unit 12, it is determined that they are the same person. If the user authentication using the face image data is successful, it obtains information included in the two-dimensional code captured by the imaging unit 11 (hereinafter referred to as two-dimensional code information) and requests the server 2 to execute a predetermined process together with the obtained two-dimensional code information.

[0028] The server 2 is a computer that executes a predetermined process. The server 2 is composed of an electronic device having at least a central processing unit (CPU) with a calculation function and a control function, and a main storage device (memory) such as a RAM having a function of storing programs and data. In addition to the main storage device, the server 2 may also have an auxiliary storage device such as a hard disk.

[0029] Functionally, the server 2 includes a storage unit 21, a communication unit 22, and a control unit 23, as shown in FIG.

[0030] The storage unit 21 stores an application program 24 to be executed by the server 2 in the information processing system 10. The storage unit 21 also stores data 25 related to a predetermined process. The data 25 is data that is registered in the storage unit 21 before the predetermined process is executed, or data that is registered after the predetermined process is executed. For example, in the case where the predetermined process is a ticket confirmation process, after a user purchases a ticket, the data 25 includes face image data of the user and ticket information of the issued ticket. The face image data of the user is, for example, face image data of the user taken by a smartphone or the like at the time of purchase. Also, in the case where the predetermined process is a time attendance management process, the data 25 is attendance data of all users who use the facility, and by executing the predetermined process, attendance data including information on the time of arrival and departure and the type of arrival and departure for each user is registered in the storage unit 21.

[0031] The communication unit 22 exchanges various information with the AI ​​camera system 1 via the communication network 3. For example, it receives a request to execute a predetermined process including two-dimensional code information from the AI ​​camera system 1, and transmits information on the result of the predetermined process to the AI ​​camera system 1.

[0032] The control unit 23 has a function of executing a predetermined process by executing the application program 24. For example, when the predetermined process is a ticket confirmation process, it determines whether or not the ticket ID included in the received two-dimensional code information matches the ticket ID of the ticket information stored in the storage unit 21, and if they match, the authenticity of the ticket purchased by the purchaser is confirmed, so the control unit 23 controls to transmit a message to the AI ​​camera system 1 indicating that the ticket has been confirmed as a processing result. Also, for example, when the predetermined process is a work attendance management process, the control unit 23 controls to register attendance data including information on the type of attendance along with the time of attendance of the corresponding user based on the received two-dimensional code information, and transmit a message to the AI ​​camera system 1 indicating that the attendance data based on the type of attendance has been registered as a processing result. Note that the time of attendance is set to the date and time when the server 2 received the request for the predetermined process.

[0033] The application program 16 is stored in a main storage device such as the above-mentioned memory or an auxiliary storage device such as a hard disk of the AI ​​camera system 1, and the application program 24 is stored in a main storage device such as the above-mentioned memory or an auxiliary storage device such as a hard disk of the server 2. The application programs 16 and 24 can be recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disc), or a BD (Blu-ray (registered trademark) Disc), or can be distributed via a communication network.

[0034] <Operation of information processing system> Next, the operation of the information processing system 10 will be described with reference to Fig. 4. Fig. 4 is a sequence diagram showing the operation of the information processing system 10 when the AI ​​camera system 1 authenticates a user using a facility based on face information, and then the server 2 executes a predetermined process based on a two-dimensional code presented by the user.

[0035] The AI ​​camera system 1 captures a facial image of a person attempting to enter a facility (step S10), and extracts feature points from the captured facial image data (step S20).

[0036] Next, the AI ​​camera system 1 acquires the feature points of the face image data 17 registered in advance in the storage unit 12 (step S30), compares the feature points of the captured face image data with the feature points of the face image data 17 registered in advance in the storage unit 12, and determines whether they match (step S40). If the feature points of the captured face image data match the feature points of the face image data 17 registered in advance in the storage unit 12 (step S40: YES), the user authentication by the face image is successful, and outputs a message prompting the user to present a two-dimensional code. For example, a message such as "Please present the QR code (registered trademark) to the camera" may be displayed on the display unit of the AI ​​camera system 1, or a voice such as "Please present the QR code (registered trademark) to the camera" may be output via a speaker, or both the message display and voice output described above may be executed.

[0037] Next, the AI ​​camera system 1 captures an image of the two-dimensional code presented by the user (step S60), acquires the information contained in the captured two-dimensional code, i.e., the two-dimensional code information (step S70), and sends a request to the server 2 to execute a specified process including the acquired two-dimensional code information (step S80).

[0038] When the server 2 receives a request to execute a predetermined process (step S90), it executes the predetermined process based on the two-dimensional code information (step S100) and transmits the result of the predetermined process to the AI ​​camera system 1 (step S110). For example, if the predetermined process is a ticket confirmation process, it confirms the ticket based on the received two-dimensional code information. Also, for example, if the predetermined process is a time attendance management process, it creates time attendance data based on the received two-dimensional code information and registers the created time attendance data in the storage unit 21.

[0039] When the AI ​​camera system 1 receives the result of the predetermined process (step S120), it outputs the result of the predetermined process to the output unit 14 (step S130). For example, if the predetermined process is attendance management process, a message such as "Ticket confirmed" is output to the display unit, and if the predetermined process is the first process, a message such as "Attendance data has been registered" is output to the display unit together with attendance type information selected by the user if the predetermined process is the second process.

[0040] As described above, according to the information processing system 10 of this embodiment, user authentication is performed and predetermined processing is executed based on the facial image data of a user visiting a facility and the two-dimensional code presented by the user, so that the user can be authenticated and the predetermined processing can be executed without touching the constituent devices of the information processing system 10. In other words, since the information processing system 10 can authenticate a user and execute predetermined processing in a contactless and simple configuration, according to this embodiment, it is possible to provide an information processing system that is very useful in preventing the spread of the new coronavirus during the recent COVID-19 pandemic.

[0041] Furthermore, when the predetermined process is a ticket verification process, the information processing system 10 can check the facial image of the ticket purchaser to verify the identity of the purchaser, thereby preventing the resale of the ticket. In other words, since the facial image of the purchaser photographed at the time of ticket purchase can be checked and the ticket obtained at the time of ticket purchase can be verified, it is possible to verify whether the user himself / herself is using the ticket, that is, whether the ticket has not been resold. Note that the facial image data is registered in advance in the storage unit 12 of the AI ​​camera system 1 after the ticket is purchased, and the ticket is issued as a two-dimensional code at the time of purchase and stored in the information processing terminal of the user.

[0042] In the information processing system 10 according to the above embodiment, the facial image data of the user is pre-stored in the storage unit 12 of the AI ​​camera system 1, but only the data of the feature points of the facial image data of the user may be pre-stored in the storage unit 12. In this case, the process of extracting the feature points of the facial image data pre-stored in the storage unit 12 can be omitted in the AI ​​camera system 1, so that the data capacity of the data stored in the AI ​​camera system 1 can be reduced, and the processing load on the AI ​​camera system 1 side can be reduced.

[0043] Furthermore, in the information processing system 10 of the above embodiment, the server 2 executes the predetermined process, but the predetermined process may be executed by the AI ​​camera system 1. In other words, the information processing system 10 may be configured only with the AI ​​camera system 1, without the server 2.

[0044] In addition, the two-dimensional code presented in the information processing system 10 of the above embodiment has been described as a two-dimensional code stored in an information processing terminal carried by a user, but is not limited thereto. For example, the two-dimensional code may be a two-dimensional code printed on a predetermined card or paper.

[0045] The AI ​​camera system 1 of the above embodiment may also be equipped with a thermal camera. That is, in addition to user authentication based on the user's face image, the thermal camera may also measure the user's body temperature. The body temperature measured by the thermal camera may be output to the display unit, and if the user's body temperature is equal to or lower than a predetermined threshold, the user may be allowed to enter the facility and a predetermined process may be executed. This contributes to preventing the spread of the new coronavirus within the facility.

[0046] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments, and various modifications and changes can be made to the embodiments of the present invention without departing from the gist of the present invention, and such modifications and changes are also included in the technical scope of the present invention. Furthermore, the actions and effects described in the embodiments of the invention are merely a list of the most preferable actions and effects resulting from the present invention, and the actions and effects of the present invention are not limited to those described in the embodiments of the present invention. [Explanation of symbols]

[0047] 1 AI camera system 2 Server 3. Communication Network 10. Information Processing Systems 11 Imaging unit 12,21 Storage section 13,22 Communications Department 14 Output section 15,23 Control section 16,24 Application Programs 17 Facial image data 25 Data

Claims

1. A storage unit that stores face image data of a person using a predetermined facility; A first imaging unit that captures an image of a face of a person who visits the predetermined facility; a second imaging unit that captures an image of the presented two-dimensional code; a face determination unit that determines whether or not the face image data captured by the first imaging unit and the face image data of the person stored in the storage unit are the same person based on feature points of the face image data; a message output unit that outputs a message to a display unit to prompt the user to present the two-dimensional code when the face determination unit determines that the users are the same person; a processing execution unit that executes a predetermined process based on information included in the two-dimensional code captured by the second imaging unit after the message output unit outputs the message to the display unit; an output unit that outputs a result of the predetermined processing to the display unit; Equipped with The storage unit stores an ID for use in the specified facility, The two-dimensional code includes an ID for using the specified facility, the predetermined process is a process of determining whether or not to permit use of the predetermined facility, The information processing system is characterized in that the processing execution unit allows use of the specified facility when the ID stored in the memory unit matches the ID contained in the two-dimensional code captured by the second imaging unit.

2. Further comprising a thermal camera for measuring the body temperature of a person using the specified facility, The information processing system according to claim 1, characterized in that the processing execution unit executes the specified processing when the face determination unit determines that the faces are the same person and when the body temperature of the person measured by the thermal camera is below a predetermined threshold value.

3. An information processing method by an information processing system having a storage unit that stores face image data of a person who uses a predetermined facility, a first imaging step of imaging a face of a person who visits the predetermined facility; a face determination step of determining whether or not the face image data captured in the first imaging step and the face image data of the person stored in the storage unit are the same person based on feature points of the face image data; a message output step of outputting, on a display unit, a message prompting the user to present a two-dimensional code when the face determination step determines that the user is the same person; a second imaging step of imaging the presented two-dimensional code; a processing execution step of executing a predetermined process based on information included in the two-dimensional code captured in the second imaging step; a result output step of outputting a result of the predetermined processing to the display unit; Equipped with The storage unit stores an ID for use in the specified facility, The two-dimensional code includes an ID for using the specified facility, the predetermined process is a process of determining whether or not to permit use of the predetermined facility, The information processing method is characterized in that the processing execution step permits use of the specified facility if the ID stored in the memory unit matches the ID contained in the two-dimensional code captured in the second imaging step.

4. An information processing program executed by an information processing system having a storage unit that stores face image data of a person using a specified facility, a first imaging step of imaging a face of a person who visits the predetermined facility; a face determination step of determining whether or not the face image data captured in the first imaging step and the face image data of the person stored in the storage unit are the same person based on feature points of the face image data; a message output step of outputting a message prompting the user to present a two-dimensional code when the face determination step determines that the two-dimensional code is the same person; a second imaging step of imaging the presented two-dimensional code; a processing execution step of executing a predetermined process based on information included in the two-dimensional code captured in the second imaging step; a result output step of outputting a result of the predetermined processing; execute the above-mentioned information processing system; The storage unit stores an ID for use in the specified facility, The two-dimensional code includes an ID for using the specified facility, the predetermined process is a process of determining whether or not to permit use of the predetermined facility, The information processing program is characterized in that the processing execution step permits use of the specified facility when the ID stored in the memory unit matches the ID contained in the two-dimensional code captured in the second imaging step.

Images