Terminal device, authentication system, authentication management device, authentication method, and authentication management method

Abstract

This terminal device for authenticating a person comprises a processor and a sensor. The processor derives, on the basis of first detection data detected by the sensor, biological feature data related to a biological feature of a first person holding the terminal device and performs first authentication of the person on the basis of the biological feature data, and if the first authentication is successful, the processor sequentially derives habit feature data related to a habit feature of the first person on the basis of second detection data detected by the sensor and sequentially performs second authentication of the first person on the basis of the sequentially derived habit feature data.

Description

Terminal device, authentication system, authentication management device, authentication method, and authentication management method

[0001] The present disclosure relates to a terminal device, an authentication system, an authentication management device, an authentication method, and an authentication management method.

[0002] Conventionally, the movement or state of a mobile terminal has been used to determine whether the holder of the mobile terminal is a legitimate user. For example, a mobile terminal is known that includes a sensor that detects the movement or state of the mobile terminal, an accumulation unit that accumulates a history of the results detected by the sensor, a memory unit that stores patterns of the results detected by the sensor when a legitimate user of the mobile terminal uses the mobile terminal for payment, and a determination unit that determines whether the holder of the mobile terminal is a legitimate user by comparing the history accumulated in the accumulation unit with the patterns stored in the memory unit (see Patent Document 1). This mobile terminal is capable of suspending the payment function using the mobile terminal if an unauthorized user is holding the mobile terminal.

[0003] Japanese Patent Application Publication No. 2015-215801

[0004] The present disclosure provides a terminal device, an authentication system, an authentication management device, an authentication method, and an authentication management method that enable improved user convenience by continuously authenticating legitimate users while maintaining security.

[0005] One aspect of the present disclosure is a terminal device for authenticating a person, comprising a processor and a sensor, wherein the processor derives biometric feature data relating to biometric features of a first person holding the terminal device based on first detection data detected by the sensor, performs a first authentication of the person based on the biometric feature data, and if the first authentication is successful, sequentially derives habit feature data relating to habit features of the first person based on second detection data detected by the sensor, and sequentially performs a second authentication of the first person based on the sequentially derived habit feature data.

[0006] One aspect of the present disclosure is an authentication system including a terminal device that authenticates a person and an authentication management device that manages authentication of the person, wherein the terminal device derives biometric feature data relating to biometric features of a first person who owns the terminal device based on first detection data detected by a sensor, and performs a first authentication of the person based on the biometric feature data, and if the first authentication is successful, sequentially derives habit feature data relating to habit features of the first person based on second detection data detected by the sensor, and sequentially performs a second authentication of the first person based on the sequentially derived habit feature data, and if the second authentication determines that the first person is not a legitimate owner of the terminal device, transmits the habit feature data to the authentication management device, and the authentication management device receives the habit feature data from the terminal device and performs a third authentication of the first person based on the habit feature data.

[0007] One aspect of the present disclosure is an authentication management device that manages authentication of a person, the authentication management device including a processor and a memory, wherein the memory holds a plurality of pieces of habit feature data related to habit characteristics of the person, and the processor classifies the plurality of habit feature data held in the memory so that habit feature data having similar feature quantities belong to the same feature group, and determines, based on a classification result of the habit feature data, reference habit feature data that is a standard related to the habit characteristics of the person and is used for a predetermined authentication.

[0008] One aspect of the present disclosure is an authentication method for authenticating a person, the authentication method including: deriving biometric feature data relating to biometric features of a first person who possesses a terminal device based on first detection data detected by a sensor; performing a first authentication of the person based on the biometric feature data; and, if the first authentication is successful, sequentially deriving habit feature data relating to habit features of the first person based on second detection data detected by the sensor; and sequentially performing a second authentication of the first person based on the sequentially derived habit feature data.

[0009] One aspect of the present disclosure is an authentication management method for managing authentication of a person, the authentication management method including: classifying a plurality of habit feature data related to habit features of the person stored in a memory such that habit feature data having similar feature quantities belong to the same feature group; and determining, based on a classification result of the habit feature data, reference habit feature data that is a standard related to the habit features of the person and is used for a predetermined authentication.

[0010] One aspect of the present disclosure is a program for causing a computer to execute the above authentication method.

[0011] One aspect of the present disclosure is a program for causing a computer to execute the above authentication management method.

[0012] According to the present disclosure, it is possible to improve convenience for users by continuously authenticating legitimate users while maintaining security.

[0013] FIG. 1 shows an example of the configuration of an authentication system in a first embodiment. FIG. 1 shows an example of the configuration of a terminal. FIG. 1 shows an example of the configuration of a first authentication management device. FIG. 1 shows an example of the configuration of a second authentication management device. FIG. 1 shows an example of an overview of a use case of the authentication system. FIG. 1 shows an example of details of a use case of the authentication system. Information showing an example of a management table held by a terminal. Waveform graph showing an example of reference habit feature data. Waveform graph showing an example of habit feature data. FIG. 1 shows an example of a management table held by a first authentication management device. FIG. 1 explains the processing overview during biometric authentication. FIG. 1 explains the processing overview during continuous authentication. FIG. 1 explains another example of the processing overview during continuous authentication. FIG. 1 explains threshold conditions. FIG. 1 explains threshold conditions. FIG. 1 explains threshold conditions. FIG. 1 explains differences between reference habit feature data held in management tables T1 and T2.

[0014] Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed description of well-known matters or redundant description of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0015] (Background to the Invention of the Embodiments of the Present Disclosure) The mobile terminal of Patent Document 1 performs identity authentication based on information about the user's movements and state, but does not perform identity authentication using information about the user's biometrics (e.g., information such as a fingerprint). Therefore, the accuracy of identity authentication may be lower than when authentication is performed using information about the biometrics.

[0016] Furthermore, in the mobile terminal of Patent Document 1, if the authentication status that determined the user to be a legitimate user is lost midway, the legitimate user may be determined to be fraudulent use by a third party. However, this mobile terminal does not take into consideration follow-up for a person who is unable to receive a service that they should have received midway through. For example, it is possible to restart the payment process from the beginning by entering a password, but this is time-consuming.

[0017] In the following embodiments, a terminal device, an authentication system, an authentication management device, an authentication method, and an authentication management method are described that enable improved user convenience by continuously authenticating legitimate users while maintaining security.

[0018] (First embodiment) <Configuration of authentication system> Fig. 1 is a diagram showing an example of the configuration of an authentication system 5 according to the first embodiment. The authentication system 5 includes a terminal 100, a first authentication management device 200, a second authentication management device 300, and a monitoring terminal 400. The terminal 100, the first authentication management device 200, and the second authentication management device 300 are communicatively connected via a network NT. The monitoring terminal 400 is communicatively connected to the second authentication management device 300, but may also be communicatively connected via the network NT.

[0019] The terminal 100 is a personal computer (PC), a tablet terminal, a smartphone, or the like. The terminal 100 authenticates a person. The terminal 100 is capable of communicating with the first authentication management device 200 via a predetermined application. The terminal 100 is owned by a predetermined person (user).

[0020] The first authentication management device 200 manages the authentication of a person. The first authentication management device 200 supports processing related to the authentication of a person. The first authentication management device 200 is, for example, a cloud server.

[0021] The second authentication management device 300 manages the authentication of a person. The second authentication management device 300 supports processing related to the authentication of a person. The second authentication management device 300 is, for example, an edge server. The second authentication management device 300 is installed, for example, at the entrance or exit of an authentication area where authentication is performed (e.g., a shopping mall), or in each store. The second authentication management device 300 may be capable of communicating directly with the terminal 100.

[0022] The monitoring terminal 400 is, for example, a terminal that monitors the authentication of people in each store. The monitoring terminal 400 may be installed in each store, or may be portable and carried by a store supervisor. For example, if there is a problem during payment at the store, the monitoring terminal 400 may receive a notification of information regarding the problem from the second authentication management device 300 and present this information (for example, by displaying or outputting sound). The monitoring terminal 400 may be a PC, a tablet terminal, a smartphone, or the like. The monitoring terminal 400 may include, for example, a processor, a memory, a communication device, an input device, a display device, a speaker, and the like.

[0023] 2 is a diagram showing an example of the configuration of the terminal 100. The terminal 100 includes a processor 110, a memory 120, a communication device 130, a camera 140, a sensor 150, and a display device 160.

[0024] The processor 110 may be configured using, for example, a Central Processing Unit (CPU) or a Digital Signal Processor (DSP). The processor 110 may also be configured using various integrated circuits (for example, a Large Scale Integration (LSI) or a Field Programmable Gate Array (FPGA)). The processor 110 realizes various functions by executing programs stored in the memory 120. The processor 110 comprehensively controls each unit of the terminal 100 and performs various processes.

[0025] The memory 120 includes a primary storage device (e.g., Random Access Memory (RAM) or Read Only Memory (ROM)). The memory 120 may include a secondary storage device (e.g., a Hard Disk Drive (HDD) or a Solid State Drive (SSD)) or a tertiary storage device (e.g., an optical disk or an SD card). The memory 120 may be an external storage medium or may be detachable from the terminal 100. The memory 120 stores various data, information, programs, etc. The memory 120 may also store, for example, various trained models.

[0026] The memory 120 also includes a management table T1, which manages information about the person (user) who holds the terminal 100. This information about the person is used for each authentication.

[0027] The communication device 130 communicates various data or information according to a wired or wireless communication method. The communication method used by the communication device may include a local area network (LAN), a wide area network (WAN), a mobile phone network, power line communication, short-range wireless communication (e.g., Bluetooth (registered trademark), NFC (registered trademark)), etc.

[0028] The camera 140 is configured to have at least a lens (not shown) and an image sensor (not shown). The image sensor is a solid-state image sensor such as a charged-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS), and converts an optical image formed on an imaging surface into an electrical signal. The camera 140 captures an image of a subject, including a person (for example, a person's face or fingerprint), and obtains a captured image.

[0029] The camera 140 may capture an image of a subject, including a person, based on, for example, a user operation via an input device. The camera 140 may also capture an image automatically according to predetermined conditions. The camera 140 may be provided external to the terminal 100. In this case, the terminal 100 and the camera 140 may be connected by wire via a USB or the like, or may be connected wirelessly via a wireless LAN or the like. The camera 140 may also be a camera that can be installed away from the terminal 100, such as a network camera.

[0030] The sensor 150 detects the movement or state of the terminal 100 or the user of the terminal 100, etc., and obtains detection data. For example, the sensor 150 includes a position detection sensor (e.g., a GPS sensor), a measurement sensor, an acceleration sensor, a gyro sensor, a vibration sensor, a microphone, a step count sensor, an infrared sensor, an ultrasonic sensor, a contact sensor, a proximity sensor, a pressure sensor, etc., and may also include other sensors. Note that the camera 140 is an example of the sensor 150.

[0031] The display device 160 is configured using a display such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display. The display device 160 displays various types of data or information. For example, the display device 160 displays information related to various types of authentication (for example, biometric authentication or continuous authentication, which will be described later).

[0032] The terminal 100 may also include an input device or the like used by a user of the terminal 100. The input device may include various buttons, keys, a keyboard, a mouse, a touch panel, a microphone, or other input devices. The input device accepts input of various data, information, and the like.

[0033] Here, the various processes executed by the processor 110 include, for example, a biometric image acquisition process, a biometric feature extraction process, a biometric information matching process, a habit information acquisition process, a habit feature extraction process, a habit feature matching process, a management table update process, and a screen control process. The habit feature matching process includes a matching score calculation process and a threshold determination process.

[0034] The processor 110 performs multiple authentications (biometric authentication and continuous authentication). Biometric authentication is authentication based on biometric feature data, which are characteristics related to a person's body. Continuous authentication is authentication that is performed continuously following biometric authentication. Continuous authentication is authentication based on habit feature data, which are characteristics related to a person's habits.

[0035] The biometric image acquisition process is a process of acquiring a captured image (biometric image) of a person captured by the camera 140. The biometric feature extraction process is a process of extracting biometric feature data (feature amounts) related to biometric features from the biometric image. The biometric feature data includes, for example, facial feature amounts and fingerprint feature amounts.

[0036] The biometric feature matching process includes a process of matching the biometric feature data with reference biometric feature data, which is reference information related to biometric features stored in the management table T1, and obtaining a matching score (biometric matching score) (e.g., degree of match). The biometric feature matching process also includes a process of determining whether the biometric matching score satisfies a predetermined threshold th for biometric feature matching (e.g., whether the biometric matching score is equal to or greater than the threshold th).

[0037] The habit information acquisition process is a process for acquiring information about a person's habits (habit information) based on detection data sequentially detected by the sensor 150. The habit information includes information such as the tilt of the terminal 100, the gait, acceleration, latitude and longitude of the terminal 100, etc. The habit information may be the same information as the detection data. Information about the person's (user's) walking, such as the gait, stride length, or walking speed, can be calculated by the processor 110 based on acceleration information (e.g., changes in acceleration waveform) over a predetermined period that can be acquired from the sensor 150. For example, the gait can be calculated (estimated) based on changes in the acceleration waveform when the user is walking. Furthermore, for example, the stride length can be calculated (estimated) by combining the calculated gait with location information (latitude and longitude) of the terminal acquired from a GPS sensor.

[0038] The habit information may also include information about a person's position, acceleration, gait (walking pattern), appearance, face, voiceprint, electroencephalogram, handwriting, blinking, keystrokes, operation method of the terminal 100, or orientation of the screen of the terminal 100 (i.e., tilt, angle), etc. Gait indicates the individual characteristics of a person's walking style, such as posture, stride length, arm swing, or asymmetry of movement. The habit information may also include information about touch characteristics of touching the terminal 100, operation characteristics of operating the mouse of the terminal 100, how the terminal 100 is held, how a doorknob is gripped, how a product is picked up, etc.

[0039] The habit feature extraction process is a process of extracting habit feature data related to the characteristics of a person's habits based on habit information. For example, the habit feature extraction process includes a process of creating a graph related to the tilt of the terminal 100 based on changes in the tilt of the terminal 100, a process of creating a graph of the gait of the terminal 100 based on changes in the gait of the terminal 100, and the like. Furthermore, for example, the habit feature data includes various graphs created by the habit feature extraction process (see FIG. 7C ). The habit feature data may include, for example, data related to the person's actions, operations, movements, or the state of the terminal held by the person, and may be data that characterizes the above-mentioned habit information. The habit feature data is detection data sequentially detected by the sensor 150 or data derived based on the detection data, and may be data whose values ​​change over time and can be represented by a time-series graph.

[0040] The habit feature matching process (first habit feature matching process) is a process in which the terminal 100 matches habit feature data. The habit feature matching process includes a matching score calculation process and a threshold determination process. The matching score calculation process is a process in which the habit feature matching process matches habit feature data with reference habit feature data, which is reference information related to the characteristics of a person's habit stored in the management table T1, to obtain a matching score (habit matching score) (e.g., degree of agreement). The threshold determination process is a process in which the habit matching score satisfies a predetermined threshold th for habit feature matching (e.g., whether the habit matching score is equal to or greater than the threshold th).

[0041] The reference habit feature data is, for example, reference data related to a person's actions, operations, movements, or the state of a device held by the person, and is data to be compared with the habit feature data (see FIG. 7B ). For example, the reference habit feature data may be data derived based on previously obtained habit feature data (e.g., an average value of multiple past habit feature data, or habit feature data that has appeared frequently in the past). For example, FIG. 7B , described below, illustrates an example of the reference habit feature data as a graph of acceleration related to a person's movements. In this case, if the waveform of the habit feature data is similar to the waveform of the reference habit feature data, the habit matching score obtained by the habit feature matching process may be increased.

[0042] The management table update process is a process of updating the management table T1 based on the authentication results of various types of authentication (e.g., biometric authentication or continuous authentication). The management table update process may also include a process of issuing an instruction via the communication device 130 to update the information in each table held on the first authentication management device 200 or the second authentication management device 300 side. The screen control process is a process of controlling the display screen displayed on the display device 160. For example, the processor 110 displays a screen prompting biometric authentication and changes the display depending on the authentication result. For example, if biometric authentication is successful, the processor 110 displays a screen notifying the user that acquisition of habit information has begun.

[0043] 3 is a diagram showing an example of the configuration of the first authentication management device 200. The first authentication management device 200 includes a processor 210, a memory 220, and a communication device 230.

[0044] The processor 210 may be configured using, for example, a CPU or a DSP. The processor 210 may also be configured using various integrated circuits (for example, an LSI or an FPGA). The processor 210 realizes various functions by executing programs stored in the memory 220. The processor 210 comprehensively controls each unit of the first authentication management device 200 and performs various processes.

[0045] The memory 220 includes a primary storage device (e.g., RAM or ROM). The memory 220 may include a secondary storage device (e.g., HDD or SSD) or a tertiary storage device (e.g., optical disk or SD card). The memory 220 may also be an external storage medium, and may be detachable from the first authentication management device 200. The memory 220 stores various types of data, information, programs, etc.

[0046] The communication device 230 communicates various data or information according to a wired or wireless communication method. The communication method used by the communication device may include a LAN, a WAN, a mobile phone network, power line communication, short-range wireless communication (e.g., Bluetooth (registered trademark), NFC (registered trademark)), etc.

[0047] Here, the various processes executed by the processor 210 include, for example, a biometric feature extraction process, a biometric feature matching process, a habit feature matching process, a management table update process, and a screen control process. The habit feature acquisition process includes a matching score calculation process and a threshold determination process.

[0048] The biometric feature extraction process is a process of acquiring (extracting) biometric feature data from the terminal 100 .

[0049] The biometric feature matching process includes a process of matching the biometric feature data with reference biometric feature data, which is reference information related to biometric features stored in the management table T2, and obtaining a matching score (biometric matching score) (e.g., degree of match). The biometric feature matching process also includes a process of determining whether the biometric matching score satisfies a predetermined threshold th for biometric feature matching (e.g., whether the biometric matching score is equal to or greater than the threshold th). Note that the threshold th for biometric feature matching may be the same or different between the terminal 100 and the first authentication management device 200.

[0050] The habit feature matching process (second habit feature matching process) is a process in which the first authentication management device 200 matches habit feature data. The habit feature matching process includes a matching score calculation process and a threshold determination process. The matching score calculation process is a process in which the habit feature matching process matches the habit feature data with reference habit feature data, which is reference information related to the habit features of a person stored in the management table T2, to obtain a matching score (habit matching score) (e.g., degree of agreement). The threshold determination process is a process in which the habit matching score satisfies a predetermined threshold th for habit feature matching (e.g., whether the habit matching score is equal to or greater than the threshold th). Note that the threshold th for habit feature matching may be the same or different between the terminal 100 and the first authentication management device 200.

[0051] The management table update process is a process for updating the management table T2 based on the results of various authentications (for example, biometric authentication or continuous authentication). The management table update process may also include a process for issuing an instruction to update the information in each table held on the terminal 100 or the second authentication management device 300 via the communication device 230. The screen control process is a process for controlling the display screen displayed on the display device 160 of the terminal 100 via the communication device 230.

[0052] 4 is a diagram showing an example of the configuration of the second authentication management device 300. The second authentication management device 300 includes a processor 310, a memory 320, and a communication device 330.

[0053] The processor 310 may be configured using, for example, a CPU or a DSP. The processor 310 may also be configured using various integrated circuits (for example, an LSI or an FPGA). The processor 310 realizes various functions by executing programs stored in the memory 320. The processor 310 comprehensively controls each unit of the second authentication management device 300 and performs various processes.

[0054] The memory 320 includes a primary storage device (for example, RAM or ROM). The memory 320 may include a secondary storage device (for example, HDD or SSD) or a tertiary storage device (for example, optical disk or SD card). The memory 320 may also be an external storage medium, and may be detachable from the second authentication management device 300. The memory 320 stores various data, information, programs, etc. The memory 320 holds, for example, a management table T3 having information on the same items as the management table T1.

[0055] The communication device 330 communicates various data or information according to a wired or wireless communication method. The communication method used by the communication device may include a LAN, a WAN, a mobile phone network, power line communication, short-range wireless communication (e.g., Bluetooth (registered trademark), NFC (registered trademark)), etc.

[0056] The camera 340 includes at least a lens (not shown) and an image sensor (not shown). The camera 340 captures an image of a subject, including a person (e.g., a person's face), to obtain a captured image. The camera 340 may capture an image of a subject, including a person's face, based on, for example, a user's operation via an input device. The camera 340 may automatically capture an image according to predetermined conditions. The camera 340 may be provided external to the second authentication management device 300. In this case, the second authentication management device 300 and the camera 340 may be connected by a cable via a USB or the like, or wirelessly via a wireless LAN or the like. The camera 340 may be a camera, such as a network camera, that can be installed away from the second authentication management device 300. The processor 310 may acquire habit information, such as the way a person grips a doorknob or picks up a product, based on the image captured by the camera 340. The acquired habit information may be sent to the first authentication management device 200 and used for matching on the first authentication management device 200 side.

[0057] The display device 360 ​​is configured using a display such as a liquid crystal display device or an organic EL display, etc. The display device 360 ​​displays various types of data or information.

[0058] Here, the various processes executed by the processor 310 include, for example, biometric image acquisition process, biometric feature extraction process, biometric feature matching process, merchandise detection process, behavior analysis process, person tracking process, management table update process, and screen control process.

[0059] The biometric image acquisition process is a process of acquiring a captured image (biometric image) of a person captured by the camera 340. The biometric feature extraction process extracts biometric feature data from the biometric image. The biometric feature matching process includes a process of matching the biometric feature data with reference biometric feature data, which is reference information related to biometric features stored in the management table T3, and obtaining a matching score (biometric matching score) (e.g., degree of match). The biometric feature matching process also includes a process of determining whether the biometric matching score satisfies a predetermined threshold th for biometric feature matching (e.g., whether the biometric matching score is equal to or greater than the threshold th).

[0060] The product detection process is, for example, a process of detecting products in a store. For example, the processor 310 detects the position and state of the products based on images captured by the camera 340 or detection data detected by the sensor 350.

[0061] The behavior analysis process is, for example, a process of analyzing the behavior of a person in a store. For example, the processor 310 detects the behavior of the person (for example, the person picking up a product) based on an image captured by the camera 340 or detection data detected by the sensor 350.

[0062] The person tracking process is a process for tracking a person in, for example, a store or an authentication area. For example, the processor 310 continuously detects the position or movement of the person in time series based on the captured image captured by the camera 340 or the detection data detected by the sensor 350.

[0063] The management table update process is a process for updating the management table T3 based on the authentication results of various authentication methods (e.g., biometric authentication or continuous authentication). The management table update process may also include a process for issuing an instruction via the communication device 330 to update the information in each table held in the first authentication management device 200 or the terminal 100. The screen control process is a process for controlling the display screen displayed by the display device 360. For example, the processor 310 displays a screen prompting biometric authentication and changes the display according to the authentication result.

[0064] <Use Cases of Authentication System> Next, a description will be given of use cases of the authentication system 5. FIG.

[0065] Case CA shows a conventional authentication method. In Case CA, when a person is repeatedly authenticated over a period of time spent in a specific area (authentication area) such as a shopping mall or a theme park, payment by handing over money, payment by touching an IC card by the person, or payment by facial recognition is required each time a purchase is made or boarding is carried out, which is time-consuming.

[0066] In contrast, Case CB illustrates an authentication method according to this embodiment. In this embodiment, a person is authenticated multiple times. In Case CB, the authentication system 5 performs strict personal authentication (biometric authentication) only during the first authentication, such as at the entrance to a specific area (authentication area). From the second authentication onward, the authentication system 5 performs continuous authentication, such as analyzing the person's behavior, allowing the person to make a purchase or board a vehicle without having to take any action for biometric authentication (for example, bringing their face close to the camera). This allows the authentication system 5 to improve user convenience and reduce the workload of employees at shopping malls, theme parks, etc. Note that the second and subsequent authentications may be performed at a predetermined timing, as described below, or may be performed compulsorily at a predetermined location.

[0067] Fig. 6 is a diagram showing an example of details of a use case of the authentication system 5. Fig. 6 illustrates an example in which a person visits a shopping mall.

[0068] For example, when a person HM carrying the terminal 100 enters a shopping mall, the authentication system 5 performs biometric authentication as person authentication. One example of the biometric authentication is facial authentication. Note that the authentication system 5 may perform biometric authentication as the first person authentication at any store in the shopping mall, rather than at the entrance of the shopping mall. If the biometric authentication is successful, the authentication system 5 performs continuous authentication as the subsequent person authentication (second and subsequent times).

[0069] A person HM who enters a shopping mall carries the terminal 100 in a pocket, bag, or the like and moves around the shopping mall. The terminal 100 continuously acquires habit information. The terminal 100 performs continuous authentication (terminal continuous authentication) as person authentication based on the habit information. If the terminal 100 fails to perform continuous authentication, it transmits feature data (habit feature data) of the person HM obtained during the continuous authentication to the first authentication management device 200. The first authentication management device 200 receives the feature data of the person HM from the terminal 100 and performs continuous authentication (server continuous authentication) as person authentication based on the feature data.

[0070] If a third party is suspected of impersonating the person HM, the terminal 100 may transmit information related to a terminal ID that identifies the terminal 100 to the second authentication management device 300. This is so that the second authentication management device 300 can re-authenticate the person HM who holds the terminal 100. The re-authentication here is biometric authentication (major authentication). When the second authentication management device 300 detects the approach of the terminal 100 identified by the acquired terminal ID, the second authentication management device 300 may instruct the terminal 100 to perform major authentication, causing the terminal 100 to perform major authentication.

[0071] In a store SP in a shopping mall, the second authentication management device 300 manages people and merchandise within the store SP. The second authentication management device 300 acquires images captured by a camera 340 and tracks people and merchandise based on the captured images. The second authentication management device 300 then associates and stores information regarding which person has taken which merchandise. Note that the merchandise may include not only physical merchandise but also services such as food and drink. When a person HM takes merchandise from the store SP and leaves the store SP, the second authentication management device 300 detects that the person HM has left the store SP, performs payment processing for the merchandise, and completes the payment. Therefore, the person HM does not need to perform any special action for payment. Note that the management, tracking, and information storage of people and merchandise by the second authentication management device 300 may be performed in conjunction with a terminal ID.

[0072] When the person HM leaves the shopping mall, the first authentication management device 200 terminates the continuous authentication. The first authentication management device 200 can detect the person HM's exit from the shopping mall, for example, by communicating a beacon signal between the first authentication management device 200 and the terminal 100. Alternatively, the second authentication management device 300 may be installed near the exit of the shopping mall, and the second authentication management device 300 may detect the person HM's exit from the shopping mall based on detection data from a sensor and notify the first authentication management device 200 of this detection information. Alternatively, the terminal 100 may instruct the first authentication management device 200 to terminate the continuous authentication by accepting a user operation via an input device, and the first authentication management device 200 may terminate the continuous authentication in response to this instruction.

[0073] Furthermore, the first authentication management device 200 requests the terminal 100 to update its log, movement history, etc. For example, the first authentication management device 200 notifies the terminal 100 to update the log information, movement history, etc. held by the terminal 100. In response to this notification, the terminal 100 updates the log information, movement history, etc.

[0074] This allows the person HM to be authenticated through continuous authentication after reliable authentication (e.g., biometric authentication) is performed without the person having to take any additional authentication action, improving the convenience of all purchasing activities.

[0075] <Details of Management Table> Next, the management table of the authentication system 5 will be described.

[0076] Fig. 7A shows information illustrating an example of a management table T1 held by the terminal 100. The management table T1 holds information about a person (user) who holds (possesses, owns) the terminal 100. A management table T1 is provided for each terminal 100, that is, for each user (person). The management table T1 shown in Fig. 7A is, for example, the management table T1 held by the terminal 100 with terminal ID "0001". The management table T1 holds habit information, biometric information, biometric authentication status information, continuous authentication status information, and suspicion flag information.

[0077] The habit information is used for continuous authentication. The habit information includes information for each habit item into which habits are classified. The habit items include, for example, information on the tilt, gait, acceleration, and movement of the terminal 100. The feature data (habit feature data) related to each habit item is accumulated to become log data. In other words, each habit item includes a tilt log, gait log, acceleration log, and movement log of the terminal 100.

[0078] The tilt of the terminal 100 can be derived based on data detected by a gyro sensor, for example. The tilt log of the terminal 100 shows the tendency of how the person HM holding the terminal 100 holds the terminal 100. Gait is an example of gait, and can be derived based on data detected by a vibration sensor, for example. The gait log shows the tendency of how the person HM holding the terminal 100 walks. The acceleration of the terminal 100 can be derived based on data detected by an acceleration sensor, for example. The movement of the terminal 100 can be derived by sequentially acquiring the position of the terminal 100 in chronological order, for example, by a GPS sensor. The movement log of the terminal 100 shows the movement tendency of the person HM holding the terminal 100.

[0079] Biometric information is used for biometric authentication. The biometric information includes information for each biometric item into which a person is classified. The biometric items include face and fingerprint. Therefore, facial feature amounts or fingerprint feature amounts, etc., are obtained as biometric feature data.

[0080] The biometric authentication status information includes information regarding the status of biometric authentication (biometric authentication state). The biometric authentication state includes before biometric authentication, authentication success, or re-authentication required (authentication failure), etc. Note that authentication failure here may include both a case where biometric authentication fails and a case where continuous authentication has been completely interrupted. The continuous authentication status information includes information regarding the status of continuous authentication (continuous authentication state). The continuous authentication state includes before continuous authentication starts, during continuous authentication, or during disconnection, etc. Note that during disconnection, continuous authentication is repeated from the time the suspicion flag is turned on until continuous authentication is successful through re-authentication. In this case, the biometric authentication status is a status where re-authentication is required. The suspicion flag information includes information regarding the suspicion flag. The suspicion flag is on when the terminal 100 is suspected to be held by an unauthorized third party, and is off when the terminal 100 is presumed to be held by a legitimate person HM.

[0081] The management table T1 includes reference feature data, feature data, threshold information, etc. for each piece of information included in the habit information. The reference feature data is data related to features that serve as a reference for each authentication. The reference feature data is used as a comparison target when calculating a matching score, as will be described in detail later. The reference feature data includes reference biometric feature data used for biometric authentication and reference habit feature data used for continuous authentication. The reference feature data is information having a representative feature pattern for the person HM holding the terminal 100 for each habit item or each biometric item. The reference feature data is stored in advance in the management table T1 before authentication, for example. The reference feature data may be represented by a waveform graph showing changes over time or may be represented by quantified information. The reference feature data may be updated at any timing. The selection of reference feature data will be described in detail later. FIG. 7B is a waveform graph showing an example of reference habit feature data. FIG. 7B illustrates reference habit feature data related to acceleration.

[0082] The feature data is derived based on detection data detected by the sensor 150 (detected in real time). The feature data includes biometric feature data used for biometric authentication and habit feature data used for continuous authentication. The feature data may be represented by a waveform graph showing changes over time, or may be represented by quantified information. Note that the feature data is derived based on detection data detected by the sensor 150 in real time, and therefore changes sequentially over time. FIG. 7C is a waveform graph showing an example of habit feature data. FIG. 7C shows habit feature data related to acceleration in real time.

[0083] The threshold information is information regarding a threshold to be compared with the matching score. The threshold information related to continuous authentication includes information regarding a threshold thA used for continuous authentication (single authentication) using one threshold and information regarding a threshold thB used for continuous authentication (combined authentication) using two thresholds. The threshold thB is smaller than the threshold thA. Note that the threshold thA is also used for combined authentication. The thresholds thA and thB related to biometric authentication are set for each habit item, for example, thresholds thA1, thA2, thA3, thA4, thB1, thB2, thB3, and thB4. Note that in the figure, th is omitted and simply indicated as "A1," "A2," ..., etc. The threshold information related to biometric authentication includes information regarding a threshold th used for biometric authentication (single authentication) using one threshold. The threshold related to biometric authentication is set for each biometric item, for example, thresholds thX and thY.

[0084] 8 is a diagram showing an example of a management table T2 held by the first authentication management device 200. The management table T2 holds information about people (users) who hold (possess) terminals 100 in the authentication system 5. There is one or more terminals 100 in the authentication system 5, and therefore there is one or more people who use the authentication system 5. In other words, the management table T2 holds information about one or more people (users) who hold (possess) one or more terminals 100. The management table T2 collectively holds information about each terminal 100 that uses the authentication system 5, that is, each user (person).

[0085] In other words, the management table T2 holds the information held in each management table T1 held by each terminal 100. The management table T2 holds the information held in the management table T1, with each terminal 100 (user) identifiable by its terminal ID (user ID). Note that although the description format of the management table T2 shown in FIG. 8 is different from the description format of the management table T1 shown in FIG. 7A, the items in the management table T2 are almost the same as those in the management table T1. Compared to the management table T1, the management table T2 holds information about multiple terminals 100 (users) (e.g., terminal IDs: "0001", "0002", ...) and additionally holds information about terminal IDs (user IDs). Furthermore, multiple pieces of reference feature data may exist for each item.

[0086] For example, when the terminal 100 enters a predetermined authentication area, the terminal 100 performs biometric authentication of the terminal 100 and transmits the biometric authentication result (biometric authentication status information) to the first authentication management device 200. The first authentication management device 200 receives the biometric authentication result from the terminal 100. The terminal 100 performs continuous authentication at predetermined time intervals and transmits the continuous authentication result (continuous authentication status information) and habit feature data to the first authentication management device 200. The first authentication management device 200 receives the continuous authentication result and habit feature data from the terminal 100. If the continuous authentication result is authentication failure, the first authentication management device 200 performs continuous authentication again. The first authentication management device 200 sequentially accumulates the habit feature data acquired from the terminal 100 in a management table T2. The first authentication management device 200 may select reference habit feature data at any time (e.g., the end of the day) based on the habit feature data accumulated in the management table T2 and update the reference habit feature data stored in the terminal 100.

[0087] <Operation of Authentication System> Next, the operation of the authentication system 5 will be described.

[0088] 9 is a diagram for explaining an outline of processing during biometric authentication. In FIG. 9, processing related to biometric authentication is performed by the terminal 100.

[0089] The processor 310 of the second authentication management device 300 installed at the entrance of a shopping mall or at a store instructs the terminal 100 to perform biometric authentication via the communication device 330. The second authentication management device 300 is connected to the terminal 100 so as to be able to communicate with it, but communication may be via a wireless LAN or the like, or may be via a beacon signal.

[0090] In the terminal 100, the communication device 130 receives an instruction to perform biometric authentication from the terminal 100. The communication device 130 sends information to the display device 160 prompting the user of the terminal 100 to perform biometric authentication. At this time, the processor 110 may prompt the user to check the screen by vibrating or the like. The display device 160 displays the information prompting the user to perform biometric authentication. This allows the user to recognize the need for biometric authentication.

[0091] Furthermore, in response to the instruction to perform biometric authentication, communication device 130 instructs camera 140 to capture an image. Camera 140 captures an image of the subject in accordance with the image capture instruction. The subject may include, for example, a person, such as the person's face. Camera 140 sends the captured image of the subject to processor 110.

[0092] In the biometric image acquisition process, the processor 110 acquires a captured image of a person (e.g., a person's face) captured by the camera 140 as a biometric image. The processor 110 performs a biometric feature extraction process on the biometric image. In the biometric feature extraction process, the processor 110 extracts biometric feature data (e.g., facial feature amounts or fingerprint feature amounts) of the person included in the biometric image.

[0093] The processor 110 acquires reference biometric feature data stored in the management table T1 of the memory 120. The processor 110 performs biometric feature matching processing based on the biometric feature data and the reference biometric feature data. In this case, the processor 110 calculates a matching score (biometric matching score) through a matching score calculation processing. In addition, the processor 110 obtains a biometric authentication result by comparing the biometric matching score with a biometric authentication threshold th (e.g., a threshold thX for facial feature amounts or a threshold thY for fingerprint feature amounts) through a threshold determination processing. For example, the processor 110 determines that the biometric authentication is successful when the matching score is equal to or greater than the biometric authentication threshold th, and determines that the biometric authentication is unsuccessful when the matching score is less than the biometric authentication threshold th.

[0094] In the screen control process, the processor 110 generates screen information for displaying the biometric authentication result obtained by the biometric feature matching process, and sends the screen information to the display device 160. The display device 160 obtains the screen information from the processor 110 and displays a screen in accordance with the screen information. In this case, the display device 160 may display information indicating the biometric authentication result or information prompting re-authentication.

[0095] In the management table update process, the processor 110 updates the management table T1. Specifically, the processor 110 stores information on the authentication result of the biometric authentication obtained by the biometric feature matching process (e.g., authentication success or authentication failure (re-authentication)) as biometric authentication status information in the management table T2.

[0096] The communication device 130 acquires the biometric authentication result from the processor 110 and transmits biometric authentication result information including the biometric authentication result and the terminal ID of the terminal 100 that performed the biometric authentication to the first authentication management device 200.

[0097] In the first authentication management device 200, the communication device 130 receives biometric authentication result information from the terminal 100 and sends it to the processor 110. The processor 110 acquires the biometric authentication result information and updates the management table T2 in a management table update process. Specifically, the processor 110 stores information on the biometric authentication result (e.g., authentication success or authentication failure (re-authentication)) included in the biometric authentication result information in the management table T2 as biometric authentication status information for the terminal 100 identified by the terminal ID included in the biometric authentication result information.

[0098] 9 illustrates an example in which biometric authentication (e.g., biometric feature matching processing) is performed by the terminal 100, but this is not limiting. In the authentication system 5, biometric authentication (e.g., biometric feature matching processing) may be performed by the first authentication management device 200.

[0099] 10 is a diagram illustrating an outline of a process for continuous authentication. In FIG. 10, continuous authentication (for example, habit characteristic matching process) is performed by the terminal 100.

[0100] In the terminal 100, the sensor 150 sequentially detects various pieces of information. The processor 110 sequentially acquires the detection data detected by the sensor 150. In the habit information acquisition process, the processor 110 acquires various pieces of habit information based on the detection data sequentially acquired after successful biometric authentication. The various pieces of habit information include, for example, information about the tilt of the terminal 100, the gait of the user carrying the terminal 100, the acceleration of the terminal 100, and the position (e.g., latitude and longitude) of the terminal 100.

[0101] In the habit feature extraction process, the processor 110 extracts habit feature data based on the habit information. In this case, for example, the processor 110 may create, as the habit feature data, information on a graph showing changes in the tilt of the terminal 100 over time. The processor 110 may also create, as the habit feature data, information on a graph showing the gait of the terminal 100. In the management table update process, the processor 110 stores the extracted habit feature data in the management table T2 for each habit item.

[0102] The processor 110 continuously stores and accumulates the habit feature data in the memory 120 until an instruction to execute continuous authentication is given. Therefore, when continuous authentication is performed, habit feature data having a time-series feature pattern is cut off, and this habit feature data is used in the subsequent continuous authentication. For example, the waveform graph-like habit feature data shown in FIG. 7C is divided into time intervals Δt. For example, in continuous authentication performed immediately after time interval Δta, the habit feature data for the divided time interval Δta is used. For example, when continuous authentication is performed frequently (e.g., every t seconds), habit feature data close to real-time data is obtained, and when continuous authentication is performed once a day, habit feature data reflecting the behavioral patterns of the user of the terminal 100 for one day is obtained.

[0103] In the continuous authentication request process, the processor 110 instructs the memory 120 to execute continuous authentication every time a predetermined time elapses. The instruction to execute continuous authentication is issued after successful biometric authentication. In response to the instruction to execute continuous authentication, the processor 110 performs a habit feature matching process.

[0104] In the habit feature matching process, the processor 110 compares the habit feature data stored in the memory 120 with the reference habit feature data in a matching score calculation process to calculate a matching score (continuous matching score) (degree of match). Then, in a threshold determination process, the processor 110 determines that the continuous authentication has been successful if the calculated continuous matching score is equal to or greater than a threshold value th for continuous authentication, and determines that the continuous authentication has failed if the continuous matching score is less than the threshold value th. Details of the method for determining continuous authentication using the threshold value th will be described later.

[0105] In the management table update process, the processor 110 stores the authentication result of the continuous authentication in the memory 120 as continuous authentication status information in the management table T1. For example, if the authentication result of the continuous authentication is successful, the processor 110 updates the continuous authentication status information to "continuing." For example, if the authentication result of the continuous authentication is unsuccessful, the processor 110 updates the continuous authentication status information to "disconnected." Note that the continuous authentication status information remains "before starting" until biometric authentication is successful.

[0106] The processor 110 sends continuous authentication result information, including the authentication result of the continuous authentication, the habit characteristic data, and the terminal ID of the terminal 100 that performed the continuous authentication, to the communication device 130. The communication device 130 transmits the continuous authentication result information to the first authentication management device 200.

[0107] In the first authentication management device 200, the communication device 230 receives the continuous authentication result information from the terminal 100 and sends it to the processor 210. The processor 210 acquires the continuous authentication result information.

[0108] The processor 210 updates the management table T2 in a management table update process. Specifically, the processor 210 updates the information of each item in the management table T2 according to the authentication result of the continuous authentication. Specifically, the processor 210 stores the authentication result of the continuous authentication included in the continuous authentication result information in the memory 220 as continuous authentication status information for the terminal 100 identified by the terminal ID included in the continuous authentication result information in the management table T2. For example, if the authentication result of the continuous authentication is successful, the processor 210 updates the continuous authentication status information to "continuing." For example, if the authentication result of the continuous authentication is unsuccessful, the processor 210 updates the continuous authentication status information to "disconnected." Note that the continuous authentication status information remains "before starting" until biometric authentication is successful. Furthermore, if the authentication result of the continuous authentication is successful, the processor 210 stores and accumulates the habit feature data included in the continuous authentication result information in the management table T2. The habit feature data accumulated in the management table T2 becomes the original information for generating reference feature data.

[0109] 11 is a diagram illustrating another example of the process outline during continuous authentication. FIG. 11 illustrates continuous authentication (e.g., habit feature matching processing) performed by the first authentication management device 200. Continuous authentication by the first authentication management device 200 is performed, for example, when continuous authentication by the terminal 100 shown in FIG. 10 fails. The first authentication management device 200 stores multiple pieces of reference habit feature data to be used as comparison targets when calculating the matching score, making matching more likely to be successful than continuous authentication performed by the terminal 100.

[0110] If the terminal 100 fails to perform continuous authentication as a result of the habit feature matching process by the processor 110, the communication device 130 transmits continuous authentication result information to the first authentication management device 200, the continuous authentication result information including the terminal ID of the terminal 100, the authentication result of the continuous authentication (authentication failed), and the habit feature data.

[0111] In the first authentication management device 200, the communications device 230 receives continuous authentication result information from the terminal 100. This continuous authentication result information includes information indicating authentication failure as the authentication result of the continuous authentication. The processor 210 acquires the continuous authentication result information from the communications device 230, and in the management table update process, updates the continuous authentication status information for the terminal 100 identified by the terminal ID to "disconnected" because the authentication result of the continuous authentication is authentication failure.

[0112] The processor 210 acquires habit feature data from the communication device 230 and acquires reference habit feature data from the management table T2 of the memory 220. The processor 210 performs habit feature matching processing in response to a failure of continuous authentication. In the habit feature matching processing, the processor 210 compares the reference habit feature data with the habit feature data in a matching score calculation processing to calculate a matching score (continuous matching score). Then, in a threshold determination processing, the processor 210 determines whether the calculated continuous matching score is equal to or greater than a threshold value th for continuous authentication. For example, the processor 210 determines that the continuous authentication has been successful if the continuous matching score is equal to or greater than the threshold value th for continuous authentication, and determines that the continuous authentication has failed if the continuous matching score is less than the threshold value th.

[0113] In the management table update process, if the authentication result of the continuous authentication at the first authentication management device 200 is authentication success, the processor 210 updates the continuous authentication status information for the terminal 100 identified by the terminal ID in the management table T2 to "ongoing." Furthermore, if the authentication result of the continuous authentication at the first authentication management device 200 is authentication failure, the processor 210 maintains the continuous authentication status information as "disconnected."

[0114] The processor 210 sends second continuous authentication result information including the authentication result of the continuous authentication in the first authentication management device 200 to the communication device 230. The communication device 230 acquires the second continuous authentication result information from the processor 210 and transmits it to the terminal 100.

[0115] In the terminal 100, the communications device 130 receives the second continuous authentication result information from the first authentication management device 200 and sends it to the processor 110. In the management table update process, the processor 110 stores the authentication result of the continuous authentication included in the second continuous authentication result information in the memory 120 as continuous authentication status information in the management table T1. For example, if the authentication result of the continuous authentication is successful, the processor 110 updates the continuous authentication status information to "continuing." For example, if the authentication result of the continuous authentication is unsuccessful, the processor 110 maintains the continuous authentication status information as "disconnected."

[0116] If the first authentication management device 200 fails to authenticate the terminal 100 during continuous authentication, the terminal 100 may transmit an instruction to perform biometric authentication to the second authentication management device 300 installed in the store before the user of the terminal 100 arrives at the store when it detects that the terminal 100 is approaching a predetermined store. The proximity of the terminal 100 to the predetermined store may be detected, for example, by a sensor 350 of the second authentication management device 300. The processor 310 of the second authentication management device 300 may perform biometric authentication on the user of the terminal 100. Alternatively, the second authentication management device 300 may instruct the terminal 100 to perform biometric authentication, and the terminal 100 may perform biometric authentication in accordance with this instruction (see FIG. 9 ). This allows the authentication system 5 to determine the authenticity of the person holding the terminal 100 through highly accurate biometric authentication, even if the authenticity of the person cannot be confirmed through continuous authentication.

[0117] <Threshold Condition> Next, the threshold condition that must be satisfied in continuous authentication will be described.

[0118] 12A, 12B, and 12C are diagrams illustrating threshold conditions. The threshold conditions include, for example, the following threshold condition TA and threshold condition TB, and may also include other threshold conditions. FIGS. 12A to 12C illustrate examples of the relationship between the matching scores for four habit items and the thresholds thA and thB. For example, it is sufficient if at least one of the threshold condition TA and the threshold condition TB is satisfied. If at least one of the threshold condition TA and the threshold condition TB is satisfied, it is determined that continuous authentication has been successful, and if neither the threshold condition TA nor the threshold condition TB is satisfied, it is determined that continuous authentication has failed.

[0119] The threshold condition TA may include the presence of a matching score that is equal to or greater than a threshold thA. For example, in FIG. 12A , the matching score related to the tilt of the terminal 100 as a habit item is equal to or greater than the threshold thA, satisfying the threshold condition TA. Therefore, continuous authentication is determined to be successful. On the other hand, in FIGS. 12B and 12C , the matching scores related to all habit items are less than the threshold thA, and therefore do not satisfy the threshold condition TA.

[0120] The threshold condition TB may include a condition that a matching score equal to or greater than the threshold thB satisfies a predetermined condition. Specifically, the threshold condition TB may include a condition that the number of matching scores equal to or greater than the threshold thB is equal to or greater than a predetermined number (e.g., two).

[0121] For example, in Fig. 12A, the number of matching scores equal to or greater than the threshold thA is one, so the threshold condition TB is not satisfied, but the threshold condition TA is satisfied, so continuous authentication is determined to be successful. In Fig. 12B, the number of matching scores equal to or greater than the threshold thB is two, so the threshold condition TA is not satisfied, but the threshold condition TB is satisfied, so continuous authentication is determined to be successful. In Fig. 12C, the number of matching scores equal to or greater than the threshold thB is only one, so the threshold condition TB is not satisfied. Therefore, even when both the threshold condition TA and the threshold condition TB are taken into consideration, continuous authentication is determined to be unsuccessful.

[0122] Next, the differences between the reference habit characteristic data held in the management tables T1 and T2 will be described.

[0123] FIG. 13 is a diagram illustrating the difference between the reference habit characteristic data held in the management tables T1 and T2.

[0124] 7A and 13 , the reference habit feature data for the tilt of the terminal 100 is the feature pattern a1, the reference habit feature data for the acceleration of the terminal 100 is the feature pattern c1, and the reference habit feature data for the movement pattern of the terminal 100 is the feature pattern d1.

[0125] The management table T1 also holds reference habit characteristic data for each habit item. For example, in Fig. 8 and Fig. 13, feature patterns a1, a2, ... are set as reference habit characteristic data for the tilt of the terminal 100, feature patterns c1, c2, ... are set as reference habit characteristic data for the acceleration of the terminal 100, and feature patterns d1, d2, ... are set as reference habit characteristic data for the movement pattern of the terminal 100. In other words, the management table T2 holds more reference habit characteristic data for each habit item than the terminal 100 does.

[0126] In the first authentication management device 200, the processor 210 obtains a large matching score when, for each habit item, the feature pattern of the habit feature data to be compared resembles one of the feature patterns of one or more reference habit feature data held in the management table T2. Thus, continuous authentication by the first authentication management device 200 is more likely to obtain a larger matching score and be more likely to be successful than continuous authentication by the terminal 100.

[0127] Therefore, if continuous authentication by the terminal 100 fails, the terminal 100 transmits the habit feature data to be compared to the first authentication management device 200, and the first authentication management device 200 performs continuous authentication using the received habit feature data. In this case, the first authentication management device 200 compares the habit feature data with each of many pieces of reference habit feature data to calculate one or more comparison scores, and determines whether, for example, the largest comparison score among the calculated comparison scores is equal to or greater than the threshold value th for continuous authentication. The one or more pieces of reference habit feature data held by the first authentication management device 200 are all based on the past behavior patterns of the user of the terminal 100. Therefore, the first authentication management device 200 can obtain more accurate continuous authentication results than the terminal 100. By transmitting the authentication results by the first authentication management device 200 to the terminal 100, the first authentication management device 200 can share the authentication results by the first authentication management device 200 with the terminal 100.

[0128] Next, a specific example of the operation relating to continuous authentication will be described.

[0129] FIG. 14 is a flowchart showing an example of the operation of the authentication system 5 when the terminal 100 performs the habit feature matching process.

[0130] In the terminal 100, the processor 110 acquires, from the management table T1, reference habit characteristic data for each habit item (each reference habit characteristic data) and habit characteristic data for each habit item (each habit characteristic data) held in the management table T1. The processor 110 compares the reference habit characteristic data with the habit characteristic data for each habit item and calculates a comparison score (S11).

[0131] The processor 110 performs a first threshold determination process. Specifically, for each habit item, the processor 110 compares the match score with a threshold thA for that habit item. The processor 110 determines whether or not there is one or more match scores that are equal to or greater than the threshold thA (S12). In other words, the processor 110 determines whether or not there is one or more habit items that have match scores that are equal to or greater than the threshold thA.

[0132] If there is one or more matching scores that are equal to or greater than the threshold thA (Yes in step S12), the processor 110 determines that a valid user is holding the terminal 100 (S13). That is, in this case, the processor 110 determines that the continuous authentication has been successful, and that the continuous authentication state is maintained as a continuous state.

[0133] On the other hand, if there is no matching score equal to or greater than the threshold value thA (No in step S12), that is, if there is no habit item having a matching score equal to or greater than the threshold value thA, the processor 110 performs a second threshold determination process. For example, the processor 110 determines whether there are a predetermined number (e.g., two) or more matching scores that are equal to the threshold value thB (S14).

[0134] If there are a predetermined number or more matching scores that are equal to or greater than the threshold value thB (Yes in step S14), the processor 110 determines that a valid user is holding the terminal 100 (S13). That is, in this case, the processor 110 determines that the continuous authentication has been successful, and that the continuous authentication state is maintained as a continuous state.

[0135] On the other hand, if there are no more than a predetermined number of matching scores that are equal to or greater than the threshold thB (No in step S14), the processor 110 determines that there is a possibility that the terminal 100 is being fraudulently used (S15). In this case, the processor 110 sets the suspicion flag held in the management table T1 to ON. If the suspicion flag is ON, the processor 110 may instruct the first authentication management device 200 via the communication device 130 to re-execute continuous authentication.

[0136] According to this habit feature matching process, the authentication system 5 determines that continuous authentication has been successful if, in the first threshold determination process, there is one or more matching scores equal to or greater than the threshold thA. In this case, the processor 110 determines that biometric authentication, which is highly accurate person authentication, is not necessary. The authentication system 5 also determines that continuous authentication has been successful if, in the second threshold determination process, there are matching scores equal to or greater than the threshold thB that satisfy a predetermined condition (for example, a predetermined number or more). In this case, the processor 110 also determines that biometric authentication, which is highly accurate person authentication, is not necessary.

[0137] The predetermined condition here may be, other than being equal to or greater than a predetermined number, for example, satisfying a predetermined ratio. Satisfying a predetermined ratio may mean, for example, that the ratio of the number of matching scores equal to or greater than a threshold value thB to the total number of matching scores calculated for each habit item included in the habit information is equal to or greater than a predetermined ratio. Furthermore, among the habit items included in the habit information, there may be habit items that must have a matching score equal to or greater than the threshold value thB.

[0138] The processor 110 may also calculate a total matching score by weighting the matching scores for each habit item and summing the weighted scores. The processor 110 may determine that the continuous authentication is successful if the total matching score is equal to or greater than a threshold value th for the total matching score, and may determine that the continuous authentication is unsuccessful if the total matching score is less than the threshold value th for the total matching score.

[0139] If the continuous authentication by the terminal 100 fails, that is, if step S14 is No, the terminal 100 may transmit the habit characteristic data to the first authentication management device 200, and the first authentication management device 200 may perform continuous authentication (that is, re-matching). In this case, the first authentication management device 200 executes the same process as that shown in Fig. 14. Note that necessary substitutions are made, such as replacing the processor 110 with the processor 210 and the management table T1 with the management table T2.

[0140] 15 and 16 are flowcharts showing an example of operation of the authentication system 5 after the habit feature matching process is performed by the terminal 100. Fig. 15 is a flowchart showing an example of operation of the authentication system 5 when continuous authentication by the terminal 100 is successful. Fig. 16 is a flowchart showing an example of operation of the authentication system 5 when continuous authentication by the terminal 100 is unsuccessful.

[0141] 15 , when the continuous authentication by the terminal 100 is successful, the communication device 130 of the terminal 100 transmits continuous authentication result information including the authentication result of the continuous authentication (information on successful matching), each habit characteristic data, and the terminal ID to the first authentication management device 200 (S21). Furthermore, the processor 110 of the terminal 100 updates the management table T1 according to the continuous authentication result information (S22). Specifically, the processor 110 updates the continuous authentication status information in the management table T1 to "ongoing."

[0142] In the first authentication management device 200, the communication device 230 receives continuous authentication result information from the terminal 100. The processor 210 updates the management table T2 in accordance with the continuous authentication result information (S23). Specifically, the processor 210 updates the continuous authentication status information for the terminal 100 identified by the terminal ID in the management table T2 to "ongoing." The processor 210 also accumulates each piece of habit characteristic data in the management table T2.

[0143] 16 , if the continuous authentication by the terminal 100 fails, the communication device 130 of the terminal 100 transmits continuous authentication result information including the authentication result of the continuous authentication, each habit characteristic data, and the terminal ID to the first authentication management device 200 (S31). The processor 110 updates the management table T1 according to the continuous authentication result information (S32). Specifically, the processor 110 updates the continuous authentication status information in the management table T1 to "disconnected."

[0144] In the first authentication management device 200, the communication device 230 receives continuous authentication result information from the terminal 100. The processor 210 acquires each habit characteristic data from the continuous authentication result information. The processor 210 acquires reference habit characteristic data (each reference habit characteristic data) for each habit item included in the habit information held in the management table T2. The processor 210 executes a habit characteristic matching process based on the acquired reference habit characteristic data and each habit characteristic data (S33), and determines whether the continuous authentication (server continuous authentication) by the first authentication management device 200 has been successful (S34). The habit characteristic matching process here is similar to the process of steps S12 and S14 of FIG. 14 , and therefore a detailed description thereof will be omitted.

[0145] If the continuous authentication by the first authentication management device 200 is successful, that is, if the matching score satisfies a predetermined condition, the processor 210 determines that the valid user is holding the terminal 100 (S35). That is, in this case, the processor 110 determines that the continuous authentication state is maintained as a continuous state. The predetermined condition here includes, for example, that there is one or more matching scores that are equal to or greater than the threshold value thA, or that there are a predetermined number or more matching scores that are equal to or greater than the threshold value thB.

[0146] On the other hand, if the continuous authentication by the first authentication management device 200 fails, that is, if the matching score does not satisfy the above-mentioned predetermined condition, the processor 210 determines that there is a possibility that the terminal 100 has been fraudulently used (S36). In this case, the processor 210 sets the suspicion flag for the terminal 100 identified by the terminal ID to ON in the management table T2. If the suspicion flag is ON, the processor 110 may instruct the terminal 100 or the second authentication management device 300 via the communication device 230 to re-execute biometric authentication.

[0147] The communication device 230 transmits second continuous authentication result information including the authentication result of the continuous authentication by the first authentication management device 200 to the terminal 100 (S37).

[0148] The processor 210 updates the management table T2 based on the second continuous authentication result information (S38). Specifically, if the authentication result of the continuous authentication at the first authentication management device 200 is authentication success, the processor 210 updates the continuous authentication status information to "ongoing". Furthermore, if the authentication result of the continuous authentication at the first authentication management device 200 is authentication failure, the processor 210 maintains the continuous authentication status information as "disconnected".

[0149] In the terminal 100, the communications device 130 receives the second continuous authentication result information from the first authentication management device 200. The processor 110 updates the management table T1 based on the second continuous authentication result information (S32). Specifically, if the authentication result of the continuous authentication at the first authentication management device 200 is authentication success, the processor 110 updates the continuous authentication status information in the management table T1 to "ongoing." Furthermore, if the authentication result of the continuous authentication at the first authentication management device 200 is authentication failure, the processor 110 maintains the continuous authentication status information in the management table T1 as "disconnected."

[0150] In this way, if continuous authentication at the terminal 100 fails, the authentication system 5 may perform continuous authentication again by performing continuous authentication using the first authentication management device 200. As described above, the reference habit characteristic data held in the management table T1 and the management table T2 of the terminal 100 are different. Therefore, even if continuous authentication at the terminal 100 fails, the authentication system 5 can increase the probability that continuous authentication by the first authentication management device 200 will be successful. Note that the reference characteristic data held in the management table T1 and the management table T2 may be updated as appropriate.

[0151] If the authentication result of the continuous authentication in the first authentication management device 200 is authentication failure, that is, if the suspicion flag in the management table T2 is on, the processor 210 of the second authentication management device 300 may transmit the terminal ID of the terminal 100 suspected of fraudulent use to the second authentication management device 300 installed in the store via the communication device 230. As a result, the processor 310 of the second authentication management device 300 may receive the terminal ID of the terminal 100 suspected of fraudulent use via the communication device 330 and present information about the terminal 100 via the display device 360. This allows a store clerk or other person who checks the display on the second authentication management device 300 in the store to take prompt action against the user of the terminal 100. For example, if the second authentication management device 300 detects the approach of the terminal ID of the terminal 100 suspected of fraudulent use, the second authentication management device 300 may transmit an instruction to vibrate the terminal 100 to inform the owner of the terminal 100 that the authentication state has been interrupted. The owner can check the terminal 100 and perform biometric authentication again if necessary.

[0152] Next, a process performed by the authentication system 5 when updating the reference feature data on the terminal 100 side will be described.

[0153] FIG. 17 is a diagram illustrating an outline of the process performed by the authentication system 5 when updating the reference habit characteristic data on the terminal 100 side.

[0154] In the terminal 100, the communication device 130 sequentially acquires the habit characteristic data from the management table T1 in the memory 120 and transmits the habit characteristic data to the first authentication management device 200. The habit characteristic data here is data that has been sequentially accumulated in the management table T1 over a predetermined time period.

[0155] In the first authentication management device 200, the communication device 230 sequentially receives the habit characteristic data from the terminal 100 and sends it to the processor 210. In the management table update process, the processor 210 sequentially holds and accumulates the habit characteristic data from the terminal 100 in the management table T2.

[0156] In the first authentication management device 200, the processor 210 instructs the memory 220 to select reference habit characteristic data to be stored on the terminal 100 side in the characteristic selection request process. The instruction to select the reference habit characteristic data is given at a predetermined timing (for example, at the end of the day (23:00, etc.)). The processor 210 continuously stores and accumulates the habit characteristic data from the terminal 100 in the memory 120 until the instruction to select the reference habit characteristic data is given. Note that the processor 210 may temporarily stop storing the habit characteristic data from the terminal 100 and may resume storage thereafter. For example, storage may be resumed when the date changes from the day storage was stopped, or at another timing.

[0157] When selecting the reference habit characteristic information data, the processor 210 acquires the habit characteristic data stored in the management table T2. This habit characteristic data is characteristic data that has continued to be stored in the memory 220 until a selection request is received. The processor 210 executes the terminal-stored characteristic selection process based on the acquired habit characteristic data.

[0158] In the terminal-stored feature selection process, the processor 210 classifies (groups) one or more pieces of habit feature data based on the similarity of the one or more pieces of habit feature data acquired from the memory 220 in the grouping process. That is, the processor 210 classifies one or more pieces of habit feature data such that habit feature data having similar feature amounts belong to the same feature group. For example, the processor 210 classifies information (e.g., graph information) indicating the waveform of the time-series change in the tilt of the terminal 100 as habit feature data of feature pattern a1, habit feature data of feature pattern a2, ..., and habit feature data of feature pattern an. As a result, for example, habit feature data related to the tilt of a plurality of pieces of terminal 100 are divided into feature groups of one of the feature patterns. The processor 210 determines the similarity between the habit feature data based on, for example, a trained model for grouping, and classifies the habit feature data. For example, this trained model can be generated by deep learning using a recurrent neural network (RNN). The processor 210 performs this grouping process for each habit item and classifies the habit feature data into several feature groups.

[0159] Furthermore, in the terminal-retained feature selection process, the processor 210 selects the habit feature data of the feature pattern that has been acquired the most times in the representative selection process. For example, if the habit feature data of feature pattern a1 is the most frequently acquired among the habit feature data of feature pattern a1, the habit feature data of feature pattern a2, ..., and the habit feature data of feature pattern an, the processor 210 determines feature pattern a1 as the representative habit feature data related to the tilt of the terminal 100 (the "reference habit feature data" to be stored in the terminal 100). The communication device 230 obtains the determined representative habit feature data (selection result) from the processor 210 and transmits it to the terminal 100.

[0160] Note that a feature pattern (e.g., feature pattern a1) of a predetermined feature group (e.g., feature group ga) may be a pattern obtained as data of the average value of a plurality of habit feature data belonging to the predetermined feature group or other statistical value data. In other words, the processor 110 may derive (e.g., calculate) a feature pattern (e.g., feature patterns a1, a2, ...) representing the characteristics of each feature group based on each piece of habit feature data belonging to each grouped feature group.

[0161] In the management table update process, the processor 210 updates the grouping information based on the results of the grouping process. For example, suppose that the management table T2 has feature patterns a1 and a2 as the tilt of the terminal 100, and a new feature pattern a3 is obtained. In this case, the processor 210 adds the feature pattern a3 to the management table T2 as the tilt of the terminal 100. Each feature pattern held in the management table T2 is each reference habit feature data held in the management table T2. Each feature pattern is obtained for each habit item.

[0162] In the terminal 100, the communication device 130 receives the representative habit characteristic data from the first authentication management device 200. The processor 110 acquires the representative habit characteristic data as a selection result and updates the reference habit characteristic data in the management table T1 in the memory 120. The processor 110 updates the reference habit characteristic data for each habit item. For example, the processor 110 updates the reference habit characteristic data for a predetermined item in the management table T1 from characteristic pattern a2 to characteristic pattern a1 based on the acquired representative habit characteristic data for the predetermined habit item.

[0163] FIG. 18 is a flowchart showing an example of the terminal-held feature selection process.

[0164] In the first authentication management device 200, the processor 210 groups (classifies) one or more pieces of habit characteristic data accumulated in the past for each habit item of the habit information held in the management table T2 based on the similarity between the pieces of habit characteristic data (S41). That is, the one or more pieces of habit characteristic data are classified so that similar characteristics are grouped together, for example, into characteristic pattern a1, characteristic pattern a2, ..., and characteristic pattern an.

[0165] The processor 210 selects, for each habit item, the habit characteristic data of the characteristic pattern that has been acquired the most frequently (S42). For example, the processor 210 selects, for each habit item, characteristic pattern a1 as the representative habit characteristic data because the number of habit characteristic data belonging to the characteristic group of characteristic pattern a1 is the largest. The communication device 230 transmits the representative habit characteristic data for each habit item to the terminal 100 (S43).

[0166] The processor 210 updates the management table T2 based on the results of the grouping process (S44). Specifically, the processor 210 updates the grouping information for each habit item in the management table T2. For example, the processor 210 adds the feature pattern a3 to the management table T2 as one of the reference habit feature data.

[0167] In the terminal 100, the communication device 130 receives the representative habit characteristic data for each habit item from the first authentication management device 200. The processor 110 acquires the representative habit characteristic data for each habit item from the communication device 130 and updates the management table T1. In this case, the processor 110 updates the reference habit characteristic data for each habit item with the representative habit characteristic data.

[0168] In this way, by executing the terminal-retained feature selection process, the authentication system 5 can sequentially accumulate feature data (biometric feature data and habit feature data) related to the biometric information and habit information of the user of the terminal 100 obtained by the terminal 100 in the management table T2. Furthermore, the authentication system 5 can set the reference feature data of the terminal 100 at a predetermined timing, that is, feature data of a representative pattern that clearly represents the habits of the user of the terminal 100 (representative habit feature data), as the reference feature data in the management table T1 of the terminal 100. Therefore, even if the habit tendencies of the user of the terminal 100 change, the characteristics of the user's habit can be tracked and updated, and continuous authentication can be performed in accordance with these habit characteristics. Therefore, the authentication accuracy of continuous authentication can be maintained.

[0169] Note that the biometric feature data used for biometric authentication rarely changes depending on the state of the terminal 100 or the user, etc., or the time. Therefore, the reference biometric feature data, which is the standard for a person's features, does not need to be updated. Note that the authentication system 5 may update the reference biometric feature data in the same way as updating the reference habit feature data. In this case, when updating the reference biometric feature data based on the captured image, the authentication system 5 may perform image quality determination processing (determining brightness, imaging angle, etc. that facilitates biometric authentication), etc.

[0170] Next, a process that takes into account fluctuations in habit characteristic data will be described.

[0171] For example, the detection data detected by the sensor 150 as habit information may fluctuate, and the same characteristic pattern of information may not always be obtained. Depending on the timing, habit characteristic data similar to the reference habit characteristic data may be obtained, or habit characteristic data dissimilar to the reference habit characteristic data may be obtained. For example, when the processor 110 continuously performs continuous authentication at predetermined time intervals, if the authentication result of the continuous authentication changes from authentication failure to authentication success, the terminal 100 may update the suspicion flag in the management table T1 from "on" to "off" and update the continuous authentication status information from "disconnected" to "continued." In this case, the processor 110 may also update the suspicion flag in the management table T2 from "on" to "off" and update the continuous authentication status information from "disconnected" to "continued" via the communication device 130.

[0172] Furthermore, if continuous authentication fails and the continuous state of person authentication is interrupted, the threshold value th for continuous authentication may be made stricter. For example, the processor 110 may make the threshold value th stricter each time continuous authentication fails, or according to the amount of time that has elapsed since the point in time when continuous authentication failed. The stricter threshold value th becomes more difficult to meet. This makes it more difficult for the authentication system 5 to succeed in continuous authentication of the terminal 100, making it more difficult to successfully authenticate a third party who happens to pick up the terminal 100, for example, and thus maintaining high security.

[0173] The processor 110 may also determine whether biometric authentication is necessary based on the reliability of the authentication result of the continuous authentication. For example, the processor 110 may calculate the reliability based on the time elapsed since the previous (last) continuous authentication or the number of authentication failures of successive continuous authentications (i.e., the number of disconnections from the continuous state). The reliability may be higher the shorter the time elapsed since the previous continuous authentication, and lower the reliability the longer the time elapsed since the previous continuous authentication. The reliability may also be higher the fewer the number of authentication failures of the continuous authentication, and lower the reliability the more the number of authentication failures of the continuous authentication. For example, the processor 110 may determine that biometric authentication is necessary if the reliability is equal to or greater than a predetermined threshold th2, and may determine that biometric authentication is not necessary if the reliability is less than the predetermined threshold th2.

[0174] Next, examples of displays (examples of GUI) on the terminal 100 and the monitoring terminal 400 will be described.

[0175] FIG. 19 is a diagram showing an example of a display when biometric authentication is performed, for example, when entering a shopping mall.

[0176] Screen G1 is a display screen containing information prompting the user to perform biometric authentication. Screen G2 is a display screen containing information displayed after biometric authentication is successful. Screen G2 may display, for example, a message indicating that continuous authentication will begin or a message encouraging the user to enjoy shopping. When the sensor 150 detects that the user is approaching the entrance to a shopping mall, the processor 110 may vibrate the terminal 100 to prompt the user to look at screen G1.

[0177] FIG. 20 shows an example of a display displayed when making a payment at a store in a shopping mall, for example.

[0178] Screen G3 is a display screen that includes information about the product to be paid for. Screen G4 is a display screen that includes information that prompts the user to perform biometric authentication at the time of payment.

[0179] The processor 110 displays screen G3 on the display device 160 during payment. If continuous authentication is successful during payment, i.e., if the continuous authentication status information in the management table T2 is "ongoing," the processor 110 may proceed with the payment without performing biometric authentication during settlement. Furthermore, if continuous authentication is successful during payment, the processor 110 may display a list of purchased items and a message such as "Thank you for your purchase" on the display screen when the user of the terminal 100 leaves the store with the items. On the other hand, if continuous authentication fails during payment, i.e., if the continuous authentication status information in the management table T2 is "disconnected," the processor 110 displays screen G4 on the display device 160 to prompt the user to perform biometric authentication. If the user of the terminal 100 succeeds in this biometric authentication, the processor 110 may proceed with the payment.

[0180] If the biometric authentication also fails, the processor 110 may display a screen indicating that the biometric authentication has failed. Furthermore, if the biometric authentication also fails, the processor 110 may cooperate with the first authentication management device 200 to prohibit the user of the terminal 100 from using applications.

[0181] Furthermore, management table T1 may previously register and store a terminal ID for identifying terminal 100 and an image (e.g., a facial image) of the user of terminal 100 in association with each other. Processor 110 may generate an extracted image by extracting a portion of the user of terminal 100 from an image captured by camera 140 of terminal 100. Processor 110 may transmit the extracted image and an image of the registered user (registered image) to monitoring terminal 400 via communication device 130.

[0182] The processor of the monitoring terminal 400 may acquire an extracted image and a registered image in which the user is captured via a communication device, and display the extracted image and the registered image side by side on a display device. This allows a monitor carrying the monitoring terminal 400 to, for example, speak to a person carrying the terminal 100 in a store as needed.

[0183] As described above, the authentication system 5 of this embodiment can perform strict authentication using biometric authentication. Furthermore, continuous authentication allows the user of the terminal 100 to easily continue authentication. Therefore, the authentication system 5 can continuously authenticate a legitimate user while maintaining security. Even if continuous continuous authentication fails and maintaining the authentication state becomes difficult, the authentication system 5 can attempt to restore the authentication state by having the first authentication management device 200 perform highly accurate continuous authentication on behalf of the terminal 100 or by prompting biometric authentication. This allows the authentication system 5 to continuously determine whether a person is a legitimate person from start to finish in an authentication area (e.g., a shopping mall or an amusement park).

[0184] In the present embodiment, the first authentication management device 200 and the second authentication management device 300 are separate entities, but this is not limiting. The first authentication management device 200 and the second authentication management device 300 may be configured as an integrated device.

[0185] In the present embodiment, both biometric authentication and continuous authentication are performed, but this is not limiting. For example, the biometric authentication may be omitted, and continuous authentication by the terminal 100 and re-authentication of continuous authentication by the first authentication management device 200 may be performed sequentially.

[0186] (Summary of the embodiment) As described above, the present disclosure describes at least the following matters. Note that, in parentheses, examples of components corresponding to the above-described embodiment are shown, but the present disclosure is not limited to these.

[0187] (Item 1) A terminal device (terminal 100) for authenticating a person, comprising: a processor (processor 110) and a sensor (sensor 150), wherein the processor: derives biometric feature data relating to biometric features of a first person (person HM) who holds the terminal device based on first detection data detected by the sensor; performs a first authentication (biometric authentication) of the person based on the biometric feature data; and, if the first authentication is successful, sequentially derives habit feature data relating to habit features of the first person based on second detection data detected by the sensor; and sequentially performs a second authentication (continuous authentication by the terminal device) of the first person based on the sequentially derived habit feature data.

[0188] This allows the terminal device to perform the first authentication based on the biometric characteristic data as well as the second authentication based on the habit characteristic data. Therefore, even if the user is not authenticated as a legitimate user by the second authentication, the first authentication can be performed as a follow-up authentication. Therefore, the terminal device can improve user convenience by continuously authenticating legitimate users while maintaining security.

[0189] (Item 2) The terminal device according to Item 1, wherein the processor derives the habit feature data for each habit item into which the person's habits are classified, calculates a matching score for each habit item based on the habit feature data in the second authentication, and determines that the first person is not a legitimate owner of the terminal device if the matching score for each habit item does not satisfy a threshold condition related to a predetermined threshold (threshold th).

[0190] This allows the terminal device to determine the legitimacy of the first person holding the terminal using the threshold value in the second authentication.

[0191] (Item 3) The terminal device according to Item 2, wherein the threshold condition includes the existence of the matching score being equal to or greater than a predetermined first threshold (threshold thA).

[0192] This allows the terminal device to determine the legitimacy of the first person holding the terminal in the second authentication using the first threshold, which is a relatively strict standard.

[0193] (Item 4) The terminal device according to Item 2, wherein the threshold condition includes that the matching score, which is equal to or greater than a second threshold (threshold thB) that is smaller than a predetermined first threshold, satisfies a predetermined condition.

[0194] As a result, in the second authentication, the terminal device can determine the legitimacy of the first person holding the terminal using a second threshold that does not satisfy the first threshold but has more relaxed standards than the first threshold.

[0195] (Item 5) The terminal device according to Item 4, wherein the threshold condition includes that the number of the matching scores that are equal to or greater than the second threshold is a predetermined number or more, or that the ratio of the number of the matching scores that are equal to or greater than the second threshold to the total number of matching scores is a predetermined ratio or more.

[0196] This allows the terminal device to determine the legitimacy of the first person who holds the terminal depending on whether or not the second threshold satisfies any of the conditions that must be satisfied in the second authentication.

[0197] (Item 6) The terminal device according to Item 1, wherein the terminal device derives reliability of the result of the second authentication based on the time elapsed since the previous second authentication processing among the second authentications that are sequentially executed, or the number of times the second authentication has failed, and determines that the first authentication needs to be executed if the reliability is equal to or less than a third threshold.

[0198] If the reliability of the second authentication result is low, there is a possibility that security will be reduced even if the second authentication is performed. In contrast, if the reliability of the second authentication result is low, the terminal device can prompt the user to perform the first authentication, which has high authentication accuracy, thereby preventing a reduction in security.

[0199] (Item 7) An authentication system (authentication system 5) comprising a terminal device that authenticates a person and an authentication management device (first authentication management device 200) that manages authentication of the person, wherein the terminal device derives biometric feature data relating to biometric features of a first person who owns the terminal device, based on first detection data detected by a sensor, performs a first authentication of the person based on the biometric feature data, and if the first authentication is successful, sequentially derives habit feature data relating to habit features of the first person, based on second detection data detected by the sensor, and sequentially performs a second authentication of the first person based on the sequentially derived habit feature data, and if it is determined by the second authentication that the first person is not a legitimate owner of the terminal device, transmits the habit feature data to the authentication management device, and the authentication management device receives the habit feature data from the terminal device, and performs a third authentication of the first person (continuous authentication by the first authentication management device) based on the habit feature data.

[0200] As a result, even if the second authentication based on the habit characteristic data by the terminal device determines that the first person is not a legitimate owner, the authentication system can re-authenticate the first person through the third authentication based on the habit characteristic data by the authentication management device. Thus, the authentication system can continuously authenticate legitimate users while maintaining security, thereby enabling improved user convenience.

[0201] (Item 8) The authentication system according to Item 7, wherein the authentication management device determines that the first person who was determined not to be the legitimate owner of the terminal device was the legitimate owner of the terminal device when the matching score obtained in the third authentication satisfies a threshold condition related to a predetermined threshold.

[0202] This allows the authentication system to overturn the authentication result of the second authentication and guarantee the legitimacy of the first person through the third authentication, thereby ensuring security through more accurate authentication based on habit feature data.

[0203] (Item 9) The authentication system according to Item 8, wherein the terminal device holds one piece of reference habit feature data having a predetermined feature pattern, which is a standard related to habit features of the first person, and calculates the matching score based on the habit feature data and the reference habit feature data; and the authentication management device holds a plurality of pieces of reference habit feature data having different feature patterns, calculates a plurality of matching scores based on the habit feature data and each of the plurality of reference habit feature data, and determines the maximum matching score of the plurality of matching scores as the matching score for the third authentication.

[0204] This allows the authentication system to obtain a larger matching score for the authentication management device compared to the terminal device, making it easier to authenticate the terminal device as a legitimate owner.

[0205] (Item 10) The authentication system according to Item 9, wherein the authentication management device sequentially acquires and stores the habit feature data from the terminal device, classifies the stored plurality of habit feature data so that habit feature data having similar feature amounts belong to the same feature group, determines the reference habit feature data to be used for the second authentication based on the classification result of the habit feature data, and transmits the determined reference habit feature data to the terminal device, and the terminal device receives and stores the reference habit feature data.

[0206] This allows the authentication system to determine reference habit feature data based on multiple pieces of habit feature data previously obtained for the person to be authenticated, and use the reference habit feature data as data to be compared with the habit feature data in the second authentication. Thus, the authentication system can derive and use reference habit feature data suitable for the person to be authenticated.

[0207] (Item 11) The authentication system according to Item 10, wherein the authentication management device sequentially acquires and stores the habit feature data from the terminal device for each habit item into which the person's habits are classified, classifies the stored plurality of habit feature data for each habit item so that habit feature data having similar features belong to the same feature group, determines the reference habit feature data to be used in the second authentication based on the classification result of the habit feature data for each habit item, and transmits the determined reference habit feature data for each habit item to the terminal device, and the terminal device receives and stores the reference habit feature data for each habit item.

[0208] This allows the authentication system to determine reference habit feature data for each habit item based on multiple pieces of habit feature data previously obtained for the person to be authenticated, and use the determined reference habit feature data as data to be compared with the habit feature data in the second authentication. Thus, the authentication system can derive and use reference habit feature data suitable for the person to be authenticated for each habit item.

[0209] (Item 12) The authentication system according to Item 10, wherein the authentication management device determines the reference habit feature data based on the habit feature data belonging to the feature group to which the largest number of the habit feature data belong, among the classified feature groups.

[0210] This allows the authentication system to use habit feature data that corresponds to the tendency of the person to be authenticated to exhibit behavior, actions, etc. as reference habit feature data. Therefore, by performing the second authentication using this reference habit feature data, the authentication system can perform the second authentication more accurately.

[0211] (Item 13) An authentication management device that manages authentication of a person, comprising: a processor (processor 210) and a memory (memory 220), wherein the memory holds a plurality of habit feature data related to habit features of the person, and the processor classifies the plurality of habit feature data held in the memory so that habit feature data having similar feature amounts belong to the same feature group, and determines, based on a classification result of the habit feature data, reference habit feature data that is a standard related to the habit features of the person and is used for a predetermined authentication (continuous authentication).

[0212] This allows the authentication management device to determine reference habit characteristic data based on the plurality of habit characteristic data accumulated for the person to be authenticated, and to use the reference habit characteristic data as data to be compared with the habit characteristic data in a predetermined authentication. Thus, the authentication management device can derive and use reference habit characteristic data suitable for the person to be authenticated.

[0213] (Item 14) An authentication method for authenticating a person, comprising: deriving biometric feature data relating to biometric features of a first person who possesses a terminal device based on first detection data detected by a sensor; performing a first authentication of the person based on the biometric feature data; if the first authentication is successful, sequentially deriving habit feature data relating to habit features of the first person based on second detection data detected by the sensor; and sequentially performing a second authentication of the first person based on the sequentially derived habit feature data.

[0214] This provides the same effect as item 1.

[0215] (Item 15) An authentication management method for managing authentication of a person, comprising: classifying a plurality of habit feature data related to habit features of the person stored in a memory so that habit feature data having similar feature amounts belong to the same feature group; and determining, based on a classification result of the habit feature data, reference habit feature data that is a standard related to the habit features of the person and is used for a predetermined authentication.

[0216] This provides the same effect as item 13.

[0217] (Item 16) A program for causing a computer to execute the authentication method according to Item 14.

[0218] This provides the same effect as item 14.

[0219] (Item 17) A program for causing a computer to execute the authentication management method according to Item 15.

[0220] This provides the same effect as item 15.

[0221] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the invention.

[0222] In addition, the above embodiment may also be applicable to a program that realizes the functions of the authentication method, which is supplied to a computer (e.g., terminal 100) via a network or various storage media, and which is read and executed by the processor of this computer, as well as to the storage media on which this program is stored.

[0223] In addition, the above embodiment may also be applicable to a program that realizes the functions of the authentication management method, which is supplied to a computer (e.g., the first authentication management device 200) via a network or various storage media, and which is read and executed by the processor of this computer, as well as to the storage media on which this program is stored.

[0224] This disclosure is based on a Japanese patent application (Patent Application No. 2023-186555) filed on October 31, 2023, the contents of which are incorporated by reference into this disclosure.

[0225] The present disclosure is useful for a terminal device, an authentication system, an authentication management device, an authentication method, an authentication management method, and the like that can continuously authenticate legitimate users while maintaining security.

[0226] 5 Authentication system 100 Terminal 110 Processor 120 Memory 130 Communication device 140 Camera 150 Sensor 160 Display device 200 First authentication management device 210 Processor 220 Memory 230 Communication device 300 Second authentication management device 310 Processor 320 Memory 330 Communication device 340 Camera 350 Sensor 360 Display device HM Person SP Shop

Claims

1. A terminal device for authenticating a person, comprising a processor and a sensor, wherein the processor derives biometric feature data relating to the biometric features of a first person who holds the terminal device based on first detection data detected by the sensor, performs a first authentication of the person based on the biometric feature data, and if the first authentication is successful, sequentially derives habit feature data relating to the habit features of the first person based on second detection data detected by the sensor, and sequentially performs a second authentication of the first person based on the sequentially derived habit feature data.

2. The terminal device described in claim 1, wherein the processor derives the habit characteristic data for each habit item into which the person's habits are classified, calculates a matching score for each habit item based on the habit characteristic data in the second authentication, and determines that the first person is not a legitimate owner of the terminal device if the matching score for each habit item does not satisfy a threshold condition related to a predetermined threshold.

3. The terminal device according to claim 2, wherein the threshold condition includes the existence of the matching score being equal to or greater than a predetermined first threshold.

4. The terminal device according to claim 2, wherein the threshold condition includes that the matching score, being equal to or greater than a second threshold that is smaller than a predetermined first threshold, satisfies a predetermined condition.

5. The terminal device described in claim 4, wherein the threshold condition includes that the number of the matching scores that are equal to or greater than the second threshold is a predetermined number or more, or that the ratio of the number of the matching scores that are equal to or greater than the second threshold to the total number of the matching scores is a predetermined ratio or more.

6. The terminal device according to claim 1, wherein the terminal device derives a reliability of the result of the second authentication based on the time elapsed since the previous processing of the second authentication among the second authentications executed sequentially or the number of times the second authentication has failed, and if the reliability is equal to or less than a third threshold, determines that it is necessary to execute the first authentication.

7. An authentication system comprising a terminal device that authenticates a person and an authentication management device that manages the authentication of the person, wherein the terminal device derives biometric feature data relating to biometric features of a first person who holds the terminal device based on first detection data detected by a sensor, performs a first authentication of the person based on the biometric feature data, and if the first authentication is successful, sequentially derives habit feature data relating to habit features of the first person based on second detection data detected by the sensor, and sequentially performs a second authentication of the first person based on the sequentially derived habit feature data, and if it is determined by the second authentication that the first person is not a legitimate owner of the terminal device, transmits the habit feature data to the authentication management device, and the authentication management device receives the habit feature data from the terminal device, and performs a third authentication of the first person based on the habit feature data.

8. The authentication system described in claim 7, wherein the authentication management device determines that the first person, who was determined to not be the legitimate owner of the terminal device, was the legitimate owner of the terminal device if the matching score obtained in the third authentication satisfies a threshold condition related to a predetermined threshold.

9. The authentication system described in claim 8, wherein the terminal device holds one reference habit feature data having a predetermined feature pattern which is a standard regarding the habit features of the first person, calculates the matching score based on the habit feature data and the reference habit feature data, and the authentication management device holds a plurality of the reference habit feature data having different feature patterns, calculates a plurality of matching scores based on the habit feature data and each of the plurality of reference habit feature data, and determines the maximum matching score of the plurality of matching scores as the matching score for the third authentication.

10. The authentication system according to claim 9, wherein the authentication management device sequentially acquires and stores the habit feature data from the terminal device, classifies the stored plurality of habit feature data so that habit feature data having similar features belong to the same feature group, determines the reference habit feature data to be used in the second authentication based on the classification result of the habit feature data, and transmits the determined reference habit feature data to the terminal device, and the terminal device receives and stores the reference habit feature data.

11. The authentication system according to claim 10, wherein the authentication management device sequentially acquires and retains the habit feature data from the terminal device for each habit item into which the person's habits are classified, classifies the retained multiple habit feature data for each habit item so that habit feature data having similar features belong to the same feature group, determines the reference habit feature data to be used in the second authentication based on the classification result of the habit feature data for each habit item, and transmits the determined reference habit feature data for each habit item to the terminal device, and the terminal device receives and retains the reference habit feature data for each habit item.

12. The authentication system according to claim 10, wherein the authentication management device determines the reference habit characteristic data based on the habit characteristic data belonging to the feature group to which the largest number of habit characteristic data belong among the classified feature groups.

13. An authentication management device that manages the authentication of a person, comprising a processor and a memory, wherein the memory holds a plurality of habit feature data relating to habit characteristics of the person, and the processor classifies the plurality of habit feature data held in the memory so that habit feature data having similar features belong to the same feature group, and determines, based on a result of the classification of the habit feature data, reference habit feature data that is a standard relating to the habit characteristics of the person and is used for a specified authentication.

14. An authentication method for authenticating a person, comprising: deriving biometric feature data relating to biometric features of a first person who possesses a terminal device based on first detection data detected by a sensor; performing a first authentication of the person based on the biometric feature data; if the first authentication is successful, sequentially deriving habit feature data relating to habit features of the first person based on second detection data detected by the sensor; and sequentially performing a second authentication of the first person based on the sequentially derived habit feature data.

15. An authentication management method for managing the authentication of a person, comprising: classifying a plurality of habit feature data relating to habit features of the person stored in a memory such that habit feature data having similar feature amounts belong to the same feature group; and determining, based on a result of the classification of the habit feature data, reference habit feature data which is a standard relating to the habit features of the person and is used for a specified authentication.

16. A program for causing a computer to execute the authentication method according to claim 14.

17. A program for causing a computer to execute the authentication management method according to claim 15.

Images