System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting facial and biometric data; means for transmitting the collected data to a server; means for analyzing the collected data at the server to learn a personal authentication model; means for storing the personal authentication model in a database and associating personal information and account information; means for authenticating a user based on facial and biometric data; means for processing transactions for the user's account if authentication is successful; and means for notifying the user of transaction results.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In everyday life, many people sometimes forget their wallets when they go out. Furthermore, in emergencies such as natural disasters, earthquakes, and fires, bankbooks and cash cards can be lost or vanished. In such situations, people often become unable to withdraw money while the reissue process is in progress, which increases their mental anxiety. To solve this problem, a system is needed that allows people to safely and quickly withdraw money from their accounts even without their cash cards, but currently, such services are insufficient. [Means for solving the problem]

[0005] The present invention provides a cash cardless system that utilizes facial recognition and biometric authentication. Specifically, it includes a means for collecting facial recognition data and biometric authentication data and transmitting them to a server. The server then analyzes the collected data and uses generative AI to train a personal authentication model. This personal authentication model is stored in a database and associated with the user's personal information and account information. Furthermore, the system authenticates the user based on the facial recognition and biometric authentication data, and if authentication is successful, processes transactions to the user's account and notifies the user of the results. This system enables safe withdrawals without a cash card, improving user convenience and peace of mind.

[0006] 1. "Facial Recognition Data" means data used to identify an individual using image information of a user's face.

[0007] 2. "Biometric Data" means data that identifies an individual using a user's physical characteristics (e.g., fingerprint, iris, etc.).

[0008] 3. "Collection instrument" means any device or software used to obtain facial recognition and biometric data.

[0009] 4. "Transmitting means" means the communications device or protocol used to send the collected data to the server.

[0010] 5. "Analyzing means" means software and algorithms used to analyze the acquired data and authenticate the user.

[0011] 6. “Generative AI” refers to systems that use artificial intelligence techniques to learn from data and generate patterns and models.

[0012] 7. “Personal Authentication Model” means a data model trained to recognize and identify users.

[0013] 8. "Database" means an information management system for storing collected data and trained personal authentication models.

[0014] 9. "Personal Information" means information that can identify a user, such as their name, address, or telephone number.

[0015] 10. "Account Information" refers to information such as bank account number, balance, and transaction history.

[0016] 11. "Means of authentication" means a device or software that uses data provided by a user to determine whether that person is the correct user.

[0017] 12. "Transaction processing instrument" means a device or software that withdraws or transfers money based on a user's request after successful authentication.

[0018] 13. "Means for notifying" means a display device, audio output device, or notification system for informing the user of the outcome of the transaction. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] System configuration

[0041] The cash cardless system of this invention is implemented in bank ATMs and smartphone apps. The system mainly consists of the following components:

[0042] 1. Terminal

[0043] An input device (such as a camera, fingerprint scanner, or iris scanner) that allows users to enter facial and biometric data.

[0044] A display or interface that guides the user through the authentication process.

[0045] A communication interface for sending data to the server.

[0046] 2. Server

[0047] Generative AI to analyze received data and generate a personal authentication model.

[0048] A database that manages personal and account information.

[0049] Software and algorithms for authentication processes and transaction processing.

[0050] Program processing procedure

[0051] The program processing of the system will be described in detail below.

[0052] User information collection and registration

[0053] 1. User: Visits a bank ATM or smartphone app and registers facial recognition and biometric data.

[0054] 2. Terminal: Have the user take a photo of their face and scan their fingerprint and iris.

[0055] 3. Terminal: The collected data is sent to the server via secure communication.

[0056] 4. Server: Analyzes the received data and uses generative AI to train a personal authentication model.

[0057] 5. Server: Stores the learned personal authentication model in a database and associates the user's personal information with their account information.

[0058] Authentication Process

[0059] 1. User: Select "Facial Recognition Transaction" from the ATM or app.

[0060] 2. Terminal: Display the face recognition and biometric authentication input interface.

[0061] 3. User: Faces the device camera and scans their fingerprint.

[0062] 4. Terminal: Sends collected data to the server.

[0063] 5. Server: Authenticates the user using the generative AI.

[0064] 6. Server: If authentication is successful, it notifies the terminal of the result.

[0065] Transaction Processing

[0066] 1. User: After successful authentication, enter the withdrawal amount.

[0067] 2. Terminal: Sends the entered amount information to the server.

[0068] 3. Server: Determines whether withdrawal is possible based on the user's account information.

[0069] 4. Server: If withdrawal is possible, withdraw the specified amount.

[0070] 5. Server: Updates the transaction record and sends a success notification to the device.

[0071] Result notification

[0072] 1. Terminal: displays the transaction results to the user.

[0073] 2. Terminal: In the case of an ATM, it dispenses cash and prints a receipt.

[0074] 3. User: Check the transaction result and receive the cash and receipt.

[0075] Specific examples

[0076] Example 1: You accidentally forget your wallet

[0077] 1. Collecting and registering user information: Register facial recognition and fingerprint data at the ATM in advance.

[0078] 2. Authentication Process:

[0079] User: Goes to a bank ATM and selects facial recognition transaction.

[0080] On the device: Have the user face the camera and scan their fingerprint.

[0081] Server: Analyzes the data and authenticates the user.

[0082] 3. Transaction Processing:

[0083] User: After successful authentication, enter the amount from the withdrawal menu.

[0084] Terminal: Sends the amount to the server.

[0085] Server: Determines whether withdrawals are possible and processes transactions.

[0086] 4. Result notification:

[0087] Terminal: Shows successful transaction and withdraws cash.

[0088] User: Take cash and receipt.

[0089] Example 2: When you lose your bankbook or cash card during a disaster

[0090] 1. User information collection and registration: Register facial recognition and biometric data at the emergency assistance center.

[0091] 2. Authentication Process:

[0092] User: Selects facial recognition transaction on the shelter terminal.

[0093] Device: Have your face face the camera and scan your fingerprint.

[0094] Server: Analyzes the data and authenticates the user.

[0095] 3. Transaction Processing:

[0096] User: After successful authentication, enter the withdrawal amount.

[0097] Terminal: Sends the amount to the server.

[0098] Server: Determines whether withdrawal is possible and processes the transaction.

[0099] 4. Result notification:

[0100] Terminal: Shows successful transaction and withdraws cash.

[0101] User: Take the cash.

[0102] The system of the present invention allows users to safely withdraw money without a cash card, improving convenience and peace of mind for users. In particular, in the event of a disaster, it allows users to quickly obtain cash, which has the effect of reducing the mental burden on victims.

[0103] The processing flow will be explained below.

[0104] Specific processing steps of the program

[0105] User information collection and registration

[0106] Step 1:

[0107] User: Opens the bank's ATM or smartphone app and opens the screen to register facial recognition and biometric data.

[0108] Step 2:

[0109] On the device: Displays an interface for taking a photo of the user's face and asks the user to face the camera.

[0110] Step 3:

[0111] User: Follow the instructions on the device to take a photo of your face in front of the camera.

[0112] Step 4:

[0113] Device: Sends the captured photo to the server.

[0114] Step 5:

[0115] On the device: Display an interface to collect fingerprint data using a fingerprint scanner and have the user scan their fingerprint.

[0116] Step 6:

[0117] User: Places finger on fingerprint scanner to provide fingerprint data.

[0118] Step 7:

[0119] Device: Sends collected fingerprint data to the server.

[0120] Step 8:

[0121] Terminal: Displays an interface for collecting iris data and prompts the user to scan their iris.

[0122] Step 9:

[0123] User: Provides iris data by pointing their iris at the device camera.

[0124] Step 10:

[0125] Terminal: Sends collected iris data to the server.

[0126] Step 11:

[0127] Server: Receives facial photos, fingerprint data, and iris data, and trains a personal authentication model using generative AI.

[0128] Step 12:

[0129] Server: The trained personal authentication model is stored in a database and associated with the user's personal and account information.

[0130] Authentication Process

[0131] Step 13:

[0132] User: Selects "Facial Recognition Transaction" from the ATM or smartphone app.

[0133] Step 14:

[0134] Terminal: Display the face recognition and biometric authentication input interface.

[0135] Step 15:

[0136] User: Face the camera and prepare to scan your fingerprint.

[0137] Step 16:

[0138] Terminal: Collects facial photo and fingerprint data and sends them to the server.

[0139] Step 17:

[0140] Server: Analyzes the received facial photo and fingerprint data and performs authentication using the generated AI.

[0141] Step 18:

[0142] Server: Notifies the terminal of the authentication result.

[0143] Transaction Processing

[0144] Step 19:

[0145] User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[0146] Step 20:

[0147] Terminal: Sends the amount data entered by the user to the server.

[0148] Step 21:

[0149] Server: Checks the user's account information and determines whether a withdrawal is possible.

[0150] Step 22:

[0151] Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[0152] Step 23:

[0153] Server: Sends the transaction results to the terminal.

[0154] Result notification

[0155] Step 24:

[0156] Terminal: Displays the transaction results to the user.

[0157] Step 25:

[0158] Terminal: In the case of an ATM, it dispenses cash and prints receipts.

[0159] Step 26:

[0160] User: Checks and receives cash and receipts.

[0161] This series of processing steps allows users to withdraw money safely and quickly using facial and biometric authentication, without using a cash card.

[0162] Example 1

[0163] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0164] In recent years, the loss and theft of cash cards and bankbooks has been increasing, and the risk of fraudulent transactions has also increased. In particular, in emergencies such as natural disasters and fires, if a bankbook or cash card is lost or lost, it can be difficult for victims to quickly secure cash. Furthermore, conventional authentication systems often use passwords or PIN codes, which still pose security risks. Therefore, to solve these issues, a system that can safely and quickly authenticate individuals and perform financial transactions is needed.

[0165] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0166] In this invention, the server includes a means for collecting facial recognition data and biometric authentication data, a means for training a personal authentication model using a generative AI, and a means for storing the personal authentication model in a database and associating personal information and financial information, thereby enabling safe and fast user authentication and financial transactions.

[0167] "Facial recognition data" is data that digitally represents a user's facial features.

[0168] "Biometric data" means a digital representation of a user's biological characteristics, such as a fingerprint or iris.

[0169] A "server" is a computer system that stores and processes data over a network.

[0170] "Generative AI" is a program that uses artificial intelligence technology to perform a specific task, and in this invention is used particularly for training a personal authentication model.

[0171] A "personal authentication model" is a model for uniquely identifying a user using collected facial recognition data and biometric authentication data.

[0172] A "database" is a system for efficiently storing and managing large amounts of data.

[0173] "Personal Information" means identifying information, such as name and address, associated with a user.

[0174] "Financial information" refers to information related to a user's account information, transaction history, etc.

[0175] "User authentication" is the process of verifying a user's identity based on collected authentication data.

[0176] A "financial transaction" refers to a monetary transaction, such as a withdrawal from or transfer to a user's account.

[0177] "Trading results" refers to information obtained as a result of a financial transaction, including the success or failure of the transaction.

[0178] The cash cardless system of this invention is implemented in bank ATMs and smartphone apps. The system is mainly composed of a terminal and a server.

[0179] System configuration

[0180] Terminal

[0181] The terminal is equipped with an input device for users to enter facial and biometric authentication data, such as a camera, fingerprint scanner, or iris scanner, and provides a display or interface that guides the user through the authentication process. The terminal also transmits the collected data to a server through a secure communication interface.

[0182] Camera: A device used to take a photo of the user's face.

[0183] Fingerprint scanner: A device used to scan a user's fingerprint.

[0184] Iris scanner: A device used to scan a user's iris.

[0185] Display: A device used to display authentication procedures and transaction details.

[0186] Communication interface: A secure protocol (e.g. HTTPS) for sending data to the server.

[0187] server

[0188] The server analyzes the received data and generates a personal authentication model using generative AI. Specifically, it uses AI frameworks such as TENSORFLOW (registered trademark) and PyTorch. It also has a database that manages personal and financial information. This database stores the trained personal authentication model and also contains software and algorithms for processing user transactions.

[0189] Generative AI: Train a person authentication model using TensorFlow and PyTorch.

[0190] Database: Personal and financial information is managed using database systems such as MySQL (registered trademark) and PostgreSQL.

[0191] Transaction Processing Algorithm: Software used to process users' financial transactions.

[0192] Specific examples

[0193] Example 1: You accidentally forget your wallet

[0194] 1. Collecting and registering user information: Register facial recognition and fingerprint data in advance at the bank ATM.

[0195] User: Accesses the ATM and registers facial recognition and fingerprint data.

[0196] Device: Takes a photo of the user's face and scans their fingerprint, sending the data to a server.

[0197] Server: Analyzes the data, uses generative AI to train a personal authentication model, and stores it in a database.

[0198] 2. Authentication Process:

[0199] User: Go to a bank ATM and select "Facial Recognition Transaction."

[0200] Device: Have your face face the camera and scan your fingerprint.

[0201] Server: Analyzes the collected data and authenticates users.

[0202] 3. Transaction Processing:

[0203] User: After successful authentication, enter the desired amount in the withdrawal menu.

[0204] Terminal: Sends the entered amount information to the server.

[0205] Server: Determines whether withdrawal is possible based on the user's account information and processes the transaction.

[0206] 4. Result notification:

[0207] Terminal: Shows successful transaction and withdraws cash.

[0208] User: Take cash and receipt.

[0209] Example 2: When you lose your bankbook or cash card during a disaster

[0210] 1. User information collection and registration: Register facial recognition and biometric data at the emergency assistance center.

[0211] User: Register facial recognition and biometric data on a terminal at the evacuation center.

[0212] Device: Face the camera, scan your fingerprint, and send the data to the server.

[0213] Server: Uses generative AI to train a personal authentication model and saves it in a database.

[0214] 2. Authentication Process:

[0215] User: Select "Facial Recognition Transaction" on the shelter terminal.

[0216] Device: Have your face face the camera and scan your fingerprint.

[0217] Server: Analyzes the data and authenticates the user.

[0218] 3. Transaction Processing:

[0219] User: After successful authentication, enter the withdrawal amount.

[0220] Terminal: Sends the amount to the server.

[0221] Server: Determines whether withdrawals are possible and processes transactions.

[0222] 4. Result notification:

[0223] Terminal: Shows successful transaction and withdraws cash.

[0224] User: Take the cash.

[0225] Prompt Sentence Examples

[0226] "Please explain your generative AI model for user authentication. Explain the process steps for analyzing the data using facial and fingerprint recognition."

[0227] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0228] Program processing flow

[0229] Step 1:

[0230] User: Accesses the bank's ATM or smartphone app and begins enrolling facial recognition and biometric data. Selects "New Enrollment" from the app's home screen and begins enrolling facial recognition and fingerprint data.

[0231] Input: Facial photo and fingerprint data

[0232] Output: Registration ready message

[0233] Specific steps: Tap "New Registration" from the app menu and follow the instructions to enter your face photo and fingerprint data.

[0234] Step 2:

[0235] Terminal: Takes a photo of the user's face and scans their fingerprint. Uses the ATM's camera to take a photo of the face and a fingerprint scanner to read the fingerprint.

[0236] Input: User's facial photo and fingerprint data

[0237] Output: Collected data

[0238] Specific actions: The app displays instructions to the user, such as "Please face the camera" and "Please place your finger on the fingerprint scanner."

[0239] Step 3:

[0240] Device: Send collected data to the server using a secure communication protocol (e.g. HTTPS).

[0241] Input: Collected data

[0242] Output: Data transmission completed message

[0243] Specific actions: Encrypt the data and send it to the server.

[0244] Step 4:

[0245] Server: Analyzes the received data and uses generative AI (using TensorFlow or PyTorch) to train a personal authentication model.

[0246] Input: Collected data

[0247] Output: Personal authentication model

[0248] Specific operations: Analyze received facial recognition and biometric data to train an AI model.

[0249] Step 5:

[0250] Server: The trained personal authentication model is stored in a database (MySQL or PostgreSQL) and associated with the user's personal and financial information.

[0251] Input: Personal authentication model and user information

[0252] Output: Updated database

[0253] Specific operation: Stores the authentication model in the database, linked to the user ID, and associates personal and financial information.

[0254] Step 6:

[0255] User: Accesses the bank's ATM or app and selects "Facial Recognition Transactions." For example, selects "Facial Recognition Transactions" from the ATM menu screen.

[0256] Input: Operation selection

[0257] Output: Authentication interface display

[0258] Specific operation: Select "Facial Recognition Transactions" on the ATM menu screen.

[0259] Step 7:

[0260] Terminal: Display the input interface for face recognition and biometric authentication. Display the camera image for face recognition and the fingerprint scanner image.

[0261] Input: Menu selection information

[0262] Output: Authentication interface

[0263] Specific actions: "Please face the camera" and "Please place your finger on the fingerprint scanner" are displayed.

[0264] Step 8:

[0265] User: Faces the device camera and scans a fingerprint, for example, by facing the camera lens and pressing a finger against the scanner.

[0266] Input: Facial photo and fingerprint data

[0267] Output: Authentication data

[0268] Specific actions: Stand in front of the camera for facial recognition and place your finger on the fingerprint scanner.

[0269] Step 9:

[0270] Terminal: Sends collected data to the server. The data is encrypted and sent using a secure communication protocol.

[0271] Input: Authentication data

[0272] Output: Data transmission completed message

[0273] Specific operation: Encrypts collected data and sends it to a server.

[0274] Step 10:

[0275] Server: Authenticates the user using generative AI, matching the stored authentication model with newly acquired data.

[0276] Input: Authentication data and personal authentication model

[0277] Output: Authentication result

[0278] Specific operation: Analyzes the received data and compares it with the stored AI model to evaluate the degree of match.

[0279] Step 11:

[0280] Server: If authentication is successful, it notifies the terminal of the result, encrypts the authentication result, and sends it to the terminal.

[0281] Input: Authentication result

[0282] Output: Authentication success message

[0283] Specific operation: Sends a message of successful authentication to the terminal.

[0284] Step 12:

[0285] User: After successful authentication, enter the amount to withdraw. Enter the desired amount on the ATM's input screen.

[0286] Input: Withdrawal amount

[0287] Output: Amount information entered

[0288] Specific operation: Enter the amount in the amount input field displayed on the ATM screen.

[0289] Step 13:

[0290] Terminal: Sends the entered amount information to the server. The entered data is encrypted and sent securely.

[0291] Input: Amount information

[0292] Output: Data transmission completed message

[0293] Specific operation: Sends the entered amount information to the server.

[0294] Step 14:

[0295] Server: Determines whether withdrawal is possible based on the user's account information. Checks the account balance and compares it with the requested amount.

[0296] Input: Amount information and account information

[0297] Output: Withdrawal decision result

[0298] Specific operation: Check the account balance and determine whether a withdrawal is possible.

[0299] Step 15:

[0300] Server: If withdrawal is possible, withdraw the specified amount. The server processes the transaction and updates the account balance.

[0301] Input: Withdrawal decision result

[0302] Output: Transaction processing results

[0303] Specific actions: Process the transaction and withdraw the specified amount.

[0304] Step 16:

[0305] Server: Updates the transaction record and sends a success notification to the terminal. Adds the transaction record to the database and sends a success message to the terminal.

[0306] Input: Transaction processing result

[0307] Output: Success message

[0308] Specific operations: Save the transaction record in the database and send a notification of successful transaction to the terminal.

[0309] Step 17:

[0310] Terminal: Display the transaction result to the user. Display a successful transaction message on the ATM screen.

[0311] Input: Success message

[0312] Output: Trade result display

[0313] Specific behavior: Display a message indicating the transaction was successful.

[0314] Step 18:

[0315] Terminal: In the case of an ATM, it dispenses cash and prints a receipt. The ATM's cash dispenser dispenses the specified amount, and the receipt printer prints the transaction details.

[0316] Input: Success message

[0317] Output: Cash and receipt

[0318] Specific actions: Dispense cash and print a receipt.

[0319] Step 19:

[0320] User: Check the transaction result and receive the cash and receipt. Take the cash and receipt that come out of the ATM.

[0321] Input: Cash and receipt

[0322] Output: Transaction completed

[0323] Specific actions: Check the transaction results and take the cash and receipt.

[0324] (Application example 1)

[0325] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0326] In conventional financial transactions using cash cards or bankbooks, if these physical media are lost, transactions often become difficult to complete. Furthermore, in emergencies such as natural disasters, earthquakes, or fires, cards and bankbooks can be damaged or lost, making it difficult to quickly obtain necessary funds. Furthermore, from a security standpoint, there is a risk of fraudulent use if cards or bankbooks are stolen. There is a need for a system that can solve these issues and enable users to conduct financial transactions quickly and with peace of mind.

[0327] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0328] In this invention, the server includes means for collecting facial authentication data and biometric authentication data, means for transmitting the collected data to the server, means for analyzing the collected data in the server and training a personal authentication model, means for saving the personal authentication model in a database and associating personal information and account information, means for authenticating a user based on the facial authentication and biometric authentication data, means for processing a transaction for the user's account if authentication is successful, means for notifying the user of the transaction result, means for confirming and approving the payment amount, means for transmitting payment amount information to the server after authentication is successful, and means for saving the transaction history of the payment amount so that the user can confirm it. This eliminates the need for physical cash cards or passbooks, allowing transactions to be carried out quickly, especially in emergencies such as natural disasters, earthquakes, and fires, while also improving security.

[0329] "Facial recognition data" is data used to analyze image data of a user's face and identify the individual.

[0330] "Biometric data" is data used to identify an individual using biometric characteristics such as fingerprints or irises.

[0331] "Means of collection" refers to devices and programs for acquiring facial recognition data and biometric authentication data.

[0332] "Transmitting means" refers to a device or program that has the function of transmitting collected data to a server.

[0333] The "analyzing means" refers to a device or program for processing the received data and generating a personal authentication model.

[0334] "Means for training" refers to devices or programs for training a personal authentication model using generative AI.

[0335] A "personal authentication model" is an algorithm or program for identifying individuals based on collected facial recognition data and biometric data.

[0336] "Database" means a storage device for storing and managing personal authentication models, personal information, and account information.

[0337] "Means for authentication" refers to devices or programs that verify a user's identity based on facial recognition and biometric data.

[0338] A "means for processing transactions" is a device or program that, after successful authentication, executes a financial transaction against a user's account.

[0339] "Means of notification" refers to devices or programs used to notify users of transaction results.

[0340] "Means for confirming and approving the payment amount" refers to a device or program that allows the user to confirm and approve the payment amount.

[0341] The "means for transmitting payment amount information" refers to a device or program for transmitting information on the payment amount to the server after successful authentication.

[0342] "Means for storing transaction history" refers to a device or program that records the results of each transaction and allows users to check them.

[0343] "Generative AI" is an artificial intelligence technology that analyzes data and trains a personal authentication model.

[0344] MODE FOR CARRYING OUT THE INVENTION

[0345] The facial recognition electronic payment system of the present invention uses facial recognition data and biometric authentication data to realize safe and fast financial transactions without the need for physical cash cards or bankbooks.

[0346] System configuration

[0347] The system mainly consists of the following components:

[0348] 1. User device (smartphone)

[0349] 2. Server

[0350] 3. Database

[0351] Hardware and software used

[0352] Smartphone: Camera for facial recognition, fingerprint scanner for fingerprint recognition

[0353] Facial recognition software: OpenCV

[0354] Generative AI model: TensorFlow

[0355] Communication interface: Secure HTTP communication

[0356] Database: SQL-based database

[0357] Program processing overview and specific examples

[0358] User information collection and registration

[0359] Users use their smartphone's camera and fingerprint scanner to collect facial and biometric data, which is then transmitted to a server through a secure communication interface.

[0360] The server analyzes the received data and uses the generative AI model to create a personal authentication model, which is stored in a database and associated with the user's personal and account information.

[0361] Authentication Process

[0362] When making a payment, users open a smartphone app and enter their facial and biometric data, which includes pointing their face at the camera and placing their fingerprint on a scanner. The smartphone then sends this data to a server, which uses a generative AI model to authenticate the user.

[0363] Payment Processing

[0364] If authentication is successful, the user confirms and approves the payment amount. The smartphone sends the confirmed payment amount to the server. The server processes the payment based on the user's account information and saves the transaction results in a database.

[0365] Result notification

[0366] The results of the transaction are notified to the user, who can check the results via a smartphone app.

[0367] Specific application

[0368] Example 1: Use in everyday life

[0369] When shopping at a supermarket, a user uses a facial recognition electronic payment app. After shopping is complete, the user opens the app at the cash register, performs facial and biometric authentication, and approves the payment. The server processes the authentication and payment, and the transaction result is notified to the user within a few seconds.

[0370] Example 2: Use during a disaster

[0371] Users who lose their bankbooks or cash cards during a disaster can use terminals installed at evacuation shelters to quickly withdraw cash using facial and biometric authentication, enabling safe and fast transactions even under certain circumstances.

[0372] Prompt Sentence Examples

[0373] "Please show us how to obtain a user's facial image and fingerprint data and perform secure authentication."

[0374] "If bankbooks and cash cards become unusable during a natural disaster, please explain how to use facial recognition as an alternative method for transactions."

[0375] As a result, physical cash cards and bankbooks are no longer necessary, allowing for quick transactions, especially in emergencies such as natural disasters, earthquakes, and fires, while also improving security.

[0376] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0377] Step 1:

[0378] The user downloads the facial recognition electronic payment app to their smartphone and launches it. The user begins facial and fingerprint authentication on the new registration screen (input: user's face and fingerprint, output: collected facial recognition data and biometric authentication data).

[0379] Step 2:

[0380] The device uses the smartphone's camera and fingerprint scanner to collect the user's facial authentication data and fingerprint data (data processing: acquisition of facial data and fingerprint data, output: collected facial authentication data and biometric authentication data).

[0381] Step 3:

[0382] The device encrypts the collected data and sends it to the server using a secure communication protocol (e.g., HTTPS) (input: collected facial recognition data and biometric authentication data, output: encrypted data sent to the server).

[0383] Step 4:

[0384] The server analyzes the received facial recognition data and biometric authentication data and creates a personal authentication model using a generative AI model (input: encrypted and transmitted facial recognition data and biometric authentication data; data calculation: training of the model using the generative AI; output: personal authentication model).

[0385] Step 5:

[0386] The server stores the personal authentication model in a database and associates it with the user's personal information and payment information (input: personal authentication model, personal information and payment information; output: personal authentication model and related information stored in the database).

[0387] Step 6:

[0388] When making a payment, the user launches the app and proceeds to the face authentication and fingerprint authentication screen (input: user operation, output: face authentication and fingerprint authentication screen).

[0389] Step 7:

[0390] The device collects the facial recognition data and biometric authentication data again and sends them to the server (input: user's face and fingerprint, data processing: recollected facial recognition data and biometric authentication data, output: data sent to the server).

[0391] Step 8:

[0392] The server compares the received data with the personal authentication model again and authenticates the user (input: recollected data, personal authentication model, data calculation: model comparison, output: authentication result).

[0393] Step 9:

[0394] If the authentication is successful, the user confirms and approves the payment amount, and this information is also sent to the server (Input: User approves the amount, Output: Payment amount information).

[0395] Step 10:

[0396] The server processes the transaction based on the user's account information and payment amount information, and records the payment results (input: payment amount information, account information, data calculation: transaction processing and recording, output: transaction results and recording).

[0397] Step 11:

[0398] The server notifies the terminal of the transaction result, and the terminal displays it to the user (input: transaction result, output: notification to user).

[0399] The above is the flow of processing steps of this system.

[0400] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0401] System configuration

[0402] The cash cardless system incorporating the emotion engine of the present invention consists of the following elements:

[0403] 1. Terminal

[0404] An input device (such as a camera, fingerprint scanner, or iris scanner) that allows users to enter facial and biometric data.

[0405] An emotion engine (including a camera and microphone) to recognize user emotions.

[0406] Displays and interfaces that show the authentication process and emotional state to the user.

[0407] A communication interface for sending data to the server.

[0408] 2. Server

[0409] A generative AI that analyzes received data (face photos, fingerprint data, iris data, emotion data) and generates a personal authentication model and emotion model.

[0410] A database that manages personal and account information.

[0411] Software and algorithms for authentication processes, emotion recognition, and transaction processing.

[0412] Program processing procedure

[0413] The program processing of the system will be described in detail below.

[0414] User information collection and registration

[0415] 1. User: Visits a bank ATM or smartphone app and registers facial recognition and biometric data.

[0416] 2. Terminal: Have the user take a photo of their face and scan their fingerprint and iris.

[0417] 3. Device: Activate the emotion engine and collect the user's facial expressions and tone of voice.

[0418] 4. Terminal: Sends collected facial recognition, biometrics, and emotion data to the server.

[0419] 5. Server: Analyzes the received data and uses generative AI to train a personal authentication model and emotion model.

[0420] 6. Server: Stores the trained model in a database and associates the user's personal and account information.

[0421] Authentication Process

[0422] 1. User: Select "Facial Recognition Transaction" from the ATM or smartphone app.

[0423] 2. Terminal: Display the face recognition and biometric authentication input interface.

[0424] 3. User: Faces the camera and prepares to scan his / her fingerprint.

[0425] 4. Terminal: Collects facial photos, fingerprint data, and emotion data and sends them to the server.

[0426] 5. Server: Analyzes the received data and performs authentication and emotion recognition using generative AI.

[0427] 6. Server: Notifies the device of the authentication result and emotional state analysis result.

[0428] Transaction Processing

[0429] 1. User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[0430] 2. Terminal: The emotion engine monitors the user's emotional state and sends the results to the server.

[0431] 3. Server: Determines whether withdrawal is possible based on the user's emotional state and account information.

[0432] 4. Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[0433] 5. Server: Sends trading results and notification methods based on sentiment analysis to the terminal.

[0434] Result notification

[0435] 1. Terminal: displays notifications to users based on trading results and emotional state.

[0436] 2. Terminal: In the case of an ATM, dispenses cash and provides appropriate notifications (audio, text, visual) as needed.

[0437] 3. User: Check the transaction result and receive the cash and receipt.

[0438] Specific examples

[0439] Example 1: You accidentally forget your wallet

[0440] 1. Collecting and registering user information: Facial recognition and fingerprint data are registered in advance at the ATM, and emotion data is also collected using the emotion engine.

[0441] 2. Authentication Process:

[0442] User: Goes to a bank ATM and selects facial recognition transaction.

[0443] Device: The device asks the user to face the camera, scans their fingerprint, and collects emotional data.

[0444] Server: Analyzes data and performs user authentication and emotion recognition.

[0445] 3. Transaction Processing:

[0446] User: After successful authentication, enter the amount from the withdrawal menu.

[0447] Terminal: Sends the amount and user emotion data to the server.

[0448] Server: Determines whether withdrawals are possible and processes transactions.

[0449] 4. Result notification:

[0450] Terminal: Shows successful transaction and withdraws cash.

[0451] User: Take cash and receipt.

[0452] Example 2: When you lose your bankbook or cash card during a disaster

[0453] 1. Collecting and registering user information: The emergency support center will register facial recognition and biometric data, and also collect emotional data from the emotion engine.

[0454] 2. Authentication Process:

[0455] User: Selects facial recognition transaction on the shelter terminal.

[0456] Device: The device asks the user to face the camera, scans their fingerprint, and collects emotional data.

[0457] Server: Analyzes data and performs user authentication and emotion recognition.

[0458] 3. Transaction Processing:

[0459] User: After successful authentication, enter the withdrawal amount.

[0460] Terminal: Sends the amount and user emotion data to the server.

[0461] Server: Determines whether withdrawal is possible and processes the transaction.

[0462] 4. Result notification:

[0463] Terminal: Displays notifications for successful transactions and emotional states, and withdraws cash.

[0464] User: Take the cash.

[0465] By using the system of this invention, users can safely and quickly withdraw money without using a cash card. Furthermore, by combining it with an emotion engine, optimal transactions can be performed according to the user's emotional state, enhancing convenience and peace of mind. In particular, in the event of a disaster, this system allows users to obtain cash quickly and appropriately, which has the effect of reducing the mental burden on victims.

[0466] The processing flow will be explained below.

[0467] Specific processing steps of the program

[0468] User information collection and registration

[0469] Step 1:

[0470] User: Opens the bank's ATM or smartphone app and opens the screen to register facial recognition and biometric data.

[0471] Step 2:

[0472] On the device: Displays an interface for taking a photo of the user's face and asks the user to face the camera.

[0473] Step 3:

[0474] User: Follow the instructions on the device to take a photo of your face in front of the camera.

[0475] Step 4:

[0476] Device: Sends the captured photo to the server.

[0477] Step 5:

[0478] On the device: Display an interface to collect fingerprint data using a fingerprint scanner and have the user scan their fingerprint.

[0479] Step 6:

[0480] User: Places finger on fingerprint scanner to provide fingerprint data.

[0481] Step 7:

[0482] Device: Sends collected fingerprint data to the server.

[0483] Step 8:

[0484] Terminal: Displays an interface for collecting iris data and prompts the user to scan their iris.

[0485] Step 9:

[0486] User: Provides iris data by pointing their iris at the device camera.

[0487] Step 10:

[0488] Terminal: Sends collected iris data and emotion data to the server.

[0489] Step 11:

[0490] Server: Receives facial photos, fingerprint data, iris data, and emotion data, and uses generative AI to train a personal authentication model and emotion model.

[0491] Step 12:

[0492] Server: Stores the trained model in a database and associates the user's personal information with their account information.

[0493] Authentication Process

[0494] Step 13:

[0495] User: Selects "Facial Recognition Transaction" from the ATM or smartphone app.

[0496] Step 14:

[0497] Terminal: Display the face recognition and biometric authentication input interface.

[0498] Step 15:

[0499] User: Face the camera and prepare to scan your fingerprint.

[0500] Step 16:

[0501] Terminal: Collects facial photos, fingerprint data, and emotion data and sends them to the server.

[0502] Step 17:

[0503] Server: Analyzes the received facial photo, fingerprint data, and emotion data, and performs authentication and emotion recognition using generative AI.

[0504] Step 18:

[0505] Server: Notifies the device of the authentication results and emotional state analysis results.

[0506] Transaction Processing

[0507] Step 19:

[0508] User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[0509] Step 20:

[0510] Terminal: The emotion engine monitors the user's emotional state and sends the results to the server.

[0511] Step 21:

[0512] Server: Determines whether a withdrawal is possible based on the user's emotional state and account information.

[0513] Step 22:

[0514] Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[0515] Step 23:

[0516] Server: Sends notifications based on trading results and sentiment analysis to the device.

[0517] Result notification

[0518] Step 24:

[0519] Terminal: Shows users notifications based on trading results and emotional state.

[0520] Step 25:

[0521] Terminal: In the case of an ATM, dispenses cash and provides appropriate notifications (audio, text, visual) as needed.

[0522] Step 26:

[0523] User: Confirm the transaction result and receive the cash and receipt.

[0524] Example 2

[0525] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0526] Conventional cash cardless systems face the risk of user authentication failure or fraudulent transactions, making it difficult to conduct transactions quickly and safely, especially during disasters. Furthermore, because they do not take into account the user's emotional state, there is a need to improve the user experience. To solve these issues, a more reliable authentication system and transaction processing that takes the user's emotional state into account are needed.

[0527] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting facial authentication data and biometric authentication data, means for collecting emotion data, means for transmitting the collected data to the server, means for analyzing the collected data in the server and training a personal authentication model and an emotion model, means for storing the personal authentication model and the emotion model in a database and associating personal information and account information, means for analyzing the user's emotional state, means for processing a transaction for the user's account if authentication is successful and the emotional state is determined to be normal, and means for notifying the user based on the transaction result and the emotional state. This improves the accuracy of user authentication and reduces the risk of fraudulent transactions. Furthermore, it allows users to withdraw cash quickly and safely even during disasters, improving their sense of security.

[0528] "Facial recognition data" is digital data obtained by analyzing the features of a user's face.

[0529] "Biometric data" is digital data collected based on biological characteristics such as fingerprints, irises, or veins.

[0530] "Emotional data" is digital data that indicates the user's emotional state, collected based on facial expressions, tone of voice, speaking rate, etc.

[0531] The "server" is a device that analyzes the collected data and generates and stores personal authentication models and emotion models.

[0532] "Generative AI" is an artificial intelligence technology that uses machine learning algorithms to train models based on collected data.

[0533] A "personal authentication model" is a user identification model trained by generative AI based on facial recognition data and biometric authentication data.

[0534] An "emotion model" is a model that determines a user's emotional state, learned by generative AI based on emotional data.

[0535] "Database" refers to the system that stores and manages personal authentication models, emotion models, and users' personal information and account information.

[0536] "Authentication" is the process of verifying a user's identity based on collected data.

[0537] A "transaction" is a financial transaction such as withdrawing or transferring cash to a user's account.

[0538] "Transaction result" refers to information on whether the transaction was successful or unsuccessful, and is the content of the notification to the user based on that information.

[0539] This invention is a cash cardless system that combines an emotion engine, allowing users to conduct transactions safely and quickly through facial recognition and biometric authentication. The emotion engine also analyzes the user's emotional state and optimizes the transaction. This system is mainly composed of a terminal and a server.

[0540] System configuration

[0541] Terminal

[0542] The device includes the following hardware and software:

[0543] Input devices (cameras, fingerprint scanners, iris scanners, etc.) that allow users to enter facial and biometric data

[0544] Emotion engine (including camera and microphone) for recognizing user emotions

[0545] Displays and interfaces that guide users through the authentication process and their emotional state

[0546] A communication interface for sending data to the server

[0547] server

[0548] The server includes the following elements:

[0549] Generative AI to analyze received data (face photo, fingerprint data, iris data, emotion data) and generate a personal authentication model and emotion model

[0550] Database for managing personal and account information

[0551] Software and algorithms for authentication processes, emotion recognition, and transaction processing

[0552] Program processing

[0553] User information collection and registration

[0554] Users access a bank's ATM or smartphone app to register their facial recognition and biometric data. The device then uses its camera to take a photo of the user's face and scan their fingerprint and iris. It also activates an emotion engine to collect the user's facial expressions and tone of voice. The collected data is sent to a server via the device. The server analyzes the received data and uses generative AI to train a personal authentication model and emotion model. These models are stored in a database and associated with the user's personal and account information.

[0555] Authentication Process

[0556] The user selects "Facial Recognition Transaction" at the ATM or via a smartphone app. The terminal displays the facial recognition and biometric authentication input interface, and the user points their face at the camera, preparing to scan their fingerprint. The terminal collects a facial photo, fingerprint data, and emotional data, and sends them to the server. The server analyzes the received data and performs authentication and emotional recognition using generative AI. The authentication results and emotional state analysis results are then notified to the terminal.

[0557] Transaction Processing

[0558] After successful authentication, the user selects "Withdraw" on the ATM or app interface and enters the amount. The terminal monitors the user's emotional state using an emotion engine and sends the results to the server. The server determines whether a withdrawal is possible based on the user's emotional state and account information, and if so, withdraws the specified amount. The transaction results and notification method based on emotion analysis are sent to the terminal.

[0559] Result notification

[0560] The terminal displays notifications to the user based on the transaction result and emotional state. In the case of an ATM, it dispenses cash and provides appropriate notifications (audio, text, visual) as needed. The user can then confirm the transaction result and collect the cash and receipt.

[0561] Specific examples

[0562] Example 1: You accidentally forget your wallet

[0563] 1. Users register their facial recognition and fingerprint data at the ATM in advance, and emotional data is also collected using an emotion engine.

[0564] 2. The user goes to a bank ATM and selects "Facial Recognition Transaction."

[0565] 3. The device will require facial recognition and fingerprint scanning, and will also collect emotional data.

[0566] 4. The server analyzes the data and performs user authentication and emotion recognition.

[0567] 5. If authentication is successful and the transaction is deemed possible, the user enters the amount and the transaction is carried out.

[0568] 6. The terminal will display a successful transaction and dispense the cash.

[0569] 7. The user receives the cash and a receipt.

[0570] Example 2: When you lose your bankbook or cash card during a disaster

[0571] 1. Users register their facial recognition and biometric data at the emergency assistance center, and emotion data is also collected through the emotion engine.

[0572] 2. The user selects "Facial Recognition Transaction" on the terminal at the evacuation center.

[0573] 3. The device will request facial recognition and fingerprint scanning, and collect emotional data.

[0574] 4. The server analyzes the data and performs user authentication and emotion recognition.

[0575] 5. If authentication is successful and the transaction is deemed possible, the user enters the amount and the transaction is carried out.

[0576] 6. The terminal will display a successful transaction and dispense the cash.

[0577] 7. The user receives cash.

[0578] Prompt Sentence Examples

[0579] "Look at the camera and make sure your face is visible."

[0580] "Place your finger on the scanner"

[0581] "Place your eyes on the iris scanner."

[0582] "Hi, I'm feeling good today."

[0583] "Select facial recognition transaction"

[0584] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0585] User information collection and registration

[0586] Step 1:

[0587] Users access the bank's ATM or smartphone app and begin registering their facial recognition and biometric data.

[0588] Input: User access operations

[0589] Output: Registration process begins

[0590] Step 2:

[0591] The device will display the instruction "Please take a photo of your face" on the display.

[0592] Input: User registration process start operation

[0593] Output: Display of face photo taking screen, display of instruction message

[0594] Specific operation: Display an instruction message on the display and start the camera.

[0595] Step 3:

[0596] The user faces the camera and takes a photo of their face.

[0597] Input: User faces the camera

[0598] Output: Facial photo data

[0599] What happens: The user turns their face in front of the camera and their image is captured.

[0600] Step 4:

[0601] The device will then display an interface that says "Please scan your fingerprint."

[0602] Input: Operation after taking face photo

[0603] Output: Display of fingerprint scan screen, display of instruction message

[0604] Specific operation: Displays an instruction message on the display and activates the fingerprint scanner.

[0605] Step 5:

[0606] The user places their finger on the scanner to scan their fingerprint.

[0607] Input: The user places their finger on the scanner

[0608] Output: Fingerprint data

[0609] What happens: A user places their finger on a fingerprint scanner and their fingerprint is captured.

[0610] Step 6:

[0611] The terminal will display the instruction "Please scan your eyes" if an iris scanner is present.

[0612] Input: Actions after fingerprint scanning

[0613] Output: Display of iris scan screen, display of instruction message

[0614] Specific operations: Display an instruction message on the display and activate the iris scanner.

[0615] Step 7:

[0616] The user positions their eye over the iris scanner and scans their iris.

[0617] Input: The user's eye movement towards the scanner

[0618] Output: Iris data

[0619] What happens: The user points their eye at the iris scanner and their iris is captured.

[0620] Step 8:

[0621] The device will launch its emotion engine and display the message, "Collecting emotion data. Please record your face and voice."

[0622] Input: Actions after iris scan

[0623] Output: Display of emotion data collection screen, display of instruction message

[0624] Specific actions: Display an instruction message on the display and activate the camera and microphone.

[0625] Step 9:

[0626] The user faces the camera and speaks into a microphone.

[0627] Input: User's facial expressions and voice

[0628] Output: Emotion data (facial expressions, tone of voice, speaking speed)

[0629] Specific actions: The user shows their facial expression in front of the camera and speaks into the microphone.

[0630] Step 10:

[0631] The device transmits the collected facial recognition, biometric authentication, and emotion data to a server.

[0632] Input: Collected authentication data and emotion data

[0633] Output: Send data to the server

[0634] Specific operation: The emotion engine analyzes the data and uses the generation AI to send it to the server. During the transmission, the message "Sending data" is displayed.

[0635] Processing on the server

[0636] Step 11:

[0637] The server analyzes the received data and uses generative AI to learn a personal authentication model and emotion model.

[0638] Input: face photo, fingerprint data, iris data, emotion data

[0639] Output: Personal authentication model, emotion model

[0640] Specific operation: Analyzes the received data and generates a model by analyzing facial feature points, fingerprint patterns, voice spectrum, etc.

[0641] Step 12:

[0642] The server stores the learned model in a database and associates it with the user's personal and account information.

[0643] Input: Trained personal authentication model, emotion model

[0644] Output: Save to database

[0645] Specific operation: Link and save the model, personal information, and account information in the database.

[0646] Authentication Process

[0647] Step 13:

[0648] Users select "Facial Recognition Transaction" at the ATM or via a smartphone app.

[0649] Input: User's transaction start operation

[0650] Output: Instructions: "Select facial recognition transaction"

[0651] Specific operation: Display the trading start menu on the interface.

[0652] Step 14:

[0653] The terminal displays an input interface for face recognition and biometric authentication.

[0654] Input: User's trade selection operation

[0655] Output: Display of face recognition and fingerprint scanning screen

[0656] Specific actions: Display messages such as "Please face the camera" and "Please place your finger on the scanner."

[0657] Step 15:

[0658] The user faces the camera and prepares to scan their fingerprint.

[0659] Input: Aligning the user's face and fingers

[0660] Output: Facial photo data, fingerprint data

[0661] Specific operation: For facial recognition, the user points their face towards the camera, and for fingerprint scanning, they place their finger on the scanner.

[0662] Step 16:

[0663] The terminal collects a facial photograph, fingerprint data, and emotion data and transmits them to a server.

[0664] Input: Newly collected authentication data and emotion data

[0665] Output: Send data to the server

[0666] Specific behavior: Displays the message "Collecting data" and the emotion engine analyzes the data.

[0667] Processing on the server

[0668] Step 17:

[0669] The server analyzes the received data and performs authentication and emotion recognition using the generative AI.

[0670] Input: Data sent from the terminal

[0671] Output: Authentication results, sentiment analysis results

[0672] Specific operation: Analyzes the transmitted data using generative AI to authenticate the user and recognize emotions.

[0673] Step 18:

[0674] The server notifies the terminal of the authentication result and the emotional state analysis result.

[0675] Input: Authentication results, sentiment analysis results

[0676] Output: Send results to the terminal

[0677] Specific operation: The success or failure of authentication and the emotional state are returned to the terminal.

[0678] Transaction Processing

[0679] Step 19:

[0680] After successful authentication, the user selects "Withdraw" from the ATM or app interface and enters the amount.

[0681] Input: User withdrawal operation

[0682] Output: Withdrawal amount data

[0683] Specific operation: The message "Please enter the withdrawal amount" will be displayed on the screen.

[0684] Step 20:

[0685] The device monitors the user's emotional state using an emotion engine and sends the results to the server.

[0686] Input: Real-time user sentiment data

[0687] Output: Emotional state data sent to server

[0688] Specific behavior: Display the message "Checking emotional state."

[0689] Processing on the server

[0690] Step 21:

[0691] The server determines whether a withdrawal is possible based on the user's emotional state and account information.

[0692] Input: Emotional state data, account information

[0693] Output: Transaction availability decision

[0694] Specific behavior: If the user is very impatient, display a warning and limit withdrawals.

[0695] Step 22:

[0696] If withdrawal is possible, the server withdraws the specified amount and updates the transaction record.

[0697] Input: Transaction authorization data, specified amount

[0698] Output: Transaction execution, transaction record updated

[0699] Specific behavior: Display the message "Transaction processing" and update the transaction database.

[0700] Step 23:

[0701] The server sends the transaction results and notification method based on sentiment analysis to the terminal.

[0702] Input: Trading results, sentiment analysis results

[0703] Output: Transaction result notification data

[0704] Specific behavior: Returns the message "Transaction successful, please accept cash."

[0705] Result notification

[0706] Step 24:

[0707] The terminal displays notifications to the user according to the trading results and emotional state.

[0708] Input: Transaction result notification data

[0709] Output: Transaction results display

[0710] Specific behavior: Display a reassuring message along with "Transaction successful."

[0711] Step 25:

[0712] In the case of an ATM, the terminal dispenses cash and provides appropriate notifications (audio, text, visual) as needed.

[0713] Input: Transaction success data

[0714] Output: Withdrawal processing, voice guide output

[0715] Specific operation: A voice message will be heard saying, "Cash has been withdrawn, please take the receipt."

[0716] Step 26:

[0717] The user confirms the transaction result and receives the cash and receipt.

[0718] Input: ATM withdrawal notification

[0719] Output: Cash and receipt received

[0720] Specific action: Withdraw cash and receipt from ATM.

[0721] (Application example 2)

[0722] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0723] Current authentication systems using cash cards require users to have their physical cards in order to be authenticated, which poses significant challenges and security issues, especially in the event of a disaster or loss. Furthermore, traditional authentication systems do not take into account the user's emotional state, which can raise doubts about the legitimacy of the transaction. Furthermore, while fast and secure payments are required for in-store purchases, traditional systems do not adequately meet these demands.

[0724] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting facial authentication data and biometric authentication data, means for training an emotion model, and means for evaluating transaction appropriateness based on the emotional state. This not only eliminates the need for users to carry cards, but also enables appropriate transactions that take the emotional state into consideration, enabling rapid response in the event of a disaster and safe and rapid payment at stores.

[0725] "Facial recognition data" is information used to identify individuals by analyzing images of their faces and facial features.

[0726] "Biometric data" is information used to analyze an individual's biometric characteristics, such as fingerprints, irises, and voice, for authentication.

[0727] An "emotion model" is a model trained to analyze a user's facial expressions and tone of voice and estimate their emotional state.

[0728] "Smart glasses" are glasses-type devices worn by the user that incorporate devices such as a camera and a display.

[0729] "Transaction information" refers to data related to a transaction conducted by a user, such as the transaction amount and transaction details.

[0730] "Communication means" refers to the network infrastructure for transmitting and receiving data between the terminal and the server.

[0731] "Generative AI" is an artificial intelligence technology that analyzes facial recognition data and biometric authentication data to create personal authentication models and emotion models.

[0732] The "transaction appropriateness evaluation means" is a means for determining whether a transaction is appropriate based on the emotional state of the user.

[0733] "Authentication means" means a means for verifying a user's identity using facial recognition and biometric data.

[0734] The "means for notifying results" is a means for notifying users of the results of trading and sentiment analysis.

[0735] We will now explain a specific example of a cash cardless system incorporating the emotion engine of the present invention. This system is based on technology that authenticates users by combining facial recognition data and biometric authentication data, and simultaneously analyzes the user's emotional state to evaluate the appropriateness of the transaction.

[0736] System configuration

[0737] The system mainly consists of the following elements:

[0738] 1. Device:

[0739] Camera: Collects user facial recognition data.

[0740] Biometric devices: Collect biometric data such as fingerprints and irises.

[0741] Emotion engine: Analyzes the user's facial expressions and tone of voice to collect emotional data.

[0742] Display: Shows the authentication process and emotional state to the user.

[0743] Communication Interface: Sends data to the server.

[0744] 2. Server:

[0745] Data analysis and generation AI: Analyzes received facial recognition data, biometric data, and emotion data to generate personal recognition models and emotion models.

[0746] Database: Stores users' personal and transaction information and manages authentication models.

[0747] Transaction suitability assessment function: Determines the suitability of a transaction based on emotional state.

[0748] Program processing

[0749] The server runs on the following hardware and software:

[0750] Hardware: If high-performance computing is required, cloud-based computing resources (e.g., Amazon Web Services, Google® Cloud Platform, Microsoft® Azure®) will be used.

[0751] Software: Emotion analysis uses emotion recognition algorithms, and personal authentication uses face and iris recognition algorithms. These are often implemented using machine learning libraries such as TensorFlow and PyTorch.

[0752] Processing flow

[0753] The device first collects the user's facial and biometric data using a camera and biometric authentication device, and then uses an emotion engine to obtain the user's emotional data. All of this data is sent to the server via a communications interface. The server then analyzes the data using generative AI to generate a personal authentication model and an emotion model, which are then stored in a database. When a transaction is made, the server uses these models to authenticate the user and simultaneously evaluate the user's emotional state to determine whether the transaction is appropriate.

[0754] Specific examples

[0755] When shopping in a physical store, users provide facial recognition and biometric data through the smart glasses and select "Facial Recognition Transaction." The smart glasses' camera recognizes the face and scans fingerprints and iris data. The emotion engine also analyzes facial expressions and tone of voice to obtain emotion data. All of this data is sent to the server in real time, where authentication and emotion evaluation are performed. If authentication is successful, the transaction amount will be displayed on the smart glasses' display, and the user can confirm and complete the payment.

[0756] Prompt Sentence Examples

[0757] "To pay at the register, you need to be authenticated using facial recognition, iris recognition, and emotional data. Just face the camera and scan your iris and fingerprint."

[0758] Through these steps, users can complete transactions quickly and safely, which is particularly useful in times of disaster or emergency.

[0759] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0760] Step 1:

[0761] The user puts on the terminal (smart glasses) and selects facial recognition transaction. The terminal instructs the user to face the camera and simultaneously starts scanning fingerprint and iris data using the biometric authentication device. Inputs include the user's facial image and biometric data (fingerprint, iris). These are acquired as real-time video and scan data. The output is the collected facial recognition data and biometric authentication data.

[0762] Step 2:

[0763] The device launches an emotion engine to collect the user's facial expressions and tone of voice. The emotion engine uses a camera and microphone to analyze the user's emotional state. Here, facial expressions and tone of voice are used as input, and the output is emotion data. This emotion data reflects the user's real-time mental state.

[0764] Step 3:

[0765] The terminal transmits the collected facial recognition data, biometric data, and emotion data to the server through a communication interface. The input is the collected authentication data and emotion data, and the output is the transmission of these data to the server.

[0766] Step 4:

[0767] The server analyzes the received data and uses generative AI to generate a personal authentication model and emotion model. It takes in facial recognition data, biometric data, and emotion data as input, performs data analysis and machine learning, and generates a personal authentication model and emotion model as output. These models are stored in the server's database.

[0768] Step 5:

[0769] The server uses the generated personal authentication model and emotion model to authenticate the user. The input is the model stored on the server and the authentication data sent from the device. The output is the result of authentication success or failure. If authentication is successful, proceed to the next step.

[0770] Step 6:

[0771] The user checks the transaction details (e.g., payment amount, purchased items, etc.) on the smart glasses display and confirms the transaction. The input is the transaction details sent from the server, which the user checks and operates on the display. The output is the transaction confirmation result.

[0772] Step 7:

[0773] The server evaluates the user's emotional state and determines the appropriateness of the transaction. Using the confirmed transaction details and emotional data as input, it performs evaluation processing based on an emotional model. The output is an evaluation result regarding the appropriateness of the transaction.

[0774] Step 8:

[0775] The server processes the transaction and notifies the user of the result. The input is the confirmation of the transaction details and the evaluation result of the emotional state. The output is a notification of whether the transaction was completed, which is displayed on the display of the user's smart glasses. At this time, the transaction information is recorded in a database.

[0776] Step 9:

[0777] The terminal notifies the user of the transaction result and displays or electronically stores the receipt as needed. The input is a notification from the server that the transaction has been completed, and the output is a visual and electronic confirmation of the result for the user.

[0778] Through these steps, users can complete transactions safely and quickly, providing a system that is extremely useful, especially in times of disaster or emergency.

[0779] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0780] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0781] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0782] [Second embodiment]

[0783] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0784] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0785] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0786] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0787] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0788] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0789] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0790] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0791] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0792] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0793] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0794] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0795] System configuration

[0796] The cash cardless system of this invention is implemented in bank ATMs and smartphone apps. The system mainly consists of the following components:

[0797] 1. Terminal

[0798] An input device (such as a camera, fingerprint scanner, or iris scanner) that allows users to enter facial and biometric data.

[0799] A display or interface that guides the user through the authentication process.

[0800] A communication interface for sending data to the server.

[0801] 2. Server

[0802] Generative AI to analyze received data and generate a personal authentication model.

[0803] A database that manages personal and account information.

[0804] Software and algorithms for authentication processes and transaction processing.

[0805] Program processing procedure

[0806] The program processing of the system will be described in detail below.

[0807] User information collection and registration

[0808] 1. User: Visits a bank ATM or smartphone app and registers facial recognition and biometric data.

[0809] 2. Terminal: Have the user take a photo of their face and scan their fingerprint and iris.

[0810] 3. Terminal: The collected data is sent to the server via secure communication.

[0811] 4. Server: Analyzes the received data and uses generative AI to train a personal authentication model.

[0812] 5. Server: Stores the learned personal authentication model in a database and associates the user's personal information with their account information.

[0813] Authentication Process

[0814] 1. User: Select "Facial Recognition Transaction" from the ATM or app.

[0815] 2. Terminal: Display the face recognition and biometric authentication input interface.

[0816] 3. User: Faces the device camera and scans their fingerprint.

[0817] 4. Terminal: Sends collected data to the server.

[0818] 5. Server: Authenticates the user using the generative AI.

[0819] 6. Server: If authentication is successful, it notifies the terminal of the result.

[0820] Transaction Processing

[0821] 1. User: After successful authentication, enter the withdrawal amount.

[0822] 2. Terminal: Sends the entered amount information to the server.

[0823] 3. Server: Determines whether withdrawal is possible based on the user's account information.

[0824] 4. Server: If withdrawal is possible, withdraw the specified amount.

[0825] 5. Server: Updates the transaction record and sends a success notification to the device.

[0826] Result notification

[0827] 1. Terminal: displays the transaction results to the user.

[0828] 2. Terminal: In the case of an ATM, it dispenses cash and prints a receipt.

[0829] 3. User: Check the transaction result and receive the cash and receipt.

[0830] Specific examples

[0831] Example 1: You accidentally forget your wallet

[0832] 1. Collecting and registering user information: Register facial recognition and fingerprint data at the ATM in advance.

[0833] 2. Authentication Process:

[0834] User: Goes to a bank ATM and selects facial recognition transaction.

[0835] On the device: Have the user face the camera and scan their fingerprint.

[0836] Server: Analyzes the data and authenticates the user.

[0837] 3. Transaction Processing:

[0838] User: After successful authentication, enter the amount from the withdrawal menu.

[0839] Terminal: Sends the amount to the server.

[0840] Server: Determines whether withdrawals are possible and processes transactions.

[0841] 4. Result notification:

[0842] Terminal: Shows successful transaction and withdraws cash.

[0843] User: Take cash and receipt.

[0844] Example 2: When you lose your bankbook or cash card during a disaster

[0845] 1. User information collection and registration: Register facial recognition and biometric data at the emergency assistance center.

[0846] 2. Authentication Process:

[0847] User: Selects facial recognition transaction on the shelter terminal.

[0848] Device: Have your face face the camera and scan your fingerprint.

[0849] Server: Analyzes the data and authenticates the user.

[0850] 3. Transaction Processing:

[0851] User: After successful authentication, enter the withdrawal amount.

[0852] Terminal: Sends the amount to the server.

[0853] Server: Determines whether withdrawal is possible and processes the transaction.

[0854] 4. Result notification:

[0855] Terminal: Shows successful transaction and withdraws cash.

[0856] User: Take the cash.

[0857] The system of the present invention allows users to safely withdraw money without a cash card, improving convenience and peace of mind for users. In particular, in the event of a disaster, it allows users to quickly obtain cash, which has the effect of reducing the mental burden on victims.

[0858] The processing flow will be explained below.

[0859] Specific processing steps of the program

[0860] User information collection and registration

[0861] Step 1:

[0862] User: Opens the bank's ATM or smartphone app and opens the screen to register facial recognition and biometric data.

[0863] Step 2:

[0864] On the device: Displays an interface for taking a photo of the user's face and asks the user to face the camera.

[0865] Step 3:

[0866] User: Follow the instructions on the device to take a photo of your face in front of the camera.

[0867] Step 4:

[0868] Device: Sends the captured photo to the server.

[0869] Step 5:

[0870] On the device: Display an interface to collect fingerprint data using a fingerprint scanner and have the user scan their fingerprint.

[0871] Step 6:

[0872] User: Places finger on fingerprint scanner to provide fingerprint data.

[0873] Step 7:

[0874] Device: Sends collected fingerprint data to the server.

[0875] Step 8:

[0876] Terminal: Displays an interface for collecting iris data and prompts the user to scan their iris.

[0877] Step 9:

[0878] User: Provides iris data by pointing their iris at the device camera.

[0879] Step 10:

[0880] Terminal: Sends collected iris data to the server.

[0881] Step 11:

[0882] Server: Receives facial photos, fingerprint data, and iris data, and trains a personal authentication model using generative AI.

[0883] Step 12:

[0884] Server: The trained personal authentication model is stored in a database and associated with the user's personal and account information.

[0885] Authentication Process

[0886] Step 13:

[0887] User: Selects "Facial Recognition Transaction" from the ATM or smartphone app.

[0888] Step 14:

[0889] Terminal: Display the face recognition and biometric authentication input interface.

[0890] Step 15:

[0891] User: Face the camera and prepare to scan your fingerprint.

[0892] Step 16:

[0893] Terminal: Collects facial photo and fingerprint data and sends them to the server.

[0894] Step 17:

[0895] Server: Analyzes the received facial photo and fingerprint data and performs authentication using the generated AI.

[0896] Step 18:

[0897] Server: Notifies the terminal of the authentication result.

[0898] Transaction Processing

[0899] Step 19:

[0900] User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[0901] Step 20:

[0902] Terminal: Sends the amount data entered by the user to the server.

[0903] Step 21:

[0904] Server: Checks the user's account information and determines whether a withdrawal is possible.

[0905] Step 22:

[0906] Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[0907] Step 23:

[0908] Server: Sends the transaction results to the terminal.

[0909] Result notification

[0910] Step 24:

[0911] Terminal: Displays the transaction results to the user.

[0912] Step 25:

[0913] Terminal: In the case of an ATM, it dispenses cash and prints receipts.

[0914] Step 26:

[0915] User: Checks and receives cash and receipts.

[0916] This series of processing steps allows users to withdraw money safely and quickly using facial and biometric authentication, without using a cash card.

[0917] Example 1

[0918] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0919] In recent years, the loss and theft of cash cards and bankbooks has been increasing, and the risk of fraudulent transactions has also increased. In particular, in emergencies such as natural disasters and fires, if a bankbook or cash card is lost or lost, it can be difficult for victims to quickly secure cash. Furthermore, conventional authentication systems often use passwords or PIN codes, which still pose security risks. Therefore, to solve these issues, a system that can safely and quickly authenticate individuals and perform financial transactions is needed.

[0920] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0921] In this invention, the server includes a means for collecting facial recognition data and biometric authentication data, a means for training a personal authentication model using a generative AI, and a means for storing the personal authentication model in a database and associating personal information and financial information, thereby enabling safe and fast user authentication and financial transactions.

[0922] "Facial recognition data" is data that digitally represents a user's facial features.

[0923] "Biometric data" means a digital representation of a user's biological characteristics, such as a fingerprint or iris.

[0924] A "server" is a computer system that stores and processes data over a network.

[0925] "Generative AI" is a program that uses artificial intelligence technology to perform a specific task, and in this invention is used particularly for training a personal authentication model.

[0926] A "personal authentication model" is a model for uniquely identifying a user using collected facial recognition data and biometric authentication data.

[0927] A "database" is a system for efficiently storing and managing large amounts of data.

[0928] "Personal Information" means identifying information, such as name and address, associated with a user.

[0929] "Financial information" refers to information related to a user's account information, transaction history, etc.

[0930] "User authentication" is the process of verifying a user's identity based on collected authentication data.

[0931] A "financial transaction" refers to a monetary transaction, such as a withdrawal from or transfer to a user's account.

[0932] "Trading results" refers to information obtained as a result of a financial transaction, including the success or failure of the transaction.

[0933] The cash cardless system of this invention is implemented in bank ATMs and smartphone apps. The system is mainly composed of a terminal and a server.

[0934] System configuration

[0935] Terminal

[0936] The terminal is equipped with an input device for users to enter facial and biometric authentication data, such as a camera, fingerprint scanner, or iris scanner, and provides a display or interface that guides the user through the authentication process. The terminal also transmits the collected data to a server through a secure communication interface.

[0937] Camera: A device used to take a photo of the user's face.

[0938] Fingerprint scanner: A device used to scan a user's fingerprint.

[0939] Iris scanner: A device used to scan a user's iris.

[0940] Display: A device used to display authentication procedures and transaction details.

[0941] Communication interface: A secure protocol (e.g. HTTPS) for sending data to the server.

[0942] server

[0943] The server analyzes the received data and generates a personal authentication model using generative AI. Specifically, it uses AI frameworks such as TensorFlow and PyTorch. It also has a database that manages personal and financial information. This database stores the trained personal authentication model and also contains software and algorithms for processing user transactions.

[0944] Generative AI: Train a person authentication model using TensorFlow and PyTorch.

[0945] Database: Manage personal and financial information using database systems such as MySQL or PostgreSQL.

[0946] Transaction Processing Algorithm: Software used to process users' financial transactions.

[0947] Specific examples

[0948] Example 1: You accidentally forget your wallet

[0949] 1. Collecting and registering user information: Register facial recognition and fingerprint data in advance at the bank ATM.

[0950] User: Accesses the ATM and registers facial recognition and fingerprint data.

[0951] Device: Takes a photo of the user's face and scans their fingerprint, sending the data to a server.

[0952] Server: Analyzes the data, uses generative AI to train a personal authentication model, and stores it in a database.

[0953] 2. Authentication Process:

[0954] User: Go to a bank ATM and select "Facial Recognition Transaction."

[0955] Device: Have your face face the camera and scan your fingerprint.

[0956] Server: Analyzes the collected data and authenticates users.

[0957] 3. Transaction Processing:

[0958] User: After successful authentication, enter the desired amount in the withdrawal menu.

[0959] Terminal: Sends the entered amount information to the server.

[0960] Server: Determines whether withdrawal is possible based on the user's account information and processes the transaction.

[0961] 4. Result notification:

[0962] Terminal: Shows successful transaction and withdraws cash.

[0963] User: Take cash and receipt.

[0964] Example 2: When you lose your bankbook or cash card during a disaster

[0965] 1. User information collection and registration: Register facial recognition and biometric data at the emergency assistance center.

[0966] User: Register facial recognition and biometric data on a terminal at the evacuation center.

[0967] Device: Face the camera, scan your fingerprint, and send the data to the server.

[0968] Server: Uses generative AI to train a personal authentication model and saves it in a database.

[0969] 2. Authentication Process:

[0970] User: Select "Facial Recognition Transaction" on the shelter terminal.

[0971] Device: Have your face face the camera and scan your fingerprint.

[0972] Server: Analyzes the data and authenticates the user.

[0973] 3. Transaction Processing:

[0974] User: After successful authentication, enter the withdrawal amount.

[0975] Terminal: Sends the amount to the server.

[0976] Server: Determines whether withdrawals are possible and processes transactions.

[0977] 4. Result notification:

[0978] Terminal: Shows successful transaction and withdraws cash.

[0979] User: Take the cash.

[0980] Prompt Sentence Examples

[0981] "Please explain your generative AI model for user authentication. Explain the process steps for analyzing the data using facial and fingerprint recognition."

[0982] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0983] Program processing flow

[0984] Step 1:

[0985] User: Accesses the bank's ATM or smartphone app and begins enrolling facial recognition and biometric data. Selects "New Enrollment" from the app's home screen and begins enrolling facial recognition and fingerprint data.

[0986] Input: Facial photo and fingerprint data

[0987] Output: Registration ready message

[0988] Specific steps: Tap "New Registration" from the app menu and follow the instructions to enter your face photo and fingerprint data.

[0989] Step 2:

[0990] Terminal: Takes a photo of the user's face and scans their fingerprint. Uses the ATM's camera to take a photo of the face and a fingerprint scanner to read the fingerprint.

[0991] Input: User's facial photo and fingerprint data

[0992] Output: Collected data

[0993] Specific actions: The app displays instructions to the user, such as "Please face the camera" and "Please place your finger on the fingerprint scanner."

[0994] Step 3:

[0995] Device: Send collected data to the server using a secure communication protocol (e.g. HTTPS).

[0996] Input: Collected data

[0997] Output: Data transmission completed message

[0998] Specific actions: Encrypt the data and send it to the server.

[0999] Step 4:

[1000] Server: Analyzes the received data and uses generative AI (using TensorFlow or PyTorch) to train a personal authentication model.

[1001] Input: Collected data

[1002] Output: Personal authentication model

[1003] Specific operations: Analyze received facial recognition and biometric data to train an AI model.

[1004] Step 5:

[1005] Server: The trained personal authentication model is stored in a database (MySQL or PostgreSQL) and associated with the user's personal and financial information.

[1006] Input: Personal authentication model and user information

[1007] Output: Updated database

[1008] Specific operation: Stores the authentication model in the database, linked to the user ID, and associates personal and financial information.

[1009] Step 6:

[1010] User: Accesses the bank's ATM or app and selects "Facial Recognition Transactions." For example, selects "Facial Recognition Transactions" from the ATM menu screen.

[1011] Input: Operation selection

[1012] Output: Authentication interface display

[1013] Specific operation: Select "Facial Recognition Transactions" on the ATM menu screen.

[1014] Step 7:

[1015] Terminal: Display the input interface for face recognition and biometric authentication. Display the camera image for face recognition and the fingerprint scanner image.

[1016] Input: Menu selection information

[1017] Output: Authentication interface

[1018] Specific actions: "Please face the camera" and "Please place your finger on the fingerprint scanner" are displayed.

[1019] Step 8:

[1020] User: Faces the device camera and scans a fingerprint, for example, by facing the camera lens and pressing a finger against the scanner.

[1021] Input: Facial photo and fingerprint data

[1022] Output: Authentication data

[1023] Specific actions: Stand in front of the camera for facial recognition and place your finger on the fingerprint scanner.

[1024] Step 9:

[1025] Terminal: Sends collected data to the server. The data is encrypted and sent using a secure communication protocol.

[1026] Input: Authentication data

[1027] Output: Data transmission completed message

[1028] Specific operation: Encrypts collected data and sends it to a server.

[1029] Step 10:

[1030] Server: Authenticates the user using generative AI, matching the stored authentication model with newly acquired data.

[1031] Input: Authentication data and personal authentication model

[1032] Output: Authentication result

[1033] Specific operation: Analyzes the received data and compares it with the stored AI model to evaluate the degree of match.

[1034] Step 11:

[1035] Server: If authentication is successful, it notifies the terminal of the result, encrypts the authentication result, and sends it to the terminal.

[1036] Input: Authentication result

[1037] Output: Authentication success message

[1038] Specific operation: Sends a message of successful authentication to the terminal.

[1039] Step 12:

[1040] User: After successful authentication, enter the amount to withdraw. Enter the desired amount on the ATM's input screen.

[1041] Input: Withdrawal amount

[1042] Output: Amount information entered

[1043] Specific operation: Enter the amount in the amount input field displayed on the ATM screen.

[1044] Step 13:

[1045] Terminal: Sends the entered amount information to the server. The entered data is encrypted and sent securely.

[1046] Input: Amount information

[1047] Output: Data transmission completed message

[1048] Specific operation: Sends the entered amount information to the server.

[1049] Step 14:

[1050] Server: Determines whether withdrawal is possible based on the user's account information. Checks the account balance and compares it with the requested amount.

[1051] Input: Amount information and account information

[1052] Output: Withdrawal decision result

[1053] Specific operation: Check the account balance and determine whether a withdrawal is possible.

[1054] Step 15:

[1055] Server: If withdrawal is possible, withdraw the specified amount. The server processes the transaction and updates the account balance.

[1056] Input: Withdrawal decision result

[1057] Output: Transaction processing results

[1058] Specific actions: Process the transaction and withdraw the specified amount.

[1059] Step 16:

[1060] Server: Updates the transaction record and sends a success notification to the terminal. Adds the transaction record to the database and sends a success message to the terminal.

[1061] Input: Transaction processing result

[1062] Output: Success message

[1063] Specific operations: Save the transaction record in the database and send a notification of successful transaction to the terminal.

[1064] Step 17:

[1065] Terminal: Display the transaction result to the user. Display a successful transaction message on the ATM screen.

[1066] Input: Success message

[1067] Output: Trade result display

[1068] Specific behavior: Display a message indicating the transaction was successful.

[1069] Step 18:

[1070] Terminal: In the case of an ATM, it dispenses cash and prints a receipt. The ATM's cash dispenser dispenses the specified amount, and the receipt printer prints the transaction details.

[1071] Input: Success message

[1072] Output: Cash and receipt

[1073] Specific actions: Dispense cash and print a receipt.

[1074] Step 19:

[1075] User: Check the transaction result and receive the cash and receipt. Take the cash and receipt that come out of the ATM.

[1076] Input: Cash and receipt

[1077] Output: Transaction completed

[1078] Specific actions: Check the transaction results and take the cash and receipt.

[1079] (Application example 1)

[1080] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1081] In conventional financial transactions using cash cards or bankbooks, if these physical media are lost, transactions often become difficult to complete. Furthermore, in emergencies such as natural disasters, earthquakes, or fires, cards and bankbooks can be damaged or lost, making it difficult to quickly obtain necessary funds. Furthermore, from a security standpoint, there is a risk of fraudulent use if cards or bankbooks are stolen. There is a need for a system that can solve these issues and enable users to conduct financial transactions quickly and with peace of mind.

[1082] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1083] In this invention, the server includes means for collecting facial authentication data and biometric authentication data, means for transmitting the collected data to the server, means for analyzing the collected data in the server and training a personal authentication model, means for saving the personal authentication model in a database and associating personal information and account information, means for authenticating a user based on the facial authentication and biometric authentication data, means for processing a transaction for the user's account if authentication is successful, means for notifying the user of the transaction result, means for confirming and approving the payment amount, means for transmitting payment amount information to the server after authentication is successful, and means for saving the transaction history of the payment amount so that the user can confirm it. This eliminates the need for physical cash cards or passbooks, allowing transactions to be carried out quickly, especially in emergencies such as natural disasters, earthquakes, and fires, while also improving security.

[1084] "Facial recognition data" is data used to analyze image data of a user's face and identify the individual.

[1085] "Biometric data" is data used to identify an individual using biometric characteristics such as fingerprints or irises.

[1086] "Means of collection" refers to devices and programs for acquiring facial recognition data and biometric authentication data.

[1087] "Transmitting means" refers to a device or program that has the function of transmitting collected data to a server.

[1088] The "analyzing means" refers to a device or program for processing the received data and generating a personal authentication model.

[1089] "Means for training" refers to devices or programs for training a personal authentication model using generative AI.

[1090] A "personal authentication model" is an algorithm or program for identifying individuals based on collected facial recognition data and biometric data.

[1091] "Database" means a storage device for storing and managing personal authentication models, personal information, and account information.

[1092] "Means for authentication" refers to devices or programs that verify a user's identity based on facial recognition and biometric data.

[1093] A "means for processing transactions" is a device or program that, after successful authentication, executes a financial transaction against a user's account.

[1094] "Means of notification" refers to devices or programs used to notify users of transaction results.

[1095] "Means for confirming and approving the payment amount" refers to a device or program that allows the user to confirm and approve the payment amount.

[1096] The "means for transmitting payment amount information" refers to a device or program for transmitting information on the payment amount to the server after successful authentication.

[1097] "Means for storing transaction history" refers to a device or program that records the results of each transaction and allows users to check them.

[1098] "Generative AI" is an artificial intelligence technology that analyzes data and trains a personal authentication model.

[1099] MODE FOR CARRYING OUT THE INVENTION

[1100] The facial recognition electronic payment system of the present invention uses facial recognition data and biometric authentication data to realize safe and fast financial transactions without the need for physical cash cards or bankbooks.

[1101] System configuration

[1102] The system mainly consists of the following components:

[1103] 1. User device (smartphone)

[1104] 2. Server

[1105] 3. Database

[1106] Hardware and software used

[1107] Smartphone: Camera for facial recognition, fingerprint scanner for fingerprint recognition

[1108] Facial recognition software: OpenCV

[1109] Generative AI model: TensorFlow

[1110] Communication interface: Secure HTTP communication

[1111] Database: SQL-based database

[1112] Program processing overview and specific examples

[1113] User information collection and registration

[1114] Users use their smartphone's camera and fingerprint scanner to collect facial and biometric data, which is then transmitted to a server through a secure communication interface.

[1115] The server analyzes the received data and uses the generative AI model to create a personal authentication model, which is stored in a database and associated with the user's personal and account information.

[1116] Authentication Process

[1117] When making a payment, users open a smartphone app and enter their facial and biometric data, which includes pointing their face at the camera and placing their fingerprint on a scanner. The smartphone then sends this data to a server, which uses a generative AI model to authenticate the user.

[1118] Payment Processing

[1119] If authentication is successful, the user confirms and approves the payment amount. The smartphone sends the confirmed payment amount to the server. The server processes the payment based on the user's account information and saves the transaction results in a database.

[1120] Result notification

[1121] The results of the transaction are notified to the user, who can check the results via a smartphone app.

[1122] Specific application

[1123] Example 1: Use in everyday life

[1124] When shopping at a supermarket, a user uses a facial recognition electronic payment app. After shopping is complete, the user opens the app at the cash register, performs facial and biometric authentication, and approves the payment. The server processes the authentication and payment, and the transaction result is notified to the user within a few seconds.

[1125] Example 2: Use during a disaster

[1126] Users who lose their bankbooks or cash cards during a disaster can use terminals installed at evacuation shelters to quickly withdraw cash using facial and biometric authentication, enabling safe and fast transactions even under certain circumstances.

[1127] Prompt Sentence Examples

[1128] "Please show us how to obtain a user's facial image and fingerprint data and perform secure authentication."

[1129] "If bankbooks and cash cards become unusable during a natural disaster, please explain how to use facial recognition as an alternative method for transactions."

[1130] As a result, physical cash cards and bankbooks are no longer necessary, allowing for quick transactions, especially in emergencies such as natural disasters, earthquakes, and fires, while also improving security.

[1131] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1132] Step 1:

[1133] The user downloads the facial recognition electronic payment app to their smartphone and launches it. The user begins facial and fingerprint authentication on the new registration screen (input: user's face and fingerprint, output: collected facial recognition data and biometric authentication data).

[1134] Step 2:

[1135] The device uses the smartphone's camera and fingerprint scanner to collect the user's facial authentication data and fingerprint data (data processing: acquisition of facial data and fingerprint data, output: collected facial authentication data and biometric authentication data).

[1136] Step 3:

[1137] The device encrypts the collected data and sends it to the server using a secure communication protocol (e.g., HTTPS) (input: collected facial recognition data and biometric authentication data, output: encrypted data sent to the server).

[1138] Step 4:

[1139] The server analyzes the received facial recognition data and biometric authentication data and creates a personal authentication model using a generative AI model (input: encrypted and transmitted facial recognition data and biometric authentication data; data calculation: training of the model using the generative AI; output: personal authentication model).

[1140] Step 5:

[1141] The server stores the personal authentication model in a database and associates it with the user's personal information and payment information (input: personal authentication model, personal information and payment information; output: personal authentication model and related information stored in the database).

[1142] Step 6:

[1143] When making a payment, the user launches the app and proceeds to the face authentication and fingerprint authentication screen (input: user operation, output: face authentication and fingerprint authentication screen).

[1144] Step 7:

[1145] The device collects the facial recognition data and biometric authentication data again and sends them to the server (input: user's face and fingerprint, data processing: recollected facial recognition data and biometric authentication data, output: data sent to the server).

[1146] Step 8:

[1147] The server compares the received data with the personal authentication model again and authenticates the user (input: recollected data, personal authentication model, data calculation: model comparison, output: authentication result).

[1148] Step 9:

[1149] If the authentication is successful, the user confirms and approves the payment amount, and this information is also sent to the server (Input: User approves the amount, Output: Payment amount information).

[1150] Step 10:

[1151] The server processes the transaction based on the user's account information and payment amount information, and records the payment results (input: payment amount information, account information, data calculation: transaction processing and recording, output: transaction results and recording).

[1152] Step 11:

[1153] The server notifies the terminal of the transaction result, and the terminal displays it to the user (input: transaction result, output: notification to user).

[1154] The above is the flow of processing steps of this system.

[1155] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1156] System configuration

[1157] The cash cardless system incorporating the emotion engine of the present invention consists of the following elements:

[1158] 1. Terminal

[1159] An input device (such as a camera, fingerprint scanner, or iris scanner) that allows users to enter facial and biometric data.

[1160] An emotion engine (including a camera and microphone) to recognize user emotions.

[1161] Displays and interfaces that show the authentication process and emotional state to the user.

[1162] A communication interface for sending data to the server.

[1163] 2. Server

[1164] A generative AI that analyzes received data (face photos, fingerprint data, iris data, emotion data) and generates a personal authentication model and emotion model.

[1165] A database that manages personal and account information.

[1166] Software and algorithms for authentication processes, emotion recognition, and transaction processing.

[1167] Program processing procedure

[1168] The program processing of the system will be described in detail below.

[1169] User information collection and registration

[1170] 1. User: Visits a bank ATM or smartphone app and registers facial recognition and biometric data.

[1171] 2. Terminal: Have the user take a photo of their face and scan their fingerprint and iris.

[1172] 3. Device: Activate the emotion engine and collect the user's facial expressions and tone of voice.

[1173] 4. Terminal: Sends collected facial recognition, biometrics, and emotion data to the server.

[1174] 5. Server: Analyzes the received data and uses generative AI to train a personal authentication model and emotion model.

[1175] 6. Server: Stores the trained model in a database and associates the user's personal and account information.

[1176] Authentication Process

[1177] 1. User: Select "Facial Recognition Transaction" from the ATM or smartphone app.

[1178] 2. Terminal: Display the face recognition and biometric authentication input interface.

[1179] 3. User: Faces the camera and prepares to scan his / her fingerprint.

[1180] 4. Terminal: Collects facial photos, fingerprint data, and emotion data and sends them to the server.

[1181] 5. Server: Analyzes the received data and performs authentication and emotion recognition using generative AI.

[1182] 6. Server: Notifies the device of the authentication result and emotional state analysis result.

[1183] Transaction Processing

[1184] 1. User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[1185] 2. Terminal: The emotion engine monitors the user's emotional state and sends the results to the server.

[1186] 3. Server: Determines whether withdrawal is possible based on the user's emotional state and account information.

[1187] 4. Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[1188] 5. Server: Sends trading results and notification methods based on sentiment analysis to the terminal.

[1189] Result notification

[1190] 1. Terminal: displays notifications to users based on trading results and emotional state.

[1191] 2. Terminal: In the case of an ATM, dispenses cash and provides appropriate notifications (audio, text, visual) as needed.

[1192] 3. User: Check the transaction result and receive the cash and receipt.

[1193] Specific examples

[1194] Example 1: You accidentally forget your wallet

[1195] 1. Collecting and registering user information: Facial recognition and fingerprint data are registered in advance at the ATM, and emotion data is also collected using the emotion engine.

[1196] 2. Authentication Process:

[1197] User: Goes to a bank ATM and selects facial recognition transaction.

[1198] Device: The device asks the user to face the camera, scans their fingerprint, and collects emotional data.

[1199] Server: Analyzes data and performs user authentication and emotion recognition.

[1200] 3. Transaction Processing:

[1201] User: After successful authentication, enter the amount from the withdrawal menu.

[1202] Terminal: Sends the amount and user emotion data to the server.

[1203] Server: Determines whether withdrawals are possible and processes transactions.

[1204] 4. Result notification:

[1205] Terminal: Shows successful transaction and withdraws cash.

[1206] User: Take cash and receipt.

[1207] Example 2: When you lose your bankbook or cash card during a disaster

[1208] 1. Collecting and registering user information: The emergency support center will register facial recognition and biometric data, and also collect emotional data from the emotion engine.

[1209] 2. Authentication Process:

[1210] User: Selects facial recognition transaction on the shelter terminal.

[1211] Device: The device asks the user to face the camera, scans their fingerprint, and collects emotional data.

[1212] Server: Analyzes data and performs user authentication and emotion recognition.

[1213] 3. Transaction Processing:

[1214] User: After successful authentication, enter the withdrawal amount.

[1215] Terminal: Sends the amount and user emotion data to the server.

[1216] Server: Determines whether withdrawal is possible and processes the transaction.

[1217] 4. Result notification:

[1218] Terminal: Displays notifications for successful transactions and emotional states, and withdraws cash.

[1219] User: Take the cash.

[1220] By using the system of this invention, users can safely and quickly withdraw money without using a cash card. Furthermore, by combining it with an emotion engine, optimal transactions can be performed according to the user's emotional state, enhancing convenience and peace of mind. In particular, in the event of a disaster, this system allows users to obtain cash quickly and appropriately, which has the effect of reducing the mental burden on victims.

[1221] The processing flow will be explained below.

[1222] Specific processing steps of the program

[1223] User information collection and registration

[1224] Step 1:

[1225] User: Opens the bank's ATM or smartphone app and opens the screen to register facial recognition and biometric data.

[1226] Step 2:

[1227] On the device: Displays an interface for taking a photo of the user's face and asks the user to face the camera.

[1228] Step 3:

[1229] User: Follow the instructions on the device to take a photo of your face in front of the camera.

[1230] Step 4:

[1231] Device: Sends the captured photo to the server.

[1232] Step 5:

[1233] On the device: Display an interface to collect fingerprint data using a fingerprint scanner and have the user scan their fingerprint.

[1234] Step 6:

[1235] User: Places finger on fingerprint scanner to provide fingerprint data.

[1236] Step 7:

[1237] Device: Sends collected fingerprint data to the server.

[1238] Step 8:

[1239] Terminal: Displays an interface for collecting iris data and prompts the user to scan their iris.

[1240] Step 9:

[1241] User: Provides iris data by pointing their iris at the device camera.

[1242] Step 10:

[1243] Terminal: Sends collected iris data and emotion data to the server.

[1244] Step 11:

[1245] Server: Receives facial photos, fingerprint data, iris data, and emotion data, and uses generative AI to train a personal authentication model and emotion model.

[1246] Step 12:

[1247] Server: Stores the trained model in a database and associates the user's personal information with their account information.

[1248] Authentication Process

[1249] Step 13:

[1250] User: Selects "Facial Recognition Transaction" from the ATM or smartphone app.

[1251] Step 14:

[1252] Terminal: Display the face recognition and biometric authentication input interface.

[1253] Step 15:

[1254] User: Face the camera and prepare to scan your fingerprint.

[1255] Step 16:

[1256] Terminal: Collects facial photos, fingerprint data, and emotion data and sends them to the server.

[1257] Step 17:

[1258] Server: Analyzes the received facial photo, fingerprint data, and emotion data, and performs authentication and emotion recognition using generative AI.

[1259] Step 18:

[1260] Server: Notifies the device of the authentication results and emotional state analysis results.

[1261] Transaction Processing

[1262] Step 19:

[1263] User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[1264] Step 20:

[1265] Terminal: The emotion engine monitors the user's emotional state and sends the results to the server.

[1266] Step 21:

[1267] Server: Determines whether a withdrawal is possible based on the user's emotional state and account information.

[1268] Step 22:

[1269] Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[1270] Step 23:

[1271] Server: Sends notifications based on trading results and sentiment analysis to the device.

[1272] Result notification

[1273] Step 24:

[1274] Terminal: Shows users notifications based on trading results and emotional state.

[1275] Step 25:

[1276] Terminal: In the case of an ATM, dispenses cash and provides appropriate notifications (audio, text, visual) as needed.

[1277] Step 26:

[1278] User: Confirm the transaction result and receive the cash and receipt.

[1279] Example 2

[1280] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1281] Conventional cash cardless systems face the risk of user authentication failure or fraudulent transactions, making it difficult to conduct transactions quickly and safely, especially during disasters. Furthermore, because they do not take into account the user's emotional state, there is a need to improve the user experience. To solve these issues, a more reliable authentication system and transaction processing that takes the user's emotional state into account are needed.

[1282] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting facial authentication data and biometric authentication data, means for collecting emotion data, means for transmitting the collected data to the server, means for analyzing the collected data in the server and training a personal authentication model and an emotion model, means for storing the personal authentication model and the emotion model in a database and associating personal information and account information, means for analyzing the user's emotional state, means for processing a transaction for the user's account if authentication is successful and the emotional state is determined to be normal, and means for notifying the user based on the transaction result and the emotional state. This improves the accuracy of user authentication and reduces the risk of fraudulent transactions. Furthermore, it allows users to withdraw cash quickly and safely even during disasters, improving their sense of security.

[1283] "Facial recognition data" is digital data obtained by analyzing the features of a user's face.

[1284] "Biometric data" is digital data collected based on biological characteristics such as fingerprints, irises, or veins.

[1285] "Emotional data" is digital data that indicates the user's emotional state, collected based on facial expressions, tone of voice, speaking rate, etc.

[1286] The "server" is a device that analyzes the collected data and generates and stores personal authentication models and emotion models.

[1287] "Generative AI" is an artificial intelligence technology that uses machine learning algorithms to train models based on collected data.

[1288] A "personal authentication model" is a user identification model trained by generative AI based on facial recognition data and biometric authentication data.

[1289] An "emotion model" is a model that determines a user's emotional state, learned by generative AI based on emotional data.

[1290] "Database" refers to the system that stores and manages personal authentication models, emotion models, and users' personal information and account information.

[1291] "Authentication" is the process of verifying a user's identity based on collected data.

[1292] A "transaction" is a financial transaction such as withdrawing or transferring cash to a user's account.

[1293] "Transaction result" refers to information on whether the transaction was successful or unsuccessful, and is the content of the notification to the user based on that information.

[1294] This invention is a cash cardless system that combines an emotion engine, allowing users to conduct transactions safely and quickly through facial recognition and biometric authentication. The emotion engine also analyzes the user's emotional state and optimizes the transaction. This system is mainly composed of a terminal and a server.

[1295] System configuration

[1296] Terminal

[1297] The device includes the following hardware and software:

[1298] Input devices (cameras, fingerprint scanners, iris scanners, etc.) that allow users to enter facial and biometric data

[1299] Emotion engine (including camera and microphone) for recognizing user emotions

[1300] Displays and interfaces that guide users through the authentication process and their emotional state

[1301] A communication interface for sending data to the server

[1302] server

[1303] The server includes the following elements:

[1304] Generative AI to analyze received data (face photo, fingerprint data, iris data, emotion data) and generate a personal authentication model and emotion model

[1305] Database for managing personal and account information

[1306] Software and algorithms for authentication processes, emotion recognition, and transaction processing

[1307] Program processing

[1308] User information collection and registration

[1309] Users access a bank's ATM or smartphone app to register their facial recognition and biometric data. The device then uses its camera to take a photo of the user's face and scan their fingerprint and iris. It also activates an emotion engine to collect the user's facial expressions and tone of voice. The collected data is sent to a server via the device. The server analyzes the received data and uses generative AI to train a personal authentication model and emotion model. These models are stored in a database and associated with the user's personal and account information.

[1310] Authentication Process

[1311] The user selects "Facial Recognition Transaction" at the ATM or via a smartphone app. The terminal displays the facial recognition and biometric authentication input interface, and the user points their face at the camera, preparing to scan their fingerprint. The terminal collects a facial photo, fingerprint data, and emotional data, and sends them to the server. The server analyzes the received data and performs authentication and emotional recognition using generative AI. The authentication results and emotional state analysis results are then notified to the terminal.

[1312] Transaction Processing

[1313] After successful authentication, the user selects "Withdraw" on the ATM or app interface and enters the amount. The terminal monitors the user's emotional state using an emotion engine and sends the results to the server. The server determines whether a withdrawal is possible based on the user's emotional state and account information, and if so, withdraws the specified amount. The transaction results and notification method based on emotion analysis are sent to the terminal.

[1314] Result notification

[1315] The terminal displays notifications to the user based on the transaction result and emotional state. In the case of an ATM, it dispenses cash and provides appropriate notifications (audio, text, visual) as needed. The user can then confirm the transaction result and collect the cash and receipt.

[1316] Specific examples

[1317] Example 1: You accidentally forget your wallet

[1318] 1. Users register their facial recognition and fingerprint data at the ATM in advance, and emotional data is also collected using an emotion engine.

[1319] 2. The user goes to a bank ATM and selects "Facial Recognition Transaction."

[1320] 3. The device will require facial recognition and fingerprint scanning, and will also collect emotional data.

[1321] 4. The server analyzes the data and performs user authentication and emotion recognition.

[1322] 5. If authentication is successful and the transaction is deemed possible, the user enters the amount and the transaction is carried out.

[1323] 6. The terminal will display a successful transaction and dispense the cash.

[1324] 7. The user receives the cash and a receipt.

[1325] Example 2: When you lose your bankbook or cash card during a disaster

[1326] 1. Users register their facial recognition and biometric data at the emergency assistance center, and emotion data is also collected through the emotion engine.

[1327] 2. The user selects "Facial Recognition Transaction" on the terminal at the evacuation center.

[1328] 3. The device will request facial recognition and fingerprint scanning, and collect emotional data.

[1329] 4. The server analyzes the data and performs user authentication and emotion recognition.

[1330] 5. If authentication is successful and the transaction is deemed possible, the user enters the amount and the transaction is carried out.

[1331] 6. The terminal will display a successful transaction and dispense the cash.

[1332] 7. The user receives cash.

[1333] Prompt Sentence Examples

[1334] "Look at the camera and make sure your face is visible."

[1335] "Place your finger on the scanner"

[1336] "Place your eyes on the iris scanner."

[1337] "Hi, I'm feeling good today."

[1338] "Select facial recognition transaction"

[1339] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1340] User information collection and registration

[1341] Step 1:

[1342] Users access the bank's ATM or smartphone app and begin registering their facial recognition and biometric data.

[1343] Input: User access operations

[1344] Output: Registration process begins

[1345] Step 2:

[1346] The device will display the instruction "Please take a photo of your face" on the display.

[1347] Input: User registration process start operation

[1348] Output: Display of face photo taking screen, display of instruction message

[1349] Specific operation: Display an instruction message on the display and start the camera.

[1350] Step 3:

[1351] The user faces the camera and takes a photo of their face.

[1352] Input: User faces the camera

[1353] Output: Facial photo data

[1354] What happens: The user turns their face in front of the camera and their image is captured.

[1355] Step 4:

[1356] The device will then display an interface that says "Please scan your fingerprint."

[1357] Input: Operation after taking face photo

[1358] Output: Display of fingerprint scan screen, display of instruction message

[1359] Specific operation: Displays an instruction message on the display and activates the fingerprint scanner.

[1360] Step 5:

[1361] The user places their finger on the scanner to scan their fingerprint.

[1362] Input: The user places their finger on the scanner

[1363] Output: Fingerprint data

[1364] What happens: A user places their finger on a fingerprint scanner and their fingerprint is captured.

[1365] Step 6:

[1366] The terminal will display the instruction "Please scan your eyes" if an iris scanner is present.

[1367] Input: Actions after fingerprint scanning

[1368] Output: Display of iris scan screen, display of instruction message

[1369] Specific operations: Display an instruction message on the display and activate the iris scanner.

[1370] Step 7:

[1371] The user positions their eye over the iris scanner and scans their iris.

[1372] Input: The user's eye movement towards the scanner

[1373] Output: Iris data

[1374] What happens: The user points their eye at the iris scanner and their iris is captured.

[1375] Step 8:

[1376] The device will launch its emotion engine and display the message, "Collecting emotion data. Please record your face and voice."

[1377] Input: Actions after iris scan

[1378] Output: Display of emotion data collection screen, display of instruction message

[1379] Specific actions: Display an instruction message on the display and activate the camera and microphone.

[1380] Step 9:

[1381] The user faces the camera and speaks into a microphone.

[1382] Input: User's facial expressions and voice

[1383] Output: Emotion data (facial expressions, tone of voice, speaking speed)

[1384] Specific actions: The user shows their facial expression in front of the camera and speaks into the microphone.

[1385] Step 10:

[1386] The device transmits the collected facial recognition, biometric authentication, and emotion data to a server.

[1387] Input: Collected authentication data and emotion data

[1388] Output: Send data to the server

[1389] Specific operation: The emotion engine analyzes the data and uses the generation AI to send it to the server. During the transmission, the message "Sending data" is displayed.

[1390] Processing on the server

[1391] Step 11:

[1392] The server analyzes the received data and uses generative AI to learn a personal authentication model and emotion model.

[1393] Input: face photo, fingerprint data, iris data, emotion data

[1394] Output: Personal authentication model, emotion model

[1395] Specific operation: Analyzes the received data and generates a model by analyzing facial feature points, fingerprint patterns, voice spectrum, etc.

[1396] Step 12:

[1397] The server stores the learned model in a database and associates it with the user's personal and account information.

[1398] Input: Trained personal authentication model, emotion model

[1399] Output: Save to database

[1400] Specific operation: Link and save the model, personal information, and account information in the database.

[1401] Authentication Process

[1402] Step 13:

[1403] Users select "Facial Recognition Transaction" at the ATM or via a smartphone app.

[1404] Input: User's transaction start operation

[1405] Output: Instructions: "Select facial recognition transaction"

[1406] Specific operation: Display the trading start menu on the interface.

[1407] Step 14:

[1408] The terminal displays an input interface for face recognition and biometric authentication.

[1409] Input: User's trade selection operation

[1410] Output: Display of face recognition and fingerprint scanning screen

[1411] Specific actions: Display messages such as "Please face the camera" and "Please place your finger on the scanner."

[1412] Step 15:

[1413] The user faces the camera and prepares to scan their fingerprint.

[1414] Input: Aligning the user's face and fingers

[1415] Output: Facial photo data, fingerprint data

[1416] Specific operation: For facial recognition, the user points their face towards the camera, and for fingerprint scanning, they place their finger on the scanner.

[1417] Step 16:

[1418] The terminal collects a facial photograph, fingerprint data, and emotion data and transmits them to a server.

[1419] Input: Newly collected authentication data and emotion data

[1420] Output: Send data to the server

[1421] Specific behavior: Displays the message "Collecting data" and the emotion engine analyzes the data.

[1422] Processing on the server

[1423] Step 17:

[1424] The server analyzes the received data and performs authentication and emotion recognition using the generative AI.

[1425] Input: Data sent from the terminal

[1426] Output: Authentication results, sentiment analysis results

[1427] Specific operation: Analyzes the transmitted data using generative AI to authenticate the user and recognize emotions.

[1428] Step 18:

[1429] The server notifies the terminal of the authentication result and the emotional state analysis result.

[1430] Input: Authentication results, sentiment analysis results

[1431] Output: Send results to the terminal

[1432] Specific operation: The success or failure of authentication and the emotional state are returned to the terminal.

[1433] Transaction Processing

[1434] Step 19:

[1435] After successful authentication, the user selects "Withdraw" from the ATM or app interface and enters the amount.

[1436] Input: User withdrawal operation

[1437] Output: Withdrawal amount data

[1438] Specific operation: The message "Please enter the withdrawal amount" will be displayed on the screen.

[1439] Step 20:

[1440] The device monitors the user's emotional state using an emotion engine and sends the results to the server.

[1441] Input: Real-time user sentiment data

[1442] Output: Emotional state data sent to server

[1443] Specific behavior: Display the message "Checking emotional state."

[1444] Processing on the server

[1445] Step 21:

[1446] The server determines whether a withdrawal is possible based on the user's emotional state and account information.

[1447] Input: Emotional state data, account information

[1448] Output: Transaction availability decision

[1449] Specific behavior: If the user is very impatient, display a warning and limit withdrawals.

[1450] Step 22:

[1451] If withdrawal is possible, the server withdraws the specified amount and updates the transaction record.

[1452] Input: Transaction authorization data, specified amount

[1453] Output: Transaction execution, transaction record updated

[1454] Specific behavior: Display the message "Transaction processing" and update the transaction database.

[1455] Step 23:

[1456] The server sends the transaction results and notification method based on sentiment analysis to the terminal.

[1457] Input: Trading results, sentiment analysis results

[1458] Output: Transaction result notification data

[1459] Specific behavior: Returns the message "Transaction successful, please accept cash."

[1460] Result notification

[1461] Step 24:

[1462] The terminal displays notifications to the user according to the trading results and emotional state.

[1463] Input: Transaction result notification data

[1464] Output: Transaction results display

[1465] Specific behavior: Display a reassuring message along with "Transaction successful."

[1466] Step 25:

[1467] In the case of an ATM, the terminal dispenses cash and provides appropriate notifications (audio, text, visual) as needed.

[1468] Input: Transaction success data

[1469] Output: Withdrawal processing, voice guide output

[1470] Specific operation: A voice message will be heard saying, "Cash has been withdrawn, please take the receipt."

[1471] Step 26:

[1472] The user confirms the transaction result and receives the cash and receipt.

[1473] Input: ATM withdrawal notification

[1474] Output: Cash and receipt received

[1475] Specific action: Withdraw cash and receipt from ATM.

[1476] (Application example 2)

[1477] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1478] Current authentication systems using cash cards require users to have their physical cards in order to be authenticated, which poses significant challenges and security issues, especially in the event of a disaster or loss. Furthermore, traditional authentication systems do not take into account the user's emotional state, which can raise doubts about the legitimacy of the transaction. Furthermore, while fast and secure payments are required for in-store purchases, traditional systems do not adequately meet these demands.

[1479] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting facial authentication data and biometric authentication data, means for training an emotion model, and means for evaluating transaction appropriateness based on the emotional state. This not only eliminates the need for users to carry cards, but also enables appropriate transactions that take the emotional state into consideration, enabling rapid response in the event of a disaster and safe and rapid payment at stores.

[1480] "Facial recognition data" is information used to identify individuals by analyzing images of their faces and facial features.

[1481] "Biometric data" is information used to analyze an individual's biometric characteristics, such as fingerprints, irises, and voice, for authentication.

[1482] An "emotion model" is a model trained to analyze a user's facial expressions and tone of voice and estimate their emotional state.

[1483] "Smart glasses" are glasses-type devices worn by the user that incorporate devices such as a camera and a display.

[1484] "Transaction information" refers to data related to a transaction conducted by a user, such as the transaction amount and transaction details.

[1485] "Communication means" refers to the network infrastructure for transmitting and receiving data between the terminal and the server.

[1486] "Generative AI" is an artificial intelligence technology that analyzes facial recognition data and biometric authentication data to create personal authentication models and emotion models.

[1487] The "transaction appropriateness evaluation means" is a means for determining whether a transaction is appropriate based on the emotional state of the user.

[1488] "Authentication means" means a means for verifying a user's identity using facial recognition and biometric data.

[1489] The "means for notifying results" is a means for notifying users of the results of trading and sentiment analysis.

[1490] We will now explain a specific example of a cash cardless system incorporating the emotion engine of the present invention. This system is based on technology that authenticates users by combining facial recognition data and biometric authentication data, and simultaneously analyzes the user's emotional state to evaluate the appropriateness of the transaction.

[1491] System configuration

[1492] The system mainly consists of the following elements:

[1493] 1. Device:

[1494] Camera: Collects user facial recognition data.

[1495] Biometric devices: Collect biometric data such as fingerprints and irises.

[1496] Emotion engine: Analyzes the user's facial expressions and tone of voice to collect emotional data.

[1497] Display: Shows the authentication process and emotional state to the user.

[1498] Communication Interface: Sends data to the server.

[1499] 2. Server:

[1500] Data analysis and generation AI: Analyzes received facial recognition data, biometric data, and emotion data to generate personal recognition models and emotion models.

[1501] Database: Stores users' personal and transaction information and manages authentication models.

[1502] Transaction suitability assessment function: Determines the suitability of a transaction based on emotional state.

[1503] Program processing

[1504] The server runs on the following hardware and software:

[1505] Hardware: If high-performance computing is required, use cloud-based computing resources (e.g., Amazon Web Services, Google Cloud Platform, Microsoft Azure).

[1506] Software: Emotion analysis uses emotion recognition algorithms, and personal authentication uses face and iris recognition algorithms. These are often implemented using machine learning libraries such as TensorFlow and PyTorch.

[1507] Processing flow

[1508] The device first collects the user's facial and biometric data using a camera and biometric authentication device, and then uses an emotion engine to obtain the user's emotional data. All of this data is sent to the server via a communications interface. The server then analyzes the data using generative AI to generate a personal authentication model and an emotion model, which are then stored in a database. When a transaction is made, the server uses these models to authenticate the user and simultaneously evaluate the user's emotional state to determine whether the transaction is appropriate.

[1509] Specific examples

[1510] When shopping in a physical store, users provide facial recognition and biometric data through the smart glasses and select "Facial Recognition Transaction." The smart glasses' camera recognizes the face and scans fingerprints and iris data. The emotion engine also analyzes facial expressions and tone of voice to obtain emotion data. All of this data is sent to the server in real time, where authentication and emotion evaluation are performed. If authentication is successful, the transaction amount will be displayed on the smart glasses' display, and the user can confirm and complete the payment.

[1511] Prompt Sentence Examples

[1512] "To pay at the register, you need to be authenticated using facial recognition, iris recognition, and emotional data. Just face the camera and scan your iris and fingerprint."

[1513] Through these steps, users can complete transactions quickly and safely, which is particularly useful in times of disaster or emergency.

[1514] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1515] Step 1:

[1516] The user puts on the terminal (smart glasses) and selects facial recognition transaction. The terminal instructs the user to face the camera and simultaneously starts scanning fingerprint and iris data using the biometric authentication device. Inputs include the user's facial image and biometric data (fingerprint, iris). These are acquired as real-time video and scan data. The output is the collected facial recognition data and biometric authentication data.

[1517] Step 2:

[1518] The device launches an emotion engine to collect the user's facial expressions and tone of voice. The emotion engine uses a camera and microphone to analyze the user's emotional state. Here, facial expressions and tone of voice are used as input, and the output is emotion data. This emotion data reflects the user's real-time mental state.

[1519] Step 3:

[1520] The terminal transmits the collected facial recognition data, biometric data, and emotion data to the server through a communication interface. The input is the collected authentication data and emotion data, and the output is the transmission of these data to the server.

[1521] Step 4:

[1522] The server analyzes the received data and uses generative AI to generate a personal authentication model and emotion model. It takes in facial recognition data, biometric data, and emotion data as input, performs data analysis and machine learning, and generates a personal authentication model and emotion model as output. These models are stored in the server's database.

[1523] Step 5:

[1524] The server uses the generated personal authentication model and emotion model to authenticate the user. The input is the model stored on the server and the authentication data sent from the device. The output is the result of authentication success or failure. If authentication is successful, proceed to the next step.

[1525] Step 6:

[1526] The user checks the transaction details (e.g., payment amount, purchased items, etc.) on the smart glasses display and confirms the transaction. The input is the transaction details sent from the server, which the user checks and operates on the display. The output is the transaction confirmation result.

[1527] Step 7:

[1528] The server evaluates the user's emotional state and determines the appropriateness of the transaction. Using the confirmed transaction details and emotional data as input, it performs evaluation processing based on an emotional model. The output is an evaluation result regarding the appropriateness of the transaction.

[1529] Step 8:

[1530] The server processes the transaction and notifies the user of the result. The input is the confirmation of the transaction details and the evaluation result of the emotional state. The output is a notification of whether the transaction was completed, which is displayed on the display of the user's smart glasses. At this time, the transaction information is recorded in a database.

[1531] Step 9:

[1532] The terminal notifies the user of the transaction result and displays or electronically stores the receipt as needed. The input is a notification from the server that the transaction has been completed, and the output is a visual and electronic confirmation of the result for the user.

[1533] Through these steps, users can complete transactions safely and quickly, providing a system that is extremely useful, especially in times of disaster or emergency.

[1534] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1535] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1536] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1537] [Third embodiment]

[1538] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1539] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1540] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1541] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1542] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1543] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1544] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1545] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1546] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1547] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1548] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1549] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1550] System configuration

[1551] The cash cardless system of this invention is implemented in bank ATMs and smartphone apps. The system mainly consists of the following components:

[1552] 1. Terminal

[1553] An input device (such as a camera, fingerprint scanner, or iris scanner) that allows users to enter facial and biometric data.

[1554] A display or interface that guides the user through the authentication process.

[1555] A communication interface for sending data to the server.

[1556] 2. Server

[1557] Generative AI to analyze received data and generate a personal authentication model.

[1558] A database that manages personal and account information.

[1559] Software and algorithms for authentication processes and transaction processing.

[1560] Program processing procedure

[1561] The program processing of the system will be described in detail below.

[1562] User information collection and registration

[1563] 1. User: Visits a bank ATM or smartphone app and registers facial recognition and biometric data.

[1564] 2. Terminal: Have the user take a photo of their face and scan their fingerprint and iris.

[1565] 3. Terminal: The collected data is sent to the server via secure communication.

[1566] 4. Server: Analyzes the received data and uses generative AI to train a personal authentication model.

[1567] 5. Server: Stores the learned personal authentication model in a database and associates the user's personal information with their account information.

[1568] Authentication Process

[1569] 1. User: Select "Facial Recognition Transaction" from the ATM or app.

[1570] 2. Terminal: Display the face recognition and biometric authentication input interface.

[1571] 3. User: Faces the device camera and scans their fingerprint.

[1572] 4. Terminal: Sends collected data to the server.

[1573] 5. Server: Authenticates the user using the generative AI.

[1574] 6. Server: If authentication is successful, it notifies the terminal of the result.

[1575] Transaction Processing

[1576] 1. User: After successful authentication, enter the withdrawal amount.

[1577] 2. Terminal: Sends the entered amount information to the server.

[1578] 3. Server: Determines whether withdrawal is possible based on the user's account information.

[1579] 4. Server: If withdrawal is possible, withdraw the specified amount.

[1580] 5. Server: Updates the transaction record and sends a success notification to the device.

[1581] Result notification

[1582] 1. Terminal: displays the transaction results to the user.

[1583] 2. Terminal: In the case of an ATM, it dispenses cash and prints a receipt.

[1584] 3. User: Check the transaction result and receive the cash and receipt.

[1585] Specific examples

[1586] Example 1: You accidentally forget your wallet

[1587] 1. Collecting and registering user information: Register facial recognition and fingerprint data at the ATM in advance.

[1588] 2. Authentication Process:

[1589] User: Goes to a bank ATM and selects facial recognition transaction.

[1590] On the device: Have the user face the camera and scan their fingerprint.

[1591] Server: Analyzes the data and authenticates the user.

[1592] 3. Transaction Processing:

[1593] User: After successful authentication, enter the amount from the withdrawal menu.

[1594] Terminal: Sends the amount to the server.

[1595] Server: Determines whether withdrawals are possible and processes transactions.

[1596] 4. Result notification:

[1597] Terminal: Shows successful transaction and withdraws cash.

[1598] User: Take cash and receipt.

[1599] Example 2: When you lose your bankbook or cash card during a disaster

[1600] 1. User information collection and registration: Register facial recognition and biometric data at the emergency assistance center.

[1601] 2. Authentication Process:

[1602] User: Selects facial recognition transaction on the shelter terminal.

[1603] Device: Have your face face the camera and scan your fingerprint.

[1604] Server: Analyzes the data and authenticates the user.

[1605] 3. Transaction Processing:

[1606] User: After successful authentication, enter the withdrawal amount.

[1607] Terminal: Sends the amount to the server.

[1608] Server: Determines whether withdrawal is possible and processes the transaction.

[1609] 4. Result notification:

[1610] Terminal: Shows successful transaction and withdraws cash.

[1611] User: Take the cash.

[1612] The system of the present invention allows users to safely withdraw money without a cash card, improving convenience and peace of mind for users. In particular, in the event of a disaster, it allows users to quickly obtain cash, which has the effect of reducing the mental burden on victims.

[1613] The processing flow will be explained below.

[1614] Specific processing steps of the program

[1615] User information collection and registration

[1616] Step 1:

[1617] User: Opens the bank's ATM or smartphone app and opens the screen to register facial recognition and biometric data.

[1618] Step 2:

[1619] On the device: Displays an interface for taking a photo of the user's face and asks the user to face the camera.

[1620] Step 3:

[1621] User: Follow the instructions on the device to take a photo of your face in front of the camera.

[1622] Step 4:

[1623] Device: Sends the captured photo to the server.

[1624] Step 5:

[1625] On the device: Display an interface to collect fingerprint data using a fingerprint scanner and have the user scan their fingerprint.

[1626] Step 6:

[1627] User: Places finger on fingerprint scanner to provide fingerprint data.

[1628] Step 7:

[1629] Device: Sends collected fingerprint data to the server.

[1630] Step 8:

[1631] Terminal: Displays an interface for collecting iris data and prompts the user to scan their iris.

[1632] Step 9:

[1633] User: Provides iris data by pointing their iris at the device camera.

[1634] Step 10:

[1635] Terminal: Sends collected iris data to the server.

[1636] Step 11:

[1637] Server: Receives facial photos, fingerprint data, and iris data, and trains a personal authentication model using generative AI.

[1638] Step 12:

[1639] Server: The trained personal authentication model is stored in a database and associated with the user's personal and account information.

[1640] Authentication Process

[1641] Step 13:

[1642] User: Selects "Facial Recognition Transaction" from the ATM or smartphone app.

[1643] Step 14:

[1644] Terminal: Display the face recognition and biometric authentication input interface.

[1645] Step 15:

[1646] User: Face the camera and prepare to scan your fingerprint.

[1647] Step 16:

[1648] Terminal: Collects facial photo and fingerprint data and sends them to the server.

[1649] Step 17:

[1650] Server: Analyzes the received facial photo and fingerprint data and performs authentication using the generated AI.

[1651] Step 18:

[1652] Server: Notifies the terminal of the authentication result.

[1653] Transaction Processing

[1654] Step 19:

[1655] User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[1656] Step 20:

[1657] Terminal: Sends the amount data entered by the user to the server.

[1658] Step 21:

[1659] Server: Checks the user's account information and determines whether a withdrawal is possible.

[1660] Step 22:

[1661] Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[1662] Step 23:

[1663] Server: Sends the transaction results to the terminal.

[1664] Result notification

[1665] Step 24:

[1666] Terminal: Displays the transaction results to the user.

[1667] Step 25:

[1668] Terminal: In the case of an ATM, it dispenses cash and prints receipts.

[1669] Step 26:

[1670] User: Checks and receives cash and receipts.

[1671] This series of processing steps allows users to withdraw money safely and quickly using facial and biometric authentication, without using a cash card.

[1672] Example 1

[1673] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1674] In recent years, the loss and theft of cash cards and bankbooks has been increasing, and the risk of fraudulent transactions has also increased. In particular, in emergencies such as natural disasters and fires, if a bankbook or cash card is lost or lost, it can be difficult for victims to quickly secure cash. Furthermore, conventional authentication systems often use passwords or PIN codes, which still pose security risks. Therefore, to solve these issues, a system that can safely and quickly authenticate individuals and perform financial transactions is needed.

[1675] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1676] In this invention, the server includes a means for collecting facial recognition data and biometric authentication data, a means for training a personal authentication model using a generative AI, and a means for storing the personal authentication model in a database and associating personal information and financial information, thereby enabling safe and fast user authentication and financial transactions.

[1677] "Facial recognition data" is data that digitally represents a user's facial features.

[1678] "Biometric data" means a digital representation of a user's biological characteristics, such as a fingerprint or iris.

[1679] A "server" is a computer system that stores and processes data over a network.

[1680] "Generative AI" is a program that uses artificial intelligence technology to perform a specific task, and in this invention is used particularly for training a personal authentication model.

[1681] A "personal authentication model" is a model for uniquely identifying a user using collected facial recognition data and biometric authentication data.

[1682] A "database" is a system for efficiently storing and managing large amounts of data.

[1683] "Personal Information" means identifying information, such as name and address, associated with a user.

[1684] "Financial information" refers to information related to a user's account information, transaction history, etc.

[1685] "User authentication" is the process of verifying a user's identity based on collected authentication data.

[1686] A "financial transaction" refers to a monetary transaction, such as a withdrawal from or transfer to a user's account.

[1687] "Trading results" refers to information obtained as a result of a financial transaction, including the success or failure of the transaction.

[1688] The cash cardless system of this invention is implemented in bank ATMs and smartphone apps. The system is mainly composed of a terminal and a server.

[1689] System configuration

[1690] Terminal

[1691] The terminal is equipped with an input device for users to enter facial and biometric authentication data, such as a camera, fingerprint scanner, or iris scanner, and provides a display or interface that guides the user through the authentication process. The terminal also transmits the collected data to a server through a secure communication interface.

[1692] Camera: A device used to take a photo of the user's face.

[1693] Fingerprint scanner: A device used to scan a user's fingerprint.

[1694] Iris scanner: A device used to scan a user's iris.

[1695] Display: A device used to display authentication procedures and transaction details.

[1696] Communication interface: A secure protocol (e.g. HTTPS) for sending data to the server.

[1697] server

[1698] The server analyzes the received data and generates a personal authentication model using generative AI. Specifically, it uses AI frameworks such as TensorFlow and PyTorch. It also has a database that manages personal and financial information. This database stores the trained personal authentication model and also contains software and algorithms for processing user transactions.

[1699] Generative AI: Train a person authentication model using TensorFlow and PyTorch.

[1700] Database: Manage personal and financial information using database systems such as MySQL or PostgreSQL.

[1701] Transaction Processing Algorithm: Software used to process users' financial transactions.

[1702] Specific examples

[1703] Example 1: You accidentally forget your wallet

[1704] 1. Collecting and registering user information: Register facial recognition and fingerprint data in advance at the bank ATM.

[1705] User: Accesses the ATM and registers facial recognition and fingerprint data.

[1706] Device: Takes a photo of the user's face and scans their fingerprint, sending the data to a server.

[1707] Server: Analyzes the data, uses generative AI to train a personal authentication model, and stores it in a database.

[1708] 2. Authentication Process:

[1709] User: Go to a bank ATM and select "Facial Recognition Transaction."

[1710] Device: Have your face face the camera and scan your fingerprint.

[1711] Server: Analyzes the collected data and authenticates users.

[1712] 3. Transaction Processing:

[1713] User: After successful authentication, enter the desired amount in the withdrawal menu.

[1714] Terminal: Sends the entered amount information to the server.

[1715] Server: Determines whether withdrawal is possible based on the user's account information and processes the transaction.

[1716] 4. Result notification:

[1717] Terminal: Shows successful transaction and withdraws cash.

[1718] User: Take cash and receipt.

[1719] Example 2: When you lose your bankbook or cash card during a disaster

[1720] 1. User information collection and registration: Register facial recognition and biometric data at the emergency assistance center.

[1721] User: Register facial recognition and biometric data on a terminal at the evacuation center.

[1722] Device: Face the camera, scan your fingerprint, and send the data to the server.

[1723] Server: Uses generative AI to train a personal authentication model and saves it in a database.

[1724] 2. Authentication Process:

[1725] User: Select "Facial Recognition Transaction" on the shelter terminal.

[1726] Device: Have your face face the camera and scan your fingerprint.

[1727] Server: Analyzes the data and authenticates the user.

[1728] 3. Transaction Processing:

[1729] User: After successful authentication, enter the withdrawal amount.

[1730] Terminal: Sends the amount to the server.

[1731] Server: Determines whether withdrawals are possible and processes transactions.

[1732] 4. Result notification:

[1733] Terminal: Shows successful transaction and withdraws cash.

[1734] User: Take the cash.

[1735] Prompt Sentence Examples

[1736] "Please explain your generative AI model for user authentication. Explain the process steps for analyzing the data using facial and fingerprint recognition."

[1737] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1738] Program processing flow

[1739] Step 1:

[1740] User: Accesses the bank's ATM or smartphone app and begins enrolling facial recognition and biometric data. Selects "New Enrollment" from the app's home screen and begins enrolling facial recognition and fingerprint data.

[1741] Input: Facial photo and fingerprint data

[1742] Output: Registration ready message

[1743] Specific steps: Tap "New Registration" from the app menu and follow the instructions to enter your face photo and fingerprint data.

[1744] Step 2:

[1745] Terminal: Takes a photo of the user's face and scans their fingerprint. Uses the ATM's camera to take a photo of the face and a fingerprint scanner to read the fingerprint.

[1746] Input: User's facial photo and fingerprint data

[1747] Output: Collected data

[1748] Specific actions: The app displays instructions to the user, such as "Please face the camera" and "Please place your finger on the fingerprint scanner."

[1749] Step 3:

[1750] Device: Send collected data to the server using a secure communication protocol (e.g. HTTPS).

[1751] Input: Collected data

[1752] Output: Data transmission completed message

[1753] Specific actions: Encrypt the data and send it to the server.

[1754] Step 4:

[1755] Server: Analyzes the received data and uses generative AI (using TensorFlow or PyTorch) to train a personal authentication model.

[1756] Input: Collected data

[1757] Output: Personal authentication model

[1758] Specific operations: Analyze received facial recognition and biometric data to train an AI model.

[1759] Step 5:

[1760] Server: The trained personal authentication model is stored in a database (MySQL or PostgreSQL) and associated with the user's personal and financial information.

[1761] Input: Personal authentication model and user information

[1762] Output: Updated database

[1763] Specific operation: Stores the authentication model in the database, linked to the user ID, and associates personal and financial information.

[1764] Step 6:

[1765] User: Accesses the bank's ATM or app and selects "Facial Recognition Transactions." For example, selects "Facial Recognition Transactions" from the ATM menu screen.

[1766] Input: Operation selection

[1767] Output: Authentication interface display

[1768] Specific operation: Select "Facial Recognition Transactions" on the ATM menu screen.

[1769] Step 7:

[1770] Terminal: Display the input interface for face recognition and biometric authentication. Display the camera image for face recognition and the fingerprint scanner image.

[1771] Input: Menu selection information

[1772] Output: Authentication interface

[1773] Specific actions: "Please face the camera" and "Please place your finger on the fingerprint scanner" are displayed.

[1774] Step 8:

[1775] User: Faces the device camera and scans a fingerprint, for example, by facing the camera lens and pressing a finger against the scanner.

[1776] Input: Facial photo and fingerprint data

[1777] Output: Authentication data

[1778] Specific actions: Stand in front of the camera for facial recognition and place your finger on the fingerprint scanner.

[1779] Step 9:

[1780] Terminal: Sends collected data to the server. The data is encrypted and sent using a secure communication protocol.

[1781] Input: Authentication data

[1782] Output: Data transmission completed message

[1783] Specific operation: Encrypts collected data and sends it to a server.

[1784] Step 10:

[1785] Server: Authenticates the user using generative AI, matching the stored authentication model with newly acquired data.

[1786] Input: Authentication data and personal authentication model

[1787] Output: Authentication result

[1788] Specific operation: Analyzes the received data and compares it with the stored AI model to evaluate the degree of match.

[1789] Step 11:

[1790] Server: If authentication is successful, it notifies the terminal of the result, encrypts the authentication result, and sends it to the terminal.

[1791] Input: Authentication result

[1792] Output: Authentication success message

[1793] Specific operation: Sends a message of successful authentication to the terminal.

[1794] Step 12:

[1795] User: After successful authentication, enter the amount to withdraw. Enter the desired amount on the ATM's input screen.

[1796] Input: Withdrawal amount

[1797] Output: Amount information entered

[1798] Specific operation: Enter the amount in the amount input field displayed on the ATM screen.

[1799] Step 13:

[1800] Terminal: Sends the entered amount information to the server. The entered data is encrypted and sent securely.

[1801] Input: Amount information

[1802] Output: Data transmission completed message

[1803] Specific operation: Sends the entered amount information to the server.

[1804] Step 14:

[1805] Server: Determines whether withdrawal is possible based on the user's account information. Checks the account balance and compares it with the requested amount.

[1806] Input: Amount information and account information

[1807] Output: Withdrawal decision result

[1808] Specific operation: Check the account balance and determine whether a withdrawal is possible.

[1809] Step 15:

[1810] Server: If withdrawal is possible, withdraw the specified amount. The server processes the transaction and updates the account balance.

[1811] Input: Withdrawal decision result

[1812] Output: Transaction processing results

[1813] Specific actions: Process the transaction and withdraw the specified amount.

[1814] Step 16:

[1815] Server: Updates the transaction record and sends a success notification to the terminal. Adds the transaction record to the database and sends a success message to the terminal.

[1816] Input: Transaction processing result

[1817] Output: Success message

[1818] Specific operations: Save the transaction record in the database and send a notification of successful transaction to the terminal.

[1819] Step 17:

[1820] Terminal: Display the transaction result to the user. Display a successful transaction message on the ATM screen.

[1821] Input: Success message

[1822] Output: Trade result display

[1823] Specific behavior: Display a message indicating the transaction was successful.

[1824] Step 18:

[1825] Terminal: In the case of an ATM, it dispenses cash and prints a receipt. The ATM's cash dispenser dispenses the specified amount, and the receipt printer prints the transaction details.

[1826] Input: Success message

[1827] Output: Cash and receipt

[1828] Specific actions: Dispense cash and print a receipt.

[1829] Step 19:

[1830] User: Check the transaction result and receive the cash and receipt. Take the cash and receipt that come out of the ATM.

[1831] Input: Cash and receipt

[1832] Output: Transaction completed

[1833] Specific actions: Check the transaction results and take the cash and receipt.

[1834] (Application example 1)

[1835] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1836] In conventional financial transactions using cash cards or bankbooks, if these physical media are lost, transactions often become difficult to complete. Furthermore, in emergencies such as natural disasters, earthquakes, or fires, cards and bankbooks can be damaged or lost, making it difficult to quickly obtain necessary funds. Furthermore, from a security standpoint, there is a risk of fraudulent use if cards or bankbooks are stolen. There is a need for a system that can solve these issues and enable users to conduct financial transactions quickly and with peace of mind.

[1837] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1838] In this invention, the server includes means for collecting facial authentication data and biometric authentication data, means for transmitting the collected data to the server, means for analyzing the collected data in the server and training a personal authentication model, means for saving the personal authentication model in a database and associating personal information and account information, means for authenticating a user based on the facial authentication and biometric authentication data, means for processing a transaction for the user's account if authentication is successful, means for notifying the user of the transaction result, means for confirming and approving the payment amount, means for transmitting payment amount information to the server after authentication is successful, and means for saving the transaction history of the payment amount so that the user can confirm it. This eliminates the need for physical cash cards or passbooks, allowing transactions to be carried out quickly, especially in emergencies such as natural disasters, earthquakes, and fires, while also improving security.

[1839] "Facial recognition data" is data used to analyze image data of a user's face and identify the individual.

[1840] "Biometric data" is data used to identify an individual using biometric characteristics such as fingerprints or irises.

[1841] "Means of collection" refers to devices and programs for acquiring facial recognition data and biometric authentication data.

[1842] "Transmitting means" refers to a device or program that has the function of transmitting collected data to a server.

[1843] The "analyzing means" refers to a device or program for processing the received data and generating a personal authentication model.

[1844] "Means for training" refers to devices or programs for training a personal authentication model using generative AI.

[1845] A "personal authentication model" is an algorithm or program for identifying individuals based on collected facial recognition data and biometric data.

[1846] "Database" means a storage device for storing and managing personal authentication models, personal information, and account information.

[1847] "Means for authentication" refers to devices or programs that verify a user's identity based on facial recognition and biometric data.

[1848] A "means for processing transactions" is a device or program that, after successful authentication, executes a financial transaction against a user's account.

[1849] "Means of notification" refers to devices or programs used to notify users of transaction results.

[1850] "Means for confirming and approving the payment amount" refers to a device or program that allows the user to confirm and approve the payment amount.

[1851] The "means for transmitting payment amount information" refers to a device or program for transmitting information on the payment amount to the server after successful authentication.

[1852] "Means for storing transaction history" refers to a device or program that records the results of each transaction and allows users to check them.

[1853] "Generative AI" is an artificial intelligence technology that analyzes data and trains a personal authentication model.

[1854] MODE FOR CARRYING OUT THE INVENTION

[1855] The facial recognition electronic payment system of the present invention uses facial recognition data and biometric authentication data to realize safe and fast financial transactions without the need for physical cash cards or bankbooks.

[1856] System configuration

[1857] The system mainly consists of the following components:

[1858] 1. User device (smartphone)

[1859] 2. Server

[1860] 3. Database

[1861] Hardware and software used

[1862] Smartphone: Camera for facial recognition, fingerprint scanner for fingerprint recognition

[1863] Facial recognition software: OpenCV

[1864] Generative AI model: TensorFlow

[1865] Communication interface: Secure HTTP communication

[1866] Database: SQL-based database

[1867] Program processing overview and specific examples

[1868] User information collection and registration

[1869] Users use their smartphone's camera and fingerprint scanner to collect facial and biometric data, which is then transmitted to a server through a secure communication interface.

[1870] The server analyzes the received data and uses the generative AI model to create a personal authentication model, which is stored in a database and associated with the user's personal and account information.

[1871] Authentication Process

[1872] When making a payment, users open a smartphone app and enter their facial and biometric data, which includes pointing their face at the camera and placing their fingerprint on a scanner. The smartphone then sends this data to a server, which uses a generative AI model to authenticate the user.

[1873] Payment Processing

[1874] If authentication is successful, the user confirms and approves the payment amount. The smartphone sends the confirmed payment amount to the server. The server processes the payment based on the user's account information and saves the transaction results in a database.

[1875] Result notification

[1876] The results of the transaction are notified to the user, who can check the results via a smartphone app.

[1877] Specific application

[1878] Example 1: Use in everyday life

[1879] When shopping at a supermarket, a user uses a facial recognition electronic payment app. After shopping is complete, the user opens the app at the cash register, performs facial and biometric authentication, and approves the payment. The server processes the authentication and payment, and the transaction result is notified to the user within a few seconds.

[1880] Example 2: Use during a disaster

[1881] Users who lose their bankbooks or cash cards during a disaster can use terminals installed at evacuation shelters to quickly withdraw cash using facial and biometric authentication, enabling safe and fast transactions even under certain circumstances.

[1882] Prompt Sentence Examples

[1883] "Please show us how to obtain a user's facial image and fingerprint data and perform secure authentication."

[1884] "If bankbooks and cash cards become unusable during a natural disaster, please explain how to use facial recognition as an alternative method for transactions."

[1885] As a result, physical cash cards and bankbooks are no longer necessary, allowing for quick transactions, especially in emergencies such as natural disasters, earthquakes, and fires, while also improving security.

[1886] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1887] Step 1:

[1888] The user downloads the facial recognition electronic payment app to their smartphone and launches it. The user begins facial and fingerprint authentication on the new registration screen (input: user's face and fingerprint, output: collected facial recognition data and biometric authentication data).

[1889] Step 2:

[1890] The device uses the smartphone's camera and fingerprint scanner to collect the user's facial authentication data and fingerprint data (data processing: acquisition of facial data and fingerprint data, output: collected facial authentication data and biometric authentication data).

[1891] Step 3:

[1892] The device encrypts the collected data and sends it to the server using a secure communication protocol (e.g., HTTPS) (input: collected facial recognition data and biometric authentication data, output: encrypted data sent to the server).

[1893] Step 4:

[1894] The server analyzes the received facial recognition data and biometric authentication data and creates a personal authentication model using a generative AI model (input: encrypted and transmitted facial recognition data and biometric authentication data; data calculation: training of the model using the generative AI; output: personal authentication model).

[1895] Step 5:

[1896] The server stores the personal authentication model in a database and associates it with the user's personal information and payment information (input: personal authentication model, personal information and payment information; output: personal authentication model and related information stored in the database).

[1897] Step 6:

[1898] When making a payment, the user launches the app and proceeds to the face authentication and fingerprint authentication screen (input: user operation, output: face authentication and fingerprint authentication screen).

[1899] Step 7:

[1900] The device collects the facial recognition data and biometric authentication data again and sends them to the server (input: user's face and fingerprint, data processing: recollected facial recognition data and biometric authentication data, output: data sent to the server).

[1901] Step 8:

[1902] The server compares the received data with the personal authentication model again and authenticates the user (input: recollected data, personal authentication model, data calculation: model comparison, output: authentication result).

[1903] Step 9:

[1904] If the authentication is successful, the user confirms and approves the payment amount, and this information is also sent to the server (Input: User approves the amount, Output: Payment amount information).

[1905] Step 10:

[1906] The server processes the transaction based on the user's account information and payment amount information, and records the payment results (input: payment amount information, account information, data calculation: transaction processing and recording, output: transaction results and recording).

[1907] Step 11:

[1908] The server notifies the terminal of the transaction result, and the terminal displays it to the user (input: transaction result, output: notification to user).

[1909] The above is the flow of processing steps of this system.

[1910] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1911] System configuration

[1912] The cash cardless system incorporating the emotion engine of the present invention consists of the following elements:

[1913] 1. Terminal

[1914] An input device (such as a camera, fingerprint scanner, or iris scanner) that allows users to enter facial and biometric data.

[1915] An emotion engine (including a camera and microphone) to recognize user emotions.

[1916] Displays and interfaces that show the authentication process and emotional state to the user.

[1917] A communication interface for sending data to the server.

[1918] 2. Server

[1919] A generative AI that analyzes received data (face photos, fingerprint data, iris data, emotion data) and generates a personal authentication model and emotion model.

[1920] A database that manages personal and account information.

[1921] Software and algorithms for authentication processes, emotion recognition, and transaction processing.

[1922] Program processing procedure

[1923] The program processing of the system will be described in detail below.

[1924] User information collection and registration

[1925] 1. User: Visits a bank ATM or smartphone app and registers facial recognition and biometric data.

[1926] 2. Terminal: Have the user take a photo of their face and scan their fingerprint and iris.

[1927] 3. Device: Activate the emotion engine and collect the user's facial expressions and tone of voice.

[1928] 4. Terminal: Sends collected facial recognition, biometrics, and emotion data to the server.

[1929] 5. Server: Analyzes the received data and uses generative AI to train a personal authentication model and emotion model.

[1930] 6. Server: Stores the trained model in a database and associates the user's personal and account information.

[1931] Authentication Process

[1932] 1. User: Select "Facial Recognition Transaction" from the ATM or smartphone app.

[1933] 2. Terminal: Display the face recognition and biometric authentication input interface.

[1934] 3. User: Faces the camera and prepares to scan his / her fingerprint.

[1935] 4. Terminal: Collects facial photos, fingerprint data, and emotion data and sends them to the server.

[1936] 5. Server: Analyzes the received data and performs authentication and emotion recognition using generative AI.

[1937] 6. Server: Notifies the device of the authentication result and emotional state analysis result.

[1938] Transaction Processing

[1939] 1. User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[1940] 2. Terminal: The emotion engine monitors the user's emotional state and sends the results to the server.

[1941] 3. Server: Determines whether withdrawal is possible based on the user's emotional state and account information.

[1942] 4. Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[1943] 5. Server: Sends trading results and notification methods based on sentiment analysis to the terminal.

[1944] Result notification

[1945] 1. Terminal: displays notifications to users based on trading results and emotional state.

[1946] 2. Terminal: In the case of an ATM, dispenses cash and provides appropriate notifications (audio, text, visual) as needed.

[1947] 3. User: Check the transaction result and receive the cash and receipt.

[1948] Specific examples

[1949] Example 1: You accidentally forget your wallet

[1950] 1. Collecting and registering user information: Facial recognition and fingerprint data are registered in advance at the ATM, and emotion data is also collected using the emotion engine.

[1951] 2. Authentication Process:

[1952] User: Goes to a bank ATM and selects facial recognition transaction.

[1953] Device: The device asks the user to face the camera, scans their fingerprint, and collects emotional data.

[1954] Server: Analyzes data and performs user authentication and emotion recognition.

[1955] 3. Transaction Processing:

[1956] User: After successful authentication, enter the amount from the withdrawal menu.

[1957] Terminal: Sends the amount and user emotion data to the server.

[1958] Server: Determines whether withdrawals are possible and processes transactions.

[1959] 4. Result notification:

[1960] Terminal: Shows successful transaction and withdraws cash.

[1961] User: Take cash and receipt.

[1962] Example 2: When you lose your bankbook or cash card during a disaster

[1963] 1. Collecting and registering user information: The emergency support center will register facial recognition and biometric data, and also collect emotional data from the emotion engine.

[1964] 2. Authentication Process:

[1965] User: Selects facial recognition transaction on the shelter terminal.

[1966] Device: The device asks the user to face the camera, scans their fingerprint, and collects emotional data.

[1967] Server: Analyzes data and performs user authentication and emotion recognition.

[1968] 3. Transaction Processing:

[1969] User: After successful authentication, enter the withdrawal amount.

[1970] Terminal: Sends the amount and user emotion data to the server.

[1971] Server: Determines whether withdrawal is possible and processes the transaction.

[1972] 4. Result notification:

[1973] Terminal: Displays notifications for successful transactions and emotional states, and withdraws cash.

[1974] User: Take the cash.

[1975] By using the system of this invention, users can safely and quickly withdraw money without using a cash card. Furthermore, by combining it with an emotion engine, optimal transactions can be performed according to the user's emotional state, enhancing convenience and peace of mind. In particular, in the event of a disaster, this system allows users to obtain cash quickly and appropriately, which has the effect of reducing the mental burden on victims.

[1976] The processing flow will be explained below.

[1977] Specific processing steps of the program

[1978] User information collection and registration

[1979] Step 1:

[1980] User: Opens the bank's ATM or smartphone app and opens the screen to register facial recognition and biometric data.

[1981] Step 2:

[1982] On the device: Displays an interface for taking a photo of the user's face and asks the user to face the camera.

[1983] Step 3:

[1984] User: Follow the instructions on the device to take a photo of your face in front of the camera.

[1985] Step 4:

[1986] Device: Sends the captured photo to the server.

[1987] Step 5:

[1988] On the device: Display an interface to collect fingerprint data using a fingerprint scanner and have the user scan their fingerprint.

[1989] Step 6:

[1990] User: Places finger on fingerprint scanner to provide fingerprint data.

[1991] Step 7:

[1992] Device: Sends collected fingerprint data to the server.

[1993] Step 8:

[1994] Terminal: Displays an interface for collecting iris data and prompts the user to scan their iris.

[1995] Step 9:

[1996] User: Provides iris data by pointing their iris at the device camera.

[1997] Step 10:

[1998] Terminal: Sends collected iris data and emotion data to the server.

[1999] Step 11:

[2000] Server: Receives facial photos, fingerprint data, iris data, and emotion data, and uses generative AI to train a personal authentication model and emotion model.

[2001] Step 12:

[2002] Server: Stores the trained model in a database and associates the user's personal information with their account information.

[2003] Authentication Process

[2004] Step 13:

[2005] User: Selects "Facial Recognition Transaction" from the ATM or smartphone app.

[2006] Step 14:

[2007] Terminal: Display the face recognition and biometric authentication input interface.

[2008] Step 15:

[2009] User: Face the camera and prepare to scan your fingerprint.

[2010] Step 16:

[2011] Terminal: Collects facial photos, fingerprint data, and emotion data and sends them to the server.

[2012] Step 17:

[2013] Server: Analyzes the received facial photo, fingerprint data, and emotion data, and performs authentication and emotion recognition using generative AI.

[2014] Step 18:

[2015] Server: Notifies the device of the authentication results and emotional state analysis results.

[2016] Transaction Processing

[2017] Step 19:

[2018] User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[2019] Step 20:

[2020] Terminal: The emotion engine monitors the user's emotional state and sends the results to the server.

[2021] Step 21:

[2022] Server: Determines whether a withdrawal is possible based on the user's emotional state and account information.

[2023] Step 22:

[2024] Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[2025] Step 23:

[2026] Server: Sends notifications based on trading results and sentiment analysis to the device.

[2027] Result notification

[2028] Step 24:

[2029] Terminal: Shows users notifications based on trading results and emotional state.

[2030] Step 25:

[2031] Terminal: In the case of an ATM, dispenses cash and provides appropriate notifications (audio, text, visual) as needed.

[2032] Step 26:

[2033] User: Confirm the transaction result and receive the cash and receipt.

[2034] Example 2

[2035] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2036] Conventional cash cardless systems face the risk of user authentication failure or fraudulent transactions, making it difficult to conduct transactions quickly and safely, especially during disasters. Furthermore, because they do not take into account the user's emotional state, there is a need to improve the user experience. To solve these issues, a more reliable authentication system and transaction processing that takes the user's emotional state into account are needed.

[2037] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting facial authentication data and biometric authentication data, means for collecting emotion data, means for transmitting the collected data to the server, means for analyzing the collected data in the server and training a personal authentication model and an emotion model, means for storing the personal authentication model and the emotion model in a database and associating personal information and account information, means for analyzing the user's emotional state, means for processing a transaction for the user's account if authentication is successful and the emotional state is determined to be normal, and means for notifying the user based on the transaction result and the emotional state. This improves the accuracy of user authentication and reduces the risk of fraudulent transactions. Furthermore, it allows users to withdraw cash quickly and safely even during disasters, improving their sense of security.

[2038] "Facial recognition data" is digital data obtained by analyzing the features of a user's face.

[2039] "Biometric data" is digital data collected based on biological characteristics such as fingerprints, irises, or veins.

[2040] "Emotional data" is digital data that indicates the user's emotional state, collected based on facial expressions, tone of voice, speaking rate, etc.

[2041] The "server" is a device that analyzes the collected data and generates and stores personal authentication models and emotion models.

[2042] "Generative AI" is an artificial intelligence technology that uses machine learning algorithms to train models based on collected data.

[2043] A "personal authentication model" is a user identification model trained by generative AI based on facial recognition data and biometric authentication data.

[2044] An "emotion model" is a model that determines a user's emotional state, learned by generative AI based on emotional data.

[2045] "Database" refers to the system that stores and manages personal authentication models, emotion models, and users' personal information and account information.

[2046] "Authentication" is the process of verifying a user's identity based on collected data.

[2047] A "transaction" is a financial transaction such as withdrawing or transferring cash to a user's account.

[2048] "Transaction result" refers to information on whether the transaction was successful or unsuccessful, and is the content of the notification to the user based on that information.

[2049] This invention is a cash cardless system that combines an emotion engine, allowing users to conduct transactions safely and quickly through facial recognition and biometric authentication. The emotion engine also analyzes the user's emotional state and optimizes the transaction. This system is mainly composed of a terminal and a server.

[2050] System configuration

[2051] Terminal

[2052] The device includes the following hardware and software:

[2053] Input devices (cameras, fingerprint scanners, iris scanners, etc.) that allow users to enter facial and biometric data

[2054] Emotion engine (including camera and microphone) for recognizing user emotions

[2055] Displays and interfaces that guide users through the authentication process and their emotional state

[2056] A communication interface for sending data to the server

[2057] server

[2058] The server includes the following elements:

[2059] Generative AI to analyze received data (face photo, fingerprint data, iris data, emotion data) and generate a personal authentication model and emotion model

[2060] Database for managing personal and account information

[2061] Software and algorithms for authentication processes, emotion recognition, and transaction processing

[2062] Program processing

[2063] User information collection and registration

[2064] Users access a bank's ATM or smartphone app to register their facial recognition and biometric data. The device then uses its camera to take a photo of the user's face and scan their fingerprint and iris. It also activates an emotion engine to collect the user's facial expressions and tone of voice. The collected data is sent to a server via the device. The server analyzes the received data and uses generative AI to train a personal authentication model and emotion model. These models are stored in a database and associated with the user's personal and account information.

[2065] Authentication Process

[2066] The user selects "Facial Recognition Transaction" at the ATM or via a smartphone app. The terminal displays the facial recognition and biometric authentication input interface, and the user points their face at the camera, preparing to scan their fingerprint. The terminal collects a facial photo, fingerprint data, and emotional data, and sends them to the server. The server analyzes the received data and performs authentication and emotional recognition using generative AI. The authentication results and emotional state analysis results are then notified to the terminal.

[2067] Transaction Processing

[2068] After successful authentication, the user selects "Withdraw" on the ATM or app interface and enters the amount. The terminal monitors the user's emotional state using an emotion engine and sends the results to the server. The server determines whether a withdrawal is possible based on the user's emotional state and account information, and if so, withdraws the specified amount. The transaction results and notification method based on emotion analysis are sent to the terminal.

[2069] Result notification

[2070] The terminal displays notifications to the user based on the transaction result and emotional state. In the case of an ATM, it dispenses cash and provides appropriate notifications (audio, text, visual) as needed. The user can then confirm the transaction result and collect the cash and receipt.

[2071] Specific examples

[2072] Example 1: You accidentally forget your wallet

[2073] 1. Users register their facial recognition and fingerprint data at the ATM in advance, and emotional data is also collected using an emotion engine.

[2074] 2. The user goes to a bank ATM and selects "Facial Recognition Transaction."

[2075] 3. The device will require facial recognition and fingerprint scanning, and will also collect emotional data.

[2076] 4. The server analyzes the data and performs user authentication and emotion recognition.

[2077] 5. If authentication is successful and the transaction is deemed possible, the user enters the amount and the transaction is carried out.

[2078] 6. The terminal will display a successful transaction and dispense the cash.

[2079] 7. The user receives the cash and a receipt.

[2080] Example 2: When you lose your bankbook or cash card during a disaster

[2081] 1. Users register their facial recognition and biometric data at the emergency assistance center, and emotion data is also collected through the emotion engine.

[2082] 2. The user selects "Facial Recognition Transaction" on the terminal at the evacuation center.

[2083] 3. The device will request facial recognition and fingerprint scanning, and collect emotional data.

[2084] 4. The server analyzes the data and performs user authentication and emotion recognition.

[2085] 5. If authentication is successful and the transaction is deemed possible, the user enters the amount and the transaction is carried out.

[2086] 6. The terminal will display a successful transaction and dispense the cash.

[2087] 7. The user receives cash.

[2088] Prompt Sentence Examples

[2089] "Look at the camera and make sure your face is visible."

[2090] "Place your finger on the scanner"

[2091] "Place your eyes on the iris scanner."

[2092] "Hi, I'm feeling good today."

[2093] "Select facial recognition transaction"

[2094] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2095] User information collection and registration

[2096] Step 1:

[2097] Users access the bank's ATM or smartphone app and begin registering their facial recognition and biometric data.

[2098] Input: User access operations

[2099] Output: Registration process begins

[2100] Step 2:

[2101] The device will display the instruction "Please take a photo of your face" on the display.

[2102] Input: User registration process start operation

[2103] Output: Display of face photo taking screen, display of instruction message

[2104] Specific operation: Display an instruction message on the display and start the camera.

[2105] Step 3:

[2106] The user faces the camera and takes a photo of their face.

[2107] Input: User faces the camera

[2108] Output: Facial photo data

[2109] What happens: The user turns their face in front of the camera and their image is captured.

[2110] Step 4:

[2111] The device will then display an interface that says "Please scan your fingerprint."

[2112] Input: Operation after taking face photo

[2113] Output: Display of fingerprint scan screen, display of instruction message

[2114] Specific operation: Displays an instruction message on the display and activates the fingerprint scanner.

[2115] Step 5:

[2116] The user places their finger on the scanner to scan their fingerprint.

[2117] Input: The user places their finger on the scanner

[2118] Output: Fingerprint data

[2119] What happens: A user places their finger on a fingerprint scanner and their fingerprint is captured.

[2120] Step 6:

[2121] The terminal will display the instruction "Please scan your eyes" if an iris scanner is present.

[2122] Input: Actions after fingerprint scanning

[2123] Output: Display of iris scan screen, display of instruction message

[2124] Specific operations: Display an instruction message on the display and activate the iris scanner.

[2125] Step 7:

[2126] The user positions their eye over the iris scanner and scans their iris.

[2127] Input: The user's eye movement towards the scanner

[2128] Output: Iris data

[2129] What happens: The user points their eye at the iris scanner and their iris is captured.

[2130] Step 8:

[2131] The device will launch its emotion engine and display the message, "Collecting emotion data. Please record your face and voice."

[2132] Input: Actions after iris scan

[2133] Output: Display of emotion data collection screen, display of instruction message

[2134] Specific actions: Display an instruction message on the display and activate the camera and microphone.

[2135] Step 9:

[2136] The user faces the camera and speaks into a microphone.

[2137] Input: User's facial expressions and voice

[2138] Output: Emotion data (facial expressions, tone of voice, speaking speed)

[2139] Specific actions: The user shows their facial expression in front of the camera and speaks into the microphone.

[2140] Step 10:

[2141] The device transmits the collected facial recognition, biometric authentication, and emotion data to a server.

[2142] Input: Collected authentication data and emotion data

[2143] Output: Send data to the server

[2144] Specific operation: The emotion engine analyzes the data and uses the generation AI to send it to the server. During the transmission, the message "Sending data" is displayed.

[2145] Processing on the server

[2146] Step 11:

[2147] The server analyzes the received data and uses generative AI to learn a personal authentication model and emotion model.

[2148] Input: face photo, fingerprint data, iris data, emotion data

[2149] Output: Personal authentication model, emotion model

[2150] Specific operation: Analyzes the received data and generates a model by analyzing facial feature points, fingerprint patterns, voice spectrum, etc.

[2151] Step 12:

[2152] The server stores the learned model in a database and associates it with the user's personal and account information.

[2153] Input: Trained personal authentication model, emotion model

[2154] Output: Save to database

[2155] Specific operation: Link and save the model, personal information, and account information in the database.

[2156] Authentication Process

[2157] Step 13:

[2158] Users select "Facial Recognition Transaction" at the ATM or via a smartphone app.

[2159] Input: User's transaction start operation

[2160] Output: Instructions: "Select facial recognition transaction"

[2161] Specific operation: Display the trading start menu on the interface.

[2162] Step 14:

[2163] The terminal displays an input interface for face recognition and biometric authentication.

[2164] Input: User's trade selection operation

[2165] Output: Display of face recognition and fingerprint scanning screen

[2166] Specific actions: Display messages such as "Please face the camera" and "Please place your finger on the scanner."

[2167] Step 15:

[2168] The user faces the camera and prepares to scan their fingerprint.

[2169] Input: Aligning the user's face and fingers

[2170] Output: Facial photo data, fingerprint data

[2171] Specific operation: For facial recognition, the user points their face towards the camera, and for fingerprint scanning, they place their finger on the scanner.

[2172] Step 16:

[2173] The terminal collects a facial photograph, fingerprint data, and emotion data and transmits them to a server.

[2174] Input: Newly collected authentication data and emotion data

[2175] Output: Send data to the server

[2176] Specific behavior: Displays the message "Collecting data" and the emotion engine analyzes the data.

[2177] Processing on the server

[2178] Step 17:

[2179] The server analyzes the received data and performs authentication and emotion recognition using the generative AI.

[2180] Input: Data sent from the terminal

[2181] Output: Authentication results, sentiment analysis results

[2182] Specific operation: Analyzes the transmitted data using generative AI to authenticate the user and recognize emotions.

[2183] Step 18:

[2184] The server notifies the terminal of the authentication result and the emotional state analysis result.

[2185] Input: Authentication results, sentiment analysis results

[2186] Output: Send results to the terminal

[2187] Specific operation: The success or failure of authentication and the emotional state are returned to the terminal.

[2188] Transaction Processing

[2189] Step 19:

[2190] After successful authentication, the user selects "Withdraw" from the ATM or app interface and enters the amount.

[2191] Input: User withdrawal operation

[2192] Output: Withdrawal amount data

[2193] Specific operation: The message "Please enter the withdrawal amount" will be displayed on the screen.

[2194] Step 20:

[2195] The device monitors the user's emotional state using an emotion engine and sends the results to the server.

[2196] Input: Real-time user sentiment data

[2197] Output: Emotional state data sent to server

[2198] Specific behavior: Display the message "Checking emotional state."

[2199] Processing on the server

[2200] Step 21:

[2201] The server determines whether a withdrawal is possible based on the user's emotional state and account information.

[2202] Input: Emotional state data, account information

[2203] Output: Transaction availability decision

[2204] Specific behavior: If the user is very impatient, display a warning and limit withdrawals.

[2205] Step 22:

[2206] If withdrawal is possible, the server withdraws the specified amount and updates the transaction record.

[2207] Input: Transaction authorization data, specified amount

[2208] Output: Transaction execution, transaction record updated

[2209] Specific behavior: Display the message "Transaction processing" and update the transaction database.

[2210] Step 23:

[2211] The server sends the transaction results and notification method based on sentiment analysis to the terminal.

[2212] Input: Trading results, sentiment analysis results

[2213] Output: Transaction result notification data

[2214] Specific behavior: Returns the message "Transaction successful, please accept cash."

[2215] Result notification

[2216] Step 24:

[2217] The terminal displays notifications to the user according to the trading results and emotional state.

[2218] Input: Transaction result notification data

[2219] Output: Transaction results display

[2220] Specific behavior: Display a reassuring message along with "Transaction successful."

[2221] Step 25:

[2222] In the case of an ATM, the terminal dispenses cash and provides appropriate notifications (audio, text, visual) as needed.

[2223] Input: Transaction success data

[2224] Output: Withdrawal processing, voice guide output

[2225] Specific operation: A voice message will be heard saying, "Cash has been withdrawn, please take the receipt."

[2226] Step 26:

[2227] The user confirms the transaction result and receives the cash and receipt.

[2228] Input: ATM withdrawal notification

[2229] Output: Cash and receipt received

[2230] Specific action: Withdraw cash and receipt from ATM.

[2231] (Application example 2)

[2232] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2233] Current authentication systems using cash cards require users to have their physical cards in order to be authenticated, which poses significant challenges and security issues, especially in the event of a disaster or loss. Furthermore, traditional authentication systems do not take into account the user's emotional state, which can raise doubts about the legitimacy of the transaction. Furthermore, while fast and secure payments are required for in-store purchases, traditional systems do not adequately meet these demands.

[2234] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting facial authentication data and biometric authentication data, means for training an emotion model, and means for evaluating transaction appropriateness based on the emotional state. This not only eliminates the need for users to carry cards, but also enables appropriate transactions that take the emotional state into consideration, enabling rapid response in the event of a disaster and safe and rapid payment at stores.

[2235] "Facial recognition data" is information used to identify individuals by analyzing images of their faces and facial features.

[2236] "Biometric data" is information used to analyze an individual's biometric characteristics, such as fingerprints, irises, and voice, for authentication.

[2237] An "emotion model" is a model trained to analyze a user's facial expressions and tone of voice and estimate their emotional state.

[2238] "Smart glasses" are glasses-type devices worn by the user that incorporate devices such as a camera and a display.

[2239] "Transaction information" refers to data related to a transaction conducted by a user, such as the transaction amount and transaction details.

[2240] "Communication means" refers to the network infrastructure for transmitting and receiving data between the terminal and the server.

[2241] "Generative AI" is an artificial intelligence technology that analyzes facial recognition data and biometric authentication data to create personal authentication models and emotion models.

[2242] The "transaction appropriateness evaluation means" is a means for determining whether a transaction is appropriate based on the emotional state of the user.

[2243] "Authentication means" means a means for verifying a user's identity using facial recognition and biometric data.

[2244] The "means for notifying results" is a means for notifying users of the results of trading and sentiment analysis.

[2245] We will now explain a specific example of a cash cardless system incorporating the emotion engine of the present invention. This system is based on technology that authenticates users by combining facial recognition data and biometric authentication data, and simultaneously analyzes the user's emotional state to evaluate the appropriateness of the transaction.

[2246] System configuration

[2247] The system mainly consists of the following elements:

[2248] 1. Device:

[2249] Camera: Collects user facial recognition data.

[2250] Biometric devices: Collect biometric data such as fingerprints and irises.

[2251] Emotion engine: Analyzes the user's facial expressions and tone of voice to collect emotional data.

[2252] Display: Shows the authentication process and emotional state to the user.

[2253] Communication Interface: Sends data to the server.

[2254] 2. Server:

[2255] Data analysis and generation AI: Analyzes received facial recognition data, biometric data, and emotion data to generate personal recognition models and emotion models.

[2256] Database: Stores users' personal and transaction information and manages authentication models.

[2257] Transaction suitability assessment function: Determines the suitability of a transaction based on emotional state.

[2258] Program processing

[2259] The server runs on the following hardware and software:

[2260] Hardware: If high-performance computing is required, use cloud-based computing resources (e.g., Amazon Web Services, Google Cloud Platform, Microsoft Azure).

[2261] Software: Emotion analysis uses emotion recognition algorithms, and personal authentication uses face and iris recognition algorithms. These are often implemented using machine learning libraries such as TensorFlow and PyTorch.

[2262] Processing flow

[2263] The device first collects the user's facial and biometric data using a camera and biometric authentication device, and then uses an emotion engine to obtain the user's emotional data. All of this data is sent to the server via a communications interface. The server then analyzes the data using generative AI to generate a personal authentication model and an emotion model, which are then stored in a database. When a transaction is made, the server uses these models to authenticate the user and simultaneously evaluate the user's emotional state to determine whether the transaction is appropriate.

[2264] Specific examples

[2265] When shopping in a physical store, users provide facial recognition and biometric data through the smart glasses and select "Facial Recognition Transaction." The smart glasses' camera recognizes the face and scans fingerprints and iris data. The emotion engine also analyzes facial expressions and tone of voice to obtain emotion data. All of this data is sent to the server in real time, where authentication and emotion evaluation are performed. If authentication is successful, the transaction amount will be displayed on the smart glasses' display, and the user can confirm and complete the payment.

[2266] Prompt Sentence Examples

[2267] "To pay at the register, you need to be authenticated using facial recognition, iris recognition, and emotional data. Just face the camera and scan your iris and fingerprint."

[2268] Through these steps, users can complete transactions quickly and safely, which is particularly useful in times of disaster or emergency.

[2269] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2270] Step 1:

[2271] The user puts on the terminal (smart glasses) and selects facial recognition transaction. The terminal instructs the user to face the camera and simultaneously starts scanning fingerprint and iris data using the biometric authentication device. Inputs include the user's facial image and biometric data (fingerprint, iris). These are acquired as real-time video and scan data. The output is the collected facial recognition data and biometric authentication data.

[2272] Step 2:

[2273] The device launches an emotion engine to collect the user's facial expressions and tone of voice. The emotion engine uses a camera and microphone to analyze the user's emotional state. Here, facial expressions and tone of voice are used as input, and the output is emotion data. This emotion data reflects the user's real-time mental state.

[2274] Step 3:

[2275] The terminal transmits the collected facial recognition data, biometric data, and emotion data to the server through a communication interface. The input is the collected authentication data and emotion data, and the output is the transmission of these data to the server.

[2276] Step 4:

[2277] The server analyzes the received data and uses generative AI to generate a personal authentication model and emotion model. It takes in facial recognition data, biometric data, and emotion data as input, performs data analysis and machine learning, and generates a personal authentication model and emotion model as output. These models are stored in the server's database.

[2278] Step 5:

[2279] The server uses the generated personal authentication model and emotion model to authenticate the user. The input is the model stored on the server and the authentication data sent from the device. The output is the result of authentication success or failure. If authentication is successful, proceed to the next step.

[2280] Step 6:

[2281] The user checks the transaction details (e.g., payment amount, purchased items, etc.) on the smart glasses display and confirms the transaction. The input is the transaction details sent from the server, which the user checks and operates on the display. The output is the transaction confirmation result.

[2282] Step 7:

[2283] The server evaluates the user's emotional state and determines the appropriateness of the transaction. Using the confirmed transaction details and emotional data as input, it performs evaluation processing based on an emotional model. The output is an evaluation result regarding the appropriateness of the transaction.

[2284] Step 8:

[2285] The server processes the transaction and notifies the user of the result. The input is the confirmation of the transaction details and the evaluation result of the emotional state. The output is a notification of whether the transaction was completed, which is displayed on the display of the user's smart glasses. At this time, the transaction information is recorded in a database.

[2286] Step 9:

[2287] The terminal notifies the user of the transaction result and displays or electronically stores the receipt as needed. The input is a notification from the server that the transaction has been completed, and the output is a visual and electronic confirmation of the result for the user.

[2288] Through these steps, users can complete transactions safely and quickly, providing a system that is extremely useful, especially in times of disaster or emergency.

[2289] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2290] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2291] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2292] [Fourth embodiment]

[2293] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2294] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2295] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2296] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2297] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2298] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2299] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2300] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2301] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2302] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2303] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2304] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2305] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2306] System configuration

[2307] The cash cardless system of this invention is implemented in bank ATMs and smartphone apps. The system mainly consists of the following components:

[2308] 1. Terminal

[2309] An input device (such as a camera, fingerprint scanner, or iris scanner) that allows users to enter facial and biometric data.

[2310] A display or interface that guides the user through the authentication process.

[2311] A communication interface for sending data to the server.

[2312] 2. Server

[2313] Generative AI to analyze received data and generate a personal authentication model.

[2314] A database that manages personal and account information.

[2315] Software and algorithms for authentication processes and transaction processing.

[2316] Program processing procedure

[2317] The program processing of the system will be described in detail below.

[2318] User information collection and registration

[2319] 1. User: Visits a bank ATM or smartphone app and registers facial recognition and biometric data.

[2320] 2. Terminal: Have the user take a photo of their face and scan their fingerprint and iris.

[2321] 3. Terminal: The collected data is sent to the server via secure communication.

[2322] 4. Server: Analyzes the received data and uses generative AI to train a personal authentication model.

[2323] 5. Server: Stores the learned personal authentication model in a database and associates the user's personal information with their account information.

[2324] Authentication Process

[2325] 1. User: Select "Facial Recognition Transaction" from the ATM or app.

[2326] 2. Terminal: Display the face recognition and biometric authentication input interface.

[2327] 3. User: Faces the device camera and scans their fingerprint.

[2328] 4. Terminal: Sends collected data to the server.

[2329] 5. Server: Authenticates the user using the generative AI.

[2330] 6. Server: If authentication is successful, it notifies the terminal of the result.

[2331] Transaction Processing

[2332] 1. User: After successful authentication, enter the withdrawal amount.

[2333] 2. Terminal: Sends the entered amount information to the server.

[2334] 3. Server: Determines whether withdrawal is possible based on the user's account information.

[2335] 4. Server: If withdrawal is possible, withdraw the specified amount.

[2336] 5. Server: Updates the transaction record and sends a success notification to the device.

[2337] Result notification

[2338] 1. Terminal: displays the transaction results to the user.

[2339] 2. Terminal: In the case of an ATM, it dispenses cash and prints a receipt.

[2340] 3. User: Check the transaction result and receive the cash and receipt.

[2341] Specific examples

[2342] Example 1: You accidentally forget your wallet

[2343] 1. Collecting and registering user information: Register facial recognition and fingerprint data at the ATM in advance.

[2344] 2. Authentication Process:

[2345] User: Goes to a bank ATM and selects facial recognition transaction.

[2346] On the device: Have the user face the camera and scan their fingerprint.

[2347] Server: Analyzes the data and authenticates the user.

[2348] 3. Transaction Processing:

[2349] User: After successful authentication, enter the amount from the withdrawal menu.

[2350] Terminal: Sends the amount to the server.

[2351] Server: Determines whether withdrawals are possible and processes transactions.

[2352] 4. Result notification:

[2353] Terminal: Shows successful transaction and withdraws cash.

[2354] User: Take cash and receipt.

[2355] Example 2: When you lose your bankbook or cash card during a disaster

[2356] 1. User information collection and registration: Register facial recognition and biometric data at the emergency assistance center.

[2357] 2. Authentication Process:

[2358] User: Selects facial recognition transaction on the shelter terminal.

[2359] Device: Have your face face the camera and scan your fingerprint.

[2360] Server: Analyzes the data and authenticates the user.

[2361] 3. Transaction Processing:

[2362] User: After successful authentication, enter the withdrawal amount.

[2363] Terminal: Sends the amount to the server.

[2364] Server: Determines whether withdrawal is possible and processes the transaction.

[2365] 4. Result notification:

[2366] Terminal: Shows successful transaction and withdraws cash.

[2367] User: Take the cash.

[2368] The system of the present invention allows users to safely withdraw money without a cash card, improving convenience and peace of mind for users. In particular, in the event of a disaster, it allows users to quickly obtain cash, which has the effect of reducing the mental burden on victims.

[2369] The processing flow will be explained below.

[2370] Specific processing steps of the program

[2371] User information collection and registration

[2372] Step 1:

[2373] User: Opens the bank's ATM or smartphone app and opens the screen to register facial recognition and biometric data.

[2374] Step 2:

[2375] On the device: Displays an interface for taking a photo of the user's face and asks the user to face the camera.

[2376] Step 3:

[2377] User: Follow the instructions on the device to take a photo of your face in front of the camera.

[2378] Step 4:

[2379] Device: Sends the captured photo to the server.

[2380] Step 5:

[2381] On the device: Display an interface to collect fingerprint data using a fingerprint scanner and have the user scan their fingerprint.

[2382] Step 6:

[2383] User: Places finger on fingerprint scanner to provide fingerprint data.

[2384] Step 7:

[2385] Device: Sends collected fingerprint data to the server.

[2386] Step 8:

[2387] Terminal: Displays an interface for collecting iris data and prompts the user to scan their iris.

[2388] Step 9:

[2389] User: Provides iris data by pointing their iris at the device camera.

[2390] Step 10:

[2391] Terminal: Sends collected iris data to the server.

[2392] Step 11:

[2393] Server: Receives facial photos, fingerprint data, and iris data, and trains a personal authentication model using generative AI.

[2394] Step 12:

[2395] Server: The trained personal authentication model is stored in a database and associated with the user's personal and account information.

[2396] Authentication Process

[2397] Step 13:

[2398] User: Selects "Facial Recognition Transaction" from the ATM or smartphone app.

[2399] Step 14:

[2400] Terminal: Display the face recognition and biometric authentication input interface.

[2401] Step 15:

[2402] User: Face the camera and prepare to scan your fingerprint.

[2403] Step 16:

[2404] Terminal: Collects facial photo and fingerprint data and sends them to the server.

[2405] Step 17:

[2406] Server: Analyzes the received facial photo and fingerprint data and performs authentication using the generated AI.

[2407] Step 18:

[2408] Server: Notifies the terminal of the authentication result.

[2409] Transaction Processing

[2410] Step 19:

[2411] User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[2412] Step 20:

[2413] Terminal: Sends the amount data entered by the user to the server.

[2414] Step 21:

[2415] Server: Checks the user's account information and determines whether a withdrawal is possible.

[2416] Step 22:

[2417] Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[2418] Step 23:

[2419] Server: Sends the transaction results to the terminal.

[2420] Result notification

[2421] Step 24:

[2422] Terminal: Displays the transaction results to the user.

[2423] Step 25:

[2424] Terminal: In the case of an ATM, it dispenses cash and prints receipts.

[2425] Step 26:

[2426] User: Checks and receives cash and receipts.

[2427] This series of processing steps allows users to withdraw money safely and quickly using facial and biometric authentication, without using a cash card.

[2428] Example 1

[2429] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2430] In recent years, the loss and theft of cash cards and bankbooks has been increasing, and the risk of fraudulent transactions has also increased. In particular, in emergencies such as natural disasters and fires, if a bankbook or cash card is lost or lost, it can be difficult for victims to quickly secure cash. Furthermore, conventional authentication systems often use passwords or PIN codes, which still pose security risks. Therefore, to solve these issues, a system that can safely and quickly authenticate individuals and perform financial transactions is needed.

[2431] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2432] In this invention, the server includes a means for collecting facial recognition data and biometric authentication data, a means for training a personal authentication model using a generative AI, and a means for storing the personal authentication model in a database and associating personal information and financial information, thereby enabling safe and fast user authentication and financial transactions.

[2433] "Facial recognition data" is data that digitally represents a user's facial features.

[2434] "Biometric data" means a digital representation of a user's biological characteristics, such as a fingerprint or iris.

[2435] A "server" is a computer system that stores and processes data over a network.

[2436] "Generative AI" is a program that uses artificial intelligence technology to perform a specific task, and in this invention is used particularly for training a personal authentication model.

[2437] A "personal authentication model" is a model for uniquely identifying a user using collected facial recognition data and biometric authentication data.

[2438] A "database" is a system for efficiently storing and managing large amounts of data.

[2439] "Personal Information" means identifying information, such as name and address, associated with a user.

[2440] "Financial information" refers to information related to a user's account information, transaction history, etc.

[2441] "User authentication" is the process of verifying a user's identity based on collected authentication data.

[2442] A "financial transaction" refers to a monetary transaction, such as a withdrawal from or transfer to a user's account.

[2443] "Trading results" refers to information obtained as a result of a financial transaction, including the success or failure of the transaction.

[2444] The cash cardless system of this invention is implemented in bank ATMs and smartphone apps. The system is mainly composed of a terminal and a server.

[2445] System configuration

[2446] Terminal

[2447] The terminal is equipped with an input device for users to enter facial and biometric authentication data, such as a camera, fingerprint scanner, or iris scanner, and provides a display or interface that guides the user through the authentication process. The terminal also transmits the collected data to a server through a secure communication interface.

[2448] Camera: A device used to take a photo of the user's face.

[2449] Fingerprint scanner: A device used to scan a user's fingerprint.

[2450] Iris scanner: A device used to scan a user's iris.

[2451] Display: A device used to display authentication procedures and transaction details.

[2452] Communication interface: A secure protocol (e.g. HTTPS) for sending data to the server.

[2453] server

[2454] The server analyzes the received data and generates a personal authentication model using generative AI. Specifically, it uses AI frameworks such as TensorFlow and PyTorch. It also has a database that manages personal and financial information. This database stores the trained personal authentication model and also contains software and algorithms for processing user transactions.

[2455] Generative AI: Train a person authentication model using TensorFlow and PyTorch.

[2456] Database: Manage personal and financial information using database systems such as MySQL or PostgreSQL.

[2457] Transaction Processing Algorithm: Software used to process users' financial transactions.

[2458] Specific examples

[2459] Example 1: You accidentally forget your wallet

[2460] 1. Collecting and registering user information: Register facial recognition and fingerprint data in advance at the bank ATM.

[2461] User: Accesses the ATM and registers facial recognition and fingerprint data.

[2462] Device: Takes a photo of the user's face and scans their fingerprint, sending the data to a server.

[2463] Server: Analyzes the data, uses generative AI to train a personal authentication model, and stores it in a database.

[2464] 2. Authentication Process:

[2465] User: Go to a bank ATM and select "Facial Recognition Transaction."

[2466] Device: Have your face face the camera and scan your fingerprint.

[2467] Server: Analyzes the collected data and authenticates users.

[2468] 3. Transaction Processing:

[2469] User: After successful authentication, enter the desired amount in the withdrawal menu.

[2470] Terminal: Sends the entered amount information to the server.

[2471] Server: Determines whether withdrawal is possible based on the user's account information and processes the transaction.

[2472] 4. Result notification:

[2473] Terminal: Shows successful transaction and withdraws cash.

[2474] User: Take cash and receipt.

[2475] Example 2: When you lose your bankbook or cash card during a disaster

[2476] 1. User information collection and registration: Register facial recognition and biometric data at the emergency assistance center.

[2477] User: Register facial recognition and biometric data on a terminal at the evacuation center.

[2478] Device: Face the camera, scan your fingerprint, and send the data to the server.

[2479] Server: Uses generative AI to train a personal authentication model and saves it in a database.

[2480] 2. Authentication Process:

[2481] User: Select "Facial Recognition Transaction" on the shelter terminal.

[2482] Device: Have your face face the camera and scan your fingerprint.

[2483] Server: Analyzes the data and authenticates the user.

[2484] 3. Transaction Processing:

[2485] User: After successful authentication, enter the withdrawal amount.

[2486] Terminal: Sends the amount to the server.

[2487] Server: Determines whether withdrawals are possible and processes transactions.

[2488] 4. Result notification:

[2489] Terminal: Shows successful transaction and withdraws cash.

[2490] User: Take the cash.

[2491] Prompt Sentence Examples

[2492] "Please explain your generative AI model for user authentication. Explain the process steps for analyzing the data using facial and fingerprint recognition."

[2493] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2494] Program processing flow

[2495] Step 1:

[2496] User: Accesses the bank's ATM or smartphone app and begins enrolling facial recognition and biometric data. Selects "New Enrollment" from the app's home screen and begins enrolling facial recognition and fingerprint data.

[2497] Input: Facial photo and fingerprint data

[2498] Output: Registration ready message

[2499] Specific steps: Tap "New Registration" from the app menu and follow the instructions to enter your face photo and fingerprint data.

[2500] Step 2:

[2501] Terminal: Takes a photo of the user's face and scans their fingerprint. Uses the ATM's camera to take a photo of the face and a fingerprint scanner to read the fingerprint.

[2502] Input: User's facial photo and fingerprint data

[2503] Output: Collected data

[2504] Specific actions: The app displays instructions to the user, such as "Please face the camera" and "Please place your finger on the fingerprint scanner."

[2505] Step 3:

[2506] Device: Send collected data to the server using a secure communication protocol (e.g. HTTPS).

[2507] Input: Collected data

[2508] Output: Data transmission completed message

[2509] Specific actions: Encrypt the data and send it to the server.

[2510] Step 4:

[2511] Server: Analyzes the received data and uses generative AI (using TensorFlow or PyTorch) to train a personal authentication model.

[2512] Input: Collected data

[2513] Output: Personal authentication model

[2514] Specific operations: Analyze received facial recognition and biometric data to train an AI model.

[2515] Step 5:

[2516] Server: The trained personal authentication model is stored in a database (MySQL or PostgreSQL) and associated with the user's personal and financial information.

[2517] Input: Personal authentication model and user information

[2518] Output: Updated database

[2519] Specific operation: Stores the authentication model in the database, linked to the user ID, and associates personal and financial information.

[2520] Step 6:

[2521] User: Accesses the bank's ATM or app and selects "Facial Recognition Transactions." For example, selects "Facial Recognition Transactions" from the ATM menu screen.

[2522] Input: Operation selection

[2523] Output: Authentication interface display

[2524] Specific operation: Select "Facial Recognition Transactions" on the ATM menu screen.

[2525] Step 7:

[2526] Terminal: Display the input interface for face recognition and biometric authentication. Display the camera image for face recognition and the fingerprint scanner image.

[2527] Input: Menu selection information

[2528] Output: Authentication interface

[2529] Specific actions: "Please face the camera" and "Please place your finger on the fingerprint scanner" are displayed.

[2530] Step 8:

[2531] User: Faces the device camera and scans a fingerprint, for example, by facing the camera lens and pressing a finger against the scanner.

[2532] Input: Facial photo and fingerprint data

[2533] Output: Authentication data

[2534] Specific actions: Stand in front of the camera for facial recognition and place your finger on the fingerprint scanner.

[2535] Step 9:

[2536] Terminal: Sends collected data to the server. The data is encrypted and sent using a secure communication protocol.

[2537] Input: Authentication data

[2538] Output: Data transmission completed message

[2539] Specific operation: Encrypts collected data and sends it to a server.

[2540] Step 10:

[2541] Server: Authenticates the user using generative AI, matching the stored authentication model with newly acquired data.

[2542] Input: Authentication data and personal authentication model

[2543] Output: Authentication result

[2544] Specific operation: Analyzes the received data and compares it with the stored AI model to evaluate the degree of match.

[2545] Step 11:

[2546] Server: If authentication is successful, it notifies the terminal of the result, encrypts the authentication result, and sends it to the terminal.

[2547] Input: Authentication result

[2548] Output: Authentication success message

[2549] Specific operation: Sends a message of successful authentication to the terminal.

[2550] Step 12:

[2551] User: After successful authentication, enter the amount to withdraw. Enter the desired amount on the ATM's input screen.

[2552] Input: Withdrawal amount

[2553] Output: Amount information entered

[2554] Specific operation: Enter the amount in the amount input field displayed on the ATM screen.

[2555] Step 13:

[2556] Terminal: Sends the entered amount information to the server. The entered data is encrypted and sent securely.

[2557] Input: Amount information

[2558] Output: Data transmission completed message

[2559] Specific operation: Sends the entered amount information to the server.

[2560] Step 14:

[2561] Server: Determines whether withdrawal is possible based on the user's account information. Checks the account balance and compares it with the requested amount.

[2562] Input: Amount information and account information

[2563] Output: Withdrawal decision result

[2564] Specific operation: Check the account balance and determine whether a withdrawal is possible.

[2565] Step 15:

[2566] Server: If withdrawal is possible, withdraw the specified amount. The server processes the transaction and updates the account balance.

[2567] Input: Withdrawal decision result

[2568] Output: Transaction processing results

[2569] Specific actions: Process the transaction and withdraw the specified amount.

[2570] Step 16:

[2571] Server: Updates the transaction record and sends a success notification to the terminal. Adds the transaction record to the database and sends a success message to the terminal.

[2572] Input: Transaction processing result

[2573] Output: Success message

[2574] Specific operations: Save the transaction record in the database and send a notification of successful transaction to the terminal.

[2575] Step 17:

[2576] Terminal: Display the transaction result to the user. Display a successful transaction message on the ATM screen.

[2577] Input: Success message

[2578] Output: Trade result display

[2579] Specific behavior: Display a message indicating the transaction was successful.

[2580] Step 18:

[2581] Terminal: In the case of an ATM, it dispenses cash and prints a receipt. The ATM's cash dispenser dispenses the specified amount, and the receipt printer prints the transaction details.

[2582] Input: Success message

[2583] Output: Cash and receipt

[2584] Specific actions: Dispense cash and print a receipt.

[2585] Step 19:

[2586] User: Check the transaction result and receive the cash and receipt. Take the cash and receipt that come out of the ATM.

[2587] Input: Cash and receipt

[2588] Output: Transaction completed

[2589] Specific actions: Check the transaction results and take the cash and receipt.

[2590] (Application example 1)

[2591] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2592] In conventional financial transactions using cash cards or bankbooks, if these physical media are lost, transactions often become difficult to complete. Furthermore, in emergencies such as natural disasters, earthquakes, or fires, cards and bankbooks can be damaged or lost, making it difficult to quickly obtain necessary funds. Furthermore, from a security standpoint, there is a risk of fraudulent use if cards or bankbooks are stolen. There is a need for a system that can solve these issues and enable users to conduct financial transactions quickly and with peace of mind.

[2593] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2594] In this invention, the server includes means for collecting facial authentication data and biometric authentication data, means for transmitting the collected data to the server, means for analyzing the collected data in the server and training a personal authentication model, means for saving the personal authentication model in a database and associating personal information and account information, means for authenticating a user based on the facial authentication and biometric authentication data, means for processing a transaction for the user's account if authentication is successful, means for notifying the user of the transaction result, means for confirming and approving the payment amount, means for transmitting payment amount information to the server after authentication is successful, and means for saving the transaction history of the payment amount so that the user can confirm it. This eliminates the need for physical cash cards or passbooks, allowing transactions to be carried out quickly, especially in emergencies such as natural disasters, earthquakes, and fires, while also improving security.

[2595] "Facial recognition data" is data used to analyze image data of a user's face and identify the individual.

[2596] "Biometric data" is data used to identify an individual using biometric characteristics such as fingerprints or irises.

[2597] "Means of collection" refers to devices and programs for acquiring facial recognition data and biometric authentication data.

[2598] "Transmitting means" refers to a device or program that has the function of transmitting collected data to a server.

[2599] The "analyzing means" refers to a device or program for processing the received data and generating a personal authentication model.

[2600] "Means for training" refers to devices or programs for training a personal authentication model using generative AI.

[2601] A "personal authentication model" is an algorithm or program for identifying individuals based on collected facial recognition data and biometric data.

[2602] "Database" means a storage device for storing and managing personal authentication models, personal information, and account information.

[2603] "Means for authentication" refers to devices or programs that verify a user's identity based on facial recognition and biometric data.

[2604] A "means for processing transactions" is a device or program that, after successful authentication, executes a financial transaction against a user's account.

[2605] "Means of notification" refers to devices or programs used to notify users of transaction results.

[2606] "Means for confirming and approving the payment amount" refers to a device or program that allows the user to confirm and approve the payment amount.

[2607] The "means for transmitting payment amount information" refers to a device or program for transmitting information on the payment amount to the server after successful authentication.

[2608] "Means for storing transaction history" refers to a device or program that records the results of each transaction and allows users to check them.

[2609] "Generative AI" is an artificial intelligence technology that analyzes data and trains a personal authentication model.

[2610] MODE FOR CARRYING OUT THE INVENTION

[2611] The facial recognition electronic payment system of the present invention uses facial recognition data and biometric authentication data to realize safe and fast financial transactions without the need for physical cash cards or bankbooks.

[2612] System configuration

[2613] The system mainly consists of the following components:

[2614] 1. User device (smartphone)

[2615] 2. Server

[2616] 3. Database

[2617] Hardware and software used

[2618] Smartphone: Camera for facial recognition, fingerprint scanner for fingerprint recognition

[2619] Facial recognition software: OpenCV

[2620] Generative AI model: TensorFlow

[2621] Communication interface: Secure HTTP communication

[2622] Database: SQL-based database

[2623] Program processing overview and specific examples

[2624] User information collection and registration

[2625] Users use their smartphone's camera and fingerprint scanner to collect facial and biometric data, which is then transmitted to a server through a secure communication interface.

[2626] The server analyzes the received data and uses the generative AI model to create a personal authentication model, which is stored in a database and associated with the user's personal and account information.

[2627] Authentication Process

[2628] When making a payment, users open a smartphone app and enter their facial and biometric data, which includes pointing their face at the camera and placing their fingerprint on a scanner. The smartphone then sends this data to a server, which uses a generative AI model to authenticate the user.

[2629] Payment Processing

[2630] If authentication is successful, the user confirms and approves the payment amount. The smartphone sends the confirmed payment amount to the server. The server processes the payment based on the user's account information and saves the transaction results in a database.

[2631] Result notification

[2632] The results of the transaction are notified to the user, who can check the results via a smartphone app.

[2633] Specific application

[2634] Example 1: Use in everyday life

[2635] When shopping at a supermarket, a user uses a facial recognition electronic payment app. After shopping is complete, the user opens the app at the cash register, performs facial and biometric authentication, and approves the payment. The server processes the authentication and payment, and the transaction result is notified to the user within a few seconds.

[2636] Example 2: Use during a disaster

[2637] Users who lose their bankbooks or cash cards during a disaster can use terminals installed at evacuation shelters to quickly withdraw cash using facial and biometric authentication, enabling safe and fast transactions even under certain circumstances.

[2638] Prompt Sentence Examples

[2639] "Please show us how to obtain a user's facial image and fingerprint data and perform secure authentication."

[2640] "If bankbooks and cash cards become unusable during a natural disaster, please explain how to use facial recognition as an alternative method for transactions."

[2641] As a result, physical cash cards and bankbooks are no longer necessary, allowing for quick transactions, especially in emergencies such as natural disasters, earthquakes, and fires, while also improving security.

[2642] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2643] Step 1:

[2644] The user downloads the facial recognition electronic payment app to their smartphone and launches it. The user begins facial and fingerprint authentication on the new registration screen (input: user's face and fingerprint, output: collected facial recognition data and biometric authentication data).

[2645] Step 2:

[2646] The device uses the smartphone's camera and fingerprint scanner to collect the user's facial authentication data and fingerprint data (data processing: acquisition of facial data and fingerprint data, output: collected facial authentication data and biometric authentication data).

[2647] Step 3:

[2648] The device encrypts the collected data and sends it to the server using a secure communication protocol (e.g., HTTPS) (input: collected facial recognition data and biometric authentication data, output: encrypted data sent to the server).

[2649] Step 4:

[2650] The server analyzes the received facial recognition data and biometric authentication data and creates a personal authentication model using a generative AI model (input: encrypted and transmitted facial recognition data and biometric authentication data; data calculation: training of the model using the generative AI; output: personal authentication model).

[2651] Step 5:

[2652] The server stores the personal authentication model in a database and associates it with the user's personal information and payment information (input: personal authentication model, personal information and payment information; output: personal authentication model and related information stored in the database).

[2653] Step 6:

[2654] When making a payment, the user launches the app and proceeds to the face authentication and fingerprint authentication screen (input: user operation, output: face authentication and fingerprint authentication screen).

[2655] Step 7:

[2656] The device collects the facial recognition data and biometric authentication data again and sends them to the server (input: user's face and fingerprint, data processing: recollected facial recognition data and biometric authentication data, output: data sent to the server).

[2657] Step 8:

[2658] The server compares the received data with the personal authentication model again and authenticates the user (input: recollected data, personal authentication model, data calculation: model comparison, output: authentication result).

[2659] Step 9:

[2660] If the authentication is successful, the user confirms and approves the payment amount, and this information is also sent to the server (Input: User approves the amount, Output: Payment amount information).

[2661] Step 10:

[2662] The server processes the transaction based on the user's account information and payment amount information, and records the payment results (input: payment amount information, account information, data calculation: transaction processing and recording, output: transaction results and recording).

[2663] Step 11:

[2664] The server notifies the terminal of the transaction result, and the terminal displays it to the user (input: transaction result, output: notification to user).

[2665] The above is the flow of processing steps of this system.

[2666] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2667] System configuration

[2668] The cash cardless system incorporating the emotion engine of the present invention consists of the following elements:

[2669] 1. Terminal

[2670] An input device (such as a camera, fingerprint scanner, or iris scanner) that allows users to enter facial and biometric data.

[2671] An emotion engine (including a camera and microphone) to recognize user emotions.

[2672] Displays and interfaces that show the authentication process and emotional state to the user.

[2673] A communication interface for sending data to the server.

[2674] 2. Server

[2675] A generative AI that analyzes received data (face photos, fingerprint data, iris data, emotion data) and generates a personal authentication model and emotion model.

[2676] A database that manages personal and account information.

[2677] Software and algorithms for authentication processes, emotion recognition, and transaction processing.

[2678] Program processing procedure

[2679] The program processing of the system will be described in detail below.

[2680] User information collection and registration

[2681] 1. User: Visits a bank ATM or smartphone app and registers facial recognition and biometric data.

[2682] 2. Terminal: Have the user take a photo of their face and scan their fingerprint and iris.

[2683] 3. Device: Activate the emotion engine and collect the user's facial expressions and tone of voice.

[2684] 4. Terminal: Sends collected facial recognition, biometrics, and emotion data to the server.

[2685] 5. Server: Analyzes the received data and uses generative AI to train a personal authentication model and emotion model.

[2686] 6. Server: Stores the trained model in a database and associates the user's personal and account information.

[2687] Authentication Process

[2688] 1. User: Select "Facial Recognition Transaction" from the ATM or smartphone app.

[2689] 2. Terminal: Display the face recognition and biometric authentication input interface.

[2690] 3. User: Faces the camera and prepares to scan his / her fingerprint.

[2691] 4. Terminal: Collects facial photos, fingerprint data, and emotion data and sends them to the server.

[2692] 5. Server: Analyzes the received data and performs authentication and emotion recognition using generative AI.

[2693] 6. Server: Notifies the device of the authentication result and emotional state analysis result.

[2694] Transaction Processing

[2695] 1. User: After successful authentication, select "Withdraw" from the ATM or app interface and enter the amount.

[2696] 2. Terminal: The emotion engine monitors the user's emotional state and sends the results to the server.

[2697] 3. Server: Determines whether withdrawal is possible based on the user's emotional state and account information.

[2698] 4. Server: If withdrawal is possible, withdraw the specified amount and update the transaction record.

[2699] 5. Server: Sends trading results and notification methods based on sentiment analysis to the terminal.

[2700] Result notification

[2701] 1. Terminal: displays notifications to users based on trading results and emotional state.

[2702] 2. Terminal: In the case of an ATM, dispenses cash and provides appropriate notifications (audio, text, visual) as needed.

[2703] 3. User: Check the transaction result and receive the cash and receipt.

[2704] Specific examples

[2705] Example 1: You accidentally forget your wallet

[2706] 1. Collecting and registering user information: Facial recognition and fingerprint data are registered in advance at the ATM, and emotion data is also collected using the emotion engine.

[2707] 2. Authentication Process:

[2708] User: Goes to a bank ATM and selects facial recognition transaction.

[2709] Device: The device asks the user to face the camera, scans their fingerprint, and collects emotional data.

[2710] Server: Analyzes data and performs user authentication and emotion recognition.

[2711] 3. Transaction Processing:

[2712] User: After successful authentication, enter the amount from the withdrawal menu.

[2713] Terminal: Sends the amount and user emotion data to the server.

[2714] Server: Determines whether withdrawals are possible and processes transactions.

[2715] 4. Result notification:

[2716] Terminal: Shows successful transaction and withdraws cash.

[2717] User: Take cash and receipt.

[2718] Example 2: When you lose your bankbook or cash card during a disaster

[2719] 1. Collecting and registering user information: The emergency support center will register facial recognition and biometric data, and also collect emotional data from the emotion engine.

[2720] 2. Authentication Process:

[2721] User: Selects facial recognition transaction on the shelter terminal.

[2722] Device: The device asks the user to face the camera, scans their fingerprint, and collects emotional data.

[2723] Server: Analyzes data and performs user authentication and emotion recognition.

[2724] 3. Transaction Processing:

[2725] User: After successful authentication, enter the withdrawal amount.

[2726] Terminal: Sends the amount and user emotion data to the server.

[2727] Server: Determines whether withdrawal is possible and processes the transaction.

[2728] 4. Result notification:

[2729] Terminal: Displays notifications for successful transactions and emotional states, and withdraws cash.

[2730] User: Take the cash.

[2731] By using the system of this invention, users can safely and quickly withdraw money without using a cash card. Furthermore, by combining it with an emotion engine, optimal transactions can be performed according to the user's emotional state, enhancing convenience and peace of mind. In particular, in the event of a disaster, this system allows users to obtain cash quickly and appropriately, which has the effect of reducing the mental burden on victims.

[2732] The processing flow will be explained below.

[2733] Specific processing steps of the program

[2734] User information collection and registration

[2735] Step 1:

[2736] User: Opens the bank's ATM or smartphone app and opens the screen to register facial recognition and biometric data.

[2737] Step 2:

[2738] On the device: Displays an interface for taking a photo of the user's face and asks the user to face the camera.

[2739] Step 3:

[2740] User: Follow the instructions on the device to take a photo of your face in front of the camera.

[2741] Step 4:

[2742] Device: Sends the captured photo to the server.

[2743] Step 5:

[2744] On the device: Display an interface to collect fingerprint data using a fingerprint scanner and have the user scan their fingerprint.

[2745] Step 6:

[2746] User: Places finger on fingerprint scanner to provide fingerprint data.

[2747] Step 7:

[2748] Device: Sends collected fingerprint data to the server.

[2749] Step 8:

[2750] Terminal: Displays an interface for collecting iris data and prompts the user to scan their iris.

[2751] Step 9:

[2752] User: Provides iris data by pointing their iris at the device camera.

[2753] Step 10:

[2754] Terminal: Sends collected iris data and emotion data to the server.

[2755] Step 11:

[2756] Server: Receives facial photos, fingerprint data, iris data, and emotion data, and uses generative AI to train a personal authentication model and emotion model.

[2757] Step 12:

[2758] Server: Stores the trained model in a database and associates the user's personal information with their account information.

[2759] Authentication Process

[2760] Step 13:

[2761] User: Selects "Facial Recognition Transaction" from the ATM or smartphone app.

[2762] Step 14:

[2763] Terminal: Display the face recognition and biometric authentication input interface.

[2764] Step 15:

[2765] User: Face the camera and prepare to scan your fingerprint.

[2766] Step 16:

[2767] Terminal: Collects facial photos, fingerprint data, and emotion data and sends them to the server.

[2768] Step 17:

[2769] Server: Analyzes the received facial photo, fingerprint data, and emotion data, and performs authentication and emotion recognition using generative AI.

[2770] Step 18:

[2771] Server: Notifies the device of the authentication results and emotional state analysis results.

[2772] Transaction Proc...

Claims

1. a means for collecting facial recognition data and biometric data; means for transmitting the collected data to a server; means for analyzing the collected data in the server and training a personal authentication model; means for storing the personal authentication model in a database and associating it with personal information and account information; a means for authenticating a user based on facial recognition and biometric data; If authentication is successful, a means for processing transactions against the user's account; A means of notifying users of the results of their transactions; A system including:

2. 2. The system according to claim 1, which is used when a bankbook and a cash card are lost or vanished due to a natural disaster, earthquake, fire, or the like.

3. The system of claim 1, which utilizes generative AI to train a personal authentication model.

Images