System

Abstract

To achieve more robust and highly reliable biometric authentication by integrally using various kinds of biological information (face image, voiceprint, fingerprint, etc.).SOLUTION: A system includes a means for acquiring biological information which is a face image, a voiceprint, or a fingerprint, a means for inputting the acquired biological information to a generation model which is a machine learning algorithm to generate a virtual profile, and a means for storing the generated virtual profile in association with a user ID. The virtual profile also includes information for a neural network emotion engine that estimates the emotional state of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional password-based authentication systems have the drawbacks of requiring users to remember passwords, making password management cumbersome, and presenting security vulnerabilities. Introducing an AI-based biometric authentication system is expected to address these issues and improve user convenience and overall system security. Furthermore, it is necessary to achieve more robust and reliable authentication by integrating a variety of biometric information (e.g., facial images, voiceprints, and fingerprints). [Means for solving the problem]

[0005] The present invention is a system that includes a means for acquiring biometric information, a means for inputting the acquired biometric information into a generative model to generate a virtual profile, and a means for storing the generated virtual profile by linking it to a user ID. Furthermore, the acquired biometric information includes facial images, voiceprints, and fingerprints, and the generative model uses a machine learning algorithm. It also includes a means for preprocessing the biometric information, generating and matching a virtual profile, and returning authentication results. This configuration provides an advanced authentication system that eliminates the hassle of password management and improves security.

[0006] "Biometric information" refers to physical or behavioral characteristics used to identify an individual, including, for example, facial images, voiceprints, and fingerprints.

[0007] "Generative model" refers to an algorithm or system that uses machine learning algorithms to generate a unique virtual profile from input biometric information.

[0008] "Virtual Profile" refers to a unique digital profile generated from biometric information by a generative model and used to authenticate a user.

[0009] "User ID" means a unique identifier used to identify a user and which is linked to a virtual profile in a database.

[0010] "Preprocessing" refers to the processing performed to improve the quality of the data or remove noise before inputting the biometric information into the generative model.

[0011] "Matching" refers to the process of comparing a virtual profile generated from newly acquired biometric information with an existing virtual profile stored in a database to confirm a match.

[0012] The "authentication result" refers to information indicating whether the user authentication was successful or unsuccessful as a result of the comparison.

[0013] A "secure communications protocol" is a protocol for securely transmitting biometric and other data that uses encryption methods, such as HTTPS.

[0014] "Database" refers to a system for regularly storing information such as generated virtual profiles and user IDs.

[0015] A "sensor" is a device used to acquire biometric information, and specifically includes a camera, microphone, fingerprint scanner, etc. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The system according to the present invention acquires biometric information, generates a virtual profile based on a generative model, and performs a series of processes to authenticate a user based on the profile. The system also comprises a server, a terminal, and a user.

[0038] Explanation of program processing

[0039] User registration process

[0040] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the system's registration screen. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0041] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[0042] 3. The device sends the collected biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0043] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0044] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0045] 6. The server stores the generated virtual profile in a database together with the corresponding user ID.

[0046] Authentication process

[0047] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the login screen. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0048] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[0049] 3. The device sends the collected biometric information to the server using a secure communication protocol.

[0050] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0051] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[0052] 6. The server matches the generated virtual profile with existing virtual profiles stored in a database.

[0053] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, the server sends a message of failed authentication and asks the user to try again.

[0054] Specific examples

[0055] For example, when User A attempts to register for the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores that profile in a database. Next, when User A accesses the system again and attempts to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[0056] The above is an embodiment of the system of the present invention, which can eliminate the hassle of passwords and provide a high level of security.

[0057] The processing flow will be explained below.

[0058] User registration process

[0059] Step 1:

[0060] The user presses the "Register" button on the registration screen and is then asked to enter biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0061] Step 2:

[0062] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[0063] Step 3:

[0064] The biometric information acquired by the terminal is transmitted to a server using a secure communication protocol (for example, HTTPS).

[0065] Step 4:

[0066] The server receives the biometric information sent from the device and performs preprocessing on the received data, such as normalization to adjust the brightness of facial images, filtering to remove noise from voiceprint data, and enhancement to clarify fingerprint data.

[0067] Step 5:

[0068] The server inputs the pre-processed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0069] Step 6:

[0070] The server stores the generated virtual profile in a database together with the corresponding user ID.

[0071] Authentication process

[0072] Step 1:

[0073] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0074] Step 2:

[0075] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the respective digital formats.

[0076] Step 3:

[0077] The terminal transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[0078] Step 4:

[0079] The server receives the biometric information sent from the device and performs preprocessing on the received data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0080] Step 5:

[0081] The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[0082] Step 6:

[0083] The server matches the generated virtual profile with existing virtual profiles stored in a database.

[0084] Step 7:

[0085] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system; if not, it sends a message of failed authentication and asks the user to try again.

[0086] The above are the specific steps in the user registration and authentication processes, which greatly improve the security and convenience of the system.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] Conventional biometric authentication systems face challenges in efficiently and securely processing a user's biometric information for user authentication. Furthermore, when using a wide variety of biometric information, preprocessing each piece of information and generating a virtual profile using a generative model can be complex, potentially reducing the security and accuracy of the authentication process.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model to generate a virtual profile, means for storing the generated virtual profile in association with a user identifier, means for transmitting the acquired biometric information to the server using a secure communication protocol, means for preprocessing the received biometric information by the server, and means for verifying the generated virtual profile by the server. This enables efficient and secure processing of a wide variety of user biometric information, enabling highly accurate user authentication.

[0092] "Biometric information" is digital data that characterizes a part of a user's body, such as a facial image, voiceprint, or fingerprint.

[0093] A "generative model" is a model that uses machine learning algorithms to generate unique virtual profiles from input data.

[0094] A "virtual profile" is a digital profile created using a generative model that uniquely represents a user's biometric information.

[0095] A "user identifier" is an identification number or ID that is uniquely assigned to each user.

[0096] A "secure communication protocol" is a safe protocol used to protect data communication channels, and specifically includes HTTPS.

[0097] "Preprocessing" refers to a series of data processing operations including noise removal, filtering, and normalization of biological information.

[0098] "Matching" is the process of comparing a newly generated virtual profile with an existing virtual profile to determine the degree of match.

[0099] A "server" is a computer system that collects biometric information, pre-processes it, applies generative models, and generates and matches virtual profiles.

[0100] The system according to the present invention comprises a user, a terminal, and a server. A specific embodiment of the system is described below.

[0101] The system of the present invention acquires a user's biometric information and performs a series of processes to authenticate the user based on that information. The hardware required includes sensors such as a camera, microphone, and fingerprint scanner. The software used includes machine learning frameworks such as TensorFlow and PyTorch, and the HTTPS protocol for secure communication.

[0102] First, the user provides biometric information such as a facial image, voiceprint, and fingerprint on the system's registration screen. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. Through these operations, the user's biometric information is captured as digital data.

[0103] The device then collects this biometric information and converts the data from each sensor into a digital format—for example, converting camera footage into image data, microphone recordings into audio data, and fingerprint scans into fingerprint data—and transmits the collected data to a server using a secure communication protocol.

[0104] The server performs preprocessing on the received biometric data, including normalizing the facial image (correcting for lighting conditions and adjusting the size), filtering the voiceprint (removing noise), and enhancing the fingerprint data (emphasizing details). The preprocessed biometric data is then input into a generative model.

[0105] The generative model uses machine learning algorithms such as TensorFlow or PyTorch to generate a unique virtual profile from the preprocessed data, which is then stored in a database along with a user identifier.

[0106] During authentication, the user again provides biometric information such as a facial image, voiceprint, and fingerprint. The device collects this information, converts it into digital data, and sends it to the server. The server then preprocesses it again and generates a new virtual profile using a generative model. This new virtual profile is then compared with existing virtual profiles stored in the database. If the results match, the user is authenticated and can access the system. If the results do not match, the user receives a message indicating authentication has failed and is asked to try again.

[0107] Specific examples

[0108] For example, if User A wants to register with the system, he or she provides a face image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information and inputs it into a generative model (a TensorFlow-based machine learning model) to generate a virtual profile, which is then stored in a database.

[0109] Next, when User A accesses the system again and tries to log in, he / she provides his / her biometric information again. The terminal collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[0110] Prompt Sentence Examples

[0111] An example prompt to create a virtual profile for User B in the system is:

[0112] Please provide User B's face image, voiceprint, and fingerprint. Follow these steps to collect biometric information:

[0113] 1. Face the camera.

[0114] 2. Speak into the microphone.

[0115] 3. Place your finger on the fingerprint scanner.

[0116] Once you have collected this information, submit it to the system.

[0117] The above is an embodiment of the present invention. This system can eliminate the hassle of passwords and provide high security.

[0118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0119] Step 1: Provide biometric information

[0120] Users provide biometric information such as a facial image, voiceprint, or fingerprint on the system registration or login screen. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. This inputs biometric information as analog data.

[0121] Step 2: Collecting and digitalizing biometric information

[0122] The device collects biometric information provided by the user from each sensor (camera, microphone, fingerprint scanner) and converts it into digital form. For example, camera footage is converted into image data, microphone recordings into voice data, and fingerprint scans into fingerprint data. Here, the input is analog data and the output is digital data.

[0123] Step 3: Sending biometric information

[0124] The device transmits the collected digital biometric information to the server using a secure communication protocol (e.g., HTTPS). Specifically, the device encrypts the data and transmits it over a secure channel. The input is digital data, and the output is the encrypted data transmitted to the server.

[0125] Step 4: Preprocessing of biological information

[0126] The server receives the digital data sent from the device and performs preprocessing on each piece of information, such as normalizing facial images (correcting for lighting conditions and adjusting size), filtering voiceprints (removing noise), and enhancing fingerprint data (emphasizing details).The input here is the received biometric information, and the output is the preprocessed data.

[0127] Step 5: Generate a Virtual Profile

[0128] The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile. The generative model uses a machine learning algorithm (e.g., TensorFlow or PyTorch). Specifically, the server inputs the preprocessed data into the generative model, and the generative model outputs a virtual profile. The input is the preprocessed data, and the output is the virtual profile.

[0129] Step 6: Save the Virtual Profile

[0130] The server stores the generated virtual profile together with the corresponding user identifier in a database. Specifically, the server executes an SQL query to insert the virtual profile and the user identifier into the database. The input is the virtual profile and the user identifier, and the output is the result stored in the database.

[0131] Step 7: Provide biometric information when logging in

[0132] The user then provides biometric information such as a facial image, voiceprint, and fingerprint on the login screen again. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. This re-enters the biometric information as analog data.

[0133] Step 8: Collecting and digitally converting biometric information during login

[0134] The device collects biometric information provided by the user from each sensor (camera, microphone, fingerprint scanner), converts it into digital form, and outputs the collected data as digital data again.

[0135] Step 9: Sending biometric information when logging in

[0136] The device sends the collected digital biometric information to a server using a secure communication protocol (HTTPS). The input is digital data, and the output is encrypted data sent to the server.

[0137] Step 10: Preprocessing biometric information at login

[0138] The server receives the digital data sent from the terminal and pre-processes each piece of information again. The input is the received biometric information, and the output is the pre-processed data.

[0139] Step 11: Create a new virtual profile

[0140] The server inputs the preprocessed biometric information into a generative model to generate a new virtual profile. The input is the preprocessed data, and the output is the new virtual profile.

[0141] Step 12: Profile Matching

[0142] The server matches the newly generated virtual profile with existing virtual profiles stored in a database. Specifically, the server executes a query to retrieve existing profiles and uses a comparison algorithm to check for a match. The inputs are the new virtual profile and the existing virtual profile, and the output is the match result.

[0143] Step 13: Sending authentication results

[0144] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If the profile does not match, the server sends a message of failed authentication and asks the user to try again. The input is the matching result, and the output is the authentication message.

[0145] (Application example 1)

[0146] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0147] Conventional biometric authentication systems aim to improve security by using biometric information such as facial images, fingerprints, and voiceprints instead of passwords. However, issues remain, such as the difficulty of securing and managing biometric information, as well as the lack of preprocessing, normalization, and secure communication methods for biometric information. For these reasons, a system that can realize a safe and efficient authentication process is needed.

[0148] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0149] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model and generating a virtual profile, means for storing the generated virtual profile in association with a user ID, means for preprocessing or normalizing the registered biometric information, means for securely transmitting the preprocessed or normalized biometric information to the server, means for securely communicating biometric data from the local device to the server, and means for storing the generated virtual profile in a database and performing re-matching. This increases the security of biometric information and enables an efficient authentication process.

[0150] "Biometric information" is digital data of biological characteristics, such as facial images, fingerprints, and voiceprints, that are used to identify individuals.

[0151] A "generative model" is a model that uses a machine learning algorithm to generate a virtual profile based on biometric information.

[0152] A "virtual profile" is a unique digital profile data created by a generative model and linked to a user ID.

[0153] "Preprocessing" is the process of converting biometric information into a format suitable for the generative model by processing it through normalization, filtering, enhancement, and other processes.

[0154] "Normalization" is the process of converting biometric information into a standard format or scale to ensure data consistency.

[0155] "Secure communications" means encrypted communications protocols, such as HTTPS, used to protect the confidentiality and integrity of information.

[0156] "Database" refers to an integrated management system for storing and matching generated virtual profiles and corresponding user IDs.

[0157] "Matching" is the process of comparing the newly generated virtual profile with existing profiles to confirm a match.

[0158] A "local device" is a terminal such as a smartphone or tablet to which an individual user provides biometric information.

[0159] The system according to the present invention acquires biometric information, generates a virtual profile using a generative model, and performs a series of processes to authenticate a user based on the profile. The system is composed of a user, a terminal, and a server.

[0160] Explanation of program processing

[0161] User registration process

[0162] 1. The user provides biometric information such as a facial image, voiceprint, and fingerprint on the system's registration screen. For example, the user faces the smartphone camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0163] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[0164] 3. The device sends the collected biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0165] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information (e.g., normalizing facial images, filtering voiceprints, enhancing fingerprint data).

[0166] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0167] 6. The server stores the generated virtual profile in a database together with the corresponding user ID.

[0168] Authentication process

[0169] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the login screen. For example, the user faces the smartphone camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0170] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[0171] 3. The device sends the collected biometric information to the server using a secure communication protocol.

[0172] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information (e.g., normalizing facial images, filtering voiceprints, enhancing fingerprint data).

[0173] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[0174] 6. The server matches the generated virtual profile with existing virtual profiles stored in a database.

[0175] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, the server sends a message of failed authentication and asks the user to try again.

[0176] Specific examples

[0177] For example, when User A attempts to register for the system, he or she provides a facial image, voiceprint, and fingerprint. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores that profile in a database. Next, when User A accesses the system again and attempts to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[0178] Prompt Sentence Examples

[0179] For example, a sentence such as, "Generate a virtual profile using user A's facial image, voiceprint, and fingerprints. Then, compare it with existing virtual profiles stored in the database to determine whether there is a match" is input into the generative AI model.

[0180] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0181] Step 1:

[0182] The user provides biometric information such as a face image, voiceprint, and fingerprint on the system's registration screen. Here, the user's face image, voiceprint, and fingerprint are used as input. Specifically, the user faces the smartphone's camera, speaks into the microphone, and places their finger on the fingerprint scanner. The output is the state in which this biometric information has been collected.

[0183] Step 2:

[0184] The device collects biometric information provided by the user and obtains digital data from each sensor (camera, microphone, fingerprint scanner). This data is then converted into digital format. The input is the biometric information provided by the user, and the output is digital facial image data, voiceprint data, and fingerprint data. Specifically, each sensor captures the biometric information and stores it as digital data inside the device.

[0185] Step 3:

[0186] The device transmits the collected digital biometric information to a server using a secure communication protocol (e.g., HTTPS). The input is the digital biometric data, and the output is the data transmitted to the server. Specifically, the device encrypts the data and transmits it to the server via a secure channel.

[0187] Step 4:

[0188] The server receives the biometric information sent from the device and performs preprocessing on each piece of information. The input is the biometric data sent from the device, and the output is the preprocessed data. Specific operations include normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0189] Step 5:

[0190] The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile. The input is the preprocessed biometric data, and the output is the virtual profile. The generative model uses a machine learning algorithm. The specific operation is to apply the preprocessed data to the generative model to generate the virtual profile.

[0191] Step 6:

[0192] The server stores the generated virtual profile together with the user ID in a database. The input is the virtual profile and the user ID, and the output is the state saved in the database. The specific operation is to uniquely associate the virtual profile with the user ID and record it in the database.

[0193] Step 7:

[0194] The user provides their biometric information again on the login screen. The input is a new face image, voiceprint, and fingerprint, and the output is the newly collected biometric information. Specifically, the smartphone's camera, microphone, and fingerprint scanner are used again to acquire the new information.

[0195] Step 8:

[0196] The terminal converts the newly collected biometric information back into digital data and transmits it to the server using a secure communication protocol. The input is the new digital biometric information data, and the output is the data transmitted to the server again. The specific operations are the same as steps 2 and 3.

[0197] Step 9:

[0198] The server preprocesses the new biometric data sent from the device and inputs it into the generative model to generate a new virtual profile. The input is the newly preprocessed biometric data, and the output is a new virtual profile. The specific operations are the same as steps 4 and 5.

[0199] Step 10:

[0200] The server matches the newly generated virtual profile with the existing virtual profiles stored in the database. The input is the data of the new virtual profile and the existing virtual profile, and the output is the matching result. The specific operation is to retrieve the existing profile from the database and compare it with the new profile.

[0201] Step 11:

[0202] If the profile matches, the server sends an authentication success message to the terminal and the user can access the system. If it does not match, it sends an authentication failure message and asks the user to try again. The input is the matching result, and the output is a message indicating authentication success or failure. The specific operation is to generate a success or failure message and send it to the terminal.

[0203] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0204] This invention combines a system that acquires biometric information, generates a virtual profile based on a generative model, and authenticates users based on that profile with an emotion engine that recognizes the user's emotions. The system is composed of a server, a terminal, and a user, and by further integrating the emotion engine, authentication can be performed taking into account the user's emotional state.

[0205] Explanation of program processing

[0206] User registration process

[0207] 1. The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0208] 2. The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, converting the biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[0209] 3. The device transmits the acquired biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0210] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalization to adjust the brightness of facial images, filtering to remove noise from voiceprint data, and enhancement to make fingerprint data clearer.

[0211] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0212] 6. The server stores the generated virtual profile in a database along with the corresponding user ID.

[0213] 7. The server simultaneously uses an emotion engine to recognize the user's emotional state through facial expression analysis of facial images or tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[0214] Authentication process

[0215] 1. The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0216] 2. The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, converting the captured biometric information into the respective digital formats.

[0217] 3. The device transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[0218] 4. The server receives the biometric data sent from the device and performs preprocessing on each piece of data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0219] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[0220] 6. The server matches the generated virtual profile with existing virtual profiles stored in the database.

[0221] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, it sends a message of failed authentication and asks the user to try again.

[0222] 8. The server also uses the emotion engine to analyze the user's current emotional state and compare it with the existing emotional state. If the emotional state matches or is within a trusted range, it will be reflected in the authentication result.

[0223] Specific examples

[0224] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then saved in a database. At this time, the emotion engine recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and saves this in the profile. Next, when User A attempts to log in to the system again, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, authenticates User A. The emotion engine also analyzes User A's current emotional state and compares it with the existing emotional state. If they match, authentication becomes more reliable.

[0225] The above is an embodiment of a system incorporating an emotion engine according to the present invention. This system can eliminate the hassle of passwords and provide a higher level of security and user experience.

[0226] The processing flow will be explained below.

[0227] User registration process

[0228] Step 1:

[0229] The user presses the "Register" button on the registration screen and is then prompted to enter biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0230] Step 2:

[0231] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[0232] Step 3:

[0233] The biometric information acquired by the terminal is transmitted to a server using a secure communication protocol (for example, HTTPS).

[0234] Step 4:

[0235] The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalization to adjust the brightness and contrast of facial images, filtering to remove noise from voiceprint data, and enhancement to make fingerprint data clearer.

[0236] Step 5:

[0237] The server inputs the pre-processed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0238] Step 6:

[0239] The server stores the generated virtual profile in a database together with the corresponding user ID.

[0240] Step 7:

[0241] The server uses an emotion engine to recognize the user's emotional state by analyzing facial expressions in facial images or tone in voiceprints, for example, by analyzing subtle facial movements and voice tone, pitch, and rhythm.

[0242] Step 8:

[0243] The server stores the recognized emotional state together with the virtual profile in a database.

[0244] Authentication process

[0245] Step 1:

[0246] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0247] Step 2:

[0248] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the respective digital formats.

[0249] Step 3:

[0250] The terminal transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[0251] Step 4:

[0252] The server receives the biometric data sent from the device and performs preprocessing on each piece of data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0253] Step 5:

[0254] The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[0255] Step 6:

[0256] The server matches the generated virtual profile with existing virtual profiles stored in a database.

[0257] Step 7:

[0258] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system; if not, it sends a message of failed authentication and asks the user to try again.

[0259] Step 8:

[0260] The server uses an emotion engine to analyze the user's current emotional state, again by analyzing, for example, facial expressions and tone of voice.

[0261] Step 9:

[0262] The server compares the current emotional state with the existing emotional state, and if they match or are within a trustworthy range, reflects the result in the authentication result. For example, if the emotional state matches and the virtual profile matches, authentication is completed.

[0263] Specific examples

[0264] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then saved in a database. At this time, the emotion engine recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and saves this in the profile. Next, when User A attempts to log in to the system again, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated. The emotion engine is also used to analyze User A's current emotional state and compares it with the existing emotional state. If they match or are within the confidence range, more reliable authentication is achieved.

[0265] The above is an embodiment of a system incorporating an emotion engine according to the present invention. This system eliminates the hassle of passwords and provides a higher level of security and user experience.

[0266] Example 2

[0267] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0268] Conventional biometric authentication systems authenticate users based on their biometric information, but they have limitations in authentication accuracy and security. Furthermore, because they do not take into account the user's psychological state or emotions, it is difficult to prevent unauthorized access and the user experience is unsatisfactory. Furthermore, there is a risk of attacks on authentication data, making it necessary to strengthen security.

[0269] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0270] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model to generate a virtual profile, means for storing the generated virtual profile in association with a user ID, means for preprocessing the biometric information, means for authenticating the user based on the virtual profile, means for analyzing the user's emotional state using an emotion engine, means for adding the analyzed emotional state to the virtual profile and storing it, and means for reacquiring the biometric information and emotional state for authentication. This enables highly accurate authentication that takes into account the user's emotional state as well as the biometric information, thereby strengthening the security of the entire authentication process and improving the user experience.

[0271] "Biometric information" is data that digitally captures a user's physical characteristics, including facial images, voiceprints, fingerprints, and the like.

[0272] A "generative model" is a computational tool that uses machine learning algorithms to generate new information from input data.

[0273] A "virtual profile" is digital information that uniquely identifies a user and is generated using machine learning algorithms based on biometric information.

[0274] A "user ID" is an identifier that uniquely identifies a user and is linked to a virtual profile.

[0275] "Preprocessing" refers to data processing means for making acquired biometric information easier to analyze, and includes brightness adjustment, noise removal, enhancement, and the like.

[0276] The "emotion engine" is a software component for analyzing a user's emotional state from facial images and voiceprints.

[0277] "Emotional state" is data that indicates the emotional state that the user is currently experiencing, and is generated as a result of analysis.

[0278] "Authentication" is the process of verifying that a user is legitimately accessing something based on the biometric information and emotional state they provide.

[0279] This invention provides a system that acquires a user's biometric information and emotional state, and generates and authenticates a virtual profile based on a generative model. This system consists of a server, a terminal, and a user, and by integrating an emotion engine, it enables advanced authentication that takes into account the user's emotional state.

[0280] Overall system configuration

[0281] The main components of the system are:

[0282] Server: Responsible for preprocessing biometric information, running generative models, generating and storing virtual profiles, authentication, and analyzing emotional states.

[0283] Terminal: Acquires biometric information from the user, converts it into digital format, and sends it to the server.

[0284] User: Provides biometric information such as facial image, voiceprint, and fingerprint.

[0285] Emotion engine: A software component that analyzes the user's emotional state from biometric information.

[0286] Hardware and software used

[0287] Hardware:

[0288] 1. Camera: Captures a face image.

[0289] 2. Microphone: Records your voiceprint.

[0290] 3. Fingerprint scanner: Reads your fingerprint.

[0291] software:

[0292] 1. Secure communication protocols (e.g. HTTPS)

[0293] 2. Generative Models Using Machine Learning Algorithms

[0294] 3. Environment-dependent image processing algorithms (brightness adjustment, noise reduction, enhancement)

[0295] 4. Emotion analysis software (facial expression analysis and voice tone analysis)

[0296] Program processing explanation

[0297] User registration process

[0298] 1. The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0299] 2. The device converts the acquired biometric information into JPEG format (face image), WAV format (voiceprint), and binary format (fingerprint).

[0300] 3. The device sends the acquired biometric information to the server using HTTPS communication.

[0301] 4. The server adjusts the brightness of the received facial image, removes noise from the voiceprint data, and performs enhancement processing on the fingerprint data.

[0302] 5. The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile.

[0303] 6. The server associates the generated virtual profile with the user ID and saves it in the database.

[0304] 7. The server uses an emotion engine to recognize the user's emotional state from facial expression analysis of facial images and tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[0305] Authentication process

[0306] 1. The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, and fingerprint, again using the camera, microphone, and fingerprint scanner.

[0307] 2. The device converts the acquired biometric information into the appropriate digital format.

[0308] 3. The device sends the acquired biometric information to the server using HTTPS communication.

[0309] 4. The server normalizes the received facial images, filters the voiceprint data, and enhances the fingerprint data.

[0310] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[0311] 6. The server matches the new virtual profile with an existing virtual profile.

[0312] 7. If there is a match, the server sends an authentication success message to the terminal and the user can access the system. If there is no match, it sends an authentication failure message and asks the user to try again.

[0313] 8. The server uses the emotion engine to analyze the current emotional state and compare it with the existing emotional states. If it matches or is within the trusted range, it is reflected in the authentication result.

[0314] Specific examples

[0315] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server then preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then stored in a database. At this time, the emotion engine also recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and stores this in the profile.

[0316] Next, when User A accesses the system again and tries to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server then compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated. The emotion engine also analyzes the current emotional state and compares it with the existing emotional state. If they match, more reliable authentication is possible.

[0317] Prompt Sentence Examples

[0318] Example of a prompt during registration:

[0319] Describe the process of taking a user's facial image, voiceprint, and fingerprint as input, generating a virtual profile based on a generative model, and storing it in a database along with biometric information and emotional state.

[0320] An example of a login prompt:

[0321] Describe the authentication process by acquiring a user's facial image, voiceprint, and fingerprints, matching a newly generated virtual profile with an existing profile, and analyzing and comparing their emotional state.

[0322] The above is an example of a biometric authentication system that integrates an emotion engine according to the present invention, which eliminates the hassle of passwords and provides a higher level of security and user experience.

[0323] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0324] User registration process

[0325] Step 1:

[0326] The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. The user then faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0327] Input: User's face image, voiceprint, fingerprint

[0328] Output: Biometric information acquired by the device

[0329] Step 2:

[0330] The device uses a camera to capture a facial image in JPEG format, a microphone to record a voiceprint in WAV format, and a fingerprint scanner to read a fingerprint in binary format.

[0331] Input: Acquired biometric information

[0332] Output: JPEG facial image, WAV voiceprint, binary fingerprint

[0333] Step 3:

[0334] The device sends the acquired biometric information to the server using HTTPS.

[0335] Input: JPEG facial image, WAV voice print, binary fingerprint

[0336] Output: Biometric information sent to the server

[0337] Step 4:

[0338] The server normalizes the brightness of the received facial images, removes noise from voiceprints, and performs enhancement processing on fingerprint data.

[0339] Input: Biometric information sent to the server

[0340] Output: Preprocessed facial images, voiceprints, and fingerprint data

[0341] Step 5:

[0342] The server inputs the pre-processed biometric information into a generative model to generate a unique virtual profile.

[0343] Input: Preprocessed face images, voiceprints, and fingerprint data

[0344] Output: Generated virtual profile

[0345] Step 6:

[0346] The server associates the generated virtual profile with the user ID and stores it in the database.

[0347] Input: Generated virtual profile, user ID

[0348] Output: Virtual profile and user ID stored in the database

[0349] Step 7:

[0350] The server uses an emotion engine to recognize the user's emotional state from facial expression analysis of facial images and tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[0351] Input: Preprocessed face images and voiceprint data

[0352] Output: Recognized emotional state, updated virtual profile

[0353] Authentication process

[0354] Step 1:

[0355] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, and fingerprint. The user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0356] Input: User's face image, voiceprint, fingerprint

[0357] Output: Biometric information acquired by the device

[0358] Step 2:

[0359] The device uses a camera to capture a facial image in JPEG format, a microphone to record a voiceprint in WAV format, and a fingerprint scanner to read a fingerprint in binary format.

[0360] Input: Acquired biometric information

[0361] Output: JPEG facial image, WAV voiceprint, binary fingerprint

[0362] Step 3:

[0363] The device sends the acquired biometric information to the server using HTTPS.

[0364] Input: JPEG facial image, WAV voice print, binary fingerprint

[0365] Output: Biometric information sent to the server

[0366] Step 4:

[0367] The server normalizes the received facial images, filters voiceprints, and enhances fingerprint data.

[0368] Input: Biometric information sent to the server

[0369] Output: Preprocessed facial images, voiceprints, and fingerprint data

[0370] Step 5:

[0371] The server inputs the preprocessed biometric information into a generative model to generate a new virtual profile.

[0372] Input: Preprocessed face images, voiceprints, and fingerprint data

[0373] Output: The newly generated virtual profile

[0374] Step 6:

[0375] The server matches the newly created virtual profile with existing virtual profiles.

[0376] Input: Newly created virtual profile, existing virtual profile

[0377] Output: Matching result

[0378] Step 7:

[0379] If they match, the server sends an authentication success message to the terminal, allowing the user to access the system. If they do not match, the server sends an authentication failure message, requesting a retry.

[0380] Input: Matching result

[0381] Output: Authentication result message

[0382] Step 8:

[0383] The server uses the emotion engine to analyze the user's current emotional state and compare it with existing emotional states. If it matches or is within a trusted range, it is reflected in the authentication result.

[0384] Input: Preprocessed face images, voiceprint data, and existing emotional states

[0385] Output: Analyzed current emotional state, final authentication result

[0386] (Application example 2)

[0387] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0388] Conventional biometric authentication systems authenticate users by capturing biometric information such as facial images, voiceprints, and fingerprints, and storing it in association with a user ID. However, this alone has limitations in the accuracy of authentication and the strength of security. In particular, there is a problem in that the risk of unauthorized access increases if someone misuses the biometric information of a genuine user. Furthermore, because the system does not take into account the user's status (for example, when they are forced to log in), the user experience may be impaired.

[0389] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0390] In this invention, the server includes a means for acquiring biometric information, a means for inputting the acquired biometric information into a generative model to generate a virtual profile, a means for storing the generated virtual profile in association with a user ID, an emotion engine for analyzing the user's emotional state, and a means for adding the user's emotional state to the virtual profile using the emotion engine. This enables highly accurate authentication based on biometric information as well as security services that take the user's emotional state into consideration.

[0391] "Biometric information" refers to data containing biological characteristics that can uniquely identify an individual, such as facial images, voiceprints, and fingerprints.

[0392] "Generative model" refers to a model that uses machine learning algorithms to generate a virtual profile based on biometric information.

[0393] A "virtual profile" is a digital user profile generated using a generative model based on biometric information and used to identify an individual user.

[0394] "User ID" means a unique identifier used to identify a particular User.

[0395] "Emotion engine" refers to software or algorithms for analyzing a user's emotional state from their biometric information (e.g., facial image, voiceprint, etc.).

[0396] "Server" refers to a computer system for collecting, processing, and storing data.

[0397] This invention provides a security application installed on smartphones. The system acquires a user's biometric information, such as facial images, voiceprints, and fingerprints, and generates a virtual profile using a generative AI model. Furthermore, it analyzes the user's emotional state using an emotion engine and adds it to the virtual profile, achieving highly accurate and secure authentication.

[0398] Hardware and Software Configuration

[0399] Hardware

[0400] Camera: A device for capturing facial images.

[0401] Microphone: A device used to capture voiceprints.

[0402] Fingerprint scanner: A device for capturing fingerprint data.

[0403] software

[0404] OpenCV: A library for acquiring and preprocessing face images.

[0405] EmotionEngine: A module that analyzes emotional states from facial images.

[0406] VoiceRecognizer: A module that acquires and preprocesses voice data.

[0407] FingerprintScanner: A module that acquires and preprocesses fingerprint data.

[0408] Requests: An HTTP request library for communicating with the server.

[0409] Generative model: A model for generating virtual profiles from biometric information using machine learning algorithms.

[0410] System configuration

[0411] 1. A user accesses a security application on their smartphone and provides a facial image, voiceprint, and fingerprint.

[0412] 2. The device captures this biometric information using the camera, microphone, and fingerprint scanner and converts it into the appropriate digital format (JPEG, WAV, binary data).

[0413] 3. The device sends the biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0414] 4. The server receives the transmitted biometric information and performs preprocessing such as normalizing the facial image, removing noise from the audio data, and enhancing the fingerprint data.

[0415] 5. The server inputs the preprocessed biometric information into the generative model to generate a virtual profile.

[0416] 6. The server associates the generated virtual profile with the user ID and stores it in the database.

[0417] 7. The server uses an emotion engine to analyze the facial expression of the facial image and the tone of the voiceprint to analyze the user's emotional state.

[0418] 8. The server adds the analyzed emotional state to the virtual profile and further stores it in a database.

[0419] Specific examples

[0420] When User A wants to register with the system, he or she will provide a facial image, voiceprint, and fingerprints as prompted by the smartphone application. For example, the following prompts will be displayed:

[0421] "Please look at the camera with a natural expression. Next, introduce yourself into the microphone. Finally, place your finger on the fingerprint scanner."

[0422] When a user provides biometric information, the device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores the profile in a database. At the same time, the emotion engine analyzes facial images and voiceprints to recognize User A's emotional state, which is then added to and saved in the profile.

[0423] Now, if User A tries to log into the system again, he will see the following prompt:

[0424] "Please face the camera again to log in. Next, speak your name into the microphone. Finally, place your finger on the fingerprint scanner."

[0425] When User A provides their biometric information, the server generates a new virtual profile and compares it with the previously saved profile. The emotion engine also analyzes the current emotional state and compares it with the previously saved emotional state to provide a more reliable authentication.

[0426] This provides highly accurate authentication based on biometric information as well as security services that take into account the user's emotional state.

[0427] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0428] Step 1:

[0429] A user accesses a security application on their smartphone and provides a facial image, voiceprint, and fingerprint.

[0430] Input: The user faces the camera, introduces themselves into the microphone, and places their finger on the fingerprint scanner.

[0431] Output: Biometric information (face image, voiceprint, fingerprint).

[0432] Specific actions: The user follows the instructions, makes a natural facial expression towards the camera, speaks a few words, and places a finger on the scanner.

[0433] Step 2:

[0434] The device captures this biometric information using a camera, microphone, and fingerprint scanner and converts it into the appropriate digital format (JPEG, WAV, binary data).

[0435] Input: Biometric information (face image, voiceprint, fingerprint).

[0436] Output: Biometric information in digital format (JPEG, WAV, binary data).

[0437] Specific operations: A facial image is taken with a camera and converted to JPEG format, voice is recorded with a microphone and converted to WAV format, and a fingerprint is read with a fingerprint scanner and converted to binary data.

[0438] Step 3:

[0439] The device transmits the biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0440] Input: Biometric data in digital format (JPEG, WAV, binary data).

[0441] Output: Success / failure of sending to server.

[0442] Specific operation: Facial image, voiceprint, and fingerprint data are sent to the server using the HTTPS protocol.

[0443] Step 4:

[0444] The server receives the transmitted biometric information and performs preprocessing such as normalizing facial images, removing noise from audio data, and enhancing fingerprint data.

[0445] Input: Biometric data in digital format (JPEG, WAV, binary data).

[0446] Output: Preprocessed biometric information in digital form.

[0447] Specific operations: Adjusts and normalizes the brightness of facial images, clears audio data with a noise reduction filter, and performs enhancement processing to clarify fingerprint data.

[0448] Step 5:

[0449] The server inputs the preprocessed biometric information into a generative AI model to generate a virtual profile.

[0450] Input: Preprocessed digital biometric information.

[0451] Output: The generated virtual profile.

[0452] How it works: Machine learning algorithms are used to analyze pre-processed biometric information and generate a unique virtual profile.

[0453] Step 6:

[0454] The server associates the generated virtual profile with the user ID and stores it in the database.

[0455] Input: Generated virtual profile and user ID.

[0456] Output: Save to database successful / failed.

[0457] Specific operation: Record the virtual profile along with the user ID in the database and check whether it was saved successfully.

[0458] Step 7:

[0459] The server uses an emotion engine to analyze the user's emotional state by analyzing facial expressions in facial images and tone in voiceprints.

[0460] Input: Preprocessed face image and voiceprint.

[0461] Output: Parsed emotional state.

[0462] Specific operation: Executes the emotion engine, estimates emotions from facial images and voiceprints, and obtains analysis results.

[0463] Step 8:

[0464] The server adds the analyzed emotional state to the virtual profile and further stores it in a database.

[0465] Input: Generated virtual profile and analyzed emotional state.

[0466] Output: Success / failure in adding to database.

[0467] Specific operation: The emotional state is incorporated into the virtual profile, saved to the database, and the success of the save is confirmed.

[0468] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0469] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0470] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0471] [Second embodiment]

[0472] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0473] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0474] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0475] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0476] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0477] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0478] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0479] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0480] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0481] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0482] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0483] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0484] The system according to the present invention acquires biometric information, generates a virtual profile based on a generative model, and performs a series of processes to authenticate a user based on the profile. The system also comprises a server, a terminal, and a user.

[0485] Explanation of program processing

[0486] User registration process

[0487] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the system's registration screen. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0488] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[0489] 3. The device sends the collected biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0490] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0491] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0492] 6. The server stores the generated virtual profile in a database together with the corresponding user ID.

[0493] Authentication process

[0494] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the login screen. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0495] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[0496] 3. The device sends the collected biometric information to the server using a secure communication protocol.

[0497] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0498] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[0499] 6. The server matches the generated virtual profile with existing virtual profiles stored in a database.

[0500] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, the server sends a message of failed authentication and asks the user to try again.

[0501] Specific examples

[0502] For example, when User A attempts to register for the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores that profile in a database. Next, when User A accesses the system again and attempts to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[0503] The above is an embodiment of the system of the present invention, which can eliminate the hassle of passwords and provide a high level of security.

[0504] The processing flow will be explained below.

[0505] User registration process

[0506] Step 1:

[0507] The user presses the "Register" button on the registration screen and is then asked to enter biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0508] Step 2:

[0509] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[0510] Step 3:

[0511] The biometric information acquired by the terminal is transmitted to a server using a secure communication protocol (for example, HTTPS).

[0512] Step 4:

[0513] The server receives the biometric information sent from the device and performs preprocessing on the received data, such as normalization to adjust the brightness of facial images, filtering to remove noise from voiceprint data, and enhancement to clarify fingerprint data.

[0514] Step 5:

[0515] The server inputs the pre-processed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0516] Step 6:

[0517] The server stores the generated virtual profile in a database together with the corresponding user ID.

[0518] Authentication process

[0519] Step 1:

[0520] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0521] Step 2:

[0522] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the respective digital formats.

[0523] Step 3:

[0524] The terminal transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[0525] Step 4:

[0526] The server receives the biometric information sent from the device and performs preprocessing on the received data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0527] Step 5:

[0528] The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[0529] Step 6:

[0530] The server matches the generated virtual profile with existing virtual profiles stored in a database.

[0531] Step 7:

[0532] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system; if not, it sends a message of failed authentication and asks the user to try again.

[0533] The above are the specific steps in the user registration and authentication processes, which greatly improve the security and convenience of the system.

[0534] Example 1

[0535] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0536] Conventional biometric authentication systems face challenges in efficiently and securely processing a user's biometric information for user authentication. Furthermore, when using a wide variety of biometric information, preprocessing each piece of information and generating a virtual profile using a generative model can be complex, potentially reducing the security and accuracy of the authentication process.

[0537] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0538] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model to generate a virtual profile, means for storing the generated virtual profile in association with a user identifier, means for transmitting the acquired biometric information to the server using a secure communication protocol, means for preprocessing the received biometric information by the server, and means for verifying the generated virtual profile by the server. This enables efficient and secure processing of a wide variety of user biometric information, enabling highly accurate user authentication.

[0539] "Biometric information" is digital data that characterizes a part of a user's body, such as a facial image, voiceprint, or fingerprint.

[0540] A "generative model" is a model that uses machine learning algorithms to generate unique virtual profiles from input data.

[0541] A "virtual profile" is a digital profile created using a generative model that uniquely represents a user's biometric information.

[0542] A "user identifier" is an identification number or ID that is uniquely assigned to each user.

[0543] A "secure communication protocol" is a safe protocol used to protect data communication channels, and specifically includes HTTPS.

[0544] "Preprocessing" refers to a series of data processing operations including noise removal, filtering, and normalization of biological information.

[0545] "Matching" is the process of comparing a newly generated virtual profile with an existing virtual profile to determine the degree of match.

[0546] A "server" is a computer system that collects biometric information, pre-processes it, applies generative models, and generates and matches virtual profiles.

[0547] The system according to the present invention comprises a user, a terminal, and a server. A specific embodiment of the system is described below.

[0548] The system of the present invention acquires a user's biometric information and performs a series of processes to authenticate the user based on that information. The hardware required includes sensors such as a camera, microphone, and fingerprint scanner. The software used includes machine learning frameworks such as TensorFlow and PyTorch, and the HTTPS protocol for secure communication.

[0549] First, the user provides biometric information such as a facial image, voiceprint, and fingerprint on the system's registration screen. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. Through these operations, the user's biometric information is captured as digital data.

[0550] The device then collects this biometric information and converts the data from each sensor into a digital format—for example, converting camera footage into image data, microphone recordings into audio data, and fingerprint scans into fingerprint data—and transmits the collected data to a server using a secure communication protocol.

[0551] The server performs preprocessing on the received biometric data, including normalizing the facial image (correcting for lighting conditions and adjusting the size), filtering the voiceprint (removing noise), and enhancing the fingerprint data (emphasizing details). The preprocessed biometric data is then input into a generative model.

[0552] The generative model uses machine learning algorithms such as TensorFlow or PyTorch to generate a unique virtual profile from the preprocessed data, which is then stored in a database along with a user identifier.

[0553] During authentication, the user again provides biometric information such as a facial image, voiceprint, and fingerprint. The device collects this information, converts it into digital data, and sends it to the server. The server then preprocesses it again and generates a new virtual profile using a generative model. This new virtual profile is then compared with existing virtual profiles stored in the database. If the results match, the user is authenticated and can access the system. If the results do not match, the user receives a message indicating authentication has failed and is asked to try again.

[0554] Specific examples

[0555] For example, if User A wants to register with the system, he or she provides a face image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information and inputs it into a generative model (a TensorFlow-based machine learning model) to generate a virtual profile, which is then stored in a database.

[0556] Next, when User A accesses the system again and tries to log in, he / she provides his / her biometric information again. The terminal collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[0557] Prompt Sentence Examples

[0558] An example prompt to create a virtual profile for User B in the system is:

[0559] Please provide User B's face image, voiceprint, and fingerprint. Follow these steps to collect biometric information:

[0560] 1. Face the camera.

[0561] 2. Speak into the microphone.

[0562] 3. Place your finger on the fingerprint scanner.

[0563] Once you have collected this information, submit it to the system.

[0564] The above is an embodiment of the present invention. This system can eliminate the hassle of passwords and provide high security.

[0565] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0566] Step 1: Provide biometric information

[0567] Users provide biometric information such as a facial image, voiceprint, or fingerprint on the system registration or login screen. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. This inputs biometric information as analog data.

[0568] Step 2: Collecting and digitalizing biometric information

[0569] The device collects biometric information provided by the user from each sensor (camera, microphone, fingerprint scanner) and converts it into digital form. For example, camera footage is converted into image data, microphone recordings into voice data, and fingerprint scans into fingerprint data. Here, the input is analog data and the output is digital data.

[0570] Step 3: Sending biometric information

[0571] The device transmits the collected digital biometric information to the server using a secure communication protocol (e.g., HTTPS). Specifically, the device encrypts the data and transmits it over a secure channel. The input is digital data, and the output is the encrypted data transmitted to the server.

[0572] Step 4: Preprocessing of biological information

[0573] The server receives the digital data sent from the device and performs preprocessing on each piece of information, such as normalizing facial images (correcting for lighting conditions and adjusting size), filtering voiceprints (removing noise), and enhancing fingerprint data (emphasizing details).The input here is the received biometric information, and the output is the preprocessed data.

[0574] Step 5: Generate a Virtual Profile

[0575] The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile. The generative model uses a machine learning algorithm (e.g., TensorFlow or PyTorch). Specifically, the server inputs the preprocessed data into the generative model, and the generative model outputs a virtual profile. The input is the preprocessed data, and the output is the virtual profile.

[0576] Step 6: Save the Virtual Profile

[0577] The server stores the generated virtual profile together with the corresponding user identifier in a database. Specifically, the server executes an SQL query to insert the virtual profile and the user identifier into the database. The input is the virtual profile and the user identifier, and the output is the result stored in the database.

[0578] Step 7: Provide biometric information when logging in

[0579] The user then provides biometric information such as a facial image, voiceprint, and fingerprint on the login screen again. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. This re-enters the biometric information as analog data.

[0580] Step 8: Collecting and digitally converting biometric information during login

[0581] The device collects biometric information provided by the user from each sensor (camera, microphone, fingerprint scanner), converts it into digital form, and outputs the collected data as digital data again.

[0582] Step 9: Sending biometric information when logging in

[0583] The device sends the collected digital biometric information to a server using a secure communication protocol (HTTPS). The input is digital data, and the output is encrypted data sent to the server.

[0584] Step 10: Preprocessing biometric information at login

[0585] The server receives the digital data sent from the terminal and pre-processes each piece of information again. The input is the received biometric information, and the output is the pre-processed data.

[0586] Step 11: Create a new virtual profile

[0587] The server inputs the preprocessed biometric information into a generative model to generate a new virtual profile. The input is the preprocessed data, and the output is the new virtual profile.

[0588] Step 12: Profile Matching

[0589] The server matches the newly generated virtual profile with existing virtual profiles stored in a database. Specifically, the server executes a query to retrieve existing profiles and uses a comparison algorithm to check for a match. The inputs are the new virtual profile and the existing virtual profile, and the output is the match result.

[0590] Step 13: Sending authentication results

[0591] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If the profile does not match, the server sends a message of failed authentication and asks the user to try again. The input is the matching result, and the output is the authentication message.

[0592] (Application example 1)

[0593] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0594] Conventional biometric authentication systems aim to improve security by using biometric information such as facial images, fingerprints, and voiceprints instead of passwords. However, issues remain, such as the difficulty of securing and managing biometric information, as well as the lack of preprocessing, normalization, and secure communication methods for biometric information. For these reasons, a system that can realize a safe and efficient authentication process is needed.

[0595] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0596] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model and generating a virtual profile, means for storing the generated virtual profile in association with a user ID, means for preprocessing or normalizing the registered biometric information, means for securely transmitting the preprocessed or normalized biometric information to the server, means for securely communicating biometric data from the local device to the server, and means for storing the generated virtual profile in a database and performing re-matching. This increases the security of biometric information and enables an efficient authentication process.

[0597] "Biometric information" is digital data of biological characteristics, such as facial images, fingerprints, and voiceprints, that are used to identify individuals.

[0598] A "generative model" is a model that uses a machine learning algorithm to generate a virtual profile based on biometric information.

[0599] A "virtual profile" is a unique digital profile data created by a generative model and linked to a user ID.

[0600] "Preprocessing" is the process of converting biometric information into a format suitable for the generative model by processing it through normalization, filtering, enhancement, and other processes.

[0601] "Normalization" is the process of converting biometric information into a standard format or scale to ensure data consistency.

[0602] "Secure communications" means encrypted communications protocols, such as HTTPS, used to protect the confidentiality and integrity of information.

[0603] "Database" refers to an integrated management system for storing and matching generated virtual profiles and corresponding user IDs.

[0604] "Matching" is the process of comparing the newly generated virtual profile with existing profiles to confirm a match.

[0605] A "local device" is a terminal such as a smartphone or tablet to which an individual user provides biometric information.

[0606] The system according to the present invention acquires biometric information, generates a virtual profile using a generative model, and performs a series of processes to authenticate a user based on the profile. The system is composed of a user, a terminal, and a server.

[0607] Explanation of program processing

[0608] User registration process

[0609] 1. The user provides biometric information such as a facial image, voiceprint, and fingerprint on the system's registration screen. For example, the user faces the smartphone camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0610] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[0611] 3. The device sends the collected biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0612] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information (e.g., normalizing facial images, filtering voiceprints, enhancing fingerprint data).

[0613] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0614] 6. The server stores the generated virtual profile in a database together with the corresponding user ID.

[0615] Authentication process

[0616] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the login screen. For example, the user faces the smartphone camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0617] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[0618] 3. The device sends the collected biometric information to the server using a secure communication protocol.

[0619] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information (e.g., normalizing facial images, filtering voiceprints, enhancing fingerprint data).

[0620] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[0621] 6. The server matches the generated virtual profile with existing virtual profiles stored in a database.

[0622] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, the server sends a message of failed authentication and asks the user to try again.

[0623] Specific examples

[0624] For example, when User A attempts to register for the system, he or she provides a facial image, voiceprint, and fingerprint. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores that profile in a database. Next, when User A accesses the system again and attempts to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[0625] Prompt Sentence Examples

[0626] For example, a sentence such as, "Generate a virtual profile using user A's facial image, voiceprint, and fingerprints. Then, compare it with existing virtual profiles stored in the database to determine whether there is a match" is input into the generative AI model.

[0627] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0628] Step 1:

[0629] The user provides biometric information such as a face image, voiceprint, and fingerprint on the system's registration screen. Here, the user's face image, voiceprint, and fingerprint are used as input. Specifically, the user faces the smartphone's camera, speaks into the microphone, and places their finger on the fingerprint scanner. The output is the state in which this biometric information has been collected.

[0630] Step 2:

[0631] The device collects biometric information provided by the user and obtains digital data from each sensor (camera, microphone, fingerprint scanner). This data is then converted into digital format. The input is the biometric information provided by the user, and the output is digital facial image data, voiceprint data, and fingerprint data. Specifically, each sensor captures the biometric information and stores it as digital data inside the device.

[0632] Step 3:

[0633] The device transmits the collected digital biometric information to a server using a secure communication protocol (e.g., HTTPS). The input is the digital biometric data, and the output is the data transmitted to the server. Specifically, the device encrypts the data and transmits it to the server via a secure channel.

[0634] Step 4:

[0635] The server receives the biometric information sent from the device and performs preprocessing on each piece of information. The input is the biometric data sent from the device, and the output is the preprocessed data. Specific operations include normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0636] Step 5:

[0637] The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile. The input is the preprocessed biometric data, and the output is the virtual profile. The generative model uses a machine learning algorithm. The specific operation is to apply the preprocessed data to the generative model to generate the virtual profile.

[0638] Step 6:

[0639] The server stores the generated virtual profile together with the user ID in a database. The input is the virtual profile and the user ID, and the output is the state saved in the database. The specific operation is to uniquely associate the virtual profile with the user ID and record it in the database.

[0640] Step 7:

[0641] The user provides their biometric information again on the login screen. The input is a new face image, voiceprint, and fingerprint, and the output is the newly collected biometric information. Specifically, the smartphone's camera, microphone, and fingerprint scanner are used again to acquire the new information.

[0642] Step 8:

[0643] The terminal converts the newly collected biometric information back into digital data and transmits it to the server using a secure communication protocol. The input is the new digital biometric information data, and the output is the data transmitted to the server again. The specific operations are the same as steps 2 and 3.

[0644] Step 9:

[0645] The server preprocesses the new biometric data sent from the device and inputs it into the generative model to generate a new virtual profile. The input is the newly preprocessed biometric data, and the output is a new virtual profile. The specific operations are the same as steps 4 and 5.

[0646] Step 10:

[0647] The server matches the newly generated virtual profile with the existing virtual profiles stored in the database. The input is the data of the new virtual profile and the existing virtual profile, and the output is the matching result. The specific operation is to retrieve the existing profile from the database and compare it with the new profile.

[0648] Step 11:

[0649] If the profile matches, the server sends an authentication success message to the terminal and the user can access the system. If it does not match, it sends an authentication failure message and asks the user to try again. The input is the matching result, and the output is a message indicating authentication success or failure. The specific operation is to generate a success or failure message and send it to the terminal.

[0650] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0651] This invention combines a system that acquires biometric information, generates a virtual profile based on a generative model, and authenticates users based on that profile with an emotion engine that recognizes the user's emotions. The system is composed of a server, a terminal, and a user, and by further integrating the emotion engine, authentication can be performed taking into account the user's emotional state.

[0652] Explanation of program processing

[0653] User registration process

[0654] 1. The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0655] 2. The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, converting the biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[0656] 3. The device transmits the acquired biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0657] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalization to adjust the brightness of facial images, filtering to remove noise from voiceprint data, and enhancement to make fingerprint data clearer.

[0658] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0659] 6. The server stores the generated virtual profile in a database along with the corresponding user ID.

[0660] 7. The server simultaneously uses an emotion engine to recognize the user's emotional state through facial expression analysis of facial images or tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[0661] Authentication process

[0662] 1. The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0663] 2. The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, converting the captured biometric information into the respective digital formats.

[0664] 3. The device transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[0665] 4. The server receives the biometric data sent from the device and performs preprocessing on each piece of data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0666] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[0667] 6. The server matches the generated virtual profile with existing virtual profiles stored in the database.

[0668] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, it sends a message of failed authentication and asks the user to try again.

[0669] 8. The server also uses the emotion engine to analyze the user's current emotional state and compare it with the existing emotional state. If the emotional state matches or is within a trusted range, it will be reflected in the authentication result.

[0670] Specific examples

[0671] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then saved in a database. At this time, the emotion engine recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and saves this in the profile. Next, when User A attempts to log in to the system again, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, authenticates User A. The emotion engine also analyzes User A's current emotional state and compares it with the existing emotional state. If they match, authentication becomes more reliable.

[0672] The above is an embodiment of a system incorporating an emotion engine according to the present invention. This system can eliminate the hassle of passwords and provide a higher level of security and user experience.

[0673] The processing flow will be explained below.

[0674] User registration process

[0675] Step 1:

[0676] The user presses the "Register" button on the registration screen and is then prompted to enter biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0677] Step 2:

[0678] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[0679] Step 3:

[0680] The biometric information acquired by the terminal is transmitted to a server using a secure communication protocol (for example, HTTPS).

[0681] Step 4:

[0682] The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalization to adjust the brightness and contrast of facial images, filtering to remove noise from voiceprint data, and enhancement to make fingerprint data clearer.

[0683] Step 5:

[0684] The server inputs the pre-processed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0685] Step 6:

[0686] The server stores the generated virtual profile in a database together with the corresponding user ID.

[0687] Step 7:

[0688] The server uses an emotion engine to recognize the user's emotional state by analyzing facial expressions in facial images or tone in voiceprints, for example, by analyzing subtle facial movements and voice tone, pitch, and rhythm.

[0689] Step 8:

[0690] The server stores the recognized emotional state together with the virtual profile in a database.

[0691] Authentication process

[0692] Step 1:

[0693] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0694] Step 2:

[0695] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the respective digital formats.

[0696] Step 3:

[0697] The terminal transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[0698] Step 4:

[0699] The server receives the biometric data sent from the device and performs preprocessing on each piece of data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0700] Step 5:

[0701] The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[0702] Step 6:

[0703] The server matches the generated virtual profile with existing virtual profiles stored in a database.

[0704] Step 7:

[0705] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system; if not, it sends a message of failed authentication and asks the user to try again.

[0706] Step 8:

[0707] The server uses an emotion engine to analyze the user's current emotional state, again by analyzing, for example, facial expressions and tone of voice.

[0708] Step 9:

[0709] The server compares the current emotional state with the existing emotional state, and if they match or are within a trustworthy range, reflects the result in the authentication result. For example, if the emotional state matches and the virtual profile matches, authentication is completed.

[0710] Specific examples

[0711] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then saved in a database. At this time, the emotion engine recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and saves this in the profile. Next, when User A attempts to log in to the system again, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated. The emotion engine is also used to analyze User A's current emotional state and compares it with the existing emotional state. If they match or are within the confidence range, more reliable authentication is achieved.

[0712] The above is an embodiment of a system incorporating an emotion engine according to the present invention. This system eliminates the hassle of passwords and provides a higher level of security and user experience.

[0713] Example 2

[0714] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0715] Conventional biometric authentication systems authenticate users based on their biometric information, but they have limitations in authentication accuracy and security. Furthermore, because they do not take into account the user's psychological state or emotions, it is difficult to prevent unauthorized access and the user experience is unsatisfactory. Furthermore, there is a risk of attacks on authentication data, making it necessary to strengthen security.

[0716] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0717] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model to generate a virtual profile, means for storing the generated virtual profile in association with a user ID, means for preprocessing the biometric information, means for authenticating the user based on the virtual profile, means for analyzing the user's emotional state using an emotion engine, means for adding the analyzed emotional state to the virtual profile and storing it, and means for reacquiring the biometric information and emotional state for authentication. This enables highly accurate authentication that takes into account the user's emotional state as well as the biometric information, thereby strengthening the security of the entire authentication process and improving the user experience.

[0718] "Biometric information" is data that digitally captures a user's physical characteristics, including facial images, voiceprints, fingerprints, and the like.

[0719] A "generative model" is a computational tool that uses machine learning algorithms to generate new information from input data.

[0720] A "virtual profile" is digital information that uniquely identifies a user and is generated using machine learning algorithms based on biometric information.

[0721] A "user ID" is an identifier that uniquely identifies a user and is linked to a virtual profile.

[0722] "Preprocessing" refers to data processing means for making acquired biometric information easier to analyze, and includes brightness adjustment, noise removal, enhancement, and the like.

[0723] The "emotion engine" is a software component for analyzing a user's emotional state from facial images and voiceprints.

[0724] "Emotional state" is data that indicates the emotional state that the user is currently experiencing, and is generated as a result of analysis.

[0725] "Authentication" is the process of verifying that a user is legitimately accessing something based on the biometric information and emotional state they provide.

[0726] This invention provides a system that acquires a user's biometric information and emotional state, and generates and authenticates a virtual profile based on a generative model. This system consists of a server, a terminal, and a user, and by integrating an emotion engine, it enables advanced authentication that takes into account the user's emotional state.

[0727] Overall system configuration

[0728] The main components of the system are:

[0729] Server: Responsible for preprocessing biometric information, running generative models, generating and storing virtual profiles, authentication, and analyzing emotional states.

[0730] Terminal: Acquires biometric information from the user, converts it into digital format, and sends it to the server.

[0731] User: Provides biometric information such as facial image, voiceprint, and fingerprint.

[0732] Emotion engine: A software component that analyzes the user's emotional state from biometric information.

[0733] Hardware and software used

[0734] Hardware:

[0735] 1. Camera: Captures a face image.

[0736] 2. Microphone: Records your voiceprint.

[0737] 3. Fingerprint scanner: Reads your fingerprint.

[0738] software:

[0739] 1. Secure communication protocols (e.g. HTTPS)

[0740] 2. Generative Models Using Machine Learning Algorithms

[0741] 3. Environment-dependent image processing algorithms (brightness adjustment, noise reduction, enhancement)

[0742] 4. Emotion analysis software (facial expression analysis and voice tone analysis)

[0743] Program processing explanation

[0744] User registration process

[0745] 1. The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0746] 2. The device converts the acquired biometric information into JPEG format (face image), WAV format (voiceprint), and binary format (fingerprint).

[0747] 3. The device sends the acquired biometric information to the server using HTTPS communication.

[0748] 4. The server adjusts the brightness of the received facial image, removes noise from the voiceprint data, and performs enhancement processing on the fingerprint data.

[0749] 5. The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile.

[0750] 6. The server associates the generated virtual profile with the user ID and saves it in the database.

[0751] 7. The server uses an emotion engine to recognize the user's emotional state from facial expression analysis of facial images and tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[0752] Authentication process

[0753] 1. The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, and fingerprint, again using the camera, microphone, and fingerprint scanner.

[0754] 2. The device converts the acquired biometric information into the appropriate digital format.

[0755] 3. The device sends the acquired biometric information to the server using HTTPS communication.

[0756] 4. The server normalizes the received facial images, filters the voiceprint data, and enhances the fingerprint data.

[0757] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[0758] 6. The server matches the new virtual profile with an existing virtual profile.

[0759] 7. If there is a match, the server sends an authentication success message to the terminal and the user can access the system. If there is no match, it sends an authentication failure message and asks the user to try again.

[0760] 8. The server uses the emotion engine to analyze the current emotional state and compare it with the existing emotional states. If it matches or is within the trusted range, it is reflected in the authentication result.

[0761] Specific examples

[0762] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server then preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then stored in a database. At this time, the emotion engine also recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and stores this in the profile.

[0763] Next, when User A accesses the system again and tries to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server then compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated. The emotion engine also analyzes the current emotional state and compares it with the existing emotional state. If they match, more reliable authentication is possible.

[0764] Prompt Sentence Examples

[0765] Example of a prompt during registration:

[0766] Describe the process of taking a user's facial image, voiceprint, and fingerprint as input, generating a virtual profile based on a generative model, and storing it in a database along with biometric information and emotional state.

[0767] An example of a login prompt:

[0768] Describe the authentication process by acquiring a user's facial image, voiceprint, and fingerprints, matching a newly generated virtual profile with an existing profile, and analyzing and comparing their emotional state.

[0769] The above is an example of a biometric authentication system that integrates an emotion engine according to the present invention, which eliminates the hassle of passwords and provides a higher level of security and user experience.

[0770] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0771] User registration process

[0772] Step 1:

[0773] The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. The user then faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0774] Input: User's face image, voiceprint, fingerprint

[0775] Output: Biometric information acquired by the device

[0776] Step 2:

[0777] The device uses a camera to capture a facial image in JPEG format, a microphone to record a voiceprint in WAV format, and a fingerprint scanner to read a fingerprint in binary format.

[0778] Input: Acquired biometric information

[0779] Output: JPEG facial image, WAV voiceprint, binary fingerprint

[0780] Step 3:

[0781] The device sends the acquired biometric information to the server using HTTPS.

[0782] Input: JPEG facial image, WAV voice print, binary fingerprint

[0783] Output: Biometric information sent to the server

[0784] Step 4:

[0785] The server normalizes the brightness of the received facial images, removes noise from voiceprints, and performs enhancement processing on fingerprint data.

[0786] Input: Biometric information sent to the server

[0787] Output: Preprocessed facial images, voiceprints, and fingerprint data

[0788] Step 5:

[0789] The server inputs the pre-processed biometric information into a generative model to generate a unique virtual profile.

[0790] Input: Preprocessed face images, voiceprints, and fingerprint data

[0791] Output: Generated virtual profile

[0792] Step 6:

[0793] The server associates the generated virtual profile with the user ID and stores it in the database.

[0794] Input: Generated virtual profile, user ID

[0795] Output: Virtual profile and user ID stored in the database

[0796] Step 7:

[0797] The server uses an emotion engine to recognize the user's emotional state from facial expression analysis of facial images and tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[0798] Input: Preprocessed face images and voiceprint data

[0799] Output: Recognized emotional state, updated virtual profile

[0800] Authentication process

[0801] Step 1:

[0802] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, and fingerprint. The user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0803] Input: User's face image, voiceprint, fingerprint

[0804] Output: Biometric information acquired by the device

[0805] Step 2:

[0806] The device uses a camera to capture a facial image in JPEG format, a microphone to record a voiceprint in WAV format, and a fingerprint scanner to read a fingerprint in binary format.

[0807] Input: Acquired biometric information

[0808] Output: JPEG facial image, WAV voiceprint, binary fingerprint

[0809] Step 3:

[0810] The device sends the acquired biometric information to the server using HTTPS.

[0811] Input: JPEG facial image, WAV voice print, binary fingerprint

[0812] Output: Biometric information sent to the server

[0813] Step 4:

[0814] The server normalizes the received facial images, filters voiceprints, and enhances fingerprint data.

[0815] Input: Biometric information sent to the server

[0816] Output: Preprocessed facial images, voiceprints, and fingerprint data

[0817] Step 5:

[0818] The server inputs the preprocessed biometric information into a generative model to generate a new virtual profile.

[0819] Input: Preprocessed face images, voiceprints, and fingerprint data

[0820] Output: The newly generated virtual profile

[0821] Step 6:

[0822] The server matches the newly created virtual profile with existing virtual profiles.

[0823] Input: Newly created virtual profile, existing virtual profile

[0824] Output: Matching result

[0825] Step 7:

[0826] If they match, the server sends an authentication success message to the terminal, allowing the user to access the system. If they do not match, the server sends an authentication failure message, requesting a retry.

[0827] Input: Matching result

[0828] Output: Authentication result message

[0829] Step 8:

[0830] The server uses the emotion engine to analyze the user's current emotional state and compare it with existing emotional states. If it matches or is within a trusted range, it is reflected in the authentication result.

[0831] Input: Preprocessed face images, voiceprint data, and existing emotional states

[0832] Output: Analyzed current emotional state, final authentication result

[0833] (Application example 2)

[0834] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0835] Conventional biometric authentication systems authenticate users by capturing biometric information such as facial images, voiceprints, and fingerprints, and storing it in association with a user ID. However, this alone has limitations in the accuracy of authentication and the strength of security. In particular, there is a problem in that the risk of unauthorized access increases if someone misuses the biometric information of a genuine user. Furthermore, because the system does not take into account the user's status (for example, when they are forced to log in), the user experience may be impaired.

[0836] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0837] In this invention, the server includes a means for acquiring biometric information, a means for inputting the acquired biometric information into a generative model to generate a virtual profile, a means for storing the generated virtual profile in association with a user ID, an emotion engine for analyzing the user's emotional state, and a means for adding the user's emotional state to the virtual profile using the emotion engine. This enables highly accurate authentication based on biometric information as well as security services that take the user's emotional state into consideration.

[0838] "Biometric information" refers to data containing biological characteristics that can uniquely identify an individual, such as facial images, voiceprints, and fingerprints.

[0839] "Generative model" refers to a model that uses machine learning algorithms to generate a virtual profile based on biometric information.

[0840] A "virtual profile" is a digital user profile generated using a generative model based on biometric information and used to identify an individual user.

[0841] "User ID" means a unique identifier used to identify a particular User.

[0842] "Emotion engine" refers to software or algorithms for analyzing a user's emotional state from their biometric information (e.g., facial image, voiceprint, etc.).

[0843] "Server" refers to a computer system for collecting, processing, and storing data.

[0844] This invention provides a security application installed on smartphones. The system acquires a user's biometric information, such as facial images, voiceprints, and fingerprints, and generates a virtual profile using a generative AI model. Furthermore, it analyzes the user's emotional state using an emotion engine and adds it to the virtual profile, achieving highly accurate and secure authentication.

[0845] Hardware and Software Configuration

[0846] Hardware

[0847] Camera: A device for capturing facial images.

[0848] Microphone: A device used to capture voiceprints.

[0849] Fingerprint scanner: A device for capturing fingerprint data.

[0850] software

[0851] OpenCV: A library for acquiring and preprocessing face images.

[0852] EmotionEngine: A module that analyzes emotional states from facial images.

[0853] VoiceRecognizer: A module that acquires and preprocesses voice data.

[0854] FingerprintScanner: A module that acquires and preprocesses fingerprint data.

[0855] Requests: An HTTP request library for communicating with the server.

[0856] Generative model: A model for generating virtual profiles from biometric information using machine learning algorithms.

[0857] System configuration

[0858] 1. A user accesses a security application on their smartphone and provides a facial image, voiceprint, and fingerprint.

[0859] 2. The device captures this biometric information using the camera, microphone, and fingerprint scanner and converts it into the appropriate digital format (JPEG, WAV, binary data).

[0860] 3. The device sends the biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0861] 4. The server receives the transmitted biometric information and performs preprocessing such as normalizing the facial image, removing noise from the audio data, and enhancing the fingerprint data.

[0862] 5. The server inputs the preprocessed biometric information into the generative model to generate a virtual profile.

[0863] 6. The server associates the generated virtual profile with the user ID and stores it in the database.

[0864] 7. The server uses an emotion engine to analyze the facial expression of the facial image and the tone of the voiceprint to analyze the user's emotional state.

[0865] 8. The server adds the analyzed emotional state to the virtual profile and further stores it in a database.

[0866] Specific examples

[0867] When User A wants to register with the system, he or she will provide a facial image, voiceprint, and fingerprints as prompted by the smartphone application. For example, the following prompts will be displayed:

[0868] "Please look at the camera with a natural expression. Next, introduce yourself into the microphone. Finally, place your finger on the fingerprint scanner."

[0869] When a user provides biometric information, the device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores the profile in a database. At the same time, the emotion engine analyzes facial images and voiceprints to recognize User A's emotional state, which is then added to and saved in the profile.

[0870] Now, if User A tries to log into the system again, he will see the following prompt:

[0871] "Please face the camera again to log in. Next, speak your name into the microphone. Finally, place your finger on the fingerprint scanner."

[0872] When User A provides their biometric information, the server generates a new virtual profile and compares it with the previously saved profile. The emotion engine also analyzes the current emotional state and compares it with the previously saved emotional state to provide a more reliable authentication.

[0873] This provides highly accurate authentication based on biometric information as well as security services that take into account the user's emotional state.

[0874] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0875] Step 1:

[0876] A user accesses a security application on their smartphone and provides a facial image, voiceprint, and fingerprint.

[0877] Input: The user faces the camera, introduces themselves into the microphone, and places their finger on the fingerprint scanner.

[0878] Output: Biometric information (face image, voiceprint, fingerprint).

[0879] Specific actions: The user follows the instructions, makes a natural facial expression towards the camera, speaks a few words, and places a finger on the scanner.

[0880] Step 2:

[0881] The device captures this biometric information using a camera, microphone, and fingerprint scanner and converts it into the appropriate digital format (JPEG, WAV, binary data).

[0882] Input: Biometric information (face image, voiceprint, fingerprint).

[0883] Output: Biometric information in digital format (JPEG, WAV, binary data).

[0884] Specific operations: A facial image is taken with a camera and converted to JPEG format, voice is recorded with a microphone and converted to WAV format, and a fingerprint is read with a fingerprint scanner and converted to binary data.

[0885] Step 3:

[0886] The device transmits the biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0887] Input: Biometric data in digital format (JPEG, WAV, binary data).

[0888] Output: Success / failure of sending to server.

[0889] Specific operation: Facial image, voiceprint, and fingerprint data are sent to the server using the HTTPS protocol.

[0890] Step 4:

[0891] The server receives the transmitted biometric information and performs preprocessing such as normalizing facial images, removing noise from audio data, and enhancing fingerprint data.

[0892] Input: Biometric data in digital format (JPEG, WAV, binary data).

[0893] Output: Preprocessed biometric information in digital form.

[0894] Specific operations: Adjusts and normalizes the brightness of facial images, clears audio data with a noise reduction filter, and performs enhancement processing to clarify fingerprint data.

[0895] Step 5:

[0896] The server inputs the preprocessed biometric information into a generative AI model to generate a virtual profile.

[0897] Input: Preprocessed digital biometric information.

[0898] Output: The generated virtual profile.

[0899] How it works: Machine learning algorithms are used to analyze pre-processed biometric information and generate a unique virtual profile.

[0900] Step 6:

[0901] The server associates the generated virtual profile with the user ID and stores it in the database.

[0902] Input: Generated virtual profile and user ID.

[0903] Output: Save to database successful / failed.

[0904] Specific operation: Record the virtual profile along with the user ID in the database and check whether it was saved successfully.

[0905] Step 7:

[0906] The server uses an emotion engine to analyze the user's emotional state by analyzing facial expressions in facial images and tone in voiceprints.

[0907] Input: Preprocessed face image and voiceprint.

[0908] Output: Parsed emotional state.

[0909] Specific operation: Executes the emotion engine, estimates emotions from facial images and voiceprints, and obtains analysis results.

[0910] Step 8:

[0911] The server adds the analyzed emotional state to the virtual profile and further stores it in a database.

[0912] Input: Generated virtual profile and analyzed emotional state.

[0913] Output: Success / failure in adding to database.

[0914] Specific operation: The emotional state is incorporated into the virtual profile, saved to the database, and the success of the save is confirmed.

[0915] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0916] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0917] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0918] [Third embodiment]

[0919] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0920] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0921] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0922] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0923] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0924] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0925] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0926] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0927] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0928] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0929] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0930] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0931] The system according to the present invention acquires biometric information, generates a virtual profile based on a generative model, and performs a series of processes to authenticate a user based on the profile. The system also comprises a server, a terminal, and a user.

[0932] Explanation of program processing

[0933] User registration process

[0934] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the system's registration screen. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0935] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[0936] 3. The device sends the collected biometric information to the server using a secure communication protocol (e.g., HTTPS).

[0937] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0938] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0939] 6. The server stores the generated virtual profile in a database together with the corresponding user ID.

[0940] Authentication process

[0941] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the login screen. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0942] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[0943] 3. The device sends the collected biometric information to the server using a secure communication protocol.

[0944] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0945] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[0946] 6. The server matches the generated virtual profile with existing virtual profiles stored in a database.

[0947] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, the server sends a message of failed authentication and asks the user to try again.

[0948] Specific examples

[0949] For example, when User A attempts to register for the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores that profile in a database. Next, when User A accesses the system again and attempts to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[0950] The above is an embodiment of the system of the present invention, which can eliminate the hassle of passwords and provide a high level of security.

[0951] The processing flow will be explained below.

[0952] User registration process

[0953] Step 1:

[0954] The user presses the "Register" button on the registration screen and is then asked to enter biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0955] Step 2:

[0956] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[0957] Step 3:

[0958] The biometric information acquired by the terminal is transmitted to a server using a secure communication protocol (for example, HTTPS).

[0959] Step 4:

[0960] The server receives the biometric information sent from the device and performs preprocessing on the received data, such as normalization to adjust the brightness of facial images, filtering to remove noise from voiceprint data, and enhancement to clarify fingerprint data.

[0961] Step 5:

[0962] The server inputs the pre-processed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[0963] Step 6:

[0964] The server stores the generated virtual profile in a database together with the corresponding user ID.

[0965] Authentication process

[0966] Step 1:

[0967] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[0968] Step 2:

[0969] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the respective digital formats.

[0970] Step 3:

[0971] The terminal transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[0972] Step 4:

[0973] The server receives the biometric information sent from the device and performs preprocessing on the received data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[0974] Step 5:

[0975] The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[0976] Step 6:

[0977] The server matches the generated virtual profile with existing virtual profiles stored in a database.

[0978] Step 7:

[0979] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system; if not, it sends a message of failed authentication and asks the user to try again.

[0980] The above are the specific steps in the user registration and authentication processes, which greatly improve the security and convenience of the system.

[0981] Example 1

[0982] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0983] Conventional biometric authentication systems face challenges in efficiently and securely processing a user's biometric information for user authentication. Furthermore, when using a wide variety of biometric information, preprocessing each piece of information and generating a virtual profile using a generative model can be complex, potentially reducing the security and accuracy of the authentication process.

[0984] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0985] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model to generate a virtual profile, means for storing the generated virtual profile in association with a user identifier, means for transmitting the acquired biometric information to the server using a secure communication protocol, means for preprocessing the received biometric information by the server, and means for verifying the generated virtual profile by the server. This enables efficient and secure processing of a wide variety of user biometric information, enabling highly accurate user authentication.

[0986] "Biometric information" is digital data that characterizes a part of a user's body, such as a facial image, voiceprint, or fingerprint.

[0987] A "generative model" is a model that uses machine learning algorithms to generate unique virtual profiles from input data.

[0988] A "virtual profile" is a digital profile created using a generative model that uniquely represents a user's biometric information.

[0989] A "user identifier" is an identification number or ID that is uniquely assigned to each user.

[0990] A "secure communication protocol" is a safe protocol used to protect data communication channels, and specifically includes HTTPS.

[0991] "Preprocessing" refers to a series of data processing operations including noise removal, filtering, and normalization of biological information.

[0992] "Matching" is the process of comparing a newly generated virtual profile with an existing virtual profile to determine the degree of match.

[0993] A "server" is a computer system that collects biometric information, pre-processes it, applies generative models, and generates and matches virtual profiles.

[0994] The system according to the present invention comprises a user, a terminal, and a server. A specific embodiment of the system is described below.

[0995] The system of the present invention acquires a user's biometric information and performs a series of processes to authenticate the user based on that information. The hardware required includes sensors such as a camera, microphone, and fingerprint scanner. The software used includes machine learning frameworks such as TensorFlow and PyTorch, and the HTTPS protocol for secure communication.

[0996] First, the user provides biometric information such as a facial image, voiceprint, and fingerprint on the system's registration screen. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. Through these operations, the user's biometric information is captured as digital data.

[0997] The device then collects this biometric information and converts the data from each sensor into a digital format—for example, converting camera footage into image data, microphone recordings into audio data, and fingerprint scans into fingerprint data—and transmits the collected data to a server using a secure communication protocol.

[0998] The server performs preprocessing on the received biometric data, including normalizing the facial image (correcting for lighting conditions and adjusting the size), filtering the voiceprint (removing noise), and enhancing the fingerprint data (emphasizing details). The preprocessed biometric data is then input into a generative model.

[0999] The generative model uses machine learning algorithms such as TensorFlow or PyTorch to generate a unique virtual profile from the preprocessed data, which is then stored in a database along with a user identifier.

[1000] During authentication, the user again provides biometric information such as a facial image, voiceprint, and fingerprint. The device collects this information, converts it into digital data, and sends it to the server. The server then preprocesses it again and generates a new virtual profile using a generative model. This new virtual profile is then compared with existing virtual profiles stored in the database. If the results match, the user is authenticated and can access the system. If the results do not match, the user receives a message indicating authentication has failed and is asked to try again.

[1001] Specific examples

[1002] For example, if User A wants to register with the system, he or she provides a face image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information and inputs it into a generative model (a TensorFlow-based machine learning model) to generate a virtual profile, which is then stored in a database.

[1003] Next, when User A accesses the system again and tries to log in, he / she provides his / her biometric information again. The terminal collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[1004] Prompt Sentence Examples

[1005] An example prompt to create a virtual profile for User B in the system is:

[1006] Please provide User B's face image, voiceprint, and fingerprint. Follow these steps to collect biometric information:

[1007] 1. Face the camera.

[1008] 2. Speak into the microphone.

[1009] 3. Place your finger on the fingerprint scanner.

[1010] Once you have collected this information, submit it to the system.

[1011] The above is an embodiment of the present invention. This system can eliminate the hassle of passwords and provide high security.

[1012] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1013] Step 1: Provide biometric information

[1014] Users provide biometric information such as a facial image, voiceprint, or fingerprint on the system registration or login screen. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. This inputs biometric information as analog data.

[1015] Step 2: Collecting and digitalizing biometric information

[1016] The device collects biometric information provided by the user from each sensor (camera, microphone, fingerprint scanner) and converts it into digital form. For example, camera footage is converted into image data, microphone recordings into voice data, and fingerprint scans into fingerprint data. Here, the input is analog data and the output is digital data.

[1017] Step 3: Sending biometric information

[1018] The device transmits the collected digital biometric information to the server using a secure communication protocol (e.g., HTTPS). Specifically, the device encrypts the data and transmits it over a secure channel. The input is digital data, and the output is the encrypted data transmitted to the server.

[1019] Step 4: Preprocessing of biological information

[1020] The server receives the digital data sent from the device and performs preprocessing on each piece of information, such as normalizing facial images (correcting for lighting conditions and adjusting size), filtering voiceprints (removing noise), and enhancing fingerprint data (emphasizing details).The input here is the received biometric information, and the output is the preprocessed data.

[1021] Step 5: Generate a Virtual Profile

[1022] The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile. The generative model uses a machine learning algorithm (e.g., TensorFlow or PyTorch). Specifically, the server inputs the preprocessed data into the generative model, and the generative model outputs a virtual profile. The input is the preprocessed data, and the output is the virtual profile.

[1023] Step 6: Save the Virtual Profile

[1024] The server stores the generated virtual profile together with the corresponding user identifier in a database. Specifically, the server executes an SQL query to insert the virtual profile and the user identifier into the database. The input is the virtual profile and the user identifier, and the output is the result stored in the database.

[1025] Step 7: Provide biometric information when logging in

[1026] The user then provides biometric information such as a facial image, voiceprint, and fingerprint on the login screen again. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. This re-enters the biometric information as analog data.

[1027] Step 8: Collecting and digitally converting biometric information during login

[1028] The device collects biometric information provided by the user from each sensor (camera, microphone, fingerprint scanner), converts it into digital form, and outputs the collected data as digital data again.

[1029] Step 9: Sending biometric information when logging in

[1030] The device sends the collected digital biometric information to a server using a secure communication protocol (HTTPS). The input is digital data, and the output is encrypted data sent to the server.

[1031] Step 10: Preprocessing biometric information at login

[1032] The server receives the digital data sent from the terminal and pre-processes each piece of information again. The input is the received biometric information, and the output is the pre-processed data.

[1033] Step 11: Create a new virtual profile

[1034] The server inputs the preprocessed biometric information into a generative model to generate a new virtual profile. The input is the preprocessed data, and the output is the new virtual profile.

[1035] Step 12: Profile Matching

[1036] The server matches the newly generated virtual profile with existing virtual profiles stored in a database. Specifically, the server executes a query to retrieve existing profiles and uses a comparison algorithm to check for a match. The inputs are the new virtual profile and the existing virtual profile, and the output is the match result.

[1037] Step 13: Sending authentication results

[1038] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If the profile does not match, the server sends a message of failed authentication and asks the user to try again. The input is the matching result, and the output is the authentication message.

[1039] (Application example 1)

[1040] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1041] Conventional biometric authentication systems aim to improve security by using biometric information such as facial images, fingerprints, and voiceprints instead of passwords. However, issues remain, such as the difficulty of securing and managing biometric information, as well as the lack of preprocessing, normalization, and secure communication methods for biometric information. For these reasons, a system that can realize a safe and efficient authentication process is needed.

[1042] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1043] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model and generating a virtual profile, means for storing the generated virtual profile in association with a user ID, means for preprocessing or normalizing the registered biometric information, means for securely transmitting the preprocessed or normalized biometric information to the server, means for securely communicating biometric data from the local device to the server, and means for storing the generated virtual profile in a database and performing re-matching. This increases the security of biometric information and enables an efficient authentication process.

[1044] "Biometric information" is digital data of biological characteristics, such as facial images, fingerprints, and voiceprints, that are used to identify individuals.

[1045] A "generative model" is a model that uses a machine learning algorithm to generate a virtual profile based on biometric information.

[1046] A "virtual profile" is a unique digital profile data created by a generative model and linked to a user ID.

[1047] "Preprocessing" is the process of converting biometric information into a format suitable for the generative model by processing it through normalization, filtering, enhancement, and other processes.

[1048] "Normalization" is the process of converting biometric information into a standard format or scale to ensure data consistency.

[1049] "Secure communications" means encrypted communications protocols, such as HTTPS, used to protect the confidentiality and integrity of information.

[1050] "Database" refers to an integrated management system for storing and matching generated virtual profiles and corresponding user IDs.

[1051] "Matching" is the process of comparing the newly generated virtual profile with existing profiles to confirm a match.

[1052] A "local device" is a terminal such as a smartphone or tablet to which an individual user provides biometric information.

[1053] The system according to the present invention acquires biometric information, generates a virtual profile using a generative model, and performs a series of processes to authenticate a user based on the profile. The system is composed of a user, a terminal, and a server.

[1054] Explanation of program processing

[1055] User registration process

[1056] 1. The user provides biometric information such as a facial image, voiceprint, and fingerprint on the system's registration screen. For example, the user faces the smartphone camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1057] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[1058] 3. The device sends the collected biometric information to the server using a secure communication protocol (e.g., HTTPS).

[1059] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information (e.g., normalizing facial images, filtering voiceprints, enhancing fingerprint data).

[1060] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[1061] 6. The server stores the generated virtual profile in a database together with the corresponding user ID.

[1062] Authentication process

[1063] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the login screen. For example, the user faces the smartphone camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1064] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[1065] 3. The device sends the collected biometric information to the server using a secure communication protocol.

[1066] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information (e.g., normalizing facial images, filtering voiceprints, enhancing fingerprint data).

[1067] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[1068] 6. The server matches the generated virtual profile with existing virtual profiles stored in a database.

[1069] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, the server sends a message of failed authentication and asks the user to try again.

[1070] Specific examples

[1071] For example, when User A attempts to register for the system, he or she provides a facial image, voiceprint, and fingerprint. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores that profile in a database. Next, when User A accesses the system again and attempts to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[1072] Prompt Sentence Examples

[1073] For example, a sentence such as, "Generate a virtual profile using user A's facial image, voiceprint, and fingerprints. Then, compare it with existing virtual profiles stored in the database to determine whether there is a match" is input into the generative AI model.

[1074] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1075] Step 1:

[1076] The user provides biometric information such as a face image, voiceprint, and fingerprint on the system's registration screen. Here, the user's face image, voiceprint, and fingerprint are used as input. Specifically, the user faces the smartphone's camera, speaks into the microphone, and places their finger on the fingerprint scanner. The output is the state in which this biometric information has been collected.

[1077] Step 2:

[1078] The device collects biometric information provided by the user and obtains digital data from each sensor (camera, microphone, fingerprint scanner). This data is then converted into digital format. The input is the biometric information provided by the user, and the output is digital facial image data, voiceprint data, and fingerprint data. Specifically, each sensor captures the biometric information and stores it as digital data inside the device.

[1079] Step 3:

[1080] The device transmits the collected digital biometric information to a server using a secure communication protocol (e.g., HTTPS). The input is the digital biometric data, and the output is the data transmitted to the server. Specifically, the device encrypts the data and transmits it to the server via a secure channel.

[1081] Step 4:

[1082] The server receives the biometric information sent from the device and performs preprocessing on each piece of information. The input is the biometric data sent from the device, and the output is the preprocessed data. Specific operations include normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[1083] Step 5:

[1084] The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile. The input is the preprocessed biometric data, and the output is the virtual profile. The generative model uses a machine learning algorithm. The specific operation is to apply the preprocessed data to the generative model to generate the virtual profile.

[1085] Step 6:

[1086] The server stores the generated virtual profile together with the user ID in a database. The input is the virtual profile and the user ID, and the output is the state saved in the database. The specific operation is to uniquely associate the virtual profile with the user ID and record it in the database.

[1087] Step 7:

[1088] The user provides their biometric information again on the login screen. The input is a new face image, voiceprint, and fingerprint, and the output is the newly collected biometric information. Specifically, the smartphone's camera, microphone, and fingerprint scanner are used again to acquire the new information.

[1089] Step 8:

[1090] The terminal converts the newly collected biometric information back into digital data and transmits it to the server using a secure communication protocol. The input is the new digital biometric information data, and the output is the data transmitted to the server again. The specific operations are the same as steps 2 and 3.

[1091] Step 9:

[1092] The server preprocesses the new biometric data sent from the device and inputs it into the generative model to generate a new virtual profile. The input is the newly preprocessed biometric data, and the output is a new virtual profile. The specific operations are the same as steps 4 and 5.

[1093] Step 10:

[1094] The server matches the newly generated virtual profile with the existing virtual profiles stored in the database. The input is the data of the new virtual profile and the existing virtual profile, and the output is the matching result. The specific operation is to retrieve the existing profile from the database and compare it with the new profile.

[1095] Step 11:

[1096] If the profile matches, the server sends an authentication success message to the terminal and the user can access the system. If it does not match, it sends an authentication failure message and asks the user to try again. The input is the matching result, and the output is a message indicating authentication success or failure. The specific operation is to generate a success or failure message and send it to the terminal.

[1097] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1098] This invention combines a system that acquires biometric information, generates a virtual profile based on a generative model, and authenticates users based on that profile with an emotion engine that recognizes the user's emotions. The system is composed of a server, a terminal, and a user, and by further integrating the emotion engine, authentication can be performed taking into account the user's emotional state.

[1099] Explanation of program processing

[1100] User registration process

[1101] 1. The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1102] 2. The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, converting the biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[1103] 3. The device transmits the acquired biometric information to the server using a secure communication protocol (e.g., HTTPS).

[1104] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalization to adjust the brightness of facial images, filtering to remove noise from voiceprint data, and enhancement to make fingerprint data clearer.

[1105] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[1106] 6. The server stores the generated virtual profile in a database along with the corresponding user ID.

[1107] 7. The server simultaneously uses an emotion engine to recognize the user's emotional state through facial expression analysis of facial images or tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[1108] Authentication process

[1109] 1. The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1110] 2. The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, converting the captured biometric information into the respective digital formats.

[1111] 3. The device transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[1112] 4. The server receives the biometric data sent from the device and performs preprocessing on each piece of data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[1113] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[1114] 6. The server matches the generated virtual profile with existing virtual profiles stored in the database.

[1115] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, it sends a message of failed authentication and asks the user to try again.

[1116] 8. The server also uses the emotion engine to analyze the user's current emotional state and compare it with the existing emotional state. If the emotional state matches or is within a trusted range, it will be reflected in the authentication result.

[1117] Specific examples

[1118] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then saved in a database. At this time, the emotion engine recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and saves this in the profile. Next, when User A attempts to log in to the system again, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, authenticates User A. The emotion engine also analyzes User A's current emotional state and compares it with the existing emotional state. If they match, authentication becomes more reliable.

[1119] The above is an embodiment of a system incorporating an emotion engine according to the present invention. This system can eliminate the hassle of passwords and provide a higher level of security and user experience.

[1120] The processing flow will be explained below.

[1121] User registration process

[1122] Step 1:

[1123] The user presses the "Register" button on the registration screen and is then prompted to enter biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1124] Step 2:

[1125] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[1126] Step 3:

[1127] The biometric information acquired by the terminal is transmitted to a server using a secure communication protocol (for example, HTTPS).

[1128] Step 4:

[1129] The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalization to adjust the brightness and contrast of facial images, filtering to remove noise from voiceprint data, and enhancement to make fingerprint data clearer.

[1130] Step 5:

[1131] The server inputs the pre-processed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[1132] Step 6:

[1133] The server stores the generated virtual profile in a database together with the corresponding user ID.

[1134] Step 7:

[1135] The server uses an emotion engine to recognize the user's emotional state by analyzing facial expressions in facial images or tone in voiceprints, for example, by analyzing subtle facial movements and voice tone, pitch, and rhythm.

[1136] Step 8:

[1137] The server stores the recognized emotional state together with the virtual profile in a database.

[1138] Authentication process

[1139] Step 1:

[1140] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1141] Step 2:

[1142] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the respective digital formats.

[1143] Step 3:

[1144] The terminal transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[1145] Step 4:

[1146] The server receives the biometric data sent from the device and performs preprocessing on each piece of data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[1147] Step 5:

[1148] The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[1149] Step 6:

[1150] The server matches the generated virtual profile with existing virtual profiles stored in a database.

[1151] Step 7:

[1152] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system; if not, it sends a message of failed authentication and asks the user to try again.

[1153] Step 8:

[1154] The server uses an emotion engine to analyze the user's current emotional state, again by analyzing, for example, facial expressions and tone of voice.

[1155] Step 9:

[1156] The server compares the current emotional state with the existing emotional state, and if they match or are within a trustworthy range, reflects the result in the authentication result. For example, if the emotional state matches and the virtual profile matches, authentication is completed.

[1157] Specific examples

[1158] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then saved in a database. At this time, the emotion engine recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and saves this in the profile. Next, when User A attempts to log in to the system again, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated. The emotion engine is also used to analyze User A's current emotional state and compares it with the existing emotional state. If they match or are within the confidence range, more reliable authentication is achieved.

[1159] The above is an embodiment of a system incorporating an emotion engine according to the present invention. This system eliminates the hassle of passwords and provides a higher level of security and user experience.

[1160] Example 2

[1161] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1162] Conventional biometric authentication systems authenticate users based on their biometric information, but they have limitations in authentication accuracy and security. Furthermore, because they do not take into account the user's psychological state or emotions, it is difficult to prevent unauthorized access and the user experience is unsatisfactory. Furthermore, there is a risk of attacks on authentication data, making it necessary to strengthen security.

[1163] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1164] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model to generate a virtual profile, means for storing the generated virtual profile in association with a user ID, means for preprocessing the biometric information, means for authenticating the user based on the virtual profile, means for analyzing the user's emotional state using an emotion engine, means for adding the analyzed emotional state to the virtual profile and storing it, and means for reacquiring the biometric information and emotional state for authentication. This enables highly accurate authentication that takes into account the user's emotional state as well as the biometric information, thereby strengthening the security of the entire authentication process and improving the user experience.

[1165] "Biometric information" is data that digitally captures a user's physical characteristics, including facial images, voiceprints, fingerprints, and the like.

[1166] A "generative model" is a computational tool that uses machine learning algorithms to generate new information from input data.

[1167] A "virtual profile" is digital information that uniquely identifies a user and is generated using machine learning algorithms based on biometric information.

[1168] A "user ID" is an identifier that uniquely identifies a user and is linked to a virtual profile.

[1169] "Preprocessing" refers to data processing means for making acquired biometric information easier to analyze, and includes brightness adjustment, noise removal, enhancement, and the like.

[1170] The "emotion engine" is a software component for analyzing a user's emotional state from facial images and voiceprints.

[1171] "Emotional state" is data that indicates the emotional state that the user is currently experiencing, and is generated as a result of analysis.

[1172] "Authentication" is the process of verifying that a user is legitimately accessing something based on the biometric information and emotional state they provide.

[1173] This invention provides a system that acquires a user's biometric information and emotional state, and generates and authenticates a virtual profile based on a generative model. This system consists of a server, a terminal, and a user, and by integrating an emotion engine, it enables advanced authentication that takes into account the user's emotional state.

[1174] Overall system configuration

[1175] The main components of the system are:

[1176] Server: Responsible for preprocessing biometric information, running generative models, generating and storing virtual profiles, authentication, and analyzing emotional states.

[1177] Terminal: Acquires biometric information from the user, converts it into digital format, and sends it to the server.

[1178] User: Provides biometric information such as facial image, voiceprint, and fingerprint.

[1179] Emotion engine: A software component that analyzes the user's emotional state from biometric information.

[1180] Hardware and software used

[1181] Hardware:

[1182] 1. Camera: Captures a face image.

[1183] 2. Microphone: Records your voiceprint.

[1184] 3. Fingerprint scanner: Reads your fingerprint.

[1185] software:

[1186] 1. Secure communication protocols (e.g. HTTPS)

[1187] 2. Generative Models Using Machine Learning Algorithms

[1188] 3. Environment-dependent image processing algorithms (brightness adjustment, noise reduction, enhancement)

[1189] 4. Emotion analysis software (facial expression analysis and voice tone analysis)

[1190] Program processing explanation

[1191] User registration process

[1192] 1. The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1193] 2. The device converts the acquired biometric information into JPEG format (face image), WAV format (voiceprint), and binary format (fingerprint).

[1194] 3. The device sends the acquired biometric information to the server using HTTPS communication.

[1195] 4. The server adjusts the brightness of the received facial image, removes noise from the voiceprint data, and performs enhancement processing on the fingerprint data.

[1196] 5. The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile.

[1197] 6. The server associates the generated virtual profile with the user ID and saves it in the database.

[1198] 7. The server uses an emotion engine to recognize the user's emotional state from facial expression analysis of facial images and tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[1199] Authentication process

[1200] 1. The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, and fingerprint, again using the camera, microphone, and fingerprint scanner.

[1201] 2. The device converts the acquired biometric information into the appropriate digital format.

[1202] 3. The device sends the acquired biometric information to the server using HTTPS communication.

[1203] 4. The server normalizes the received facial images, filters the voiceprint data, and enhances the fingerprint data.

[1204] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[1205] 6. The server matches the new virtual profile with an existing virtual profile.

[1206] 7. If there is a match, the server sends an authentication success message to the terminal and the user can access the system. If there is no match, it sends an authentication failure message and asks the user to try again.

[1207] 8. The server uses the emotion engine to analyze the current emotional state and compare it with the existing emotional states. If it matches or is within the trusted range, it is reflected in the authentication result.

[1208] Specific examples

[1209] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server then preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then stored in a database. At this time, the emotion engine also recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and stores this in the profile.

[1210] Next, when User A accesses the system again and tries to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server then compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated. The emotion engine also analyzes the current emotional state and compares it with the existing emotional state. If they match, more reliable authentication is possible.

[1211] Prompt Sentence Examples

[1212] Example of a prompt during registration:

[1213] Describe the process of taking a user's facial image, voiceprint, and fingerprint as input, generating a virtual profile based on a generative model, and storing it in a database along with biometric information and emotional state.

[1214] An example of a login prompt:

[1215] Describe the authentication process by acquiring a user's facial image, voiceprint, and fingerprints, matching a newly generated virtual profile with an existing profile, and analyzing and comparing their emotional state.

[1216] The above is an example of a biometric authentication system that integrates an emotion engine according to the present invention, which eliminates the hassle of passwords and provides a higher level of security and user experience.

[1217] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1218] User registration process

[1219] Step 1:

[1220] The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. The user then faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1221] Input: User's face image, voiceprint, fingerprint

[1222] Output: Biometric information acquired by the device

[1223] Step 2:

[1224] The device uses a camera to capture a facial image in JPEG format, a microphone to record a voiceprint in WAV format, and a fingerprint scanner to read a fingerprint in binary format.

[1225] Input: Acquired biometric information

[1226] Output: JPEG facial image, WAV voiceprint, binary fingerprint

[1227] Step 3:

[1228] The device sends the acquired biometric information to the server using HTTPS.

[1229] Input: JPEG facial image, WAV voice print, binary fingerprint

[1230] Output: Biometric information sent to the server

[1231] Step 4:

[1232] The server normalizes the brightness of the received facial images, removes noise from voiceprints, and performs enhancement processing on fingerprint data.

[1233] Input: Biometric information sent to the server

[1234] Output: Preprocessed facial images, voiceprints, and fingerprint data

[1235] Step 5:

[1236] The server inputs the pre-processed biometric information into a generative model to generate a unique virtual profile.

[1237] Input: Preprocessed face images, voiceprints, and fingerprint data

[1238] Output: Generated virtual profile

[1239] Step 6:

[1240] The server associates the generated virtual profile with the user ID and stores it in the database.

[1241] Input: Generated virtual profile, user ID

[1242] Output: Virtual profile and user ID stored in the database

[1243] Step 7:

[1244] The server uses an emotion engine to recognize the user's emotional state from facial expression analysis of facial images and tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[1245] Input: Preprocessed face images and voiceprint data

[1246] Output: Recognized emotional state, updated virtual profile

[1247] Authentication process

[1248] Step 1:

[1249] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, and fingerprint. The user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1250] Input: User's face image, voiceprint, fingerprint

[1251] Output: Biometric information acquired by the device

[1252] Step 2:

[1253] The device uses a camera to capture a facial image in JPEG format, a microphone to record a voiceprint in WAV format, and a fingerprint scanner to read a fingerprint in binary format.

[1254] Input: Acquired biometric information

[1255] Output: JPEG facial image, WAV voiceprint, binary fingerprint

[1256] Step 3:

[1257] The device sends the acquired biometric information to the server using HTTPS.

[1258] Input: JPEG facial image, WAV voice print, binary fingerprint

[1259] Output: Biometric information sent to the server

[1260] Step 4:

[1261] The server normalizes the received facial images, filters voiceprints, and enhances fingerprint data.

[1262] Input: Biometric information sent to the server

[1263] Output: Preprocessed facial images, voiceprints, and fingerprint data

[1264] Step 5:

[1265] The server inputs the preprocessed biometric information into a generative model to generate a new virtual profile.

[1266] Input: Preprocessed face images, voiceprints, and fingerprint data

[1267] Output: The newly generated virtual profile

[1268] Step 6:

[1269] The server matches the newly created virtual profile with existing virtual profiles.

[1270] Input: Newly created virtual profile, existing virtual profile

[1271] Output: Matching result

[1272] Step 7:

[1273] If they match, the server sends an authentication success message to the terminal, allowing the user to access the system. If they do not match, the server sends an authentication failure message, requesting a retry.

[1274] Input: Matching result

[1275] Output: Authentication result message

[1276] Step 8:

[1277] The server uses the emotion engine to analyze the user's current emotional state and compare it with existing emotional states. If it matches or is within a trusted range, it is reflected in the authentication result.

[1278] Input: Preprocessed face images, voiceprint data, and existing emotional states

[1279] Output: Analyzed current emotional state, final authentication result

[1280] (Application example 2)

[1281] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1282] Conventional biometric authentication systems authenticate users by capturing biometric information such as facial images, voiceprints, and fingerprints, and storing it in association with a user ID. However, this alone has limitations in the accuracy of authentication and the strength of security. In particular, there is a problem in that the risk of unauthorized access increases if someone misuses the biometric information of a genuine user. Furthermore, because the system does not take into account the user's status (for example, when they are forced to log in), the user experience may be impaired.

[1283] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1284] In this invention, the server includes a means for acquiring biometric information, a means for inputting the acquired biometric information into a generative model to generate a virtual profile, a means for storing the generated virtual profile in association with a user ID, an emotion engine for analyzing the user's emotional state, and a means for adding the user's emotional state to the virtual profile using the emotion engine. This enables highly accurate authentication based on biometric information as well as security services that take the user's emotional state into consideration.

[1285] "Biometric information" refers to data containing biological characteristics that can uniquely identify an individual, such as facial images, voiceprints, and fingerprints.

[1286] "Generative model" refers to a model that uses machine learning algorithms to generate a virtual profile based on biometric information.

[1287] A "virtual profile" is a digital user profile generated using a generative model based on biometric information and used to identify an individual user.

[1288] "User ID" means a unique identifier used to identify a particular User.

[1289] "Emotion engine" refers to software or algorithms for analyzing a user's emotional state from their biometric information (e.g., facial image, voiceprint, etc.).

[1290] "Server" refers to a computer system for collecting, processing, and storing data.

[1291] This invention provides a security application installed on smartphones. The system acquires a user's biometric information, such as facial images, voiceprints, and fingerprints, and generates a virtual profile using a generative AI model. Furthermore, it analyzes the user's emotional state using an emotion engine and adds it to the virtual profile, achieving highly accurate and secure authentication.

[1292] Hardware and Software Configuration

[1293] Hardware

[1294] Camera: A device for capturing facial images.

[1295] Microphone: A device used to capture voiceprints.

[1296] Fingerprint scanner: A device for capturing fingerprint data.

[1297] software

[1298] OpenCV: A library for acquiring and preprocessing face images.

[1299] EmotionEngine: A module that analyzes emotional states from facial images.

[1300] VoiceRecognizer: A module that acquires and preprocesses voice data.

[1301] FingerprintScanner: A module that acquires and preprocesses fingerprint data.

[1302] Requests: An HTTP request library for communicating with the server.

[1303] Generative model: A model for generating virtual profiles from biometric information using machine learning algorithms.

[1304] System configuration

[1305] 1. A user accesses a security application on their smartphone and provides a facial image, voiceprint, and fingerprint.

[1306] 2. The device captures this biometric information using the camera, microphone, and fingerprint scanner and converts it into the appropriate digital format (JPEG, WAV, binary data).

[1307] 3. The device sends the biometric information to the server using a secure communication protocol (e.g., HTTPS).

[1308] 4. The server receives the transmitted biometric information and performs preprocessing such as normalizing the facial image, removing noise from the audio data, and enhancing the fingerprint data.

[1309] 5. The server inputs the preprocessed biometric information into the generative model to generate a virtual profile.

[1310] 6. The server associates the generated virtual profile with the user ID and stores it in the database.

[1311] 7. The server uses an emotion engine to analyze the facial expression of the facial image and the tone of the voiceprint to analyze the user's emotional state.

[1312] 8. The server adds the analyzed emotional state to the virtual profile and further stores it in a database.

[1313] Specific examples

[1314] When User A wants to register with the system, he or she will provide a facial image, voiceprint, and fingerprints as prompted by the smartphone application. For example, the following prompts will be displayed:

[1315] "Please look at the camera with a natural expression. Next, introduce yourself into the microphone. Finally, place your finger on the fingerprint scanner."

[1316] When a user provides biometric information, the device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores the profile in a database. At the same time, the emotion engine analyzes facial images and voiceprints to recognize User A's emotional state, which is then added to and saved in the profile.

[1317] Now, if User A tries to log into the system again, he will see the following prompt:

[1318] "Please face the camera again to log in. Next, speak your name into the microphone. Finally, place your finger on the fingerprint scanner."

[1319] When User A provides their biometric information, the server generates a new virtual profile and compares it with the previously saved profile. The emotion engine also analyzes the current emotional state and compares it with the previously saved emotional state to provide a more reliable authentication.

[1320] This provides highly accurate authentication based on biometric information as well as security services that take into account the user's emotional state.

[1321] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1322] Step 1:

[1323] A user accesses a security application on their smartphone and provides a facial image, voiceprint, and fingerprint.

[1324] Input: The user faces the camera, introduces themselves into the microphone, and places their finger on the fingerprint scanner.

[1325] Output: Biometric information (face image, voiceprint, fingerprint).

[1326] Specific actions: The user follows the instructions, makes a natural facial expression towards the camera, speaks a few words, and places a finger on the scanner.

[1327] Step 2:

[1328] The device captures this biometric information using a camera, microphone, and fingerprint scanner and converts it into the appropriate digital format (JPEG, WAV, binary data).

[1329] Input: Biometric information (face image, voiceprint, fingerprint).

[1330] Output: Biometric information in digital format (JPEG, WAV, binary data).

[1331] Specific operations: A facial image is taken with a camera and converted to JPEG format, voice is recorded with a microphone and converted to WAV format, and a fingerprint is read with a fingerprint scanner and converted to binary data.

[1332] Step 3:

[1333] The device transmits the biometric information to the server using a secure communication protocol (e.g., HTTPS).

[1334] Input: Biometric data in digital format (JPEG, WAV, binary data).

[1335] Output: Success / failure of sending to server.

[1336] Specific operation: Facial image, voiceprint, and fingerprint data are sent to the server using the HTTPS protocol.

[1337] Step 4:

[1338] The server receives the transmitted biometric information and performs preprocessing such as normalizing facial images, removing noise from audio data, and enhancing fingerprint data.

[1339] Input: Biometric data in digital format (JPEG, WAV, binary data).

[1340] Output: Preprocessed biometric information in digital form.

[1341] Specific operations: Adjusts and normalizes the brightness of facial images, clears audio data with a noise reduction filter, and performs enhancement processing to clarify fingerprint data.

[1342] Step 5:

[1343] The server inputs the preprocessed biometric information into a generative AI model to generate a virtual profile.

[1344] Input: Preprocessed digital biometric information.

[1345] Output: The generated virtual profile.

[1346] How it works: Machine learning algorithms are used to analyze pre-processed biometric information and generate a unique virtual profile.

[1347] Step 6:

[1348] The server associates the generated virtual profile with the user ID and stores it in the database.

[1349] Input: Generated virtual profile and user ID.

[1350] Output: Save to database successful / failed.

[1351] Specific operation: Record the virtual profile along with the user ID in the database and check whether it was saved successfully.

[1352] Step 7:

[1353] The server uses an emotion engine to analyze the user's emotional state by analyzing facial expressions in facial images and tone in voiceprints.

[1354] Input: Preprocessed face image and voiceprint.

[1355] Output: Parsed emotional state.

[1356] Specific operation: Executes the emotion engine, estimates emotions from facial images and voiceprints, and obtains analysis results.

[1357] Step 8:

[1358] The server adds the analyzed emotional state to the virtual profile and further stores it in a database.

[1359] Input: Generated virtual profile and analyzed emotional state.

[1360] Output: Success / failure in adding to database.

[1361] Specific operation: The emotional state is incorporated into the virtual profile, saved to the database, and the success of the save is confirmed.

[1362] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1363] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1364] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1365] [Fourth embodiment]

[1366] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1367] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1368] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1369] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1370] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1371] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1372] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1373] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1374] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1375] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1376] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1377] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1378] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1379] The system according to the present invention acquires biometric information, generates a virtual profile based on a generative model, and performs a series of processes to authenticate a user based on the profile. The system also comprises a server, a terminal, and a user.

[1380] Explanation of program processing

[1381] User registration process

[1382] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the system's registration screen. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1383] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[1384] 3. The device sends the collected biometric information to the server using a secure communication protocol (e.g., HTTPS).

[1385] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[1386] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[1387] 6. The server stores the generated virtual profile in a database together with the corresponding user ID.

[1388] Authentication process

[1389] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the login screen. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1390] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[1391] 3. The device sends the collected biometric information to the server using a secure communication protocol.

[1392] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[1393] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[1394] 6. The server matches the generated virtual profile with existing virtual profiles stored in a database.

[1395] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, the server sends a message of failed authentication and asks the user to try again.

[1396] Specific examples

[1397] For example, when User A attempts to register for the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores that profile in a database. Next, when User A accesses the system again and attempts to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[1398] The above is an embodiment of the system of the present invention, which can eliminate the hassle of passwords and provide a high level of security.

[1399] The processing flow will be explained below.

[1400] User registration process

[1401] Step 1:

[1402] The user presses the "Register" button on the registration screen and is then asked to enter biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1403] Step 2:

[1404] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[1405] Step 3:

[1406] The biometric information acquired by the terminal is transmitted to a server using a secure communication protocol (for example, HTTPS).

[1407] Step 4:

[1408] The server receives the biometric information sent from the device and performs preprocessing on the received data, such as normalization to adjust the brightness of facial images, filtering to remove noise from voiceprint data, and enhancement to clarify fingerprint data.

[1409] Step 5:

[1410] The server inputs the pre-processed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[1411] Step 6:

[1412] The server stores the generated virtual profile in a database together with the corresponding user ID.

[1413] Authentication process

[1414] Step 1:

[1415] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1416] Step 2:

[1417] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the respective digital formats.

[1418] Step 3:

[1419] The terminal transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[1420] Step 4:

[1421] The server receives the biometric information sent from the device and performs preprocessing on the received data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[1422] Step 5:

[1423] The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[1424] Step 6:

[1425] The server matches the generated virtual profile with existing virtual profiles stored in a database.

[1426] Step 7:

[1427] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system; if not, it sends a message of failed authentication and asks the user to try again.

[1428] The above are the specific steps in the user registration and authentication processes, which greatly improve the security and convenience of the system.

[1429] Example 1

[1430] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1431] Conventional biometric authentication systems face challenges in efficiently and securely processing a user's biometric information for user authentication. Furthermore, when using a wide variety of biometric information, preprocessing each piece of information and generating a virtual profile using a generative model can be complex, potentially reducing the security and accuracy of the authentication process.

[1432] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1433] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model to generate a virtual profile, means for storing the generated virtual profile in association with a user identifier, means for transmitting the acquired biometric information to the server using a secure communication protocol, means for preprocessing the received biometric information by the server, and means for verifying the generated virtual profile by the server. This enables efficient and secure processing of a wide variety of user biometric information, enabling highly accurate user authentication.

[1434] "Biometric information" is digital data that characterizes a part of a user's body, such as a facial image, voiceprint, or fingerprint.

[1435] A "generative model" is a model that uses machine learning algorithms to generate unique virtual profiles from input data.

[1436] A "virtual profile" is a digital profile created using a generative model that uniquely represents a user's biometric information.

[1437] A "user identifier" is an identification number or ID that is uniquely assigned to each user.

[1438] A "secure communication protocol" is a safe protocol used to protect data communication channels, and specifically includes HTTPS.

[1439] "Preprocessing" refers to a series of data processing operations including noise removal, filtering, and normalization of biological information.

[1440] "Matching" is the process of comparing a newly generated virtual profile with an existing virtual profile to determine the degree of match.

[1441] A "server" is a computer system that collects biometric information, pre-processes it, applies generative models, and generates and matches virtual profiles.

[1442] The system according to the present invention comprises a user, a terminal, and a server. A specific embodiment of the system is described below.

[1443] The system of the present invention acquires a user's biometric information and performs a series of processes to authenticate the user based on that information. The hardware required includes sensors such as a camera, microphone, and fingerprint scanner. The software used includes machine learning frameworks such as TensorFlow and PyTorch, and the HTTPS protocol for secure communication.

[1444] First, the user provides biometric information such as a facial image, voiceprint, and fingerprint on the system's registration screen. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. Through these operations, the user's biometric information is captured as digital data.

[1445] The device then collects this biometric information and converts the data from each sensor into a digital format—for example, converting camera footage into image data, microphone recordings into audio data, and fingerprint scans into fingerprint data—and transmits the collected data to a server using a secure communication protocol.

[1446] The server performs preprocessing on the received biometric data, including normalizing the facial image (correcting for lighting conditions and adjusting the size), filtering the voiceprint (removing noise), and enhancing the fingerprint data (emphasizing details). The preprocessed biometric data is then input into a generative model.

[1447] The generative model uses machine learning algorithms such as TensorFlow or PyTorch to generate a unique virtual profile from the preprocessed data, which is then stored in a database along with a user identifier.

[1448] During authentication, the user again provides biometric information such as a facial image, voiceprint, and fingerprint. The device collects this information, converts it into digital data, and sends it to the server. The server then preprocesses it again and generates a new virtual profile using a generative model. This new virtual profile is then compared with existing virtual profiles stored in the database. If the results match, the user is authenticated and can access the system. If the results do not match, the user receives a message indicating authentication has failed and is asked to try again.

[1449] Specific examples

[1450] For example, if User A wants to register with the system, he or she provides a face image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information and inputs it into a generative model (a TensorFlow-based machine learning model) to generate a virtual profile, which is then stored in a database.

[1451] Next, when User A accesses the system again and tries to log in, he / she provides his / her biometric information again. The terminal collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[1452] Prompt Sentence Examples

[1453] An example prompt to create a virtual profile for User B in the system is:

[1454] Please provide User B's face image, voiceprint, and fingerprint. Follow these steps to collect biometric information:

[1455] 1. Face the camera.

[1456] 2. Speak into the microphone.

[1457] 3. Place your finger on the fingerprint scanner.

[1458] Once you have collected this information, submit it to the system.

[1459] The above is an embodiment of the present invention. This system can eliminate the hassle of passwords and provide high security.

[1460] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1461] Step 1: Provide biometric information

[1462] Users provide biometric information such as a facial image, voiceprint, or fingerprint on the system registration or login screen. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. This inputs biometric information as analog data.

[1463] Step 2: Collecting and digitalizing biometric information

[1464] The device collects biometric information provided by the user from each sensor (camera, microphone, fingerprint scanner) and converts it into digital form. For example, camera footage is converted into image data, microphone recordings into voice data, and fingerprint scans into fingerprint data. Here, the input is analog data and the output is digital data.

[1465] Step 3: Sending biometric information

[1466] The device transmits the collected digital biometric information to the server using a secure communication protocol (e.g., HTTPS). Specifically, the device encrypts the data and transmits it over a secure channel. The input is digital data, and the output is the encrypted data transmitted to the server.

[1467] Step 4: Preprocessing of biological information

[1468] The server receives the digital data sent from the device and performs preprocessing on each piece of information, such as normalizing facial images (correcting for lighting conditions and adjusting size), filtering voiceprints (removing noise), and enhancing fingerprint data (emphasizing details).The input here is the received biometric information, and the output is the preprocessed data.

[1469] Step 5: Generate a Virtual Profile

[1470] The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile. The generative model uses a machine learning algorithm (e.g., TensorFlow or PyTorch). Specifically, the server inputs the preprocessed data into the generative model, and the generative model outputs a virtual profile. The input is the preprocessed data, and the output is the virtual profile.

[1471] Step 6: Save the Virtual Profile

[1472] The server stores the generated virtual profile together with the corresponding user identifier in a database. Specifically, the server executes an SQL query to insert the virtual profile and the user identifier into the database. The input is the virtual profile and the user identifier, and the output is the result stored in the database.

[1473] Step 7: Provide biometric information when logging in

[1474] The user then provides biometric information such as a facial image, voiceprint, and fingerprint on the login screen again. Specifically, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner. This re-enters the biometric information as analog data.

[1475] Step 8: Collecting and digitally converting biometric information during login

[1476] The device collects biometric information provided by the user from each sensor (camera, microphone, fingerprint scanner), converts it into digital form, and outputs the collected data as digital data again.

[1477] Step 9: Sending biometric information when logging in

[1478] The device sends the collected digital biometric information to a server using a secure communication protocol (HTTPS). The input is digital data, and the output is encrypted data sent to the server.

[1479] Step 10: Preprocessing biometric information at login

[1480] The server receives the digital data sent from the terminal and pre-processes each piece of information again. The input is the received biometric information, and the output is the pre-processed data.

[1481] Step 11: Create a new virtual profile

[1482] The server inputs the preprocessed biometric information into a generative model to generate a new virtual profile. The input is the preprocessed data, and the output is the new virtual profile.

[1483] Step 12: Profile Matching

[1484] The server matches the newly generated virtual profile with existing virtual profiles stored in a database. Specifically, the server executes a query to retrieve existing profiles and uses a comparison algorithm to check for a match. The inputs are the new virtual profile and the existing virtual profile, and the output is the match result.

[1485] Step 13: Sending authentication results

[1486] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If the profile does not match, the server sends a message of failed authentication and asks the user to try again. The input is the matching result, and the output is the authentication message.

[1487] (Application example 1)

[1488] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1489] Conventional biometric authentication systems aim to improve security by using biometric information such as facial images, fingerprints, and voiceprints instead of passwords. However, issues remain, such as the difficulty of securing and managing biometric information, as well as the lack of preprocessing, normalization, and secure communication methods for biometric information. For these reasons, a system that can realize a safe and efficient authentication process is needed.

[1490] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1491] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model and generating a virtual profile, means for storing the generated virtual profile in association with a user ID, means for preprocessing or normalizing the registered biometric information, means for securely transmitting the preprocessed or normalized biometric information to the server, means for securely communicating biometric data from the local device to the server, and means for storing the generated virtual profile in a database and performing re-matching. This increases the security of biometric information and enables an efficient authentication process.

[1492] "Biometric information" is digital data of biological characteristics, such as facial images, fingerprints, and voiceprints, that are used to identify individuals.

[1493] A "generative model" is a model that uses a machine learning algorithm to generate a virtual profile based on biometric information.

[1494] A "virtual profile" is a unique digital profile data created by a generative model and linked to a user ID.

[1495] "Preprocessing" is the process of converting biometric information into a format suitable for the generative model by processing it through normalization, filtering, enhancement, and other processes.

[1496] "Normalization" is the process of converting biometric information into a standard format or scale to ensure data consistency.

[1497] "Secure communications" means encrypted communications protocols, such as HTTPS, used to protect the confidentiality and integrity of information.

[1498] "Database" refers to an integrated management system for storing and matching generated virtual profiles and corresponding user IDs.

[1499] "Matching" is the process of comparing the newly generated virtual profile with existing profiles to confirm a match.

[1500] A "local device" is a terminal such as a smartphone or tablet to which an individual user provides biometric information.

[1501] The system according to the present invention acquires biometric information, generates a virtual profile using a generative model, and performs a series of processes to authenticate a user based on the profile. The system is composed of a user, a terminal, and a server.

[1502] Explanation of program processing

[1503] User registration process

[1504] 1. The user provides biometric information such as a facial image, voiceprint, and fingerprint on the system's registration screen. For example, the user faces the smartphone camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1505] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[1506] 3. The device sends the collected biometric information to the server using a secure communication protocol (e.g., HTTPS).

[1507] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information (e.g., normalizing facial images, filtering voiceprints, enhancing fingerprint data).

[1508] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[1509] 6. The server stores the generated virtual profile in a database together with the corresponding user ID.

[1510] Authentication process

[1511] 1. The user provides biometric information such as a facial image, voiceprint, or fingerprint on the login screen. For example, the user faces the smartphone camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1512] 2. The device collects the biometric information provided by the user, obtains data from each sensor (camera, microphone, fingerprint scanner) and converts it into a digital format.

[1513] 3. The device sends the collected biometric information to the server using a secure communication protocol.

[1514] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information (e.g., normalizing facial images, filtering voiceprints, enhancing fingerprint data).

[1515] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[1516] 6. The server matches the generated virtual profile with existing virtual profiles stored in a database.

[1517] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, the server sends a message of failed authentication and asks the user to try again.

[1518] Specific examples

[1519] For example, when User A attempts to register for the system, he or she provides a facial image, voiceprint, and fingerprint. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores that profile in a database. Next, when User A accesses the system again and attempts to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated.

[1520] Prompt Sentence Examples

[1521] For example, a sentence such as, "Generate a virtual profile using user A's facial image, voiceprint, and fingerprints. Then, compare it with existing virtual profiles stored in the database to determine whether there is a match" is input into the generative AI model.

[1522] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1523] Step 1:

[1524] The user provides biometric information such as a face image, voiceprint, and fingerprint on the system's registration screen. Here, the user's face image, voiceprint, and fingerprint are used as input. Specifically, the user faces the smartphone's camera, speaks into the microphone, and places their finger on the fingerprint scanner. The output is the state in which this biometric information has been collected.

[1525] Step 2:

[1526] The device collects biometric information provided by the user and obtains digital data from each sensor (camera, microphone, fingerprint scanner). This data is then converted into digital format. The input is the biometric information provided by the user, and the output is digital facial image data, voiceprint data, and fingerprint data. Specifically, each sensor captures the biometric information and stores it as digital data inside the device.

[1527] Step 3:

[1528] The device transmits the collected digital biometric information to a server using a secure communication protocol (e.g., HTTPS). The input is the digital biometric data, and the output is the data transmitted to the server. Specifically, the device encrypts the data and transmits it to the server via a secure channel.

[1529] Step 4:

[1530] The server receives the biometric information sent from the device and performs preprocessing on each piece of information. The input is the biometric data sent from the device, and the output is the preprocessed data. Specific operations include normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[1531] Step 5:

[1532] The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile. The input is the preprocessed biometric data, and the output is the virtual profile. The generative model uses a machine learning algorithm. The specific operation is to apply the preprocessed data to the generative model to generate the virtual profile.

[1533] Step 6:

[1534] The server stores the generated virtual profile together with the user ID in a database. The input is the virtual profile and the user ID, and the output is the state saved in the database. The specific operation is to uniquely associate the virtual profile with the user ID and record it in the database.

[1535] Step 7:

[1536] The user provides their biometric information again on the login screen. The input is a new face image, voiceprint, and fingerprint, and the output is the newly collected biometric information. Specifically, the smartphone's camera, microphone, and fingerprint scanner are used again to acquire the new information.

[1537] Step 8:

[1538] The terminal converts the newly collected biometric information back into digital data and transmits it to the server using a secure communication protocol. The input is the new digital biometric information data, and the output is the data transmitted to the server again. The specific operations are the same as steps 2 and 3.

[1539] Step 9:

[1540] The server preprocesses the new biometric data sent from the device and inputs it into the generative model to generate a new virtual profile. The input is the newly preprocessed biometric data, and the output is a new virtual profile. The specific operations are the same as steps 4 and 5.

[1541] Step 10:

[1542] The server matches the newly generated virtual profile with the existing virtual profiles stored in the database. The input is the data of the new virtual profile and the existing virtual profile, and the output is the matching result. The specific operation is to retrieve the existing profile from the database and compare it with the new profile.

[1543] Step 11:

[1544] If the profile matches, the server sends an authentication success message to the terminal and the user can access the system. If it does not match, it sends an authentication failure message and asks the user to try again. The input is the matching result, and the output is a message indicating authentication success or failure. The specific operation is to generate a success or failure message and send it to the terminal.

[1545] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1546] This invention combines a system that acquires biometric information, generates a virtual profile based on a generative model, and authenticates users based on that profile with an emotion engine that recognizes the user's emotions. The system is composed of a server, a terminal, and a user, and by further integrating the emotion engine, authentication can be performed taking into account the user's emotional state.

[1547] Explanation of program processing

[1548] User registration process

[1549] 1. The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1550] 2. The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, converting the biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[1551] 3. The device transmits the acquired biometric information to the server using a secure communication protocol (e.g., HTTPS).

[1552] 4. The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalization to adjust the brightness of facial images, filtering to remove noise from voiceprint data, and enhancement to make fingerprint data clearer.

[1553] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[1554] 6. The server stores the generated virtual profile in a database along with the corresponding user ID.

[1555] 7. The server simultaneously uses an emotion engine to recognize the user's emotional state through facial expression analysis of facial images or tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[1556] Authentication process

[1557] 1. The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1558] 2. The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, converting the captured biometric information into the respective digital formats.

[1559] 3. The device transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[1560] 4. The server receives the biometric data sent from the device and performs preprocessing on each piece of data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[1561] 5. The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[1562] 6. The server matches the generated virtual profile with existing virtual profiles stored in the database.

[1563] 7. If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system. If not, it sends a message of failed authentication and asks the user to try again.

[1564] 8. The server also uses the emotion engine to analyze the user's current emotional state and compare it with the existing emotional state. If the emotional state matches or is within a trusted range, it will be reflected in the authentication result.

[1565] Specific examples

[1566] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then saved in a database. At this time, the emotion engine recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and saves this in the profile. Next, when User A attempts to log in to the system again, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, authenticates User A. The emotion engine also analyzes User A's current emotional state and compares it with the existing emotional state. If they match, authentication becomes more reliable.

[1567] The above is an embodiment of a system incorporating an emotion engine according to the present invention. This system can eliminate the hassle of passwords and provide a higher level of security and user experience.

[1568] The processing flow will be explained below.

[1569] User registration process

[1570] Step 1:

[1571] The user presses the "Register" button on the registration screen and is then prompted to enter biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1572] Step 2:

[1573] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the appropriate digital format (facial images are JPEG, voiceprints are WAV, and fingerprints are binary data).

[1574] Step 3:

[1575] The biometric information acquired by the terminal is transmitted to a server using a secure communication protocol (for example, HTTPS).

[1576] Step 4:

[1577] The server receives the biometric information sent from the device and performs preprocessing on each piece of information, such as normalization to adjust the brightness and contrast of facial images, filtering to remove noise from voiceprint data, and enhancement to make fingerprint data clearer.

[1578] Step 5:

[1579] The server inputs the pre-processed biometric information into a generative model, which uses machine learning algorithms to generate a unique virtual profile.

[1580] Step 6:

[1581] The server stores the generated virtual profile in a database together with the corresponding user ID.

[1582] Step 7:

[1583] The server uses an emotion engine to recognize the user's emotional state by analyzing facial expressions in facial images or tone in voiceprints, for example, by analyzing subtle facial movements and voice tone, pitch, and rhythm.

[1584] Step 8:

[1585] The server stores the recognized emotional state together with the virtual profile in a database.

[1586] Authentication process

[1587] Step 1:

[1588] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, or fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1589] Step 2:

[1590] The device captures a facial image with a camera, records a voiceprint with a microphone, and reads a fingerprint with a fingerprint scanner, then converts the captured biometric information into the respective digital formats.

[1591] Step 3:

[1592] The terminal transmits the collected biometric information to a server using a secure communication protocol (e.g., HTTPS).

[1593] Step 4:

[1594] The server receives the biometric data sent from the device and performs preprocessing on each piece of data, such as normalizing facial images, filtering voiceprints, and enhancing fingerprint data.

[1595] Step 5:

[1596] The server inputs the preprocessed biometric information into a generative model, which uses machine learning algorithms to generate a new virtual profile.

[1597] Step 6:

[1598] The server matches the generated virtual profile with existing virtual profiles stored in a database.

[1599] Step 7:

[1600] If the profile matches, the server sends a message of successful authentication to the terminal and the user can access the system; if not, it sends a message of failed authentication and asks the user to try again.

[1601] Step 8:

[1602] The server uses an emotion engine to analyze the user's current emotional state, again by analyzing, for example, facial expressions and tone of voice.

[1603] Step 9:

[1604] The server compares the current emotional state with the existing emotional state, and if they match or are within a trustworthy range, reflects the result in the authentication result. For example, if the emotional state matches and the virtual profile matches, authentication is completed.

[1605] Specific examples

[1606] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then saved in a database. At this time, the emotion engine recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and saves this in the profile. Next, when User A attempts to log in to the system again, he or she provides biometric information again. The device collects this information and sends it to the server. The server compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated. The emotion engine is also used to analyze User A's current emotional state and compares it with the existing emotional state. If they match or are within the confidence range, more reliable authentication is achieved.

[1607] The above is an embodiment of a system incorporating an emotion engine according to the present invention. This system eliminates the hassle of passwords and provides a higher level of security and user experience.

[1608] Example 2

[1609] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1610] Conventional biometric authentication systems authenticate users based on their biometric information, but they have limitations in authentication accuracy and security. Furthermore, because they do not take into account the user's psychological state or emotions, it is difficult to prevent unauthorized access and the user experience is unsatisfactory. Furthermore, there is a risk of attacks on authentication data, making it necessary to strengthen security.

[1611] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1612] In this invention, the server includes means for acquiring biometric information, means for inputting the acquired biometric information into a generative model to generate a virtual profile, means for storing the generated virtual profile in association with a user ID, means for preprocessing the biometric information, means for authenticating the user based on the virtual profile, means for analyzing the user's emotional state using an emotion engine, means for adding the analyzed emotional state to the virtual profile and storing it, and means for reacquiring the biometric information and emotional state for authentication. This enables highly accurate authentication that takes into account the user's emotional state as well as the biometric information, thereby strengthening the security of the entire authentication process and improving the user experience.

[1613] "Biometric information" is data that digitally captures a user's physical characteristics, including facial images, voiceprints, fingerprints, and the like.

[1614] A "generative model" is a computational tool that uses machine learning algorithms to generate new information from input data.

[1615] A "virtual profile" is digital information that uniquely identifies a user and is generated using machine learning algorithms based on biometric information.

[1616] A "user ID" is an identifier that uniquely identifies a user and is linked to a virtual profile.

[1617] "Preprocessing" refers to data processing means for making acquired biometric information easier to analyze, and includes brightness adjustment, noise removal, enhancement, and the like.

[1618] The "emotion engine" is a software component for analyzing a user's emotional state from facial images and voiceprints.

[1619] "Emotional state" is data that indicates the emotional state that the user is currently experiencing, and is generated as a result of analysis.

[1620] "Authentication" is the process of verifying that a user is legitimately accessing something based on the biometric information and emotional state they provide.

[1621] This invention provides a system that acquires a user's biometric information and emotional state, and generates and authenticates a virtual profile based on a generative model. This system consists of a server, a terminal, and a user, and by integrating an emotion engine, it enables advanced authentication that takes into account the user's emotional state.

[1622] Overall system configuration

[1623] The main components of the system are:

[1624] Server: Responsible for preprocessing biometric information, running generative models, generating and storing virtual profiles, authentication, and analyzing emotional states.

[1625] Terminal: Acquires biometric information from the user, converts it into digital format, and sends it to the server.

[1626] User: Provides biometric information such as facial image, voiceprint, and fingerprint.

[1627] Emotion engine: A software component that analyzes the user's emotional state from biometric information.

[1628] Hardware and software used

[1629] Hardware:

[1630] 1. Camera: Captures a face image.

[1631] 2. Microphone: Records your voiceprint.

[1632] 3. Fingerprint scanner: Reads your fingerprint.

[1633] software:

[1634] 1. Secure communication protocols (e.g. HTTPS)

[1635] 2. Generative Models Using Machine Learning Algorithms

[1636] 3. Environment-dependent image processing algorithms (brightness adjustment, noise reduction, enhancement)

[1637] 4. Emotion analysis software (facial expression analysis and voice tone analysis)

[1638] Program processing explanation

[1639] User registration process

[1640] 1. The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. For example, the user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1641] 2. The device converts the acquired biometric information into JPEG format (face image), WAV format (voiceprint), and binary format (fingerprint).

[1642] 3. The device sends the acquired biometric information to the server using HTTPS communication.

[1643] 4. The server adjusts the brightness of the received facial image, removes noise from the voiceprint data, and performs enhancement processing on the fingerprint data.

[1644] 5. The server inputs the preprocessed biometric information into a generative model to generate a unique virtual profile.

[1645] 6. The server associates the generated virtual profile with the user ID and saves it in the database.

[1646] 7. The server uses an emotion engine to recognize the user's emotional state from facial expression analysis of facial images and tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[1647] Authentication process

[1648] 1. The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, and fingerprint, again using the camera, microphone, and fingerprint scanner.

[1649] 2. The device converts the acquired biometric information into the appropriate digital format.

[1650] 3. The device sends the acquired biometric information to the server using HTTPS communication.

[1651] 4. The server normalizes the received facial images, filters the voiceprint data, and enhances the fingerprint data.

[1652] 5. The server inputs the preprocessed biometric information into the generative model to generate a new virtual profile.

[1653] 6. The server matches the new virtual profile with an existing virtual profile.

[1654] 7. If there is a match, the server sends an authentication success message to the terminal and the user can access the system. If there is no match, it sends an authentication failure message and asks the user to try again.

[1655] 8. The server uses the emotion engine to analyze the current emotional state and compare it with the existing emotional states. If it matches or is within the trusted range, it is reflected in the authentication result.

[1656] Specific examples

[1657] For example, when User A attempts to register with the system, he or she provides a facial image, voiceprint, and fingerprint on the registration screen. The device collects this information and sends it to the server. The server then preprocesses the received information, inputs it into a generative model, and generates a virtual profile, which is then stored in a database. At this time, the emotion engine also recognizes User A's emotional state by analyzing the facial expression in the facial image or the tone of the voiceprint, and stores this in the profile.

[1658] Next, when User A accesses the system again and tries to log in, he or she provides biometric information again. The device collects this information and sends it to the server. The server then compares the newly generated virtual profile with the existing profile, and if they match, User A is authenticated. The emotion engine also analyzes the current emotional state and compares it with the existing emotional state. If they match, more reliable authentication is possible.

[1659] Prompt Sentence Examples

[1660] Example of a prompt during registration:

[1661] Describe the process of taking a user's facial image, voiceprint, and fingerprint as input, generating a virtual profile based on a generative model, and storing it in a database along with biometric information and emotional state.

[1662] An example of a login prompt:

[1663] Describe the authentication process by acquiring a user's facial image, voiceprint, and fingerprints, matching a newly generated virtual profile with an existing profile, and analyzing and comparing their emotional state.

[1664] The above is an example of a biometric authentication system that integrates an emotion engine according to the present invention, which eliminates the hassle of passwords and provides a higher level of security and user experience.

[1665] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1666] User registration process

[1667] Step 1:

[1668] The user presses the "Register" button on the registration screen and enters biometric information such as a facial image, voiceprint, and fingerprint. The user then faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1669] Input: User's face image, voiceprint, fingerprint

[1670] Output: Biometric information acquired by the device

[1671] Step 2:

[1672] The device uses a camera to capture a facial image in JPEG format, a microphone to record a voiceprint in WAV format, and a fingerprint scanner to read a fingerprint in binary format.

[1673] Input: Acquired biometric information

[1674] Output: JPEG facial image, WAV voiceprint, binary fingerprint

[1675] Step 3:

[1676] The device sends the acquired biometric information to the server using HTTPS.

[1677] Input: JPEG facial image, WAV voice print, binary fingerprint

[1678] Output: Biometric information sent to the server

[1679] Step 4:

[1680] The server normalizes the brightness of the received facial images, removes noise from voiceprints, and performs enhancement processing on fingerprint data.

[1681] Input: Biometric information sent to the server

[1682] Output: Preprocessed facial images, voiceprints, and fingerprint data

[1683] Step 5:

[1684] The server inputs the pre-processed biometric information into a generative model to generate a unique virtual profile.

[1685] Input: Preprocessed face images, voiceprints, and fingerprint data

[1686] Output: Generated virtual profile

[1687] Step 6:

[1688] The server associates the generated virtual profile with the user ID and stores it in the database.

[1689] Input: Generated virtual profile, user ID

[1690] Output: Virtual profile and user ID stored in the database

[1691] Step 7:

[1692] The server uses an emotion engine to recognize the user's emotional state from facial expression analysis of facial images and tone analysis of voiceprints, and stores this emotional state together with the virtual profile.

[1693] Input: Preprocessed face images and voiceprint data

[1694] Output: Recognized emotional state, updated virtual profile

[1695] Authentication process

[1696] Step 1:

[1697] The user presses the "Login" button on the login screen and provides biometric information such as a facial image, voiceprint, and fingerprint. The user faces the camera, speaks into the microphone, and places their finger on the fingerprint scanner.

[1698] Input: User's face image, voiceprint, fingerprint

[1699] Output: Biometric information acquired by the device

[1700] Step 2:

[1701] The device uses a camera to capture a facial image in JPEG format, a microphone to record a voiceprint in WAV format, and a fingerprint scanner to read a fingerprint in binary format.

[1702] Input: Acquired biometric information

[1703] Output: JPEG facial image, WAV voiceprint, binary fingerprint

[1704] Step 3:

[1705] The device sends the acquired biometric information to the server using HTTPS.

[1706] Input: JPEG facial image, WAV voice print, binary fingerprint

[1707] Output: Biometric information sent to the server

[1708] Step 4:

[1709] The server normalizes the received facial images, filters voiceprints, and enhances fingerprint data.

[1710] Input: Biometric information sent to the server

[1711] Output: Preprocessed facial images, voiceprints, and fingerprint data

[1712] Step 5:

[1713] The server inputs the preprocessed biometric information into a generative model to generate a new virtual profile.

[1714] Input: Preprocessed face images, voiceprints, and fingerprint data

[1715] Output: The newly generated virtual profile

[1716] Step 6:

[1717] The server matches the newly created virtual profile with existing virtual profiles.

[1718] Input: Newly created virtual profile, existing virtual profile

[1719] Output: Matching result

[1720] Step 7:

[1721] If they match, the server sends an authentication success message to the terminal, allowing the user to access the system. If they do not match, the server sends an authentication failure message, requesting a retry.

[1722] Input: Matching result

[1723] Output: Authentication result message

[1724] Step 8:

[1725] The server uses the emotion engine to analyze the user's current emotional state and compare it with existing emotional states. If it matches or is within a trusted range, it is reflected in the authentication result.

[1726] Input: Preprocessed face images, voiceprint data, and existing emotional states

[1727] Output: Analyzed current emotional state, final authentication result

[1728] (Application example 2)

[1729] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1730] Conventional biometric authentication systems authenticate users by capturing biometric information such as facial images, voiceprints, and fingerprints, and storing it in association with a user ID. However, this alone has limitations in the accuracy of authentication and the strength of security. In particular, there is a problem in that the risk of unauthorized access increases if someone misuses the biometric information of a genuine user. Furthermore, because the system does not take into account the user's status (for example, when they are forced to log in), the user experience may be impaired.

[1731] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1732] In this invention, the server includes a means for acquiring biometric information, a means for inputting the acquired biometric information into a generative model to generate a virtual profile, a means for storing the generated virtual profile in association with a user ID, an emotion engine for analyzing the user's emotional state, and a means for adding the user's emotional state to the virtual profile using the emotion engine. This enables highly accurate authentication based on biometric information as well as security services that take the user's emotional state into consideration.

[1733] "Biometric information" refers to data containing biological characteristics that can uniquely identify an individual, such as facial images, voiceprints, and fingerprints.

[1734] "Generative model" refers to a model that uses machine learning algorithms to generate a virtual profile based on biometric information.

[1735] A "virtual profile" is a digital user profile generated using a generative model based on biometric information and used to identify an individual user.

[1736] "User ID" means a unique identifier used to identify a particular User.

[1737] "Emotion engine" refers to software or algorithms for analyzing a user's emotional state from their biometric information (e.g., facial image, voiceprint, etc.).

[1738] "Server" refers to a computer system for collecting, processing, and storing data.

[1739] This invention provides a security application installed on smartphones. The system acquires a user's biometric information, such as facial images, voiceprints, and fingerprints, and generates a virtual profile using a generative AI model. Furthermore, it analyzes the user's emotional state using an emotion engine and adds it to the virtual profile, achieving highly accurate and secure authentication.

[1740] Hardware and Software Configuration

[1741] Hardware

[1742] Camera: A device for capturing facial images.

[1743] Microphone: A device used to capture voiceprints.

[1744] Fingerprint scanner: A device for capturing fingerprint data.

[1745] software

[1746] OpenCV: A library for acquiring and preprocessing face images.

[1747] EmotionEngine: A module that analyzes emotional states from facial images.

[1748] VoiceRecognizer: A module that acquires and preprocesses voice data.

[1749] FingerprintScanner: A module that acquires and preprocesses fingerprint data.

[1750] Requests: An HTTP request library for communicating with the server.

[1751] Generative model: A model for generating virtual profiles from biometric information using machine learning algorithms.

[1752] System configuration

[1753] 1. A user accesses a security application on their smartphone and provides a facial image, voiceprint, and fingerprint.

[1754] 2. The device captures this biometric information using the camera, microphone, and fingerprint scanner and converts it into the appropriate digital format (JPEG, WAV, binary data).

[1755] 3. The device sends the biometric information to the server using a secure communication protocol (e.g., HTTPS).

[1756] 4. The server receives the transmitted biometric information and performs preprocessing such as normalizing the facial image, removing noise from the audio data, and enhancing the fingerprint data.

[1757] 5. The server inputs the preprocessed biometric information into the generative model to generate a virtual profile.

[1758] 6. The server associates the generated virtual profile with the user ID and stores it in the database.

[1759] 7. The server uses an emotion engine to analyze the facial expression of the facial image and the tone of the voiceprint to analyze the user's emotional state.

[1760] 8. The server adds the analyzed emotional state to the virtual profile and further stores it in a database.

[1761] Specific examples

[1762] When User A wants to register with the system, he or she will provide a facial image, voiceprint, and fingerprints as prompted by the smartphone application. For example, the following prompts will be displayed:

[1763] "Please look at the camera with a natural expression. Next, introduce yourself into the microphone. Finally, place your finger on the fingerprint scanner."

[1764] When a user provides biometric information, the device collects this information and sends it to the server. The server preprocesses the received information, inputs it into a generative model to generate a virtual profile, and stores the profile in a database. At the same time, the emotion engine analyzes facial images and voiceprints to recognize User A's emotional state, which is then added to and saved in the profile.

[1765] Now, if User A tries to log into the system again, he will see the following prompt:

[1766] "Please face the camera again to log in. Next, speak your name into the microphone. Finally, place your finger on the fingerprint scanner."

[1767] When User A provides their biometric information, the server generates a new virtual profile and compares it with the previously saved profile. The emotion engine also analyzes the current emotional state and compares it with the previously saved emotional state to provide a more reliable authentication.

[1768] This provides highly accurate authentication based on biometric information as well as security services that take into account the user's emotional state.

[1769] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1770] Step 1:

[1771] A user accesses a security application on their smartphone and provides a facial image, voiceprint, and fingerprint.

[1772] Input: The user faces the camera, introduces themselves into the microphone, and places their finger on the fingerprint scanner.

[1773] Output: Biometric information (face image, voiceprint, fingerprint).

[1774] Specific actions: The user follows the instructions, makes a natural facial expression towards the camera, speaks a few words, and places a finger on the scanner.

[1775] Step 2:

[1776] The device captures this biometric information using a camera, microphone, and fingerprint scanner and converts it into the appropriate digital format (JPEG, WAV, binary data).

[1777] Input: Biometric information (face image, voiceprint, fingerprint).

[1778] Output: Biometric information in digital format (JPEG, WAV, binary data).

[1779] Specific operations: A facial image is taken with a camera and converted to JPEG format, voice is recorded with a microphone and converted to WAV format, and a fingerprint is read with a fingerprint scanner and converted to binary data.

[1780] Step 3:

[1781] The device transmits the biometric information to the server using a secure communication protocol (e.g., HTTPS).

[1782] Input: Biometric data in digital format (JPEG, WAV, binary data).

[1783] Output: Success / failure of sending to server.

[1784] Specific operation: Facial image, voiceprint, and fingerprint data are sent to the server using the HTTPS protocol.

[1785] Step 4:

[1786] The server receives the transmitted biometric information and performs preprocessing such as normalizing facial images, removing noise from audio data, and enhancing fingerprint data.

[1787] Input: Biometric data in digital format (JPEG, WAV, binary data).

[1788] Output: Preprocessed biometric information in digital form.

[1789] Specific operations: Adjusts and normalizes the brightness of facial images, clears audio data with a noise reduction filter, and performs enhancement processing to clarify fingerprint data.

[1790] Step 5:

[1791] The server inputs the preprocessed biometric information into a generative AI model to generate a virtual profile.

[1792] Input: Preprocessed digital biometric information.

[1793] Output: The generated virtual profile.

[1794] How it works: Machine learning algorithms are used to analyze pre-processed biometric information and generate a unique virtual profile.

[1795] Step 6:

[1796] The server associates the generated virtual profile with the user ID and stores it in the database.

[1797] Input: Generated virtual profile and user ID.

[1798] Output: Save to database successful / failed.

[1799] Specific operation: Record the virtual profile along with the user ID in the database and check whether it was saved successfully.

[1800] Step 7:

[1801] The server uses an emotion engine to analyze the user's emotional state by analyzing facial expressions in facial images and tone in voiceprints.

[1802] Input: Preprocessed face image and voiceprint.

[1803] Output: Parsed emotional state.

[1804] Specific operation: Executes the emotion engine, estimates emotions from facial images and voiceprints, and obtains analysis results.

[1805] Step 8:

[1806] The server adds the analyzed emotional state to the virtual profile and further stores it in a database.

[1807] Input: Generated virtual profile and analyzed emotional state.

[1808] Output: Success / failure in adding to database.

[1809] Specific operation: The emotional state is incorporated into the virtual profile, saved to the database, and the success of the save is confirmed.

[1810] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1811] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1812] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1813] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1814] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1815] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1816] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1817] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensa...

Claims

1. A means for acquiring biometric information; means for inputting the acquired biometric information into a generative model to generate a virtual profile; A means for storing the generated virtual profile by linking it to a user ID; A system including:

2. The system according to claim 1, wherein the biometric information is a facial image, a voiceprint, or a fingerprint.

3. The system of claim 1 , wherein the generative model uses a machine learning algorithm.

4. The system of claim 1 further comprising means for preprocessing the biometric information.

5. 2. The system of claim 1, further comprising means for matching said virtual profile and returning an authentication result.

Images