system

Abstract

To provide a system configured to enable a prompt to be input quickly from a smartphone to a generative AI, resulting in more efficient utilization of the generative AI.SOLUTION: A system includes an accessory device which can be connected to a smart device, the system including: user authentication means which authenticates a user who operates the accessory device, by fingerprint authentication; a button provided in the accessory device, and pushed by a user authenticated by the user authentication means for inputting an instruction to a generative AI; voice recognition means which recognizes voice of the user in accordance with depression of the button; and means for acquiring an output of the generative AI using the generative AI and a prompt text including voice data acquired by the voice recognition means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When utilizing generative AI, the time required to input prompts is cumbersome, creating a digital divide for people who have difficulty using the service, particularly the elderly. [Means for solving the problem]

[0005] We provide an accessory device that can be connected to a smartphone. This device includes a means for enabling prompt input to generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. The user authentication means is a fingerprint authentication means, and the operation means is a one-click operation means. This allows prompt input to generative AI immediately from a smartphone, realizing smarter use of generative AI. It also contributes to bridging the digital divide for those who have difficulty using services, such as the elderly. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the disclosed technique will be described below with reference to the accompanying drawings.

[0008] do.

[0009] First, the terms used in the following description will be explained.

[0010] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0011] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0012] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0013] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0014] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0015] [First embodiment]

[0016] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0026] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0027] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0028] "Example 1"

[0029] One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. Specifically, the accessory device is connected to the smartphone using wireless communication means such as Bluetooth, and prompt input to the generative AI is performed through an application running on the smartphone. The user authentication means includes a fingerprint authentication sensor built into the device, which performs authentication by comparing the user's fingerprint information with registered user information. The operation means includes a button on the device that the user presses to input prompts to the generative AI.

[0030] "Example 2"

[0031] In a specific example of use, a user authenticates by placing their finger on the fingerprint authentication sensor of the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is sent to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

[0032] The processing flow of each embodiment will be described below.

[0033] "Example 1"

[0034] Step 1: A user connects an accessory device connectable to a smartphone to the smartphone using a wireless communication means such as Bluetooth.

[0035] Step 2: The user places their finger on the fingerprint authentication sensor of the accessory device, and the fingerprint authentication sensor authenticates the user by matching it with the fingerprint information of the registered user.

[0036] Step 3: If authentication is successful, the device's buttons are enabled.

[0037] Step 4: When the user presses a button on the device, an application on the smartphone is launched, prompting the generative AI to input the desired information.

[0038] "Example 2"

[0039] Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

[0040] Step 2: If authentication is successful, the device's buttons are enabled.

[0041] Step 3: When the user presses the button, an application on the smartphone is launched and prompts the generative AI to input the desired information.

[0042] Step 4: This allows users to easily utilize generative AI without having to operate their smartphones.

[0043] Example 1

[0044] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0045] In conventional systems using generative AI models, users are required to perform complex operations and undergo multi-step authentication when entering prompts, resulting in a poor user experience. Furthermore, there is a lack of efficient means for quickly and accurately obtaining generated results. This reduces the convenience of using generative AI models.

[0046] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0047] In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for running an application on the smartphone for inputting prompts to the generative AI; and means for sending prompts to the generative AI model and receiving generation results. This allows the user to input prompts to the generative AI model with simple operations and quickly and accurately obtain generation results.

[0048] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer, and can be used to install applications and utilize a variety of functions.

[0049] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[0050] "Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompts.

[0051] "Prompt input" is the act of inputting instructions or questions to a generative AI.

[0052] "User authentication means" refers to technology or devices used to verify a user's identity.

[0053] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and verifies the user's identity based on that information.

[0054] The "operation means" refers to an interface or device that allows a user to operate an accessory device or a smartphone.

[0055] A "one-click operation means" is an interface or device that allows a user to perform a specific function with a single click.

[0056] An "application" is software that runs on a smartphone and provides specific functions or services.

[0057] "Generation results" are output such as text and images generated by the generative AI based on the prompt text.

[0058] MODE FOR CARRYING OUT THE INVENTION

[0059] This invention is a system that uses an accessory device connectable to a smartphone to easily input prompts to a generative AI model. Specific embodiments of this system are described below.

[0060] Hardware and Software Configuration

[0061] Accessory equipment:

[0062] An accessory device is an external device that connects to a smartphone and has the following main functions:

[0063] User authentication method: Equipped with a fingerprint authentication sensor that reads the user's fingerprint information.

[0064] Operation: Equipped with a one-click button for easy user operation.

[0065] Smartphone:

[0066] The smartphone is connected to the accessory device via wireless communication means such as Bluetooth, and an application for providing prompt input to the generative AI model is installed on the smartphone.

[0067] Generative AI models:

[0068] The generative AI model runs on a server and generates text and images based on prompts sent from a smartphone.

[0069] System Operation

[0070] User authentication:

[0071] The user places their finger on the fingerprint authentication sensor installed in the accessory device. The accessory device reads the fingerprint information and compares it with fingerprint information stored in its internal database. If authentication is successful, the accessory device sends a signal indicating successful authentication to the smartphone.

[0072] Prompt Input:

[0073] After successful authentication, the user presses a button on the accessory device, which launches the smartphone application and displays a prompt input screen. The user then inputs a prompt for the generative AI model.

[0074] Send to generative AI model:

[0075] The smartphone sends the input prompt to the generative AI model, and the server operates the generative AI model based on the received prompt to generate the generated results.

[0076] View the generated results:

[0077] The server sends the generated results to the smartphone, which then displays them to the user. The user can then check the displayed results and save or share them as needed.

[0078] Specific examples

[0079] If a user wants to have a generative AI create a new poem, the specific steps are as follows:

[0080] 1. User authentication:

[0081] The user places their finger on the fingerprint authentication sensor of the accessory device.

[0082] The accessory device reads the fingerprint and performs authentication.

[0083] If the authentication is successful, a signal indicating successful authentication is sent to the smartphone.

[0084] 2. Prompt input:

[0085] The user presses a button on the accessory device.

[0086] The smartphone application will launch and a prompt input screen will appear.

[0087] The user enters the prompt text "Create a new poem."

[0088] 3. Send to generative AI model:

[0089] The smartphone sends the input prompt to the generative AI model.

[0090] The server runs a generative AI model based on the prompt text to generate a poem.

[0091] 4. View the generated results:

[0092] The server sends the generated poem to the smartphone.

[0093] The smartphone displays the received poem to the user.

[0094] The user can review the displayed poem and save or share it as desired.

[0095] In this way, a user can use the accessory device and a smartphone to provide prompt input to the generative AI model and obtain the generated results.

[0096] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0097] Step 1: Connect accessory devices

[0098] Input: User opens smartphone Bluetooth settings and selects accessory device.

[0099] How it works: Your smartphone establishes a connection with an accessory device using Bluetooth.

[0100] Output: The smartphone will inform you that the connection was successful.

[0101] Step 2: User authentication

[0102] Input: The user places their finger on the fingerprint sensor on the accessory device.

[0103] How it works: The accessory device reads your fingerprint and matches it with fingerprints stored in an internal database.

[0104] Output: If authentication is successful, the accessory device sends a signal to the smartphone indicating successful authentication.

[0105] Step 3: Prepare for prompt input

[0106] Input: The smartphone receives a signal that authentication was successful.

[0107] How it works: The smartphone displays a prompt input screen for the generative AI.

[0108] Output: The user sees the prompt input screen and begins to enter the prompt statement.

[0109] Step 4: Prompt Input

[0110] Input: The user inputs a prompt for the generative AI through a smartphone application.

[0111] How it works: The smartphone receives the input prompt text and formats it internally.

[0112] Output: The prepared prompt is ready to be sent to the generative AI model.

[0113] Step 5: Send to generative AI model

[0114] Input: The smartphone sends a prepared prompt to the generative AI model.

[0115] Operation: The server runs the generative AI model based on the received prompt sentence and creates a generated result.

[0116] Output: The generated results are sent from the server to the smartphone.

[0117] Step 6: View the generated results

[0118] Input: The smartphone receives the generated results from the server.

[0119] How it works: The smartphone displays the generated results to the user.

[0120] Output: The user checks the displayed generated results and saves or shares them as needed.

[0121] (Application example 1)

[0122] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0123] To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and lack sufficient security for user authentication, which creates the risk of operational errors and unauthorized access. This can lead to reduced productivity and safety in factories.

[0124] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0125] In this invention, the server is an accessory device connectable to a smartphone and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI, a means for generating instructions for managing and operating robots in a factory, and a means for transmitting the instructions to the robots via wireless communication means, thereby enabling efficient and safe management and operation of robots in a factory.

[0126] An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

[0127] "Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

[0128] "Means to enable prompt input" refers to a function that provides an interface for users to input instructions or questions to generative AI.

[0129] "User authentication means" is a function for confirming and authenticating the identity of a user.

[0130] "Fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

[0131] The "operation means" is an interface that allows the user to input prompts to the generative AI.

[0132] The "one-click operation means" is a function that allows users to input prompts to generative AI with a single click.

[0133] "Means for generating instructions for managing and operating robots in a factory" refers to a function for generating specific work instructions for robots in a factory.

[0134] "Wireless communication means" refers to a function for sending and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

[0135] The "means for transmitting to the robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

[0136] This invention is a system for efficiently and safely managing and operating robots in a factory using an accessory device connectable to a smartphone. Specific embodiments of this system are described below.

[0137] System Configuration

[0138] The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

[0139] Hardware

[0140] Smartphone: Functions as a user interface and connects to accessory devices via wireless communication means.

[0141] Accessory device: Connected to a smartphone, it includes a fingerprint authentication means and an operation means.

[0142] Fingerprint authentication method: Scans the user's fingerprint for authentication.

[0143] Wireless communication means: Data is sent and received between the smartphone and the factory robot using Bluetooth, Wi-Fi, etc.

[0144] Factory robot: Performs actions according to instructions.

[0145] software

[0146] Generative AI model: Generates specific instructions for factory robots based on user prompts.

[0147] Smartphone application: Provides the user interface and manages communication with the generative AI model.

[0148] Operating procedure

[0149] 1. User authentication: The user places a finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

[0150] 2. Prompt input: An authenticated user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[0151] 3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

[0152] 4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

[0153] 5. Robot operation: The factory robot performs the operation according to the received instructions.

[0154] Specific examples

[0155] For example, when a robot performs a specific task in a factory, a manager uses a smartphone and accessory devices to give instructions to the robot. The following is an example of a prompt sentence:

[0156] "Move to assembly line 3 and begin welding."

[0157] "Move to the inspection area and perform quality checks."

[0158] "Go get part A and take it to assembly line 2."

[0159] This enables efficient and safe management and operation of robots in factories. The system utilizes user authentication and generative AI to reduce the risk of misoperation and unauthorized access, improving factory productivity and safety.

[0160] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0161] Step 1:

[0162] The user places a finger on the accessory device and is authenticated by the fingerprint authentication means.

[0163] Input: User's fingerprint

[0164] Data processing: The fingerprint authentication means scans the fingerprint and compares it with registered fingerprint data.

[0165] Output: Authentication result (success or failure)

[0166] How it works: The fingerprint sensor scans the user's fingerprint and matches it with an internal database for authentication.

[0167] Step 2:

[0168] If the authentication is successful, the user uses the operation means (one-click operation means) to input a prompt to the generative AI model.

[0169] Input: Authentication result (success), user prompt

[0170] Data processing: If the authentication result is successful, send the prompt text to the generative AI model.

[0171] Output: prompt input to the generative AI model

[0172] Specific operation: When the user presses the button, the smartphone application sends a prompt sentence to the generative AI model.

[0173] Step 3:

[0174] Based on the prompts, a generative AI model generates specific instructions for the factory robots.

[0175] Input: User prompt text

[0176] Data processing: A generative AI model analyzes the prompt and generates appropriate instructions.

[0177] Output: Instructions to factory robots

[0178] Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

[0179] Step 4:

[0180] The generated instructions are transmitted to the factory robot using wireless communication means.

[0181] Input: Instructions from a generative AI model

[0182] Data processing: Instructions are sent to the factory robot via wireless communication means.

[0183] Output: Sending instructions to factory robots

[0184] Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth or Wi-Fi.

[0185] Step 5:

[0186] The factory robot operates according to the received instructions.

[0187] Input: Instructions to the factory robot

[0188] Data processing: Factory robots analyze instructions and perform corresponding actions.

[0189] Output: Robot movement

[0190] Specific operation: The factory robot operates according to instructions such as "move to assembly line 3 and start welding."

[0191] Example 2

[0192] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0193] In order to use conventional generative AI models, users had to operate their smart devices directly and input prompts. This operation was time-consuming and inconvenient, especially when their hands were full or in situations where a quick response was required. Furthermore, from a security perspective, user authentication was sometimes insufficient, creating the risk of unauthorized use by third parties. There is a need for a system that can solve these issues and make it easier and safer to use generative AI models.

[0194] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0195] In this invention, the server includes a user authentication means, an operation means for operation by an authenticated user, a means for sending a signal to the smart device using the operation means to launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without directly operating the smart device, and the user authentication means also improves security.

[0196] "User authentication means" means a means for verifying a user's identity and granting access to a system.

[0197] "Operation means" refers to the interface or device that allows the user to operate the system.

[0198] A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

[0199] The "means for transmitting a signal" is a means for transmitting specific instructions or data from the operating means to the smart device.

[0200] A "specific application" is a software program designed to provide a particular function or service.

[0201] A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

[0202] A "prompt" is a text input used to give instructions or ask questions to a generative AI model.

[0203] A "means for generating a response" is a means by which a generative AI model creates an appropriate response based on a prompt sentence.

[0204] The "means for sending a response" is a means for sending the generated response to the smart device.

[0205] MODE FOR CARRYING OUT THE INVENTION

[0206] This invention provides a system for utilizing generative AI models without requiring a user to directly operate a smart device. Specific embodiments of this system are described below.

[0207] Hardware and software used

[0208] Hardware: Accessory devices (with fingerprint authentication sensors), smart devices (smartphones, etc.)

[0209] Software: Fingerprint authentication system, smart device applications, generative AI models

[0210] System configuration

[0211] 1. User authentication method:

[0212] The user places their finger on the fingerprint sensor on the accessory device.

[0213] The fingerprint authentication system captures fingerprint data and authenticates the user.

[0214] 2. Operating means:

[0215] If authentication is successful, the buttons on the accessory device are enabled.

[0216] The user presses an enabled button.

[0217] 3. Signal transmission means:

[0218] When the button is pressed, the accessory device sends a signal to the smart device.

[0219] The smart device receives the signal and launches a specific application.

[0220] 4. Prompt sending method:

[0221] The launched application sends a prompt to the generative AI model.

[0222] Prompts are pre-defined and contain instructions or questions that the user wants to ask the generative AI model.

[0223] 5. Response Generation Method:

[0224] The server uses a generative AI model to generate a response based on the received prompt.

[0225] 6. Response sending method:

[0226] The server generates a response and sends it to the smart device.

[0227] The smart device receives the response and displays it to the user.

[0228] Specific examples

[0229] When a user places their finger on the fingerprint authentication sensor of the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device to launch an application. The application launches and sends a prompt sentence, "Tell me what the weather is today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can see the information, "Today's weather is sunny," on the screen.

[0230] Prompt Sentence Examples

[0231] "Tell me about today's weather."

[0232] "Check the schedule for the next meeting."

[0233] "Tell me the latest news"

[0234] This system allows users to easily use generated AI models without directly operating their smart devices, and also improves security through user authentication measures.

[0235] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0236] Program processing flow

[0237] Step 1:

[0238] The user places their finger on the fingerprint authentication sensor on the accessory device.

[0239] Input: User's fingerprint

[0240] Data processing: The fingerprint sensor captures fingerprint data.

[0241] Output: Captured fingerprint data

[0242] Step 2:

[0243] The device captures the fingerprint data and performs authentication.

[0244] Input: Captured fingerprint data

[0245] Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

[0246] Output: Authentication result (success or failure)

[0247] Step 3:

[0248] If authentication is successful, the device's buttons will be enabled.

[0249] Input: Authentication result (success)

[0250] Data processing: If authentication is successful, activate the device button.

[0251] Output: Enabled button

[0252] Step 4:

[0253] The user presses a button.

[0254] Input: Enabled button

[0255] Data processing: The device detects that a button has been pressed.

[0256] Output: Button press signal

[0257] Step 5:

[0258] The terminal sends a signal to the smart device to launch the application.

[0259] Input: Button press signal

[0260] Data processing: The terminal sends a signal to the smart device to launch a specific application.

[0261] Output: Launched application

[0262] Step 6:

[0263] The application sends a prompt to the generative AI model.

[0264] Input: Launched application

[0265] Data processing: The application sends pre-defined prompts to the generative AI model.

[0266] Output: The prompt sent to the generative AI model

[0267] Step 7:

[0268] The server generates a response using a generative AI model.

[0269] Input: The prompt sent to the generative AI model

[0270] Data computation: A generative AI model analyzes the prompt and generates an appropriate response.

[0271] Output: The generated response

[0272] Step 8:

[0273] The server sends the response to the smart device.

[0274] Input: The generated response

[0275] Data processing: The server generates a response and sends it to the smart device.

[0276] Output: Response sent to smart device

[0277] Step 9:

[0278] The smart device displays the response.

[0279] Input: Response sent to smart device

[0280] Data processing: The smart device displays the received response to the user.

[0281] Output: The response displayed to the user

[0282] In this way, users can utilize generative AI models without directly operating their smart devices.

[0283] (Application example 2)

[0284] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0285] Conventional food delivery systems require users to use their smartphones to place orders, which makes operation cumbersome. In addition, there are security risks due to insufficient user authentication. Furthermore, ordering systems using voice recognition are not widespread, resulting in low user convenience. There is a need to provide a system that solves these problems and allows users to use food delivery services easily and safely.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0287] An accessory device connectable to a smart device includes a user authentication means for authenticating a user operating the accessory device through fingerprint authentication, a button provided on the accessory device that a user authenticated by the user authentication means presses to input instructions to the generative AI, and a voice recognition means for recognizing the user's voice in response to the button being pressed. The server includes a means for acquiring the output of the generative AI using a prompt statement including voice data acquired by the voice recognition means and the generative AI. The prompt statement instructs the user to place an order through a food delivery service, and the acquiring means acquires the order results using a food delivery API. This allows users to easily and safely use food delivery services using only their voice, without having to operate their smartphone.

[0288] A "smart device" is a portable electronic device, such as a smartphone or tablet, that has Internet connectivity and can run applications.

[0289] An "accessory device" is additional hardware that is connected to a smart device to provide specific functionality.

[0290] "Generative AI" refers to artificial intelligence techniques that generate text and other content based on user instructions or prompts.

[0291] "Instruction input" refers to commands or requests that a user inputs to a generative AI.

[0292] "User authentication means" refers to technology or devices for verifying and authenticating the identity of a user.

[0293] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and authenticates the user based on that information.

[0294] "Operation means" refers to the interface and buttons that allow the user to operate the accessory device or smart device.

[0295] "Speech recognition means" refers to technology or devices that analyze a user's voice and convert it into text data.

[0296] "Voice data" refers to data that is a digital recording of a user's voice.

[0297] "Transmission means" refers to the technology or devices used to transmit data to other devices or systems.

[0298] This invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. Specific embodiments of this system are described below.

[0299] System configuration

[0300] The system consists of the following main components:

[0301] 1. Smart Device: A portable electronic device, such as a smartphone or tablet, that has internet connectivity and can run applications.

[0302] 2. Accessory device: Additional hardware that is connected to a smart device and includes fingerprint authentication means, operation means, and voice recognition means.

[0303] 3. Generative AI: Artificial intelligence techniques that generate text and other content based on user instructions or prompts.

[0304] Program processing

[0305] Hardware

[0306] Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

[0307] Smart device: Functions as the execution environment for applications.

[0308] Microphone: Captures the user's voice as a means of voice recognition.

[0309] software

[0310] Fingerprint authentication library: A software module for performing fingerprint authentication.

[0311] Speech Recognition Library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

[0312] Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3®).

[0313] Food Delivery API: Place orders using the API of food delivery services.

[0314] Processing flow

[0315] 1. User authentication: The user authenticates by placing their finger on the fingerprint authentication sensor of the accessory device. If authentication is successful, the next step is performed.

[0316] 2. Voice input: The user presses the operation means (one-click button) of the accessory device and speaks the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

[0317] 3. Prompt generation: Based on the acquired text data, a prompt sentence is generated for the generative AI.

[0318] 4. AI response generation: The generative AI model generates the optimal response based on the prompt.

[0319] 5. Order fulfillment: Based on the generated response, the order is completed using the food delivery API.

[0320] Specific examples

[0321] When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model receives the prompt "The user wants to order: pizza," the model then suggests the best pizza restaurant and menu item, and completes the order using a food delivery API.

[0322] Prompt Sentence Examples

[0323] "User wants to order: Pizza"

[0324] In this way, users can easily and safely use food delivery services using only their voice, without having to operate their smartphones.

[0325] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0326] Step 1:

[0327] The user places their finger on the fingerprint sensor.

[0328] Input: User's fingerprint data.

[0329] Processing: The fingerprint authentication library analyzes the fingerprint data and matches it with registered fingerprint data.

[0330] Output: Authentication result (success or failure).

[0331] Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

[0332] Step 2:

[0333] The user presses the operating means (one-click button) of the accessory device.

[0334] Input: User action (button press).

[0335] Processing: Detects that the operating means has been pressed and activates the voice recognition mode.

[0336] Output: Activate speech recognition mode.

[0337] Specific operation: The sensor detects that a button has been pressed, and the speech recognition library begins preparing for voice input.

[0338] Step 3:

[0339] The user verbally states the order details.

[0340] Input: User's voice data.

[0341] Processing: The microphone captures the voice data, which is then converted into text data by a speech recognition library.

[0342] Output: Text data (order details).

[0343] What it does: A microphone captures the user's voice, and a speech recognition library analyzes it and converts it into text.

[0344] Step 4:

[0345] The terminal generates a prompt sentence based on the acquired text data.

[0346] Input: Text data (order details).

[0347] Processing: Convert the text data into a prompt sentence of the form "The user wants to order: [Order details]".

[0348] Output: The prompt statement.

[0349] Specific operation: Converts text data into prompt sentences according to the format.

[0350] Step 5:

[0351] The device sends a prompt to the generative AI model.

[0352] Input: prompt statement.

[0353] Processing: A generative AI model analyzes the prompt and generates the optimal response.

[0354] Output: AI response (optimal order content).

[0355] Specific operation: The prompt sentence is sent to the generative AI model, and the AI ​​generates a response.

[0356] Step 6:

[0357] The device uses the generated response to complete the order using the food delivery API.

[0358] Input: AI response (optimal order content).

[0359] Processing: Fulfilling orders using food delivery APIs.

[0360] Output: Order confirmation (data indicating the order was successfully completed).

[0361] Specific behavior: Sends AI response to food delivery API and confirms order.

[0362] The server may further include means for recognizing the user's emotion using an emotion engine (described later), and may obtain an output from the generative AI using the voice data obtained by the voice recognition means, a prompt including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," the server may generate a prompt such as "User wants to order: pizza, user's emotion: sadness," and the generative AI model may suggest an optimal pizza restaurant and menu item taking the user's emotion into consideration, and complete the order using a food delivery API.

[0363] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0364] "Example 1"

[0365] One embodiment of the present invention provides an accessory device connectable to a smartphone. The device includes a means for enabling prompt input to a generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. The device also includes an emotion engine that recognizes the user's emotions. Specifically, the emotion engine recognizes the user's emotions from their tone of voice and facial expression, and feeds that information back to the generative AI. This enables the generative AI to generate an appropriate response based on the user's emotions.

[0366] "Example 2"

[0367] Another embodiment of the present invention provides a system in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone within the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotion from the tone of the voice data and feeds the results back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotion.

[0368] The processing flow of each embodiment will be described below.

[0369] "Example 1"

[0370] Step 1: The user operates an accessory device that can be connected to a smartphone.

[0371] Step 2: The user authentication means of the device authenticates the user. One possible authentication method is fingerprint authentication.

[0372] Step 3: The authenticated user prompts the generative AI.

[0373] Step 4: The device's emotion engine recognizes the user's emotions from their tone of voice and facial expressions.

[0374] Step 5: The emotion engine feeds that information back to the generative AI.

[0375] Step 6: The generative AI generates an appropriate response based on the user's emotions.

[0376] "Example 2"

[0377] Step 1: The user speaks to the accessory device.

[0378] Step 2: The microphone in the device captures the user's voice and sends the audio data to the emotion engine.

[0379] Step 3: The emotion engine analyzes the user's emotions from the tone of the voice data.

[0380] Step 4: The emotion engine feeds the results back to the generative AI.

[0381] Step 5: The generative AI generates an appropriate response based on the user's emotions.

[0382] Example 1

[0383] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0384] While conventional accessory devices that can be connected to smartphones allow prompt input to generative AI models, they have the problem of being unable to generate responses that take the user's emotional state into account. This makes it difficult to obtain appropriate responses that correspond to the user's intentions and emotions, resulting in a lack of improvement in the user experience.

[0385] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0386] In this invention, the server is an accessory device connectable to a smartphone, and includes: means for enabling prompt input to the generative AI model; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI model; emotion recognition means for recognizing the user's emotion; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the user's emotional state.

[0387] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer.

[0388] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[0389] A "generative AI model" is an artificial intelligence system that generates output such as text or speech based on input prompts.

[0390] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[0391] "User Authentication Measures" means technical measures for verifying a user's identity.

[0392] A "fingerprint authentication means" is a technical means for reading a user's fingerprint and authenticating them based on that information.

[0393] The "operation means" is an interface that allows the user to operate the accessory device.

[0394] A "one-click operation means" is an interface that allows a user to complete an operation with a single click.

[0395] "Emotion recognition means" refers to a technical means for recognizing emotions by analyzing the user's tone of voice and facial expressions.

[0396] "Emotion information" is data relating to the user's emotional state obtained by emotion recognition means.

[0397] "Feedback means" refers to the technical means for transmitting acquired emotional information to the generative AI model.

[0398] MODE FOR CARRYING OUT THE INVENTION

[0399] This invention is a system that uses an accessory device connectable to a smartphone to provide prompt input to a generative AI model and obtain a response that takes into account the user's emotional state. Specific embodiments of this system are described below.

[0400] Hardware and Software Configuration

[0401] Accessory Devices

[0402] The accessory device connects to the smartphone using wireless communication means such as Bluetooth, and includes the following main components:

[0403] Fingerprint authentication sensor: Reads the user's fingerprint for authentication.

[0404] Action button: The interface that allows the user to initiate prompt input.

[0405] Emotion recognition engine: Analyzes the user's tone of voice and facial expressions to obtain emotional information.

[0406] Smartphone

[0407] The smartphone runs a dedicated application that works in conjunction with the accessory device and provides the following functions:

[0408] Prompt input interface: A screen where users can input instructions or questions to a generative AI model.

[0409] Data transmission function: Sends prompts and emotion information to the server.

[0410] server

[0411] The server provides back-end services that communicate data between smartphones and generative AI models. The server has the following functions:

[0412] Data relay function: Sends prompts and emotional information received from the smartphone to the generative AI model.

[0413] Response reception function: Receives responses from the generative AI model and forwards them to the smartphone.

[0414] Specific examples

[0415] Consider a scenario in which a user uses an accessory device connected to a smartphone to ask a generative AI model a question.

[0416] 1. User Authentication

[0417] The user places their finger on the accessory device's fingerprint authentication sensor to authenticate. The device (smartphone) receives the fingerprint data from the accessory device and compares it with registered fingerprint data. If authentication is successful, the user can proceed to the next step.

[0418] 2. Prompt Input

[0419] The user opens a dedicated application on their smartphone and enters a prompt, for example, "What's the weather like today?" The device prepares to send this prompt to the generative AI model.

[0420] 3. Emotion recognition

[0421] The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions, and the device prepares this emotion data for transmission to a generative AI model.

[0422] 4. Sending prompts to the generative AI model

[0423] The device sends the prompt and emotion data entered by the user to the server, which then sends this data to the generative AI model, which then generates an appropriate response based on the prompt and emotion data.

[0424] 5. Receiving a response from the generative AI model

[0425] The server receives the response from the generative AI model and forwards it to the device, which prepares the response for display to the user.

[0426] 6. Displaying the response

[0427] The device displays the response received from the generative AI model to the user, for example, "It's sunny today. Have a nice day." The user can review this response and enter further prompts as needed.

[0428] Prompt Sentence Examples

[0429] "How's the weather today?"

[0430] "Please let me know when the next meeting is scheduled."

[0431] "Please tell me the latest news."

[0432] In this way, users can naturally interact with generative AI models through accessory devices.

[0433] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0434] Step 1: User authentication

[0435] The user places their finger on the fingerprint authentication sensor of the accessory device. The terminal receives the fingerprint data from the accessory device and compares it with the registered fingerprint data. The input is the user's fingerprint data, and the output is the authentication result. The terminal analyzes the fingerprint data and checks whether it matches the registered data. If authentication is successful, the terminal displays "Authentication successful."

[0436] Step 2: Connect your accessory device to your smartphone

[0437] The terminal establishes a connection with an accessory device via Bluetooth communication. The input is a Bluetooth connection request, and the output is the connection status. The terminal receives a connection request from the accessory device and establishes the connection. The server monitors this connection and relays data as necessary. If the connection is successful, the terminal displays "Connection successful."

[0438] Step 3: Prompt Input

[0439] The user opens a dedicated application on their smartphone and inputs a prompt. For example, they might input "What's the weather like today?" The input is the user's prompt sentence, and the output is the prompt data. The device prepares to send this prompt to the generative AI model. When the user has completed their input, the device displays "Prompt input completed."

[0440] Step 4: Emotion Recognition

[0441] The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions. The input is the user's tone of voice and facial expression data, and the output is emotion data. The device prepares this emotion data to send to the generative AI model. Once the emotion data is acquired, the device displays "Emotion data acquisition successful."

[0442] Step 5: Sending prompts to the generative AI model

[0443] The device sends the prompt and emotion data entered by the user to the server. The input is the prompt data and emotion data, and the output is the result of transmission to the server. The server then sends this data to the generative AI model. If the data transmission is successful, the device displays "Data transmission successful."

[0444] Step 6: Receiving a response from the generative AI model

[0445] The server receives the response from the generative AI model and transfers it to the device. The input is the response data from the generative AI model, and the output is the transfer result to the device. The device prepares to display this response to the user. When the response is received, the device displays "Response received successfully."

[0446] Step 7: View the response

[0447] The device displays the response received from the generative AI model to the user. For example, it displays "It's sunny today. Have a nice day." The input is the response data from the generative AI model, and the output is the display result to the user. The user can review this response and enter further prompts as needed.

[0448] (Application example 1)

[0449] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0450] Traditional food delivery services struggled to provide customized recommendations based on users' emotions and circumstances. They also lacked a means for users to use generative AI to simplify the ordering process. This could result in a poor user experience and reduced frequency of service use.

[0451] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0452] In this invention, the server is an accessory device connectable to a smartphone and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI, an emotion recognition means for recognizing the user's emotions, and a means for feeding back the emotion information recognized by the emotion recognition means to the generative AI. This makes it possible to propose customized food delivery according to the user's emotions.

[0453] An "accessory device connectable to a smartphone" is an external device that is connected to a smartphone using wireless communication means and provides a specific function.

[0454] "Generative AI" is an artificial intelligence system that generates responses and suggestions based on prompt input from the user.

[0455] "Means to enable prompt input" refers to a function that provides an interface for users to input instructions or questions to generative AI.

[0456] "User authentication means" refers to technology for verifying the identity of a user, ensuring that only specific users can operate the system.

[0457] "Fingerprint authentication means" is a technology that scans a user's fingerprint and authenticates the user based on that information.

[0458] "Operation means" refers to the physical or software interface that allows the user to operate the system.

[0459] A "one-click operation means" is an interface that allows a user to perform a specific function with a single click.

[0460] "Emotion recognition means" is a technology that analyzes the user's tone of voice and facial expressions to recognize their emotions.

[0461] "Means for feeding back emotional information to generative AI" refers to a function that sends recognized emotional information to generative AI, allowing the generative AI to generate responses or suggestions based on that information.

[0462] This invention is a system that uses an accessory device connectable to a smartphone to simplify the ordering process when a user uses a food delivery service through generative AI and makes customized suggestions based on the user's emotions.

[0463] System configuration

[0464] The system consists of the following main components:

[0465] 1. Accessory devices that can be connected to smartphones

[0466] Wireless communication means: Connected to a smartphone using Bluetooth or similar.

[0467] User authentication method: A fingerprint sensor is used to authenticate the user.

[0468] Control: A one-click control is used by the user to prompt the generative AI.

[0469] Emotion recognition: Analyzes the user's tone of voice and facial expressions to recognize their emotions.

[0470] 2. Generative AI

[0471] Prompt input: Accepts instructions or questions from the user and generates responses or suggestions.

[0472] Emotional feedback: Generate customized responses and suggestions based on recognized emotional information.

[0473] Program processing

[0474] The server does the following:

[0475] 1. Bluetooth connection: Connect your smartphone and accessory device using Bluetooth.

[0476] 2. User authentication: The fingerprint sensor scans the user's fingerprint for authentication.

[0477] 3. Emotion Recognition: Recognizes emotions by analyzing the user's tone of voice and facial expressions.

[0478] 4. Prompt generation: Generate prompts for the generative AI based on the recognized emotions.

[0479] 5. AI response generation: Use generative AI to generate responses and suggestions to prompts.

[0480] Hardware and software used

[0481] Hardware: Bluetooth module, fingerprint sensor, camera (for face recognition), microphone (for voice recognition)

[0482] Software: Bluetooth connection library, fingerprint authentication library, emotion recognition engine, generative AI model

[0483] Specific examples

[0484] If the user is recognized as tired, the generative AI will suggest "food delivery options for when you're tired." For example, the following prompts will be generated:

[0485] The user is feeling tired. Please suggest food delivery options.

[0486] In this way, it becomes possible to offer customized food delivery suggestions based on the user's emotions.

[0487] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0488] Step 1:

[0489] The server connects the smartphone and the accessory device using a Bluetooth module. The input is the pairing information between the smartphone and the accessory device, and the output is the establishment of the connection. Specifically, the server uses the Bluetooth connection library to establish communication between the smartphone and the accessory device.

[0490] Step 2:

[0491] The device scans the user's fingerprint using a fingerprint authentication sensor to perform user authentication. The input is the user's fingerprint data, and the output is the authentication result. Specifically, the fingerprint authentication library is used to compare the scanned fingerprint data with registered data to determine whether authentication was successful.

[0492] Step 3:

[0493] The device uses a microphone and camera to capture the user's tone of voice and facial expressions to perform emotion recognition. The input is voice data and image data, and the output is recognized emotion information. Specifically, an emotion recognition engine is used to analyze the voice data and image data to identify the user's emotion.

[0494] Step 4:

[0495] The server generates prompts for the generative AI based on the recognized emotional information. The input is emotional information, and the output is a prompt. Specifically, the server converts the emotional information into text and creates a prompt suitable for the generative AI.

[0496] Step 5:

[0497] The server uses a generative AI model to generate responses and suggestions to prompts. The input is the prompt, and the output is the generated response or suggestion. Specifically, the prompt is input into the generative AI model, and the AI ​​generates the response or suggestion.

[0498] Step 6:

[0499] The terminal presents the generated response or suggestion to the user. The input is the generated response or suggestion, and the output is what is displayed to the user. Specifically, the response or suggestion is displayed on the smartphone display so that the user can check it.

[0500] Example 2

[0501] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0502] Conventional systems using generative AI models require users to operate their smartphones, making them difficult to use. Furthermore, they are unable to generate appropriate responses based on the user's emotions, limiting the user experience. This makes it difficult for users to effectively utilize generative AI models.

[0503] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0504] In this invention, the server includes a user authentication means, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI model, an emotion recognition means for capturing the user's voice data and analyzing the emotions, and a means for feeding back the emotion information analyzed by the emotion recognition means to the generative AI model. This allows the user to easily use the generative AI model without operating a smartphone, and further enables the user to receive an appropriate response according to the user's emotions.

[0505] "User authentication means" refers to a means for verifying a user's identity, and includes biometric authentication technologies such as fingerprint authentication and facial recognition.

[0506] "Operation means" refers to the means by which a user inputs prompts to the generated AI model, and includes interfaces such as buttons and touchscreens.

[0507] An "emotion recognition means" is a means for capturing a user's voice data and analyzing the user's emotions from the tone and content of the data.

[0508] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on prompts from the user.

[0509] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[0510] The "feedback means" is a means for transmitting the emotional information analyzed by the emotion recognition means to the generative AI model.

[0511] The present invention provides a system including a user authentication means, an operation means, an emotion recognition means, and a feedback means. A specific embodiment of this system will be described below.

[0512] First, the user places their finger on the accessory device's fingerprint sensor. The fingerprint sensor captures the user's fingerprint data and sends it to the device. The device then sends this data to the server, which matches the fingerprint data with an existing database to authenticate the user. If authentication is successful, the device's buttons are enabled.

[0513] Next, when the user presses the enabled button, the device launches a dedicated application on the smartphone. This application has the function of inputting prompt sentences to the generative AI model. For example, the prompt sentence "Tell me about today's weather" can be input to the generative AI model.

[0514] Furthermore, when the user speaks to the accessory device, the microphone in the terminal captures voice data. This voice data is sent to the emotion recognition means, which analyzes the tone of the voice data to analyze the user's emotion. The analysis result is sent to the generative AI model through the feedback means. The generative AI model generates an appropriate response based on this emotional information. For example, in response to the prompt "The user seems a little tired. Please generate an encouraging message," the generative AI model generates a response such as "Take a short rest today to refresh yourself."

[0515] This system allows users to easily use generative AI models without operating a smartphone, and improves the user experience by providing appropriate responses based on the user's emotions.

[0516] As a concrete example, when a user places their finger on the fingerprint authentication sensor of an accessory device and presses a button after successful authentication, an application on the smartphone is launched and the following prompt sentence is input into the generative AI model:

[0517] "Tell me about the weather today."

[0518] As another example, when a user speaks to an accessory device saying, "How are you feeling today?", the emotion recognition means recognizes the emotion from the tone of the user's voice and inputs the following prompt sentence into the generative AI model:

[0519] "The user seems a little tired. Please generate an encouraging message."

[0520] This allows users to easily utilize generative AI without having to operate a smartphone.

[0521] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0522] Step 1:

[0523] The user places their finger on the fingerprint sensor.

[0524] Input: User's fingerprint

[0525] Specific operation: When the user places their finger on the fingerprint authentication sensor on the accessory device, the sensor will glow blue.

[0526] Output: Captured fingerprint data

[0527] Step 2:

[0528] The device captures the fingerprint data and sends it to the server.

[0529] Input: Captured fingerprint data

[0530] Specific operation: The device captures fingerprint data from the fingerprint authentication sensor and when it sends the data to the server, the device screen displays "Authentication in progress..."

[0531] Output: Fingerprint data sent to the server

[0532] Step 3:

[0533] The server authenticates the fingerprint data.

[0534] Input: Fingerprint data sent to the server

[0535] Specific operation: The server compares the received fingerprint data with an existing database and authenticates the user.

[0536] Output: Authentication result (success or failure)

[0537] Step 4:

[0538] If authentication is successful, the device's buttons are enabled.

[0539] Input: Authentication result (success)

[0540] Specific operation: If the server is successful in authentication, it sends a signal to the terminal to activate the button. The button on the terminal lights up green.

[0541] Output: Enabled button

[0542] Step 5:

[0543] The user presses a button.

[0544] Input: Enabled button

[0545] Specific Action: The user presses an enabled button, which triggers the next step.

[0546] Output: Button press signal

[0547] Step 6:

[0548] The device launches the application on the smartphone.

[0549] Input: Button press signal

[0550] Specific operation: The device launches a dedicated application on the smartphone, and the application logo appears on the smartphone screen.

[0551] Output: Launched application

[0552] Step 7:

[0553] The application prompts input to the generative AI model.

[0554] Input: Launched application

[0555] Specific operation: The application inputs a prompt sentence to the generative AI model. The screen displays "Sending prompt...". For example, the prompt sentence is "Tell me about the weather today."

[0556] Output: The prompt sent to the generative AI model

[0557] Step 8:

[0558] The user speaks to the accessory device.

[0559] Input: User's voice

[0560] Specific Actions: The user speaks to the accessory device, for example, "How are you feeling today?" The microphone on the device lights up red, indicating that it is capturing voice data.

[0561] Output: Captured audio data

[0562] Step 9:

[0563] The device captures the voice data and sends it to the emotion engine.

[0564] Input: Captured audio data

[0565] Specific operation: The device uses the built-in microphone to capture voice data and sends the data to the emotion engine.

[0566] Output: Audio data sent to the emotion engine

[0567] Step 10:

[0568] The emotion engine analyzes the user's emotions from the tone of the voice data.

[0569] Input: Audio data sent to the emotion engine

[0570] Specific operation: The emotion engine analyzes the tone of the voice data and analyzes the user's emotion. The device screen displays "Analyzing emotion..."

[0571] Output: Analyzed emotion information

[0572] Step 11:

[0573] The emotion engine feeds the analysis results back into the generative AI model.

[0574] Input: Analyzed emotion information

[0575] Specific operation: The emotion engine feeds the analysis results back to the generative AI model, for example, sending information such as "The user seems a little tired."

[0576] Output: Emotion information sent to the generative AI model

[0577] Step 12:

[0578] The generative AI model generates responses based on the user's emotions.

[0579] Input: Emotion information sent to the generative AI model

[0580] Specific operation: The generative AI model generates an appropriate response based on the emotional feedback. For example, in response to the prompt "Generate an encouraging message," it generates the response "Take a break today and refresh yourself." The response is displayed on the smartphone screen.

[0581] Output: The generated response

[0582] (Application example 2)

[0583] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0584] In conventional food delivery services, when users order using their smartphones, the operation is complicated and the service does not provide appropriate suggestions based on the user's emotions. In particular, there is a problem that the user experience is not improved for tired or stressed users, as there are not enough suggestions for relaxing meals.

[0585] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0586] In this invention, the server includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI, a voice capture means for capturing the user's voice and recognizing their emotion, an emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the user's emotion, and a means for the generative AI to generate an appropriate response based on the emotion recognized by the emotion recognition means. This enables the user to receive appropriate food delivery suggestions through voice and emotion recognition without operating a smartphone.

[0587] "Generative AI" is an artificial intelligence system that generates appropriate responses and suggestions based on user input.

[0588] "Prompt input" refers to the act of inputting instructions or questions to generative AI.

[0589] An "accessory device" is an external device that can be connected to a smartphone and that provides a specific function.

[0590] "User authentication means" means a means for verifying the identity of a user, ensuring that only specific users can operate the system.

[0591] The "fingerprint authentication means" is a means for reading a user's fingerprint and performing authentication.

[0592] "Operation means" refers to means such as interfaces and buttons that allow users to operate the system.

[0593] A "one-click operation means" is a means that allows a user to complete an operation with a single click.

[0594] "Audio capture means" means for recording or capturing a user's voice.

[0595] The "emotion recognition means" is a means for analyzing the captured voice data and recognizing the user's emotion.

[0596] "Response" refers to the response or suggestion that a generative AI generates in response to user input.

[0597] A system for implementing this invention is configured as follows: First, a user uses an accessory device connected to a smartphone. This accessory device is equipped with a fingerprint authentication means, and authentication is performed by the user placing their finger on it. If authentication is successful, the operation means of the accessory device is enabled, and the user can use the one-click operation means to input prompts to the generative AI.

[0598] Next, the voice capture means captures the user's voice. This voice data is sent to the emotion recognition means, which analyzes the user's emotions. The emotion recognition means recognizes the user's emotions from the tone and content of the voice data and feeds the results back to the generative AI. The generative AI generates an appropriate response based on the recognized emotions.

[0599] Specifically, the following hardware and software are used: A fingerprint sensor is used as the fingerprint authentication method; A microphone and voice recognition software (e.g., speech_recognition library) are used as the voice capture method; An emotion engine (e.g., EmotionEngine) is used as the emotion recognition method; and An AI generation model (e.g., AIGenerator) is used as the generative AI.

[0600] For example, if a user says, "I'm tired today," the voice capture means captures the voice, and the emotion recognition means detects "fatigue." Based on this emotion, the generative AI generates a response suggesting a "relaxing meal." In this way, appropriate food delivery suggestions are made according to the user's emotions.

[0601] Examples of prompts to input to a generative AI model include:

[0602] User Emotion: Fatigue

[0603] Prompt for generative AI: "Suggest a relaxing meal for a user who is tired."

[0604] This system allows users to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate a smartphone.

[0605] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0606] Step 1:

[0607] The user places their finger on the fingerprint authentication sensor of an accessory device connected to a smartphone. The input is the user's fingerprint data, and the output is the authentication result. The accessory device reads the fingerprint data and compares it with its internal database for authentication. If authentication is successful, the device proceeds to the next step.

[0608] Step 2:

[0609] If authentication is successful, the operation means of the accessory device is enabled. The user uses the one-click operation means to input a prompt to the generative AI. The input is the user's click operation, and the output is the trigger for the prompt input. The accessory device detects the click operation and starts inputting a prompt to the generative AI.

[0610] Step 3:

[0611] The device's voice capture means captures the user's voice. The input is the user's voice data and the output is the captured voice data. The device's microphone records the user's voice and voice recognition software converts the data into a digital format.

[0612] Step 4:

[0613] The captured voice data is sent to an emotion recognition means. The input is the captured voice data and the output is the recognized emotion data. The emotion engine analyzes the voice data and recognizes the user's emotion from the tone and content.

[0614] Step 5:

[0615] The recognized emotional data is fed back to the generative AI. The input is the recognized emotional data, and the output is a prompt to the generative AI. The generative AI then creates a prompt based on the emotional data to generate an appropriate response.

[0616] Step 6:

[0617] The generative AI generates an appropriate response based on the prompt. The input is the prompt and the output is the generated response. The generative AI analyzes the prompt and generates food delivery suggestions based on the user's emotions.

[0618] Step 7:

[0619] The generated response is provided to the user. The input is the generated response, and the output is a suggestion to the user. Appropriate food delivery suggestions are displayed or spoken to the user through the terminal's display or audio output device.

[0620] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0621] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generative AIs. Data generation models 58

[0622] , obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in the data format of voice data, text data, etc. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0623] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0624] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0625] [Second embodiment]

[0626] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0627] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0628] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0629] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0630] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0631] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0632] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0633] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0634] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0635] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0636] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0637] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0638] "Example 1"

[0639] One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. Specifically, the accessory device is connected to the smartphone using wireless communication means such as Bluetooth, and prompt input to the generative AI is performed through an application running on the smartphone. The user authentication means includes a fingerprint authentication sensor built into the device, which performs authentication by comparing the user's fingerprint information with registered user information. The operation means includes a button on the device that the user presses to input prompts to the generative AI.

[0640] "Example 2"

[0641] In a specific example of use, a user authenticates by placing their finger on the fingerprint authentication sensor of the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is sent to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

[0642] The processing flow of each embodiment will be described below.

[0643] "Example 1"

[0644] Step 1: A user connects an accessory device connectable to a smartphone to the smartphone using a wireless communication means such as Bluetooth.

[0645] Step 2: The user places their finger on the fingerprint authentication sensor of the accessory device, and the fingerprint authentication sensor authenticates the user by matching it with the fingerprint information of the registered user.

[0646] Step 3: If authentication is successful, the device's buttons are enabled.

[0647] Step 4: When the user presses a button on the device, an application on the smartphone is launched, prompting the generative AI to input the desired information.

[0648] "Example 2"

[0649] Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

[0650] Step 2: If authentication is successful, the device's buttons are enabled.

[0651] Step 3: When the user presses the button, an application on the smartphone is launched and prompts the generative AI to input the desired information.

[0652] Step 4: This allows users to easily utilize generative AI without having to operate their smartphones.

[0653] Example 1

[0654] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0655] In conventional systems using generative AI models, users are required to perform complex operations and undergo multi-step authentication when entering prompts, resulting in a poor user experience. Furthermore, there is a lack of efficient means for quickly and accurately obtaining generated results. This reduces the convenience of using generative AI models.

[0656] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0657] In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for running an application on the smartphone for inputting prompts to the generative AI; and means for sending prompts to the generative AI model and receiving generation results. This allows the user to input prompts to the generative AI model with simple operations and quickly and accurately obtain generation results.

[0658] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer, and can be used to install applications and utilize a variety of functions.

[0659] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[0660] "Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompts.

[0661] "Prompt input" is the act of inputting instructions or questions to a generative AI.

[0662] "User authentication means" refers to technology or devices used to verify a user's identity.

[0663] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and verifies the user's identity based on that information.

[0664] The "operation means" refers to an interface or device that allows a user to operate an accessory device or a smartphone.

[0665] A "one-click operation means" is an interface or device that allows a user to perform a specific function with a single click.

[0666] An "application" is software that runs on a smartphone and provides specific functions or services.

[0667] "Generation results" are output such as text and images generated by the generative AI based on the prompt text.

[0668] MODE FOR CARRYING OUT THE INVENTION

[0669] This invention is a system that uses an accessory device connectable to a smartphone to easily input prompts to a generative AI model. Specific embodiments of this system are described below.

[0670] Hardware and Software Configuration

[0671] Accessory equipment:

[0672] An accessory device is an external device that connects to a smartphone and has the following main functions:

[0673] User authentication method: Equipped with a fingerprint authentication sensor that reads the user's fingerprint information.

[0674] Operation: Equipped with a one-click button for easy user operation.

[0675] Smartphone:

[0676] The smartphone is connected to the accessory device via wireless communication means such as Bluetooth, and an application for providing prompt input to the generative AI model is installed on the smartphone.

[0677] Generative AI models:

[0678] The generative AI model runs on a server and generates text and images based on prompts sent from a smartphone.

[0679] System Operation

[0680] User authentication:

[0681] The user places their finger on the fingerprint authentication sensor installed in the accessory device. The accessory device reads the fingerprint information and compares it with fingerprint information stored in its internal database. If authentication is successful, the accessory device sends a signal indicating successful authentication to the smartphone.

[0682] Prompt Input:

[0683] After successful authentication, the user presses a button on the accessory device, which launches the smartphone application and displays a prompt input screen. The user then inputs a prompt for the generative AI model.

[0684] Send to generative AI model:

[0685] The smartphone sends the input prompt to the generative AI model, and the server operates the generative AI model based on the received prompt to generate the generated results.

[0686] View the generated results:

[0687] The server sends the generated results to the smartphone, which then displays them to the user. The user can then check the displayed results and save or share them as needed.

[0688] Specific examples

[0689] If a user wants to have a generative AI create a new poem, the specific steps are as follows:

[0690] 1. User authentication:

[0691] The user places their finger on the fingerprint authentication sensor of the accessory device.

[0692] The accessory device reads the fingerprint and performs authentication.

[0693] If the authentication is successful, a signal indicating successful authentication is sent to the smartphone.

[0694] 2. Prompt input:

[0695] The user presses a button on the accessory device.

[0696] The smartphone application will launch and a prompt input screen will appear.

[0697] The user enters the prompt text "Create a new poem."

[0698] 3. Send to generative AI model:

[0699] The smartphone sends the input prompt to the generative AI model.

[0700] The server runs a generative AI model based on the prompt text to generate a poem.

[0701] 4. View the generated results:

[0702] The server sends the generated poem to the smartphone.

[0703] The smartphone displays the received poem to the user.

[0704] The user can review the displayed poem and save or share it as desired.

[0705] In this way, a user can use the accessory device and a smartphone to provide prompt input to the generative AI model and obtain the generated results.

[0706] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0707] Step 1: Connect accessory devices

[0708] Input: User opens smartphone Bluetooth settings and selects accessory device.

[0709] How it works: Your smartphone establishes a connection with an accessory device using Bluetooth.

[0710] Output: The smartphone will inform you that the connection was successful.

[0711] Step 2: User authentication

[0712] Input: The user places their finger on the fingerprint sensor on the accessory device.

[0713] How it works: The accessory device reads your fingerprint and matches it with fingerprints stored in an internal database.

[0714] Output: If authentication is successful, the accessory device sends a signal to the smartphone indicating successful authentication.

[0715] Step 3: Prepare for prompt input

[0716] Input: The smartphone receives a signal that authentication was successful.

[0717] How it works: The smartphone displays a prompt input screen for the generative AI.

[0718] Output: The user sees the prompt input screen and begins to enter the prompt statement.

[0719] Step 4: Prompt Input

[0720] Input: The user inputs a prompt for the generative AI through a smartphone application.

[0721] How it works: The smartphone receives the input prompt text and formats it internally.

[0722] Output: The prepared prompt is ready to be sent to the generative AI model.

[0723] Step 5: Send to generative AI model

[0724] Input: The smartphone sends a prepared prompt to the generative AI model.

[0725] Operation: The server runs the generative AI model based on the received prompt sentence and creates a generated result.

[0726] Output: The generated results are sent from the server to the smartphone.

[0727] Step 6: View the generated results

[0728] Input: The smartphone receives the generated results from the server.

[0729] How it works: The smartphone displays the generated results to the user.

[0730] Output: The user checks the displayed generated results and saves or shares them as needed.

[0731] (Application example 1)

[0732] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0733] To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and lack sufficient security for user authentication, which creates the risk of operational errors and unauthorized access. This can lead to reduced productivity and safety in factories.

[0734] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0735] In this invention, the server is an accessory device connectable to a smartphone and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI, a means for generating instructions for managing and operating robots in a factory, and a means for transmitting the instructions to the robots via wireless communication means, thereby enabling efficient and safe management and operation of robots in a factory.

[0736] An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

[0737] "Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

[0738] "Means to enable prompt input" refers to a function that provides an interface for users to input instructions or questions to generative AI.

[0739] "User authentication means" is a function for confirming and authenticating the identity of a user.

[0740] "Fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

[0741] The "operation means" is an interface that allows the user to input prompts to the generative AI.

[0742] The "one-click operation means" is a function that allows users to input prompts to generative AI with a single click.

[0743] "Means for generating instructions for managing and operating robots in a factory" refers to a function for generating specific work instructions for robots in a factory.

[0744] "Wireless communication means" refers to a function for sending and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

[0745] The "means for transmitting to the robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

[0746] This invention is a system for efficiently and safely managing and operating robots in a factory using an accessory device connectable to a smartphone. Specific embodiments of this system are described below.

[0747] System Configuration

[0748] The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

[0749] Hardware

[0750] Smartphone: Functions as a user interface and connects to accessory devices via wireless communication means.

[0751] Accessory device: Connected to a smartphone, it includes a fingerprint authentication means and an operation means.

[0752] Fingerprint authentication method: Scans the user's fingerprint for authentication.

[0753] Wireless communication means: Data is sent and received between the smartphone and the factory robot using Bluetooth, Wi-Fi, etc.

[0754] Factory robot: Performs actions according to instructions.

[0755] software

[0756] Generative AI model: Generates specific instructions for factory robots based on user prompts.

[0757] Smartphone application: Provides the user interface and manages communication with the generative AI model.

[0758] Operating procedure

[0759] 1. User authentication: The user places a finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

[0760] 2. Prompt input: An authenticated user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[0761] 3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

[0762] 4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

[0763] 5. Robot operation: The factory robot performs the operation according to the received instructions.

[0764] Specific examples

[0765] For example, when a robot performs a specific task in a factory, a manager uses a smartphone and accessory devices to give instructions to the robot. The following is an example of a prompt sentence:

[0766] "Move to assembly line 3 and begin welding."

[0767] "Move to the inspection area and perform quality checks."

[0768] "Go get part A and take it to assembly line 2."

[0769] This enables efficient and safe management and operation of robots in factories. The system utilizes user authentication and generative AI to reduce the risk of misoperation and unauthorized access, improving factory productivity and safety.

[0770] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0771] Step 1:

[0772] The user places a finger on the accessory device and is authenticated by the fingerprint authentication means.

[0773] Input: User's fingerprint

[0774] Data processing: The fingerprint authentication means scans the fingerprint and compares it with registered fingerprint data.

[0775] Output: Authentication result (success or failure)

[0776] How it works: The fingerprint sensor scans the user's fingerprint and matches it with an internal database for authentication.

[0777] Step 2:

[0778] If the authentication is successful, the user uses the operation means (one-click operation means) to input a prompt to the generative AI model.

[0779] Input: Authentication result (success), user prompt

[0780] Data processing: If the authentication result is successful, send the prompt text to the generative AI model.

[0781] Output: prompt input to the generative AI model

[0782] Specific operation: When the user presses the button, the smartphone application sends a prompt sentence to the generative AI model.

[0783] Step 3:

[0784] Based on the prompts, a generative AI model generates specific instructions for the factory robots.

[0785] Input: User prompt text

[0786] Data processing: A generative AI model analyzes the prompt and generates appropriate instructions.

[0787] Output: Instructions to factory robots

[0788] Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

[0789] Step 4:

[0790] The generated instructions are transmitted to the factory robot using wireless communication means.

[0791] Input: Instructions from a generative AI model

[0792] Data processing: Instructions are sent to the factory robot via wireless communication means.

[0793] Output: Sending instructions to factory robots

[0794] Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth or Wi-Fi.

[0795] Step 5:

[0796] The factory robot operates according to the received instructions.

[0797] Input: Instructions to the factory robot

[0798] Data processing: Factory robots analyze instructions and perform corresponding actions.

[0799] Output: Robot movement

[0800] Specific operation: The factory robot operates according to instructions such as "move to assembly line 3 and start welding."

[0801] Example 2

[0802] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0803] In order to use conventional generative AI models, users had to operate their smart devices directly and input prompts. This operation was time-consuming and inconvenient, especially when their hands were full or in situations where a quick response was required. Furthermore, from a security perspective, user authentication was sometimes insufficient, creating the risk of unauthorized use by third parties. There is a need for a system that can solve these issues and make it easier and safer to use generative AI models.

[0804] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0805] In this invention, the server includes a user authentication means, an operation means for operation by an authenticated user, a means for sending a signal to the smart device using the operation means to launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without directly operating the smart device, and the user authentication means also improves security.

[0806] "User authentication means" means a means for verifying a user's identity and granting access to a system.

[0807] "Operation means" refers to the interface or device that allows the user to operate the system.

[0808] A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

[0809] The "means for transmitting a signal" is a means for transmitting specific instructions or data from the operating means to the smart device.

[0810] A "specific application" is a software program designed to provide a particular function or service.

[0811] A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

[0812] A "prompt" is a text input used to give instructions or ask questions to a generative AI model.

[0813] A "means for generating a response" is a means by which a generative AI model creates an appropriate response based on a prompt sentence.

[0814] The "means for sending a response" is a means for sending the generated response to the smart device.

[0815] MODE FOR CARRYING OUT THE INVENTION

[0816] This invention provides a system for utilizing generative AI models without requiring a user to directly operate a smart device. Specific embodiments of this system are described below.

[0817] Hardware and software used

[0818] Hardware: Accessory devices (with fingerprint authentication sensors), smart devices (smartphones, etc.)

[0819] Software: Fingerprint authentication system, smart device applications, generative AI models

[0820] System configuration

[0821] 1. User authentication method:

[0822] The user places their finger on the fingerprint sensor on the accessory device.

[0823] The fingerprint authentication system captures fingerprint data and authenticates the user.

[0824] 2. Operating means:

[0825] If authentication is successful, the buttons on the accessory device are enabled.

[0826] The user presses an enabled button.

[0827] 3. Signal transmission means:

[0828] When the button is pressed, the accessory device sends a signal to the smart device.

[0829] The smart device receives the signal and launches a specific application.

[0830] 4. Prompt sending method:

[0831] The launched application sends a prompt to the generative AI model.

[0832] Prompts are pre-defined and contain instructions or questions that the user wants to ask the generative AI model.

[0833] 5. Response Generation Method:

[0834] The server uses a generative AI model to generate a response based on the received prompt.

[0835] 6. Response sending method:

[0836] The server generates a response and sends it to the smart device.

[0837] The smart device receives the response and displays it to the user.

[0838] Specific examples

[0839] When a user places their finger on the fingerprint authentication sensor of the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device to launch an application. The application launches and sends a prompt sentence, "Tell me what the weather is today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can see the information, "Today's weather is sunny," on the screen.

[0840] Prompt Sentence Examples

[0841] "Tell me about today's weather."

[0842] "Check the schedule for the next meeting."

[0843] "Tell me the latest news"

[0844] This system allows users to easily use generated AI models without directly operating their smart devices, and also improves security through user authentication measures.

[0845] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0846] Program processing flow

[0847] Step 1:

[0848] The user places their finger on the fingerprint authentication sensor on the accessory device.

[0849] Input: User's fingerprint

[0850] Data processing: The fingerprint sensor captures fingerprint data.

[0851] Output: Captured fingerprint data

[0852] Step 2:

[0853] The device captures the fingerprint data and performs authentication.

[0854] Input: Captured fingerprint data

[0855] Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

[0856] Output: Authentication result (success or failure)

[0857] Step 3:

[0858] If authentication is successful, the device's buttons will be enabled.

[0859] Input: Authentication result (success)

[0860] Data processing: If authentication is successful, activate the device button.

[0861] Output: Enabled button

[0862] Step 4:

[0863] The user presses a button.

[0864] Input: Enabled button

[0865] Data processing: The device detects that a button has been pressed.

[0866] Output: Button press signal

[0867] Step 5:

[0868] The terminal sends a signal to the smart device to launch the application.

[0869] Input: Button press signal

[0870] Data processing: The terminal sends a signal to the smart device to launch a specific application.

[0871] Output: Launched application

[0872] Step 6:

[0873] The application sends a prompt to the generative AI model.

[0874] Input: Launched application

[0875] Data processing: The application sends pre-defined prompts to the generative AI model.

[0876] Output: The prompt sent to the generative AI model

[0877] Step 7:

[0878] The server generates a response using a generative AI model.

[0879] Input: The prompt sent to the generative AI model

[0880] Data computation: A generative AI model analyzes the prompt and generates an appropriate response.

[0881] Output: The generated response

[0882] Step 8:

[0883] The server sends the response to the smart device.

[0884] Input: The generated response

[0885] Data processing: The server generates a response and sends it to the smart device.

[0886] Output: Response sent to smart device

[0887] Step 9:

[0888] The smart device displays the response.

[0889] Input: Response sent to smart device

[0890] Data processing: The smart device displays the received response to the user.

[0891] Output: The response displayed to the user

[0892] In this way, users can utilize generative AI models without directly operating their smart devices.

[0893] (Application example 2)

[0894] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0895] Conventional food delivery systems require users to use their smartphones to place orders, which makes operation cumbersome. In addition, there are security risks due to insufficient user authentication. Furthermore, ordering systems using voice recognition are not widespread, resulting in low user convenience. There is a need to provide a system that solves these problems and allows users to use food delivery services easily and safely.

[0896] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0897] An accessory device connectable to a smart device includes a user authentication means for authenticating a user operating the accessory device through fingerprint authentication, a button provided on the accessory device that a user authenticated by the user authentication means presses to input instructions to the generative AI, and a voice recognition means for recognizing the user's voice in response to the button being pressed. The server includes a means for acquiring the output of the generative AI using a prompt statement including voice data acquired by the voice recognition means and the generative AI. The prompt statement instructs the user to place an order through a food delivery service, and the acquiring means acquires the order results using a food delivery API. This allows users to easily and safely use food delivery services using only their voice, without having to operate their smartphone.

[0898] A "smart device" is a portable electronic device, such as a smartphone or tablet, that has Internet connectivity and can run applications.

[0899] An "accessory device" is additional hardware that is connected to a smart device to provide specific functionality.

[0900] "Generative AI" refers to artificial intelligence techniques that generate text and other content based on user instructions or prompts.

[0901] "Instruction input" refers to commands or requests that a user inputs to a generative AI.

[0902] "User authentication means" refers to technology or devices for verifying and authenticating the identity of a user.

[0903] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and authenticates the user based on that information.

[0904] "Operation means" refers to the interface and buttons that allow the user to operate the accessory device or smart device.

[0905] "Speech recognition means" refers to technology or devices that analyze a user's voice and convert it into text data.

[0906] "Voice data" refers to data that is a digital recording of a user's voice.

[0907] "Transmission means" refers to the technology or devices used to transmit data to other devices or systems.

[0908] This invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. Specific embodiments of this system are described below.

[0909] System configuration

[0910] The system consists of the following main components:

[0911] 1. Smart Device: A portable electronic device, such as a smartphone or tablet, that has internet connectivity and can run applications.

[0912] 2. Accessory device: Additional hardware that is connected to a smart device and includes fingerprint authentication means, operation means, and voice recognition means.

[0913] 3. Generative AI: Artificial intelligence techniques that generate text and other content based on user instructions or prompts.

[0914] Program processing

[0915] Hardware

[0916] Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

[0917] Smart device: Functions as the execution environment for applications.

[0918] Microphone: Captures the user's voice as a means of voice recognition.

[0919] software

[0920] Fingerprint authentication library: A software module for performing fingerprint authentication.

[0921] Speech Recognition Library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

[0922] Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3®).

[0923] Food Delivery API: Place orders using the API of food delivery services.

[0924] Processing flow

[0925] 1. User authentication: The user authenticates by placing their finger on the fingerprint authentication sensor of the accessory device. If authentication is successful, the next step is performed.

[0926] 2. Voice input: The user presses the operation means (one-click button) of the accessory device and speaks the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

[0927] 3. Prompt generation: Based on the acquired text data, a prompt sentence is generated for the generative AI.

[0928] 4. AI response generation: The generative AI model generates the optimal response based on the prompt.

[0929] 5. Order fulfillment: Based on the generated response, the order is completed using the food delivery API.

[0930] Specific examples

[0931] When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model receives the prompt "The user wants to order: pizza," the model then suggests the best pizza restaurant and menu item, and completes the order using a food delivery API.

[0932] Prompt Sentence Examples

[0933] "User wants to order: Pizza"

[0934] In this way, users can easily and safely use food delivery services using only their voice, without having to operate their smartphones.

[0935] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0936] Step 1:

[0937] The user places their finger on the fingerprint sensor.

[0938] Input: User's fingerprint data.

[0939] Processing: The fingerprint authentication library analyzes the fingerprint data and matches it with registered fingerprint data.

[0940] Output: Authentication result (success or failure).

[0941] Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

[0942] Step 2:

[0943] The user presses the operating means (one-click button) of the accessory device.

[0944] Input: User action (button press).

[0945] Processing: Detects that the operating means has been pressed and activates the voice recognition mode.

[0946] Output: Activate speech recognition mode.

[0947] Specific operation: The sensor detects that a button has been pressed, and the speech recognition library begins preparing for voice input.

[0948] Step 3:

[0949] The user verbally states the order details.

[0950] Input: User's voice data.

[0951] Processing: The microphone captures the voice data, which is then converted into text data by a speech recognition library.

[0952] Output: Text data (order details).

[0953] What it does: A microphone captures the user's voice, and a speech recognition library analyzes it and converts it into text.

[0954] Step 4:

[0955] The terminal generates a prompt sentence based on the acquired text data.

[0956] Input: Text data (order details).

[0957] Processing: Convert the text data into a prompt sentence of the form "The user wants to order: [Order details]".

[0958] Output: The prompt statement.

[0959] Specific operation: Converts text data into prompt sentences according to the format.

[0960] Step 5:

[0961] The device sends a prompt to the generative AI model.

[0962] Input: prompt statement.

[0963] Processing: A generative AI model analyzes the prompt and generates the optimal response.

[0964] Output: AI response (optimal order content).

[0965] Specific operation: The prompt sentence is sent to the generative AI model, and the AI ​​generates a response.

[0966] Step 6:

[0967] The device uses the generated response to complete the order using the food delivery API.

[0968] Input: AI response (optimal order content).

[0969] Processing: Fulfilling orders using food delivery APIs.

[0970] Output: Order confirmation (data indicating the order was successfully completed).

[0971] Specific behavior: Sends AI response to food delivery API and confirms order.

[0972] The server may further include means for recognizing the user's emotion using an emotion engine (described later), and may obtain an output from the generative AI using the voice data obtained by the voice recognition means, a prompt including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," the server may generate a prompt such as "User wants to order: pizza, user's emotion: sadness," and the generative AI model may suggest an optimal pizza restaurant and menu item taking the user's emotion into consideration, and complete the order using a food delivery API.

[0973] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0974] "Example 1"

[0975] One embodiment of the present invention provides an accessory device connectable to a smartphone. The device includes a means for enabling prompt input to a generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. The device also includes an emotion engine that recognizes the user's emotions. Specifically, the emotion engine recognizes the user's emotions from their tone of voice and facial expression, and feeds that information back to the generative AI. This enables the generative AI to generate an appropriate response based on the user's emotions.

[0976] "Example 2"

[0977] Another embodiment of the present invention provides a system in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone within the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotion from the tone of the voice data and feeds the results back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotion.

[0978] The processing flow of each embodiment will be described below.

[0979] "Example 1"

[0980] Step 1: The user operates an accessory device that can be connected to a smartphone.

[0981] Step 2: The user authentication means of the device authenticates the user. One possible authentication method is fingerprint authentication.

[0982] Step 3: The authenticated user prompts the generative AI.

[0983] Step 4: The device's emotion engine recognizes the user's emotions from their tone of voice and facial expressions.

[0984] Step 5: The emotion engine feeds that information back to the generative AI.

[0985] Step 6: The generative AI generates an appropriate response based on the user's emotions.

[0986] "Example 2"

[0987] Step 1: The user speaks to the accessory device.

[0988] Step 2: The microphone in the device captures the user's voice and sends the audio data to the emotion engine.

[0989] Step 3: The emotion engine analyzes the user's emotions from the tone of the voice data.

[0990] Step 4: The emotion engine feeds the results back to the generative AI.

[0991] Step 5: The generative AI generates an appropriate response based on the user's emotions.

[0992] Example 1

[0993] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0994] While conventional accessory devices that can be connected to smartphones allow prompt input to generative AI models, they have the problem of being unable to generate responses that take the user's emotional state into account. This makes it difficult to obtain appropriate responses that correspond to the user's intentions and emotions, resulting in a lack of improvement in the user experience.

[0995] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0996] In this invention, the server is an accessory device connectable to a smartphone, and includes: means for enabling prompt input to the generative AI model; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI model; emotion recognition means for recognizing the user's emotion; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the user's emotional state.

[0997] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer.

[0998] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[0999] A "generative AI model" is an artificial intelligence system that generates output such as text or speech based on input prompts.

[1000] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[1001] "User Authentication Measures" means technical measures for verifying a user's identity.

[1002] A "fingerprint authentication means" is a technical means for reading a user's fingerprint and authenticating them based on that information.

[1003] The "operation means" is an interface that allows the user to operate the accessory device.

[1004] A "one-click operation means" is an interface that allows a user to complete an operation with a single click.

[1005] "Emotion recognition means" refers to a technical means for recognizing emotions by analyzing the user's tone of voice and facial expressions.

[1006] "Emotion information" is data relating to the user's emotional state obtained by emotion recognition means.

[1007] "Feedback means" refers to the technical means for transmitting acquired emotional information to the generative AI model.

[1008] MODE FOR CARRYING OUT THE INVENTION

[1009] This invention is a system that uses an accessory device connectable to a smartphone to provide prompt input to a generative AI model and obtain a response that takes into account the user's emotional state. Specific embodiments of this system are described below.

[1010] Hardware and Software Configuration

[1011] Accessory Devices

[1012] The accessory device connects to the smartphone using wireless communication means such as Bluetooth, and includes the following main components:

[1013] Fingerprint authentication sensor: Reads the user's fingerprint for authentication.

[1014] Action button: The interface that allows the user to initiate prompt input.

[1015] Emotion recognition engine: Analyzes the user's tone of voice and facial expressions to obtain emotional information.

[1016] Smartphone

[1017] The smartphone runs a dedicated application that works in conjunction with the accessory device and provides the following functions:

[1018] Prompt input interface: A screen where users can input instructions or questions to a generative AI model.

[1019] Data transmission function: Sends prompts and emotion information to the server.

[1020] server

[1021] The server provides back-end services that communicate data between smartphones and generative AI models. The server has the following functions:

[1022] Data relay function: Sends prompts and emotional information received from the smartphone to the generative AI model.

[1023] Response reception function: Receives responses from the generative AI model and forwards them to the smartphone.

[1024] Specific examples

[1025] Consider a scenario in which a user uses an accessory device connected to a smartphone to ask a generative AI model a question.

[1026] 1. User Authentication

[1027] The user places their finger on the accessory device's fingerprint authentication sensor to authenticate. The device (smartphone) receives the fingerprint data from the accessory device and compares it with registered fingerprint data. If authentication is successful, the user can proceed to the next step.

[1028] 2. Prompt Input

[1029] The user opens a dedicated application on their smartphone and enters a prompt, for example, "What's the weather like today?" The device prepares to send this prompt to the generative AI model.

[1030] 3. Emotion recognition

[1031] The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions, and the device prepares this emotion data for transmission to a generative AI model.

[1032] 4. Sending prompts to the generative AI model

[1033] The device sends the prompt and emotion data entered by the user to the server, which then sends this data to the generative AI model, which then generates an appropriate response based on the prompt and emotion data.

[1034] 5. Receiving a response from the generative AI model

[1035] The server receives the response from the generative AI model and forwards it to the device, which prepares the response for display to the user.

[1036] 6. Displaying the response

[1037] The device displays the response received from the generative AI model to the user, for example, "It's sunny today. Have a nice day." The user can review this response and enter further prompts as needed.

[1038] Prompt Sentence Examples

[1039] "How's the weather today?"

[1040] "Please let me know when the next meeting is scheduled."

[1041] "Please tell me the latest news."

[1042] In this way, users can naturally interact with generative AI models through accessory devices.

[1043] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1044] Step 1: User authentication

[1045] The user places their finger on the fingerprint authentication sensor of the accessory device. The terminal receives the fingerprint data from the accessory device and compares it with the registered fingerprint data. The input is the user's fingerprint data, and the output is the authentication result. The terminal analyzes the fingerprint data and checks whether it matches the registered data. If authentication is successful, the terminal displays "Authentication successful."

[1046] Step 2: Connect your accessory device to your smartphone

[1047] The terminal establishes a connection with an accessory device via Bluetooth communication. The input is a Bluetooth connection request, and the output is the connection status. The terminal receives a connection request from the accessory device and establishes the connection. The server monitors this connection and relays data as necessary. If the connection is successful, the terminal displays "Connection successful."

[1048] Step 3: Prompt Input

[1049] The user opens a dedicated application on their smartphone and inputs a prompt. For example, they might input "What's the weather like today?" The input is the user's prompt sentence, and the output is the prompt data. The device prepares to send this prompt to the generative AI model. When the user has completed their input, the device displays "Prompt input completed."

[1050] Step 4: Emotion Recognition

[1051] The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions. The input is the user's tone of voice and facial expression data, and the output is emotion data. The device prepares this emotion data to send to the generative AI model. Once the emotion data is acquired, the device displays "Emotion data acquisition successful."

[1052] Step 5: Sending prompts to the generative AI model

[1053] The device sends the prompt and emotion data entered by the user to the server. The input is the prompt data and emotion data, and the output is the result of transmission to the server. The server then sends this data to the generative AI model. If the data transmission is successful, the device displays "Data transmission successful."

[1054] Step 6: Receiving a response from the generative AI model

[1055] The server receives the response from the generative AI model and transfers it to the device. The input is the response data from the generative AI model, and the output is the transfer result to the device. The device prepares to display this response to the user. When the response is received, the device displays "Response received successfully."

[1056] Step 7: View the response

[1057] The device displays the response received from the generative AI model to the user. For example, it displays "It's sunny today. Have a nice day." The input is the response data from the generative AI model, and the output is the display result to the user. The user can review this response and enter further prompts as needed.

[1058] (Application example 1)

[1059] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1060] Traditional food delivery services struggled to provide customized recommendations based on users' emotions and circumstances. They also lacked a means for users to use generative AI to simplify the ordering process. This could result in a poor user experience and reduced frequency of service use.

[1061] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1062] In this invention, the server is an accessory device connectable to a smartphone and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI, an emotion recognition means for recognizing the user's emotions, and a means for feeding back the emotion information recognized by the emotion recognition means to the generative AI. This makes it possible to propose customized food delivery according to the user's emotions.

[1063] An "accessory device connectable to a smartphone" is an external device that is connected to a smartphone using wireless communication means and provides a specific function.

[1064] "Generative AI" is an artificial intelligence system that generates responses and suggestions based on prompt input from the user.

[1065] "Means to enable prompt input" refers to a function that provides an interface for users to input instructions or questions to generative AI.

[1066] "User authentication means" refers to technology for verifying the identity of a user, ensuring that only specific users can operate the system.

[1067] "Fingerprint authentication means" is a technology that scans a user's fingerprint and authenticates the user based on that information.

[1068] "Operation means" refers to the physical or software interface that allows the user to operate the system.

[1069] A "one-click operation means" is an interface that allows a user to perform a specific function with a single click.

[1070] "Emotion recognition means" is a technology that analyzes the user's tone of voice and facial expressions to recognize their emotions.

[1071] "Means for feeding back emotional information to generative AI" refers to a function that sends recognized emotional information to generative AI, allowing the generative AI to generate responses or suggestions based on that information.

[1072] This invention is a system that uses an accessory device connectable to a smartphone to simplify the ordering process when a user uses a food delivery service through generative AI and makes customized suggestions based on the user's emotions.

[1073] System configuration

[1074] The system consists of the following main components:

[1075] 1. Accessory devices that can be connected to smartphones

[1076] Wireless communication means: Connected to a smartphone using Bluetooth or similar.

[1077] User authentication method: A fingerprint sensor is used to authenticate the user.

[1078] Control: A one-click control is used by the user to prompt the generative AI.

[1079] Emotion recognition: Analyzes the user's tone of voice and facial expressions to recognize their emotions.

[1080] 2. Generative AI

[1081] Prompt input: Accepts instructions or questions from the user and generates responses or suggestions.

[1082] Emotional feedback: Generate customized responses and suggestions based on recognized emotional information.

[1083] Program processing

[1084] The server does the following:

[1085] 1. Bluetooth connection: Connect your smartphone and accessory device using Bluetooth.

[1086] 2. User authentication: The fingerprint sensor scans the user's fingerprint for authentication.

[1087] 3. Emotion Recognition: Recognizes emotions by analyzing the user's tone of voice and facial expressions.

[1088] 4. Prompt generation: Generate prompts for the generative AI based on the recognized emotions.

[1089] 5. AI response generation: Use generative AI to generate responses and suggestions to prompts.

[1090] Hardware and software used

[1091] Hardware: Bluetooth module, fingerprint sensor, camera (for face recognition), microphone (for voice recognition)

[1092] Software: Bluetooth connection library, fingerprint authentication library, emotion recognition engine, generative AI model

[1093] Specific examples

[1094] If the user is recognized as tired, the generative AI will suggest "food delivery options for when you're tired." For example, the following prompts will be generated:

[1095] The user is feeling tired. Please suggest food delivery options.

[1096] In this way, it becomes possible to offer customized food delivery suggestions based on the user's emotions.

[1097] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1098] Step 1:

[1099] The server connects the smartphone and the accessory device using a Bluetooth module. The input is the pairing information between the smartphone and the accessory device, and the output is the establishment of the connection. Specifically, the server uses the Bluetooth connection library to establish communication between the smartphone and the accessory device.

[1100] Step 2:

[1101] The device scans the user's fingerprint using a fingerprint authentication sensor to perform user authentication. The input is the user's fingerprint data, and the output is the authentication result. Specifically, the fingerprint authentication library is used to compare the scanned fingerprint data with registered data to determine whether authentication was successful.

[1102] Step 3:

[1103] The device uses a microphone and camera to capture the user's tone of voice and facial expressions to perform emotion recognition. The input is voice data and image data, and the output is recognized emotion information. Specifically, an emotion recognition engine is used to analyze the voice data and image data to identify the user's emotion.

[1104] Step 4:

[1105] The server generates prompts for the generative AI based on the recognized emotional information. The input is emotional information, and the output is a prompt. Specifically, the server converts the emotional information into text and creates a prompt suitable for the generative AI.

[1106] Step 5:

[1107] The server uses a generative AI model to generate responses and suggestions to prompts. The input is the prompt, and the output is the generated response or suggestion. Specifically, the prompt is input into the generative AI model, and the AI ​​generates the response or suggestion.

[1108] Step 6:

[1109] The terminal presents the generated response or suggestion to the user. The input is the generated response or suggestion, and the output is what is displayed to the user. Specifically, the response or suggestion is displayed on the smartphone display so that the user can check it.

[1110] Example 2

[1111] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1112] Conventional systems using generative AI models require users to operate their smartphones, making them difficult to use. Furthermore, they are unable to generate appropriate responses based on the user's emotions, limiting the user experience. This makes it difficult for users to effectively utilize generative AI models.

[1113] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1114] In this invention, the server includes a user authentication means, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI model, an emotion recognition means for capturing the user's voice data and analyzing the emotions, and a means for feeding back the emotion information analyzed by the emotion recognition means to the generative AI model. This allows the user to easily use the generative AI model without operating a smartphone, and further enables the user to receive an appropriate response according to the user's emotions.

[1115] "User authentication means" refers to a means for verifying a user's identity, and includes biometric authentication technologies such as fingerprint authentication and facial recognition.

[1116] "Operation means" refers to the means by which a user inputs prompts to the generated AI model, and includes interfaces such as buttons and touchscreens.

[1117] An "emotion recognition means" is a means for capturing a user's voice data and analyzing the user's emotions from the tone and content of the data.

[1118] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on prompts from the user.

[1119] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[1120] The "feedback means" is a means for transmitting the emotional information analyzed by the emotion recognition means to the generative AI model.

[1121] The present invention provides a system including a user authentication means, an operation means, an emotion recognition means, and a feedback means. A specific embodiment of this system will be described below.

[1122] First, the user places their finger on the accessory device's fingerprint sensor. The fingerprint sensor captures the user's fingerprint data and sends it to the device. The device then sends this data to the server, which matches the fingerprint data with an existing database to authenticate the user. If authentication is successful, the device's buttons are enabled.

[1123] Next, when the user presses the enabled button, the device launches a dedicated application on the smartphone. This application has the function of inputting prompt sentences to the generative AI model. For example, the prompt sentence "Tell me about today's weather" can be input to the generative AI model.

[1124] Furthermore, when the user speaks to the accessory device, the microphone in the terminal captures voice data. This voice data is sent to the emotion recognition means, which analyzes the tone of the voice data to analyze the user's emotion. The analysis result is sent to the generative AI model through the feedback means. The generative AI model generates an appropriate response based on this emotional information. For example, in response to the prompt "The user seems a little tired. Please generate an encouraging message," the generative AI model generates a response such as "Take a short rest today to refresh yourself."

[1125] This system allows users to easily use generative AI models without operating a smartphone, and improves the user experience by providing appropriate responses based on the user's emotions.

[1126] As a concrete example, when a user places their finger on the fingerprint authentication sensor of an accessory device and presses a button after successful authentication, an application on the smartphone is launched and the following prompt sentence is input into the generative AI model:

[1127] "Tell me about the weather today."

[1128] As another example, when a user speaks to an accessory device saying, "How are you feeling today?", the emotion recognition means recognizes the emotion from the tone of the user's voice and inputs the following prompt sentence into the generative AI model:

[1129] "The user seems a little tired. Please generate an encouraging message."

[1130] This allows users to easily utilize generative AI without having to operate a smartphone.

[1131] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1132] Step 1:

[1133] The user places their finger on the fingerprint sensor.

[1134] Input: User's fingerprint

[1135] Specific operation: When the user places their finger on the fingerprint authentication sensor on the accessory device, the sensor will glow blue.

[1136] Output: Captured fingerprint data

[1137] Step 2:

[1138] The device captures the fingerprint data and sends it to the server.

[1139] Input: Captured fingerprint data

[1140] Specific operation: The device captures fingerprint data from the fingerprint authentication sensor and when it sends the data to the server, the device screen displays "Authentication in progress..."

[1141] Output: Fingerprint data sent to the server

[1142] Step 3:

[1143] The server authenticates the fingerprint data.

[1144] Input: Fingerprint data sent to the server

[1145] Specific operation: The server compares the received fingerprint data with an existing database and authenticates the user.

[1146] Output: Authentication result (success or failure)

[1147] Step 4:

[1148] If authentication is successful, the device's buttons are enabled.

[1149] Input: Authentication result (success)

[1150] Specific operation: If the server is successful in authentication, it sends a signal to the terminal to activate the button. The button on the terminal lights up green.

[1151] Output: Enabled button

[1152] Step 5:

[1153] The user presses a button.

[1154] Input: Enabled button

[1155] Specific Action: The user presses an enabled button, which triggers the next step.

[1156] Output: Button press signal

[1157] Step 6:

[1158] The device launches the application on the smartphone.

[1159] Input: Button press signal

[1160] Specific operation: The device launches a dedicated application on the smartphone, and the application logo appears on the smartphone screen.

[1161] Output: Launched application

[1162] Step 7:

[1163] The application prompts input to the generative AI model.

[1164] Input: Launched application

[1165] Specific operation: The application inputs a prompt sentence to the generative AI model. The screen displays "Sending prompt...". For example, the prompt sentence is "Tell me about the weather today."

[1166] Output: The prompt sent to the generative AI model

[1167] Step 8:

[1168] The user speaks to the accessory device.

[1169] Input: User's voice

[1170] Specific Actions: The user speaks to the accessory device, for example, "How are you feeling today?" The microphone on the device lights up red, indicating that it is capturing voice data.

[1171] Output: Captured audio data

[1172] Step 9:

[1173] The device captures the voice data and sends it to the emotion engine.

[1174] Input: Captured audio data

[1175] Specific operation: The device uses the built-in microphone to capture voice data and sends the data to the emotion engine.

[1176] Output: Audio data sent to the emotion engine

[1177] Step 10:

[1178] The emotion engine analyzes the user's emotions from the tone of the voice data.

[1179] Input: Audio data sent to the emotion engine

[1180] Specific operation: The emotion engine analyzes the tone of the voice data and analyzes the user's emotion. The device screen displays "Analyzing emotion..."

[1181] Output: Analyzed emotion information

[1182] Step 11:

[1183] The emotion engine feeds the analysis results back into the generative AI model.

[1184] Input: Analyzed emotion information

[1185] Specific operation: The emotion engine feeds the analysis results back to the generative AI model, for example, sending information such as "The user seems a little tired."

[1186] Output: Emotion information sent to the generative AI model

[1187] Step 12:

[1188] The generative AI model generates responses based on the user's emotions.

[1189] Input: Emotion information sent to the generative AI model

[1190] Specific operation: The generative AI model generates an appropriate response based on the emotional feedback. For example, in response to the prompt "Generate an encouraging message," it generates the response "Take a break today and refresh yourself." The response is displayed on the smartphone screen.

[1191] Output: The generated response

[1192] (Application example 2)

[1193] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1194] In conventional food delivery services, when users order using their smartphones, the operation is complicated and the service does not provide appropriate suggestions based on the user's emotions. In particular, there is a problem that the user experience is not improved for tired or stressed users, as there are not enough suggestions for relaxing meals.

[1195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1196] In this invention, the server includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI, a voice capture means for capturing the user's voice and recognizing their emotion, an emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the user's emotion, and a means for the generative AI to generate an appropriate response based on the emotion recognized by the emotion recognition means. This enables the user to receive appropriate food delivery suggestions through voice and emotion recognition without operating a smartphone.

[1197] "Generative AI" is an artificial intelligence system that generates appropriate responses and suggestions based on user input.

[1198] "Prompt input" refers to the act of inputting instructions or questions to generative AI.

[1199] An "accessory device" is an external device that can be connected to a smartphone and that provides a specific function.

[1200] "User authentication means" means a means for verifying the identity of a user, ensuring that only specific users can operate the system.

[1201] The "fingerprint authentication means" is a means for reading a user's fingerprint and performing authentication.

[1202] "Operation means" refers to means such as interfaces and buttons that allow users to operate the system.

[1203] A "one-click operation means" is a means that allows a user to complete an operation with a single click.

[1204] "Audio capture means" means for recording or capturing a user's voice.

[1205] The "emotion recognition means" is a means for analyzing the captured voice data and recognizing the user's emotion.

[1206] "Response" refers to the response or suggestion that a generative AI generates in response to user input.

[1207] A system for implementing this invention is configured as follows: First, a user uses an accessory device connected to a smartphone. This accessory device is equipped with a fingerprint authentication means, and authentication is performed by the user placing their finger on it. If authentication is successful, the operation means of the accessory device is enabled, and the user can use the one-click operation means to input prompts to the generative AI.

[1208] Next, the voice capture means captures the user's voice. This voice data is sent to the emotion recognition means, which analyzes the user's emotions. The emotion recognition means recognizes the user's emotions from the tone and content of the voice data and feeds the results back to the generative AI. The generative AI generates an appropriate response based on the recognized emotions.

[1209] Specifically, the following hardware and software are used: A fingerprint sensor is used as the fingerprint authentication method; A microphone and voice recognition software (e.g., speech_recognition library) are used as the voice capture method; An emotion engine (e.g., EmotionEngine) is used as the emotion recognition method; and An AI generation model (e.g., AIGenerator) is used as the generative AI.

[1210] For example, if a user says, "I'm tired today," the voice capture means captures the voice, and the emotion recognition means detects "fatigue." Based on this emotion, the generative AI generates a response suggesting a "relaxing meal." In this way, appropriate food delivery suggestions are made according to the user's emotions.

[1211] Examples of prompts to input to a generative AI model include:

[1212] User Emotion: Fatigue

[1213] Prompt for generative AI: "Suggest a relaxing meal for a user who is tired."

[1214] This system allows users to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate a smartphone.

[1215] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1216] Step 1:

[1217] The user places their finger on the fingerprint authentication sensor of an accessory device connected to a smartphone. The input is the user's fingerprint data, and the output is the authentication result. The accessory device reads the fingerprint data and compares it with its internal database for authentication. If authentication is successful, the device proceeds to the next step.

[1218] Step 2:

[1219] If authentication is successful, the operation means of the accessory device is enabled. The user uses the one-click operation means to input a prompt to the generative AI. The input is the user's click operation, and the output is the trigger for the prompt input. The accessory device detects the click operation and starts inputting a prompt to the generative AI.

[1220] Step 3:

[1221] The device's voice capture means captures the user's voice. The input is the user's voice data and the output is the captured voice data. The device's microphone records the user's voice and voice recognition software converts the data into a digital format.

[1222] Step 4:

[1223] The captured voice data is sent to an emotion recognition means. The input is the captured voice data and the output is the recognized emotion data. The emotion engine analyzes the voice data and recognizes the user's emotion from the tone and content.

[1224] Step 5:

[1225] The recognized emotional data is fed back to the generative AI. The input is the recognized emotional data, and the output is a prompt to the generative AI. The generative AI then creates a prompt based on the emotional data to generate an appropriate response.

[1226] Step 6:

[1227] The generative AI generates an appropriate response based on the prompt. The input is the prompt and the output is the generated response. The generative AI analyzes the prompt and generates food delivery suggestions based on the user's emotions.

[1228] Step 7:

[1229] The generated response is provided to the user. The input is the generated response, and the output is a suggestion to the user. Appropriate food delivery suggestions are displayed or spoken to the user through the terminal's display or audio output device.

[1230] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1231] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1232] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1233] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1234] [Third embodiment]

[1235] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1236] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1237] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1238] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1239] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1240] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1241] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1242] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1243] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1244] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1245] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1246] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1247] "Example 1"

[1248] One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. Specifically, the accessory device is connected to the smartphone using wireless communication means such as Bluetooth, and prompt input to the generative AI is performed through an application running on the smartphone. The user authentication means includes a fingerprint authentication sensor built into the device, which performs authentication by comparing the user's fingerprint information with registered user information. The operation means includes a button on the device that the user presses to input prompts to the generative AI.

[1249] "Example 2"

[1250] In a specific example of use, a user authenticates by placing their finger on the fingerprint authentication sensor of the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is sent to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

[1251] The processing flow of each embodiment will be described below.

[1252] "Example 1"

[1253] Step 1: A user connects an accessory device connectable to a smartphone to the smartphone using a wireless communication means such as Bluetooth.

[1254] Step 2: The user places their finger on the fingerprint authentication sensor of the accessory device, and the fingerprint authentication sensor authenticates the user by matching it with the fingerprint information of the registered user.

[1255] Step 3: If authentication is successful, the device's buttons are enabled.

[1256] Step 4: When the user presses a button on the device, an application on the smartphone is launched, prompting the generative AI to input the desired information.

[1257] "Example 2"

[1258] Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

[1259] Step 2: If authentication is successful, the device's buttons are enabled.

[1260] Step 3: When the user presses the button, an application on the smartphone is launched and prompts the generative AI to input the desired information.

[1261] Step 4: This allows users to easily utilize generative AI without having to operate their smartphones.

[1262] Example 1

[1263] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1264] In conventional systems using generative AI models, users are required to perform complex operations and undergo multi-step authentication when entering prompts, resulting in a poor user experience. Furthermore, there is a lack of efficient means for quickly and accurately obtaining generated results. This reduces the convenience of using generative AI models.

[1265] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1266] In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for running an application on the smartphone for inputting prompts to the generative AI; and means for sending prompts to the generative AI model and receiving generation results. This allows the user to input prompts to the generative AI model with simple operations and quickly and accurately obtain generation results.

[1267] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer, and can be used to install applications and utilize a variety of functions.

[1268] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[1269] "Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompts.

[1270] "Prompt input" is the act of inputting instructions or questions to a generative AI.

[1271] "User authentication means" refers to technology or devices used to verify a user's identity.

[1272] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and verifies the user's identity based on that information.

[1273] The "operation means" refers to an interface or device that allows a user to operate an accessory device or a smartphone.

[1274] A "one-click operation means" is an interface or device that allows a user to perform a specific function with a single click.

[1275] An "application" is software that runs on a smartphone and provides specific functions or services.

[1276] "Generation results" are output such as text and images generated by the generative AI based on the prompt text.

[1277] MODE FOR CARRYING OUT THE INVENTION

[1278] This invention is a system that uses an accessory device connectable to a smartphone to easily input prompts to a generative AI model. Specific embodiments of this system are described below.

[1279] Hardware and Software Configuration

[1280] Accessory equipment:

[1281] An accessory device is an external device that connects to a smartphone and has the following main functions:

[1282] User authentication method: Equipped with a fingerprint authentication sensor that reads the user's fingerprint information.

[1283] Operation: Equipped with a one-click button for easy user operation.

[1284] Smartphone:

[1285] The smartphone is connected to the accessory device via wireless communication means such as Bluetooth, and an application for providing prompt input to the generative AI model is installed on the smartphone.

[1286] Generative AI models:

[1287] The generative AI model runs on a server and generates text and images based on prompts sent from a smartphone.

[1288] System Operation

[1289] User authentication:

[1290] The user places their finger on the fingerprint authentication sensor installed in the accessory device. The accessory device reads the fingerprint information and compares it with fingerprint information stored in its internal database. If authentication is successful, the accessory device sends a signal indicating successful authentication to the smartphone.

[1291] Prompt Input:

[1292] After successful authentication, the user presses a button on the accessory device, which launches the smartphone application and displays a prompt input screen. The user then inputs a prompt for the generative AI model.

[1293] Send to generative AI model:

[1294] The smartphone sends the input prompt to the generative AI model, and the server operates the generative AI model based on the received prompt to generate the generated results.

[1295] View the generated results:

[1296] The server sends the generated results to the smartphone, which then displays them to the user. The user can then check the displayed results and save or share them as needed.

[1297] Specific examples

[1298] If a user wants to have a generative AI create a new poem, the specific steps are as follows:

[1299] 1. User authentication:

[1300] The user places their finger on the fingerprint authentication sensor of the accessory device.

[1301] The accessory device reads the fingerprint and performs authentication.

[1302] If the authentication is successful, a signal indicating successful authentication is sent to the smartphone.

[1303] 2. Prompt input:

[1304] The user presses a button on the accessory device.

[1305] The smartphone application will launch and a prompt input screen will appear.

[1306] The user enters the prompt text "Create a new poem."

[1307] 3. Send to generative AI model:

[1308] The smartphone sends the input prompt to the generative AI model.

[1309] The server runs a generative AI model based on the prompt text to generate a poem.

[1310] 4. View the generated results:

[1311] The server sends the generated poem to the smartphone.

[1312] The smartphone displays the received poem to the user.

[1313] The user can review the displayed poem and save or share it as desired.

[1314] In this way, a user can use the accessory device and a smartphone to provide prompt input to the generative AI model and obtain the generated results.

[1315] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1316] Step 1: Connect accessory devices

[1317] Input: User opens smartphone Bluetooth settings and selects accessory device.

[1318] How it works: Your smartphone establishes a connection with an accessory device using Bluetooth.

[1319] Output: The smartphone will inform you that the connection was successful.

[1320] Step 2: User authentication

[1321] Input: The user places their finger on the fingerprint sensor on the accessory device.

[1322] How it works: The accessory device reads your fingerprint and matches it with fingerprints stored in an internal database.

[1323] Output: If authentication is successful, the accessory device sends a signal to the smartphone indicating successful authentication.

[1324] Step 3: Prepare for prompt input

[1325] Input: The smartphone receives a signal that authentication was successful.

[1326] How it works: The smartphone displays a prompt input screen for the generative AI.

[1327] Output: The user sees the prompt input screen and begins to enter the prompt statement.

[1328] Step 4: Prompt Input

[1329] Input: The user inputs a prompt for the generative AI through a smartphone application.

[1330] How it works: The smartphone receives the input prompt text and formats it internally.

[1331] Output: The prepared prompt is ready to be sent to the generative AI model.

[1332] Step 5: Send to generative AI model

[1333] Input: The smartphone sends a prepared prompt to the generative AI model.

[1334] Operation: The server runs the generative AI model based on the received prompt sentence and creates a generated result.

[1335] Output: The generated results are sent from the server to the smartphone.

[1336] Step 6: View the generated results

[1337] Input: The smartphone receives the generated results from the server.

[1338] How it works: The smartphone displays the generated results to the user.

[1339] Output: The user checks the displayed generated results and saves or shares them as needed.

[1340] (Application example 1)

[1341] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1342] To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and lack sufficient security for user authentication, which creates the risk of operational errors and unauthorized access. This can lead to reduced productivity and safety in factories.

[1343] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1344] In this invention, the server is an accessory device connectable to a smartphone and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI, a means for generating instructions for managing and operating robots in a factory, and a means for transmitting the instructions to the robots via wireless communication means, thereby enabling efficient and safe management and operation of robots in a factory.

[1345] An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

[1346] "Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

[1347] "Means to enable prompt input" refers to a function that provides an interface for users to input instructions or questions to generative AI.

[1348] "User authentication means" is a function for confirming and authenticating the identity of a user.

[1349] "Fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

[1350] The "operation means" is an interface that allows the user to input prompts to the generative AI.

[1351] The "one-click operation means" is a function that allows users to input prompts to generative AI with a single click.

[1352] "Means for generating instructions for managing and operating robots in a factory" refers to a function for generating specific work instructions for robots in a factory.

[1353] "Wireless communication means" refers to a function for sending and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

[1354] The "means for transmitting to the robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

[1355] This invention is a system for efficiently and safely managing and operating robots in a factory using an accessory device connectable to a smartphone. Specific embodiments of this system are described below.

[1356] System Configuration

[1357] The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

[1358] Hardware

[1359] Smartphone: Functions as a user interface and connects to accessory devices via wireless communication means.

[1360] Accessory device: Connected to a smartphone, it includes a fingerprint authentication means and an operation means.

[1361] Fingerprint authentication method: Scans the user's fingerprint for authentication.

[1362] Wireless communication means: Data is sent and received between the smartphone and the factory robot using Bluetooth, Wi-Fi, etc.

[1363] Factory robot: Performs actions according to instructions.

[1364] software

[1365] Generative AI model: Generates specific instructions for factory robots based on user prompts.

[1366] Smartphone application: Provides the user interface and manages communication with the generative AI model.

[1367] Operating procedure

[1368] 1. User authentication: The user places a finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

[1369] 2. Prompt input: An authenticated user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[1370] 3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

[1371] 4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

[1372] 5. Robot operation: The factory robot performs the operation according to the received instructions.

[1373] Specific examples

[1374] For example, when a robot performs a specific task in a factory, a manager uses a smartphone and accessory devices to give instructions to the robot. The following is an example of a prompt sentence:

[1375] "Move to assembly line 3 and begin welding."

[1376] "Move to the inspection area and perform quality checks."

[1377] "Go get part A and take it to assembly line 2."

[1378] This enables efficient and safe management and operation of robots in factories. The system utilizes user authentication and generative AI to reduce the risk of misoperation and unauthorized access, improving factory productivity and safety.

[1379] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1380] Step 1:

[1381] The user places a finger on the accessory device and is authenticated by the fingerprint authentication means.

[1382] Input: User's fingerprint

[1383] Data processing: The fingerprint authentication means scans the fingerprint and compares it with registered fingerprint data.

[1384] Output: Authentication result (success or failure)

[1385] How it works: The fingerprint sensor scans the user's fingerprint and matches it with an internal database for authentication.

[1386] Step 2:

[1387] If the authentication is successful, the user uses the operation means (one-click operation means) to input a prompt to the generative AI model.

[1388] Input: Authentication result (success), user prompt

[1389] Data processing: If the authentication result is successful, send the prompt text to the generative AI model.

[1390] Output: prompt input to the generative AI model

[1391] Specific operation: When the user presses the button, the smartphone application sends a prompt sentence to the generative AI model.

[1392] Step 3:

[1393] Based on the prompts, a generative AI model generates specific instructions for the factory robots.

[1394] Input: User prompt text

[1395] Data processing: A generative AI model analyzes the prompt and generates appropriate instructions.

[1396] Output: Instructions to factory robots

[1397] Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

[1398] Step 4:

[1399] The generated instructions are transmitted to the factory robot using wireless communication means.

[1400] Input: Instructions from a generative AI model

[1401] Data processing: Instructions are sent to the factory robot via wireless communication means.

[1402] Output: Sending instructions to factory robots

[1403] Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth or Wi-Fi.

[1404] Step 5:

[1405] The factory robot operates according to the received instructions.

[1406] Input: Instructions to the factory robot

[1407] Data processing: Factory robots analyze instructions and perform corresponding actions.

[1408] Output: Robot movement

[1409] Specific operation: The factory robot operates according to instructions such as "move to assembly line 3 and start welding."

[1410] Example 2

[1411] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1412] In order to use conventional generative AI models, users had to operate their smart devices directly and input prompts. This operation was time-consuming and inconvenient, especially when their hands were full or in situations where a quick response was required. Furthermore, from a security perspective, user authentication was sometimes insufficient, creating the risk of unauthorized use by third parties. There is a need for a system that can solve these issues and make it easier and safer to use generative AI models.

[1413] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1414] In this invention, the server includes a user authentication means, an operation means for operation by an authenticated user, a means for sending a signal to the smart device using the operation means to launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without directly operating the smart device, and the user authentication means also improves security.

[1415] "User authentication means" means a means for verifying a user's identity and granting access to a system.

[1416] "Operation means" refers to the interface or device that allows the user to operate the system.

[1417] A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

[1418] The "means for transmitting a signal" is a means for transmitting specific instructions or data from the operating means to the smart device.

[1419] A "specific application" is a software program designed to provide a particular function or service.

[1420] A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

[1421] A "prompt" is a text input used to give instructions or ask questions to a generative AI model.

[1422] A "means for generating a response" is a means by which a generative AI model creates an appropriate response based on a prompt sentence.

[1423] The "means for sending a response" is a means for sending the generated response to the smart device.

[1424] MODE FOR CARRYING OUT THE INVENTION

[1425] This invention provides a system for utilizing generative AI models without requiring a user to directly operate a smart device. Specific embodiments of this system are described below.

[1426] Hardware and software used

[1427] Hardware: Accessory devices (with fingerprint authentication sensors), smart devices (smartphones, etc.)

[1428] Software: Fingerprint authentication system, smart device applications, generative AI models

[1429] System configuration

[1430] 1. User authentication method:

[1431] The user places their finger on the fingerprint sensor on the accessory device.

[1432] The fingerprint authentication system captures fingerprint data and authenticates the user.

[1433] 2. Operating means:

[1434] If authentication is successful, the buttons on the accessory device are enabled.

[1435] The user presses an enabled button.

[1436] 3. Signal transmission means:

[1437] When the button is pressed, the accessory device sends a signal to the smart device.

[1438] The smart device receives the signal and launches a specific application.

[1439] 4. Prompt sending method:

[1440] The launched application sends a prompt to the generative AI model.

[1441] Prompts are pre-defined and contain instructions or questions that the user wants to ask the generative AI model.

[1442] 5. Response Generation Method:

[1443] The server uses a generative AI model to generate a response based on the received prompt.

[1444] 6. Response sending method:

[1445] The server generates a response and sends it to the smart device.

[1446] The smart device receives the response and displays it to the user.

[1447] Specific examples

[1448] When a user places their finger on the fingerprint authentication sensor of the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device to launch an application. The application launches and sends a prompt sentence, "Tell me what the weather is today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can see the information, "Today's weather is sunny," on the screen.

[1449] Prompt Sentence Examples

[1450] "Tell me about today's weather."

[1451] "Check the schedule for the next meeting."

[1452] "Tell me the latest news"

[1453] This system allows users to easily use generated AI models without directly operating their smart devices, and also improves security through user authentication measures.

[1454] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1455] Program processing flow

[1456] Step 1:

[1457] The user places their finger on the fingerprint authentication sensor on the accessory device.

[1458] Input: User's fingerprint

[1459] Data processing: The fingerprint sensor captures fingerprint data.

[1460] Output: Captured fingerprint data

[1461] Step 2:

[1462] The device captures the fingerprint data and performs authentication.

[1463] Input: Captured fingerprint data

[1464] Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

[1465] Output: Authentication result (success or failure)

[1466] Step 3:

[1467] If authentication is successful, the device's buttons will be enabled.

[1468] Input: Authentication result (success)

[1469] Data processing: If authentication is successful, activate the device button.

[1470] Output: Enabled button

[1471] Step 4:

[1472] The user presses a button.

[1473] Input: Enabled button

[1474] Data processing: The device detects that a button has been pressed.

[1475] Output: Button press signal

[1476] Step 5:

[1477] The terminal sends a signal to the smart device to launch the application.

[1478] Input: Button press signal

[1479] Data processing: The terminal sends a signal to the smart device to launch a specific application.

[1480] Output: Launched application

[1481] Step 6:

[1482] The application sends a prompt to the generative AI model.

[1483] Input: Launched application

[1484] Data processing: The application sends pre-defined prompts to the generative AI model.

[1485] Output: The prompt sent to the generative AI model

[1486] Step 7:

[1487] The server generates a response using a generative AI model.

[1488] Input: The prompt sent to the generative AI model

[1489] Data computation: A generative AI model analyzes the prompt and generates an appropriate response.

[1490] Output: The generated response

[1491] Step 8:

[1492] The server sends the response to the smart device.

[1493] Input: The generated response

[1494] Data processing: The server generates a response and sends it to the smart device.

[1495] Output: Response sent to smart device

[1496] Step 9:

[1497] The smart device displays the response.

[1498] Input: Response sent to smart device

[1499] Data processing: The smart device displays the received response to the user.

[1500] Output: The response displayed to the user

[1501] In this way, users can utilize generative AI models without directly operating their smart devices.

[1502] (Application example 2)

[1503] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1504] Conventional food delivery systems require users to use their smartphones to place orders, which makes operation cumbersome. In addition, there are security risks due to insufficient user authentication. Furthermore, ordering systems using voice recognition are not widespread, resulting in low user convenience. There is a need to provide a system that solves these problems and allows users to use food delivery services easily and safely.

[1505] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1506] An accessory device connectable to a smart device includes a user authentication means for authenticating a user operating the accessory device through fingerprint authentication, a button provided on the accessory device that a user authenticated by the user authentication means presses to input instructions to the generative AI, and a voice recognition means for recognizing the user's voice in response to the button being pressed. The server includes a means for acquiring the output of the generative AI using a prompt statement including voice data acquired by the voice recognition means and the generative AI. The prompt statement instructs the user to place an order through a food delivery service, and the acquiring means acquires the order results using a food delivery API. This allows users to easily and safely use food delivery services using only their voice, without having to operate their smartphone.

[1507] A "smart device" is a portable electronic device, such as a smartphone or tablet, that has Internet connectivity and can run applications.

[1508] An "accessory device" is additional hardware that is connected to a smart device to provide specific functionality.

[1509] "Generative AI" refers to artificial intelligence techniques that generate text and other content based on user instructions or prompts.

[1510] "Instruction input" refers to commands or requests that a user inputs to a generative AI.

[1511] "User authentication means" refers to technology or devices for verifying and authenticating the identity of a user.

[1512] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and authenticates the user based on that information.

[1513] "Operation means" refers to the interface and buttons that allow the user to operate the accessory device or smart device.

[1514] "Speech recognition means" refers to technology or devices that analyze a user's voice and convert it into text data.

[1515] "Voice data" refers to data that is a digital recording of a user's voice.

[1516] "Transmission means" refers to the technology or devices used to transmit data to other devices or systems.

[1517] This invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. Specific embodiments of this system are described below.

[1518] System configuration

[1519] The system consists of the following main components:

[1520] 1. Smart Device: A portable electronic device, such as a smartphone or tablet, that has internet connectivity and can run applications.

[1521] 2. Accessory device: Additional hardware that is connected to a smart device and includes fingerprint authentication means, operation means, and voice recognition means.

[1522] 3. Generative AI: Artificial intelligence techniques that generate text and other content based on user instructions or prompts.

[1523] Program processing

[1524] Hardware

[1525] Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

[1526] Smart device: Functions as the execution environment for applications.

[1527] Microphone: Captures the user's voice as a means of voice recognition.

[1528] software

[1529] Fingerprint authentication library: A software module for performing fingerprint authentication.

[1530] Speech Recognition Library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

[1531] Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3®).

[1532] Food Delivery API: Place orders using the API of food delivery services.

[1533] Processing flow

[1534] 1. User authentication: The user authenticates by placing their finger on the fingerprint authentication sensor of the accessory device. If authentication is successful, the next step is performed.

[1535] 2. Voice input: The user presses the operation means (one-click button) of the accessory device and speaks the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

[1536] 3. Prompt generation: Based on the acquired text data, a prompt sentence is generated for the generative AI.

[1537] 4. AI response generation: The generative AI model generates the optimal response based on the prompt.

[1538] 5. Order fulfillment: Based on the generated response, the order is completed using the food delivery API.

[1539] Specific examples

[1540] When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model receives the prompt "The user wants to order: pizza," the model then suggests the best pizza restaurant and menu item, and completes the order using a food delivery API.

[1541] Prompt Sentence Examples

[1542] "User wants to order: Pizza"

[1543] In this way, users can easily and safely use food delivery services using only their voice, without having to operate their smartphones.

[1544] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1545] Step 1:

[1546] The user places their finger on the fingerprint sensor.

[1547] Input: User's fingerprint data.

[1548] Processing: The fingerprint authentication library analyzes the fingerprint data and matches it with registered fingerprint data.

[1549] Output: Authentication result (success or failure).

[1550] Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

[1551] Step 2:

[1552] The user presses the operating means (one-click button) of the accessory device.

[1553] Input: User action (button press).

[1554] Processing: Detects that the operating means has been pressed and activates the voice recognition mode.

[1555] Output: Activate speech recognition mode.

[1556] Specific operation: The sensor detects that a button has been pressed, and the speech recognition library begins preparing for voice input.

[1557] Step 3:

[1558] The user verbally states the order details.

[1559] Input: User's voice data.

[1560] Processing: The microphone captures the voice data, which is then converted into text data by a speech recognition library.

[1561] Output: Text data (order details).

[1562] What it does: A microphone captures the user's voice, and a speech recognition library analyzes it and converts it into text.

[1563] Step 4:

[1564] The terminal generates a prompt sentence based on the acquired text data.

[1565] Input: Text data (order details).

[1566] Processing: Convert the text data into a prompt sentence of the form "The user wants to order: [Order details]".

[1567] Output: The prompt statement.

[1568] Specific operation: Converts text data into prompt sentences according to the format.

[1569] Step 5:

[1570] The device sends a prompt to the generative AI model.

[1571] Input: prompt statement.

[1572] Processing: A generative AI model analyzes the prompt and generates the optimal response.

[1573] Output: AI response (optimal order content).

[1574] Specific operation: The prompt sentence is sent to the generative AI model, and the AI ​​generates a response.

[1575] Step 6:

[1576] The device uses the generated response to complete the order using the food delivery API.

[1577] Input: AI response (optimal order content).

[1578] Processing: Fulfilling orders using food delivery APIs.

[1579] Output: Order confirmation (data indicating the order was successfully completed).

[1580] Specific behavior: Sends AI response to food delivery API and confirms order.

[1581] The server may further include means for recognizing the user's emotion using an emotion engine (described later), and may obtain an output from the generative AI using the voice data obtained by the voice recognition means, a prompt including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," the server may generate a prompt such as "User wants to order: pizza, user's emotion: sadness," and the generative AI model may suggest an optimal pizza restaurant and menu item taking the user's emotion into consideration, and complete the order using a food delivery API.

[1582] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1583] "Example 1"

[1584] One embodiment of the present invention provides an accessory device connectable to a smartphone. The device includes a means for enabling prompt input to a generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. The device also includes an emotion engine that recognizes the user's emotions. Specifically, the emotion engine recognizes the user's emotions from their tone of voice and facial expression, and feeds that information back to the generative AI. This enables the generative AI to generate an appropriate response based on the user's emotions.

[1585] "Example 2"

[1586] Another embodiment of the present invention provides a system in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone within the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotion from the tone of the voice data and feeds the results back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotion.

[1587] The processing flow of each embodiment will be described below.

[1588] "Example 1"

[1589] Step 1: The user operates an accessory device that can be connected to a smartphone.

[1590] Step 2: The user authentication means of the device authenticates the user. One possible authentication method is fingerprint authentication.

[1591] Step 3: The authenticated user prompts the generative AI.

[1592] Step 4: The device's emotion engine recognizes the user's emotions from their tone of voice and facial expressions.

[1593] Step 5: The emotion engine feeds that information back to the generative AI.

[1594] Step 6: The generative AI generates an appropriate response based on the user's emotions.

[1595] "Example 2"

[1596] Step 1: The user speaks to the accessory device.

[1597] Step 2: The microphone in the device captures the user's voice and sends the audio data to the emotion engine.

[1598] Step 3: The emotion engine analyzes the user's emotions from the tone of the voice data.

[1599] Step 4: The emotion engine feeds the results back to the generative AI.

[1600] Step 5: The generative AI generates an appropriate response based on the user's emotions.

[1601] Example 1

[1602] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1603] While conventional accessory devices that can be connected to smartphones allow prompt input to generative AI models, they have the problem of being unable to generate responses that take the user's emotional state into account. This makes it difficult to obtain appropriate responses that correspond to the user's intentions and emotions, resulting in a lack of improvement in the user experience.

[1604] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1605] In this invention, the server is an accessory device connectable to a smartphone, and includes: means for enabling prompt input to the generative AI model; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI model; emotion recognition means for recognizing the user's emotion; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the user's emotional state.

[1606] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer.

[1607] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[1608] A "generative AI model" is an artificial intelligence system that generates output such as text or speech based on input prompts.

[1609] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[1610] "User Authentication Measures" means technical measures for verifying a user's identity.

[1611] A "fingerprint authentication means" is a technical means for reading a user's fingerprint and authenticating them based on that information.

[1612] The "operation means" is an interface that allows the user to operate the accessory device.

[1613] A "one-click operation means" is an interface that allows a user to complete an operation with a single click.

[1614] "Emotion recognition means" refers to a technical means for recognizing emotions by analyzing the user's tone of voice and facial expressions.

[1615] "Emotion information" is data relating to the user's emotional state obtained by emotion recognition means.

[1616] "Feedback means" refers to the technical means for transmitting acquired emotional information to the generative AI model.

[1617] MODE FOR CARRYING OUT THE INVENTION

[1618] This invention is a system that uses an accessory device connectable to a smartphone to provide prompt input to a generative AI model and obtain a response that takes into account the user's emotional state. Specific embodiments of this system are described below.

[1619] Hardware and Software Configuration

[1620] Accessory Devices

[1621] The accessory device connects to the smartphone using wireless communication means such as Bluetooth, and includes the following main components:

[1622] Fingerprint authentication sensor: Reads the user's fingerprint for authentication.

[1623] Action button: The interface that allows the user to initiate prompt input.

[1624] Emotion recognition engine: Analyzes the user's tone of voice and facial expressions to obtain emotional information.

[1625] Smartphone

[1626] The smartphone runs a dedicated application that works in conjunction with the accessory device and provides the following functions:

[1627] Prompt input interface: A screen where users can input instructions or questions to a generative AI model.

[1628] Data transmission function: Sends prompts and emotion information to the server.

[1629] server

[1630] The server provides back-end services that communicate data between smartphones and generative AI models. The server has the following functions:

[1631] Data relay function: Sends prompts and emotional information received from the smartphone to the generative AI model.

[1632] Response reception function: Receives responses from the generative AI model and forwards them to the smartphone.

[1633] Specific examples

[1634] Consider a scenario in which a user uses an accessory device connected to a smartphone to ask a generative AI model a question.

[1635] 1. User Authentication

[1636] The user places their finger on the accessory device's fingerprint authentication sensor to authenticate. The device (smartphone) receives the fingerprint data from the accessory device and compares it with registered fingerprint data. If authentication is successful, the user can proceed to the next step.

[1637] 2. Prompt Input

[1638] The user opens a dedicated application on their smartphone and enters a prompt, for example, "What's the weather like today?" The device prepares to send this prompt to the generative AI model.

[1639] 3. Emotion recognition

[1640] The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions, and the device prepares this emotion data for transmission to a generative AI model.

[1641] 4. Sending prompts to the generative AI model

[1642] The device sends the prompt and emotion data entered by the user to the server, which then sends this data to the generative AI model, which then generates an appropriate response based on the prompt and emotion data.

[1643] 5. Receiving a response from the generative AI model

[1644] The server receives the response from the generative AI model and forwards it to the device, which prepares the response for display to the user.

[1645] 6. Displaying the response

[1646] The device displays the response received from the generative AI model to the user, for example, "It's sunny today. Have a nice day." The user can review this response and enter further prompts as needed.

[1647] Prompt Sentence Examples

[1648] "How's the weather today?"

[1649] "Please let me know when the next meeting is scheduled."

[1650] "Please tell me the latest news."

[1651] In this way, users can naturally interact with generative AI models through accessory devices.

[1652] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1653] Step 1: User authentication

[1654] The user places their finger on the fingerprint authentication sensor of the accessory device. The terminal receives the fingerprint data from the accessory device and compares it with the registered fingerprint data. The input is the user's fingerprint data, and the output is the authentication result. The terminal analyzes the fingerprint data and checks whether it matches the registered data. If authentication is successful, the terminal displays "Authentication successful."

[1655] Step 2: Connect your accessory device to your smartphone

[1656] The terminal establishes a connection with an accessory device via Bluetooth communication. The input is a Bluetooth connection request, and the output is the connection status. The terminal receives a connection request from the accessory device and establishes the connection. The server monitors this connection and relays data as necessary. If the connection is successful, the terminal displays "Connection successful."

[1657] Step 3: Prompt Input

[1658] The user opens a dedicated application on their smartphone and inputs a prompt. For example, they might input "What's the weather like today?" The input is the user's prompt sentence, and the output is the prompt data. The device prepares to send this prompt to the generative AI model. When the user has completed their input, the device displays "Prompt input completed."

[1659] Step 4: Emotion Recognition

[1660] The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions. The input is the user's tone of voice and facial expression data, and the output is emotion data. The device prepares this emotion data to send to the generative AI model. Once the emotion data is acquired, the device displays "Emotion data acquisition successful."

[1661] Step 5: Sending prompts to the generative AI model

[1662] The device sends the prompt and emotion data entered by the user to the server. The input is the prompt data and emotion data, and the output is the result of transmission to the server. The server then sends this data to the generative AI model. If the data transmission is successful, the device displays "Data transmission successful."

[1663] Step 6: Receiving a response from the generative AI model

[1664] The server receives the response from the generative AI model and transfers it to the device. The input is the response data from the generative AI model, and the output is the transfer result to the device. The device prepares to display this response to the user. When the response is received, the device displays "Response received successfully."

[1665] Step 7: View the response

[1666] The device displays the response received from the generative AI model to the user. For example, it displays "It's sunny today. Have a nice day." The input is the response data from the generative AI model, and the output is the display result to the user. The user can review this response and enter further prompts as needed.

[1667] (Application example 1)

[1668] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1669] Traditional food delivery services struggled to provide customized recommendations based on users' emotions and circumstances. They also lacked a means for users to use generative AI to simplify the ordering process. This could result in a poor user experience and reduced frequency of service use.

[1670] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1671] In this invention, the server is an accessory device connectable to a smartphone and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI, an emotion recognition means for recognizing the user's emotions, and a means for feeding back the emotion information recognized by the emotion recognition means to the generative AI. This makes it possible to propose customized food delivery according to the user's emotions.

[1672] An "accessory device connectable to a smartphone" is an external device that is connected to a smartphone using wireless communication means and provides a specific function.

[1673] "Generative AI" is an artificial intelligence system that generates responses and suggestions based on prompt input from the user.

[1674] "Means to enable prompt input" refers to a function that provides an interface for users to input instructions or questions to generative AI.

[1675] "User authentication means" refers to technology for verifying the identity of a user, ensuring that only specific users can operate the system.

[1676] "Fingerprint authentication means" is a technology that scans a user's fingerprint and authenticates the user based on that information.

[1677] "Operation means" refers to the physical or software interface that allows the user to operate the system.

[1678] A "one-click operation means" is an interface that allows a user to perform a specific function with a single click.

[1679] "Emotion recognition means" is a technology that analyzes the user's tone of voice and facial expressions to recognize their emotions.

[1680] "Means for feeding back emotional information to generative AI" refers to a function that sends recognized emotional information to generative AI, allowing the generative AI to generate responses or suggestions based on that information.

[1681] This invention is a system that uses an accessory device connectable to a smartphone to simplify the ordering process when a user uses a food delivery service through generative AI and makes customized suggestions based on the user's emotions.

[1682] System configuration

[1683] The system consists of the following main components:

[1684] 1. Accessory devices that can be connected to smartphones

[1685] Wireless communication means: Connected to a smartphone using Bluetooth or similar.

[1686] User authentication method: A fingerprint sensor is used to authenticate the user.

[1687] Control: A one-click control is used by the user to prompt the generative AI.

[1688] Emotion recognition: Analyzes the user's tone of voice and facial expressions to recognize their emotions.

[1689] 2. Generative AI

[1690] Prompt input: Accepts instructions or questions from the user and generates responses or suggestions.

[1691] Emotional feedback: Generate customized responses and suggestions based on recognized emotional information.

[1692] Program processing

[1693] The server does the following:

[1694] 1. Bluetooth connection: Connect your smartphone and accessory device using Bluetooth.

[1695] 2. User authentication: The fingerprint sensor scans the user's fingerprint for authentication.

[1696] 3. Emotion Recognition: Recognizes emotions by analyzing the user's tone of voice and facial expressions.

[1697] 4. Prompt generation: Generate prompts for the generative AI based on the recognized emotions.

[1698] 5. AI response generation: Use generative AI to generate responses and suggestions to prompts.

[1699] Hardware and software used

[1700] Hardware: Bluetooth module, fingerprint sensor, camera (for face recognition), microphone (for voice recognition)

[1701] Software: Bluetooth connection library, fingerprint authentication library, emotion recognition engine, generative AI model

[1702] Specific examples

[1703] If the user is recognized as tired, the generative AI will suggest "food delivery options for when you're tired." For example, the following prompts will be generated:

[1704] The user is feeling tired. Please suggest food delivery options.

[1705] In this way, it becomes possible to offer customized food delivery suggestions based on the user's emotions.

[1706] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1707] Step 1:

[1708] The server connects the smartphone and the accessory device using a Bluetooth module. The input is the pairing information between the smartphone and the accessory device, and the output is the establishment of the connection. Specifically, the server uses the Bluetooth connection library to establish communication between the smartphone and the accessory device.

[1709] Step 2:

[1710] The device scans the user's fingerprint using a fingerprint authentication sensor to perform user authentication. The input is the user's fingerprint data, and the output is the authentication result. Specifically, the fingerprint authentication library is used to compare the scanned fingerprint data with registered data to determine whether authentication was successful.

[1711] Step 3:

[1712] The device uses a microphone and camera to capture the user's tone of voice and facial expressions to perform emotion recognition. The input is voice data and image data, and the output is recognized emotion information. Specifically, an emotion recognition engine is used to analyze the voice data and image data to identify the user's emotion.

[1713] Step 4:

[1714] The server generates prompts for the generative AI based on the recognized emotional information. The input is emotional information, and the output is a prompt. Specifically, the server converts the emotional information into text and creates a prompt suitable for the generative AI.

[1715] Step 5:

[1716] The server uses a generative AI model to generate responses and suggestions to prompts. The input is the prompt, and the output is the generated response or suggestion. Specifically, the prompt is input into the generative AI model, and the AI ​​generates the response or suggestion.

[1717] Step 6:

[1718] The terminal presents the generated response or suggestion to the user. The input is the generated response or suggestion, and the output is what is displayed to the user. Specifically, the response or suggestion is displayed on the smartphone display so that the user can check it.

[1719] Example 2

[1720] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1721] Conventional systems using generative AI models require users to operate their smartphones, making them difficult to use. Furthermore, they are unable to generate appropriate responses based on the user's emotions, limiting the user experience. This makes it difficult for users to effectively utilize generative AI models.

[1722] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1723] In this invention, the server includes a user authentication means, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI model, an emotion recognition means for capturing the user's voice data and analyzing the emotions, and a means for feeding back the emotion information analyzed by the emotion recognition means to the generative AI model. This allows the user to easily use the generative AI model without operating a smartphone, and further enables the user to receive an appropriate response according to the user's emotions.

[1724] "User authentication means" refers to a means for verifying a user's identity, and includes biometric authentication technologies such as fingerprint authentication and facial recognition.

[1725] "Operation means" refers to the means by which a user inputs prompts to the generated AI model, and includes interfaces such as buttons and touchscreens.

[1726] An "emotion recognition means" is a means for capturing a user's voice data and analyzing the user's emotions from the tone and content of the data.

[1727] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on prompts from the user.

[1728] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[1729] The "feedback means" is a means for transmitting the emotional information analyzed by the emotion recognition means to the generative AI model.

[1730] The present invention provides a system including a user authentication means, an operation means, an emotion recognition means, and a feedback means. A specific embodiment of this system will be described below.

[1731] First, the user places their finger on the accessory device's fingerprint sensor. The fingerprint sensor captures the user's fingerprint data and sends it to the device. The device then sends this data to the server, which matches the fingerprint data with an existing database to authenticate the user. If authentication is successful, the device's buttons are enabled.

[1732] Next, when the user presses the enabled button, the device launches a dedicated application on the smartphone. This application has the function of inputting prompt sentences to the generative AI model. For example, the prompt sentence "Tell me about today's weather" can be input to the generative AI model.

[1733] Furthermore, when the user speaks to the accessory device, the microphone in the terminal captures voice data. This voice data is sent to the emotion recognition means, which analyzes the tone of the voice data to analyze the user's emotion. The analysis result is sent to the generative AI model through the feedback means. The generative AI model generates an appropriate response based on this emotional information. For example, in response to the prompt "The user seems a little tired. Please generate an encouraging message," the generative AI model generates a response such as "Take a short rest today to refresh yourself."

[1734] This system allows users to easily use generative AI models without operating a smartphone, and improves the user experience by providing appropriate responses based on the user's emotions.

[1735] As a concrete example, when a user places their finger on the fingerprint authentication sensor of an accessory device and presses a button after successful authentication, an application on the smartphone is launched and the following prompt sentence is input into the generative AI model:

[1736] "Tell me about the weather today."

[1737] As another example, when a user speaks to an accessory device saying, "How are you feeling today?", the emotion recognition means recognizes the emotion from the tone of the user's voice and inputs the following prompt sentence into the generative AI model:

[1738] "The user seems a little tired. Please generate an encouraging message."

[1739] This allows users to easily utilize generative AI without having to operate a smartphone.

[1740] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1741] Step 1:

[1742] The user places their finger on the fingerprint sensor.

[1743] Input: User's fingerprint

[1744] Specific operation: When the user places their finger on the fingerprint authentication sensor on the accessory device, the sensor will glow blue.

[1745] Output: Captured fingerprint data

[1746] Step 2:

[1747] The device captures the fingerprint data and sends it to the server.

[1748] Input: Captured fingerprint data

[1749] Specific operation: The device captures fingerprint data from the fingerprint authentication sensor and when it sends the data to the server, the device screen displays "Authentication in progress..."

[1750] Output: Fingerprint data sent to the server

[1751] Step 3:

[1752] The server authenticates the fingerprint data.

[1753] Input: Fingerprint data sent to the server

[1754] Specific operation: The server compares the received fingerprint data with an existing database and authenticates the user.

[1755] Output: Authentication result (success or failure)

[1756] Step 4:

[1757] If authentication is successful, the device's buttons are enabled.

[1758] Input: Authentication result (success)

[1759] Specific operation: If the server is successful in authentication, it sends a signal to the terminal to activate the button. The button on the terminal lights up green.

[1760] Output: Enabled button

[1761] Step 5:

[1762] The user presses a button.

[1763] Input: Enabled button

[1764] Specific Action: The user presses an enabled button, which triggers the next step.

[1765] Output: Button press signal

[1766] Step 6:

[1767] The device launches the application on the smartphone.

[1768] Input: Button press signal

[1769] Specific operation: The device launches a dedicated application on the smartphone, and the application logo appears on the smartphone screen.

[1770] Output: Launched application

[1771] Step 7:

[1772] The application prompts input to the generative AI model.

[1773] Input: Launched application

[1774] Specific operation: The application inputs a prompt sentence to the generative AI model. The screen displays "Sending prompt...". For example, the prompt sentence is "Tell me about the weather today."

[1775] Output: The prompt sent to the generative AI model

[1776] Step 8:

[1777] The user speaks to the accessory device.

[1778] Input: User's voice

[1779] Specific Actions: The user speaks to the accessory device, for example, "How are you feeling today?" The microphone on the device lights up red, indicating that it is capturing voice data.

[1780] Output: Captured audio data

[1781] Step 9:

[1782] The device captures the voice data and sends it to the emotion engine.

[1783] Input: Captured audio data

[1784] Specific operation: The device uses the built-in microphone to capture voice data and sends the data to the emotion engine.

[1785] Output: Audio data sent to the emotion engine

[1786] Step 10:

[1787] The emotion engine analyzes the user's emotions from the tone of the voice data.

[1788] Input: Audio data sent to the emotion engine

[1789] Specific operation: The emotion engine analyzes the tone of the voice data and analyzes the user's emotion. The device screen displays "Analyzing emotion..."

[1790] Output: Analyzed emotion information

[1791] Step 11:

[1792] The emotion engine feeds the analysis results back into the generative AI model.

[1793] Input: Analyzed emotion information

[1794] Specific operation: The emotion engine feeds the analysis results back to the generative AI model, for example, sending information such as "The user seems a little tired."

[1795] Output: Emotion information sent to the generative AI model

[1796] Step 12:

[1797] The generative AI model generates responses based on the user's emotions.

[1798] Input: Emotion information sent to the generative AI model

[1799] Specific operation: The generative AI model generates an appropriate response based on the emotional feedback. For example, in response to the prompt "Generate an encouraging message," it generates the response "Take a break today and refresh yourself." The response is displayed on the smartphone screen.

[1800] Output: The generated response

[1801] (Application example 2)

[1802] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1803] In conventional food delivery services, when users order using their smartphones, the operation is complicated and the service does not provide appropriate suggestions based on the user's emotions. In particular, there is a problem that the user experience is not improved for tired or stressed users, as there are not enough suggestions for relaxing meals.

[1804] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1805] In this invention, the server includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI, a voice capture means for capturing the user's voice and recognizing their emotion, an emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the user's emotion, and a means for the generative AI to generate an appropriate response based on the emotion recognized by the emotion recognition means. This enables the user to receive appropriate food delivery suggestions through voice and emotion recognition without operating a smartphone.

[1806] "Generative AI" is an artificial intelligence system that generates appropriate responses and suggestions based on user input.

[1807] "Prompt input" refers to the act of inputting instructions or questions to generative AI.

[1808] An "accessory device" is an external device that can be connected to a smartphone and that provides a specific function.

[1809] "User authentication means" means a means for verifying the identity of a user, ensuring that only specific users can operate the system.

[1810] The "fingerprint authentication means" is a means for reading a user's fingerprint and performing authentication.

[1811] "Operation means" refers to means such as interfaces and buttons that allow users to operate the system.

[1812] A "one-click operation means" is a means that allows a user to complete an operation with a single click.

[1813] "Audio capture means" means for recording or capturing a user's voice.

[1814] The "emotion recognition means" is a means for analyzing the captured voice data and recognizing the user's emotion.

[1815] "Response" refers to the response or suggestion that a generative AI generates in response to user input.

[1816] A system for implementing this invention is configured as follows: First, a user uses an accessory device connected to a smartphone. This accessory device is equipped with a fingerprint authentication means, and authentication is performed by the user placing their finger on it. If authentication is successful, the operation means of the accessory device is enabled, and the user can use the one-click operation means to input prompts to the generative AI.

[1817] Next, the voice capture means captures the user's voice. This voice data is sent to the emotion recognition means, which analyzes the user's emotions. The emotion recognition means recognizes the user's emotions from the tone and content of the voice data and feeds the results back to the generative AI. The generative AI generates an appropriate response based on the recognized emotions.

[1818] Specifically, the following hardware and software are used: A fingerprint sensor is used as the fingerprint authentication method; A microphone and voice recognition software (e.g., speech_recognition library) are used as the voice capture method; An emotion engine (e.g., EmotionEngine) is used as the emotion recognition method; and An AI generation model (e.g., AIGenerator) is used as the generative AI.

[1819] For example, if a user says, "I'm tired today," the voice capture means captures the voice, and the emotion recognition means detects "fatigue." Based on this emotion, the generative AI generates a response suggesting a "relaxing meal." In this way, appropriate food delivery suggestions are made according to the user's emotions.

[1820] Examples of prompts to input to a generative AI model include:

[1821] User Emotion: Fatigue

[1822] Prompt for generative AI: "Suggest a relaxing meal for a user who is tired."

[1823] This system allows users to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate a smartphone.

[1824] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1825] Step 1:

[1826] The user places their finger on the fingerprint authentication sensor of an accessory device connected to a smartphone. The input is the user's fingerprint data, and the output is the authentication result. The accessory device reads the fingerprint data and compares it with its internal database for authentication. If authentication is successful, the device proceeds to the next step.

[1827] Step 2:

[1828] If authentication is successful, the operation means of the accessory device is enabled. The user uses the one-click operation means to input a prompt to the generative AI. The input is the user's click operation, and the output is the trigger for the prompt input. The accessory device detects the click operation and starts inputting a prompt to the generative AI.

[1829] Step 3:

[1830] The device's voice capture means captures the user's voice. The input is the user's voice data and the output is the captured voice data. The device's microphone records the user's voice and voice recognition software converts the data into a digital format.

[1831] Step 4:

[1832] The captured voice data is sent to an emotion recognition means. The input is the captured voice data and the output is the recognized emotion data. The emotion engine analyzes the voice data and recognizes the user's emotion from the tone and content.

[1833] Step 5:

[1834] The recognized emotional data is fed back to the generative AI. The input is the recognized emotional data, and the output is a prompt to the generative AI. The generative AI then creates a prompt based on the emotional data to generate an appropriate response.

[1835] Step 6:

[1836] The generative AI generates an appropriate response based on the prompt. The input is the prompt and the output is the generated response. The generative AI analyzes the prompt and generates food delivery suggestions based on the user's emotions.

[1837] Step 7:

[1838] The generated response is provided to the user. The input is the generated response, and the output is a suggestion to the user. Appropriate food delivery suggestions are displayed or spoken to the user through the terminal's display or audio output device.

[1839] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1840] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generative AIs. Data generation models 58

[1841] , obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in the data format of voice data, text data, etc. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1842] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1843] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1844] [Fourth embodiment]

[1845] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1846] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1847] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1848] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1849] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1850] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1851] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1852] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1853] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1854] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1855] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1856] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1857] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1858] "Example 1"

[1859] One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. Specifically, the accessory device is connected to the smartphone using wireless communication means such as Bluetooth, and prompt input to the generative AI is performed through an application running on the smartphone. The user authentication means includes a fingerprint authentication sensor built into the device, which performs authentication by comparing the user's fingerprint information with registered user information. The operation means includes a button on the device that the user presses to input prompts to the generative AI.

[1860] "Example 2"

[1861] In a specific example of use, a user authenticates by placing their finger on the fingerprint authentication sensor of the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is sent to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

[1862] The processing flow of each embodiment will be described below.

[1863] "Example 1"

[1864] Step 1: A user connects an accessory device connectable to a smartphone to the smartphone using a wireless communication means such as Bluetooth.

[1865] Step 2: The user places their finger on the fingerprint authentication sensor of the accessory device, and the fingerprint authentication sensor authenticates the user by matching it with the fingerprint information of the registered user.

[1866] Step 3: If authentication is successful, the device's buttons are enabled.

[1867] Step 4: When the user presses a button on the device, an application on the smartphone is launched, prompting the generative AI to input the desired information.

[1868] "Example 2"

[1869] Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

[1870] Step 2: If authentication is successful, the device's buttons are enabled.

[1871] Step 3: When the user presses the button, an application on the smartphone is launched and prompts the generative AI to input the desired information.

[1872] Step 4: This allows users to easily utilize generative AI without having to operate their smartphones.

[1873] Example 1

[1874] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1875] In conventional systems using generative AI models, users are required to perform complex operations and undergo multi-step authentication when entering prompts, resulting in a poor user experience. Furthermore, there is a lack of efficient means for quickly and accurately obtaining generated results. This reduces the convenience of using generative AI models.

[1876] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1877] In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for running an application on the smartphone for inputting prompts to the generative AI; and means for sending prompts to the generative AI model and receiving generation results. This allows the user to input prompts to the generative AI model with simple operations and quickly and accurately obtain generation results.

[1878] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer, and can be used to install applications and utilize a variety of functions.

[1879] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[1880] "Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompts.

[1881] "Prompt input" is the act of inputting instructions or questions to a generative AI.

[1882] "User authentication means" refers to technology or devices used to verify a user's identity.

[1883] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and verifies the user's identity based on that information.

[1884] The "operation means" refers to an interface or device that allows a user to operate an accessory device or a smartphone.

[1885] A "one-click operation means" is an interface or device that allows a user to perform a specific function with a single click.

[1886] An "application" is software that runs on a smartphone and provides specific functions or services.

[1887] "Generation results" are output such as text and images generated by the generative AI based on the prompt text.

[1888] MODE FOR CARRYING OUT THE INVENTION

[1889] This invention is a system that uses an accessory device connectable to a smartphone to easily input prompts to a generative AI model. Specific embodiments of this system are described below.

[1890] Hardware and Software Configuration

[1891] Accessory equipment:

[1892] An accessory device is an external device that connects to a smartphone and has the following main functions:

[1893] User authentication method: Equipped with a fingerprint authentication sensor that reads the user's fingerprint information.

[1894] Operation: Equipped with a one-click button for easy user operation.

[1895] Smartphone:

[1896] The smartphone is connected to the accessory device via wireless communication means such as Bluetooth, and an application for providing prompt input to the generative AI model is installed on the smartphone.

[1897] Generative AI models:

[1898] The generative AI model runs on a server and generates text and images based on prompts sent from a smartphone.

[1899] System Operation

[1900] User authentication:

[1901] The user places their finger on the fingerprint authentication sensor installed in the accessory device. The accessory device reads the fingerprint information and compares it with fingerprint information stored in its internal database. If authentication is successful, the accessory device sends a signal indicating successful authentication to the smartphone.

[1902] Prompt Input:

[1903] After successful authentication, the user presses a button on the accessory device, which launches the smartphone application and displays a prompt input screen. The user then inputs a prompt for the generative AI model.

[1904] Send to generative AI model:

[1905] The smartphone sends the input prompt to the generative AI model, and the server operates the generative AI model based on the received prompt to generate the generated results.

[1906] View the generated results:

[1907] The server sends the generated results to the smartphone, which then displays them to the user. The user can then check the displayed results and save or share them as needed.

[1908] Specific examples

[1909] If a user wants to have a generative AI create a new poem, the specific steps are as follows:

[1910] 1. User authentication:

[1911] The user places their finger on the fingerprint authentication sensor of the accessory device.

[1912] The accessory device reads the fingerprint and performs authentication.

[1913] If the authentication is successful, a signal indicating successful authentication is sent to the smartphone.

[1914] 2. Prompt input:

[1915] The user presses a button on the accessory device.

[1916] The smartphone application will launch and a prompt input screen will appear.

[1917] The user enters the prompt text "Create a new poem."

[1918] 3. Send to generative AI model:

[1919] The smartphone sends the input prompt to the generative AI model.

[1920] The server runs a generative AI model based on the prompt text to generate a poem.

[1921] 4. View the generated results:

[1922] The server sends the generated poem to the smartphone.

[1923] The smartphone displays the received poem to the user.

[1924] The user can review the displayed poem and save or share it as desired.

[1925] In this way, a user can use the accessory device and a smartphone to provide prompt input to the generative AI model and obtain the generated results.

[1926] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1927] Step 1: Connect accessory devices

[1928] Input: User opens smartphone Bluetooth settings and selects accessory device.

[1929] How it works: Your smartphone establishes a connection with an accessory device using Bluetooth.

[1930] Output: The smartphone will inform you that the connection was successful.

[1931] Step 2: User authentication

[1932] Input: The user places their finger on the fingerprint sensor on the accessory device.

[1933] How it works: The accessory device reads your fingerprint and matches it with fingerprints stored in an internal database.

[1934] Output: If authentication is successful, the accessory device sends a signal to the smartphone indicating successful authentication.

[1935] Step 3: Prepare for prompt input

[1936] Input: The smartphone receives a signal that authentication was successful.

[1937] How it works: The smartphone displays a prompt input screen for the generative AI.

[1938] Output: The user sees the prompt input screen and begins to enter the prompt statement.

[1939] Step 4: Prompt Input

[1940] Input: The user inputs a prompt for the generative AI through a smartphone application.

[1941] How it works: The smartphone receives the input prompt text and formats it internally.

[1942] Output: The prepared prompt is ready to be sent to the generative AI model.

[1943] Step 5: Send to generative AI model

[1944] Input: The smartphone sends a prepared prompt to the generative AI model.

[1945] Operation: The server runs the generative AI model based on the received prompt sentence and creates a generated result.

[1946] Output: The generated results are sent from the server to the smartphone.

[1947] Step 6: View the generated results

[1948] Input: The smartphone receives the generated results from the server.

[1949] How it works: The smartphone displays the generated results to the user.

[1950] Output: The user checks the displayed generated results and saves or shares them as needed.

[1951] (Application example 1)

[1952] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1953] To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and lack sufficient security for user authentication, which creates the risk of operational errors and unauthorized access. This can lead to reduced productivity and safety in factories.

[1954] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1955] In this invention, the server is an accessory device connectable to a smartphone and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI, a means for generating instructions for managing and operating robots in a factory, and a means for transmitting the instructions to the robots via wireless communication means, thereby enabling efficient and safe management and operation of robots in a factory.

[1956] An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

[1957] "Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

[1958] "Means to enable prompt input" refers to a function that provides an interface for users to input instructions or questions to generative AI.

[1959] "User authentication means" is a function for confirming and authenticating the identity of a user.

[1960] "Fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

[1961] The "operation means" is an interface that allows the user to input prompts to the generative AI.

[1962] The "one-click operation means" is a function that allows users to input prompts to generative AI with a single click.

[1963] "Means for generating instructions for managing and operating robots in a factory" refers to a function for generating specific work instructions for robots in a factory.

[1964] "Wireless communication means" refers to a function for sending and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

[1965] The "means for transmitting to the robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

[1966] This invention is a system for efficiently and safely managing and operating robots in a factory using an accessory device connectable to a smartphone. Specific embodiments of this system are described below.

[1967] System Configuration

[1968] The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

[1969] Hardware

[1970] Smartphone: Functions as a user interface and connects to accessory devices via wireless communication means.

[1971] Accessory device: Connected to a smartphone, it includes a fingerprint authentication means and an operation means.

[1972] Fingerprint authentication method: Scans the user's fingerprint for authentication.

[1973] Wireless communication means: Data is sent and received between the smartphone and the factory robot using Bluetooth, Wi-Fi, etc.

[1974] Factory robot: Performs actions according to instructions.

[1975] software

[1976] Generative AI model: Generates specific instructions for factory robots based on user prompts.

[1977] Smartphone application: Provides the user interface and manages communication with the generative AI model.

[1978] Operating procedure

[1979] 1. User authentication: The user places a finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

[1980] 2. Prompt input: An authenticated user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[1981] 3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

[1982] 4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

[1983] 5. Robot operation: The factory robot performs the operation according to the received instructions.

[1984] Specific examples

[1985] For example, when a robot performs a specific task in a factory, a manager uses a smartphone and accessory devices to give instructions to the robot. The following is an example of a prompt sentence:

[1986] "Move to assembly line 3 and begin welding."

[1987] "Move to the inspection area and perform quality checks."

[1988] "Go get part A and take it to assembly line 2."

[1989] This enables efficient and safe management and operation of robots in factories. The system utilizes user authentication and generative AI to reduce the risk of misoperation and unauthorized access, improving factory productivity and safety.

[1990] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1991] Step 1:

[1992] The user places a finger on the accessory device and is authenticated by the fingerprint authentication means.

[1993] Input: User's fingerprint

[1994] Data processing: The fingerprint authentication means scans the fingerprint and compares it with registered fingerprint data.

[1995] Output: Authentication result (success or failure)

[1996] How it works: The fingerprint sensor scans the user's fingerprint and matches it with an internal database for authentication.

[1997] Step 2:

[1998] If the authentication is successful, the user uses the operation means (one-click operation means) to input a prompt to the generative AI model.

[1999] Input: Authentication result (success), user prompt

[2000] Data processing: If the authentication result is successful, send the prompt text to the generative AI model.

[2001] Output: prompt input to the generative AI model

[2002] Specific operation: When the user presses the button, the smartphone application sends a prompt sentence to the generative AI model.

[2003] Step 3:

[2004] Based on the prompts, a generative AI model generates specific instructions for the factory robots.

[2005] Input: User prompt text

[2006] Data processing: A generative AI model analyzes the prompt and generates appropriate instructions.

[2007] Output: Instructions to factory robots

[2008] Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

[2009] Step 4:

[2010] The generated instructions are transmitted to the factory robot using wireless communication means.

[2011] Input: Instructions from a generative AI model

[2012] Data processing: Instructions are sent to the factory robot via wireless communication means.

[2013] Output: Sending instructions to factory robots

[2014] Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth or Wi-Fi.

[2015] Step 5:

[2016] The factory robot operates according to the received instructions.

[2017] Input: Instructions to the factory robot

[2018] Data processing: Factory robots analyze instructions and perform corresponding actions.

[2019] Output: Robot movement

[2020] Specific operation: The factory robot operates according to instructions such as "move to assembly line 3 and start welding."

[2021] Example 2

[2022] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2023] In order to use conventional generative AI models, users had to operate their smart devices directly and input prompts. This operation was time-consuming and inconvenient, especially when their hands were full or in situations where a quick response was required. Furthermore, from a security perspective, user authentication was sometimes insufficient, creating the risk of unauthorized use by third parties. There is a need for a system that can solve these issues and make it easier and safer to use generative AI models.

[2024] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2025] In this invention, the server includes a user authentication means, an operation means for operation by an authenticated user, a means for sending a signal to the smart device using the operation means to launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without directly operating the smart device, and the user authentication means also improves security.

[2026] "User authentication means" means a means for verifying a user's identity and granting access to a system.

[2027] "Operation means" refers to the interface or device that allows the user to operate the system.

[2028] A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

[2029] The "means for transmitting a signal" is a means for transmitting specific instructions or data from the operating means to the smart device.

[2030] A "specific application" is a software program designed to provide a particular function or service.

[2031] A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

[2032] A "prompt" is a text input used to give instructions or ask questions to a generative AI model.

[2033] A "means for generating a response" is a means by which a generative AI model creates an appropriate response based on a prompt sentence.

[2034] The "means for sending a response" is a means for sending the generated response to the smart device.

[2035] MODE FOR CARRYING OUT THE INVENTION

[2036] This invention provides a system for utilizing generative AI models without requiring a user to directly operate a smart device. Specific embodiments of this system are described below.

[2037] Hardware and software used

[2038] Hardware: Accessory devices (with fingerprint authentication sensors), smart devices (smartphones, etc.)

[2039] Software: Fingerprint authentication system, smart device applications, generative AI models

[2040] System configuration

[2041] 1. User authentication method:

[2042] The user places their finger on the fingerprint sensor on the accessory device.

[2043] The fingerprint authentication system captures fingerprint data and authenticates the user.

[2044] 2. Operating means:

[2045] If authentication is successful, the buttons on the accessory device are enabled.

[2046] The user presses an enabled button.

[2047] 3. Signal transmission means:

[2048] When the button is pressed, the accessory device sends a signal to the smart device.

[2049] The smart device receives the signal and launches a specific application.

[2050] 4. Prompt sending method:

[2051] The launched application sends a prompt to the generative AI model.

[2052] Prompts are pre-defined and contain instructions or questions that the user wants to ask the generative AI model.

[2053] 5. Response Generation Method:

[2054] The server uses a generative AI model to generate a response based on the received prompt.

[2055] 6. Response sending method:

[2056] The server generates a response and sends it to the smart device.

[2057] The smart device receives the response and displays it to the user.

[2058] Specific examples

[2059] When a user places their finger on the fingerprint authentication sensor of the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device to launch an application. The application launches and sends a prompt sentence, "Tell me what the weather is today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can see the information, "Today's weather is sunny," on the screen.

[2060] Prompt Sentence Examples

[2061] "Tell me about today's weather."

[2062] "Check the schedule for the next meeting."

[2063] "Tell me the latest news"

[2064] This system allows users to easily use generated AI models without directly operating their smart devices, and also improves security through user authentication measures.

[2065] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2066] Program processing flow

[2067] Step 1:

[2068] The user places their finger on the fingerprint authentication sensor on the accessory device.

[2069] Input: User's fingerprint

[2070] Data processing: The fingerprint sensor captures fingerprint data.

[2071] Output: Captured fingerprint data

[2072] Step 2:

[2073] The device captures the fingerprint data and performs authentication.

[2074] Input: Captured fingerprint data

[2075] Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

[2076] Output: Authentication result (success or failure)

[2077] Step 3:

[2078] If authentication is successful, the device's buttons will be enabled.

[2079] Input: Authentication result (success)

[2080] Data processing: If authentication is successful, activate the device button.

[2081] Output: Enabled button

[2082] Step 4:

[2083] The user presses a button.

[2084] Input: Enabled button

[2085] Data processing: The device detects that a button has been pressed.

[2086] Output: Button press signal

[2087] Step 5:

[2088] The terminal sends a signal to the smart device to launch the application.

[2089] Input: Button press signal

[2090] Data processing: The terminal sends a signal to the smart device to launch a specific application.

[2091] Output: Launched application

[2092] Step 6:

[2093] The application sends a prompt to the generative AI model.

[2094] Input: Launched application

[2095] Data processing: The application sends pre-defined prompts to the generative AI model.

[2096] Output: The prompt sent to the generative AI model

[2097] Step 7:

[2098] The server generates a response using a generative AI model.

[2099] Input: The prompt sent to the generative AI model

[2100] Data computation: A generative AI model analyzes the prompt and generates an appropriate response.

[2101] Output: The generated response

[2102] Step 8:

[2103] The server sends the response to the smart device.

[2104] Input: The generated response

[2105] Data processing: The server generates a response and sends it to the smart device.

[2106] Output: Response sent to smart device

[2107] Step 9:

[2108] The smart device displays the response.

[2109] Input: Response sent to smart device

[2110] Data processing: The smart device displays the received response to the user.

[2111] Output: The response displayed to the user

[2112] In this way, users can utilize generative AI models without directly operating their smart devices.

[2113] (Application example 2)

[2114] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2115] Conventional food delivery systems require users to use their smartphones to place orders, which makes operation cumbersome. In addition, there are security risks due to insufficient user authentication. Furthermore, ordering systems using voice recognition are not widespread, resulting in low user convenience. There is a need to provide a system that solves these problems and allows users to use food delivery services easily and safely.

[2116] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2117] An accessory device connectable to a smart device includes a user authentication means for authenticating a user operating the accessory device through fingerprint authentication, a button provided on the accessory device that a user authenticated by the user authentication means presses to input instructions to the generative AI, and a voice recognition means for recognizing the user's voice in response to the button being pressed. The server includes a means for acquiring the output of the generative AI using a prompt statement including voice data acquired by the voice recognition means and the generative AI. The prompt statement instructs the user to place an order through a food delivery service, and the acquiring means acquires the order results using a food delivery API. This allows users to easily and safely use food delivery services using only their voice, without having to operate their smartphone.

[2118] A "smart device" is a portable electronic device, such as a smartphone or tablet, that has Internet connectivity and can run applications.

[2119] An "accessory device" is additional hardware that is connected to a smart device to provide specific functionality.

[2120] "Generative AI" refers to artificial intelligence techniques that generate text and other content based on user instructions or prompts.

[2121] "Instruction input" refers to commands or requests that a user inputs to a generative AI.

[2122] "User authentication means" refers to technology or devices for verifying and authenticating the identity of a user.

[2123] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and authenticates the user based on that information.

[2124] "Operation means" refers to the interface and buttons that allow the user to operate the accessory device or smart device.

[2125] "Speech recognition means" refers to technology or devices that analyze a user's voice and convert it into text data.

[2126] "Voice data" refers to data that is a digital recording of a user's voice.

[2127] "Transmission means" refers to the technology or devices used to transmit data to other devices or systems.

[2128] This invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. Specific embodiments of this system are described below.

[2129] System configuration

[2130] The system consists of the following main components:

[2131] 1. Smart Device: A portable electronic device, such as a smartphone or tablet, that has internet connectivity and can run applications.

[2132] 2. Accessory device: Additional hardware that is connected to a smart device and includes fingerprint authentication means, operation means, and voice recognition means.

[2133] 3. Generative AI: Artificial intelligence techniques that generate text and other content based on user instructions or prompts.

[2134] Program processing

[2135] Hardware

[2136] Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

[2137] Smart device: Functions as the execution environment for applications.

[2138] Microphone: Captures the user's voice as a means of voice recognition.

[2139] software

[2140] Fingerprint authentication library: A software module for performing fingerprint authentication.

[2141] Speech Recognition Library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

[2142] Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3®).

[2143] Food Delivery API: Place orders using the API of food delivery services.

[2144] Processing flow

[2145] 1. User authentication: The user authenticates by placing their finger on the fingerprint authentication sensor of the accessory device. If authentication is successful, the next step is performed.

[2146] 2. Voice input: The user presses the operation means (one-click button) of the accessory device and speaks the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

[2147] 3. Prompt generation: Based on the acquired text data, a prompt sentence is generated for the generative AI.

[2148] 4. AI response generation: The generative AI model generates the optimal response based on the prompt.

[2149] 5. Order fulfillment: Based on the generated response, the order is completed using the food delivery API.

[2150] Specific examples

[2151] When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model receives the prompt "The user wants to order: pizza," the model then suggests the best pizza restaurant and menu item, and completes the order using a food delivery API.

[2152] Prompt Sentence Examples

[2153] "User wants to order: Pizza"

[2154] In this way, users can easily and safely use food delivery services using only their voice, without having to operate their smartphones.

[2155] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2156] Step 1:

[2157] The user places their finger on the fingerprint sensor.

[2158] Input: User's fingerprint data.

[2159] Processing: The fingerprint authentication library analyzes the fingerprint data and matches it with registered fingerprint data.

[2160] Output: Authentication result (success or failure).

[2161] Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

[2162] Step 2:

[2163] The user presses the operating means (one-click button) of the accessory device.

[2164] Input: User action (button press).

[2165] Processing: Detects that the operating means has been pressed and activates the voice recognition mode.

[2166] Output: Activate speech recognition mode.

[2167] Specific operation: The sensor detects that a button has been pressed, and the speech recognition library begins preparing for voice input.

[2168] Step 3:

[2169] The user verbally states the order details.

[2170] Input: User's voice data.

[2171] Processing: The microphone captures the voice data, which is then converted into text data by a speech recognition library.

[2172] Output: Text data (order details).

[2173] What it does: A microphone captures the user's voice, and a speech recognition library analyzes it and converts it into text.

[2174] Step 4:

[2175] The terminal generates a prompt sentence based on the acquired text data.

[2176] Input: Text data (order details).

[2177] Processing: Convert the text data into a prompt sentence of the form "The user wants to order: [Order details]".

[2178] Output: The prompt statement.

[2179] Specific operation: Converts text data into prompt sentences according to the format.

[2180] Step 5:

[2181] The device sends a prompt to the generative AI model.

[2182] Input: prompt statement.

[2183] Processing: A generative AI model analyzes the prompt and generates the optimal response.

[2184] Output: AI response (optimal order content).

[2185] Specific operation: The prompt sentence is sent to the generative AI model, and the AI ​​generates a response.

[2186] Step 6:

[2187] The device uses the generated response to complete the order using the food delivery API.

[2188] Input: AI response (optimal order content).

[2189] Processing: Fulfilling orders using food delivery APIs.

[2190] Output: Order confirmation (data indicating the order was successfully completed).

[2191] Specific behavior: Sends AI response to food delivery API and confirms order.

[2192] The server may further include means for recognizing the user's emotion using an emotion engine (described later), and may obtain an output from the generative AI using the voice data obtained by the voice recognition means, a prompt including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," the server may generate a prompt such as "User wants to order: pizza, user's emotion: sadness," and the generative AI model may suggest an optimal pizza restaurant and menu item taking the user's emotion into consideration, and complete the order using a food delivery API.

[2193] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2194] "Example 1"

[2195] One embodiment of the present invention provides an accessory device connectable to a smartphone. The device includes a means for enabling prompt input to a generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. The device also includes an emotion engine that recognizes the user's emotions. Specifically, the emotion engine recognizes the user's emotions from their tone of voice and facial expression, and feeds that information back to the generative AI. This enables the generative AI to generate an appropriate response based on the user's emotions.

[2196] "Example 2"

[2197] Another embodiment of the present invention provides a system in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone within the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotion from the tone of the voice data and feeds the results back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotion.

[2198] The processing flow of each embodiment will be described below.

[2199] "Example 1"

[2200] Step 1: The user operates an accessory device that can be connected to a smartphone.

[2201] Step 2: The user authentication means of the device authenticates the user. One possible authentication method is fingerprint authentication.

[2202] Step 3: The authenticated user prompts the generative AI.

[2203] Step 4: The device's emotion engine recognizes the user's emotions from their tone of voice and facial expressions.

[2204] Step 5: The emotion engine feeds that information back to the generative AI.

[2205] Step 6: The generative AI generates an appropriate response based on the user's emotions.

[2206] "Example 2"

[2207] Step 1: The user speaks to the accessory device.

[2208] Step 2: The microphone in the device captures the user's voice and sends the audio data to the emotion engine.

[2209] Step 3: The emotion engine analyzes the user's emotions from the tone of the voice data.

[2210] Step 4: The emotion engine feeds the results back to the generative AI.

[2211] Step 5: The generative AI generates an appropriate response based on the user's emotions.

[2212] Example 1

[2213] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2214] While conventional accessory devices that can be connected to smartphones allow prompt input to generative AI models, they have the problem of being unable to generate responses that take the user's emotional state into account. This makes it difficult to obtain appropriate responses that correspond to the user's intentions and emotions, resulting in a lack of improvement in the user experience.

[2215] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2216] In this invention, the server is an accessory device connectable to a smartphone, and includes: means for enabling prompt input to the generative AI model; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI model; emotion recognition means for recognizing the user's emotion; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the user's emotional state.

[2217] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer.

[2218] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[2219] A "generative AI model" is an artificial intelligence system that generates output such as text or speech based on input prompts.

[2220] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[2221] "User Authentication Measures" means technical measures for verifying a user's identity.

[2222] A "fingerprint authentication means" is a technical means for reading a user's fingerprint and authenticating them based on that information.

[2223] The "operation me...

Claims

[Claim 1] 1. A system including an accessory device connectable to a smart device, a user authentication means for authenticating a user who operates the accessory device by fingerprint authentication; an operation means for enabling an operation means of the accessory device when authentication by the user authentication means is successful; a button provided on the accessory device, the button being pressed to input an instruction to the generative AI after the operation means is enabled; a voice recognition means for recognizing a user's voice in response to pressing of the button; a means for acquiring an output of the generative AI using a prompt sentence including the voice data acquired by the voice recognition means and the generative AI; means for analyzing the voice data acquired by the voice recognition means and recognizing the user's emotion using an emotion engine that recognizes the user's emotion from the tone and content of the voice data; The acquiring means includes: generating a prompt sentence including the voice data acquired by the voice recognition means and the user's emotion recognized by the emotion engine; the prompt instructs the user to order from a food delivery service; The acquiring means acquires the order result using the generative AI and the food delivery API. system.

Images