System

Abstract

To provide a system configured to enable a prompt to be input quickly from a smartphone to a generative AI, resulting in more efficient utilization of the generative AI.SOLUTION: A system includes an accessory device which can be connected to a smart device, the system including: user authentication means which authenticates a user who operates the accessory device, by fingerprint authentication; a button provided in the accessory device, and pushed by a user authenticated by the user authentication means for inputting an instruction to a generative AI; voice recognition means which recognizes voice of the user in accordance with depression of the button; and means for acquiring an output of the generative AI using the generative AI and a prompt text including voice data acquired by the voice recognition means.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] When using generative AI, the time required to enter prompts is cumbersome, creating a digital divide for people who have difficulty using services, particularly the elderly. [Means for solving the problem]

[0005] An accessory device that can be connected to a smartphone is provided. This device includes a means for enabling prompt input to a generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI. The user authentication means is a fingerprint authentication means, and the operation means is a one-click operation means. This allows prompt input to the generative AI immediately from a smartphone, realizing smarter use of generative AI. It also contributes to bridging the digital divide for people who have difficulty using services, such as the elderly. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] do.

[0009] First, the terms used in the following description will be explained.

[0010] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0011] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0012] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0013] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0014] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0015] [First embodiment]

[0016] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0026] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0027] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0028] "Example 1"

[0029] As one embodiment of the present invention, an accessory device connectable to a smartphone is provided. This device includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI. Specifically, the accessory device is connected to a smartphone using a wireless communication means such as Bluetooth, and the prompt input to the generative AI is performed through an application running on the smartphone. As the user authentication means, a fingerprint authentication sensor is built into the device, and authentication is performed by matching the fingerprint information of a registered user. As the operation means, a prompt input to the generative AI is performed by a user pressing a button provided on the device.

[0030] "Example 2"

[0031] As a specific example of use, a user authenticates themselves by placing their finger on the fingerprint authentication sensor of the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is made to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

[0032] The process flow of each embodiment will be described below.

[0033] "Example 1"

[0034] Step 1: A user connects an accessory device connectable to a smartphone to the smartphone using a wireless communication means such as Bluetooth.

[0035] Step 2: The user places his or her finger on the fingerprint authentication sensor of the accessory device, and the fingerprint authentication sensor authenticates the user by matching the fingerprint information with that of the registered user.

[0036] Step 3: If the authentication is successful, the device's buttons are enabled.

[0037] Step 4: When the user presses a button on the device, an application on the smartphone is launched and a prompt input is sent to the generative AI.

[0038] "Example 2"

[0039] Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

[0040] Step 2: If authentication is successful, the device's buttons are enabled.

[0041] Step 3: When the user presses the button, an application on the smartphone is launched and prompts the generative AI to input the information.

[0042] Step 4: This will enable users to easily utilize generative AI without having to operate a smartphone.

[0043] Example 1

[0044] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0045] In conventional systems using generative AI models, users had to go through complicated operations and multi-step authentication when entering a prompt, which resulted in poor user experience. In addition, there was a lack of efficient means to quickly and accurately obtain the generated results. This reduced the convenience of users when using generative AI models.

[0046] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0047] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a means for running an application for inputting a prompt to the generative AI on a smartphone, and a means for sending a prompt sentence to the generative AI model and receiving a generation result. This allows a user to input a prompt sentence to the generative AI model with a simple operation and quickly and accurately obtain a generation result.

[0048] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer, and can have a variety of functions installed on it.

[0049] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[0050] "Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompt sentences.

[0051] "Prompt input" is the act of inputting instructions or questions to a generative AI.

[0052] "User authentication means" refers to technology or devices used to verify a user's identity.

[0053] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and verifies the user's identity based on that information.

[0054] The "operation means" refers to an interface or device that allows the user to operate the accessory device or smartphone.

[0055] A "one-click operation means" is an interface or device that allows a user to execute a specific function with a single click.

[0056] An "application" is software that runs on a smartphone and provides specific functions or services.

[0057] "Generated results" are output such as text and images that the generative AI generates based on the prompt text.

[0058] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0059] The present invention is a system for easily inputting prompts to a generative AI model using an accessory device connectable to a smartphone. A specific embodiment of the system will be described below.

[0060] Hardware and Software Configuration

[0061] Accessory equipment:

[0062] An accessory device is an external device that is connected to a smartphone and has the following main functions:

[0063] User authentication method: Equipped with a fingerprint authentication sensor to read the user's fingerprint information.

[0064] Operation: Equipped with a one-click button, users can easily operate it.

[0065] Smartphone:

[0066] The smartphone is connected to the accessory device using a wireless communication means such as Bluetooth. An application for inputting prompts to the generative AI model is installed on the smartphone.

[0067] Generative AI models:

[0068] The generative AI model runs on a server and generates text and images based on prompts sent from the smartphone.

[0069] System Operation

[0070] User authentication:

[0071] The user places his / her finger on the fingerprint authentication sensor installed in the accessory device. The accessory device reads the fingerprint information and compares it with the fingerprint information stored in the internal database. If the authentication is successful, the accessory device sends a signal of authentication success to the smartphone.

[0072] Prompt Input:

[0073] After successful authentication, the user presses a button on the accessory device. The smartphone application starts and a prompt input screen is displayed. The user inputs a prompt sentence for the generative AI model.

[0074] Send to generative AI model:

[0075] The smartphone sends the input prompt sentence to the generative AI model. The server operates the generative AI model based on the received prompt sentence and creates the generated result.

[0076] View the generated results:

[0077] The server sends the generated results to the smartphone. The smartphone displays the received generated results to the user. The user checks the displayed generated results and saves or shares them as necessary.

[0078] Examples

[0079] The specific steps a user takes if they want a generative AI to create a new poem are as follows:

[0080] 1. User Authentication:

[0081] The user places his or her finger on the fingerprint authentication sensor of the accessory device.

[0082] The accessory device reads the fingerprint and performs authentication.

[0083] If the authentication is successful, a signal indicating successful authentication is sent to the smartphone.

[0084] 2. Prompt input:

[0085] The user presses a button on the accessory device.

[0086] The smartphone application will start and a prompt input screen will be displayed.

[0087] The user enters the prompt text "Create a new poem."

[0088] 3. Send to generative AI model:

[0089] The smartphone sends the entered prompt text to the generative AI model.

[0090] The server runs a generative AI model based on the prompt text to generate a poem.

[0091] 4. View the generated results:

[0092] The server sends the generated poem to the smartphone.

[0093] The smartphone displays the received poem to the user.

[0094] The user can review the displayed poem and save or share it as desired.

[0095] In this way, a user can use the accessory device and a smartphone to provide prompt input to the generative AI model and obtain generative results.

[0096] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0097] Step 1: Connect accessory devices

[0098] Input: A user opens the Bluetooth settings on their smartphone and selects an accessory device.

[0099] How it works: A smartphone establishes a connection with an accessory device using Bluetooth.

[0100] Output: The smartphone will inform you that the connection was successful.

[0101] Step 2: User authentication

[0102] Input: The user places their finger on the fingerprint sensor on the accessory device.

[0103] How it works: The accessory device reads your fingerprint and matches it with fingerprint information stored in an internal database.

[0104] Output: If authentication is successful, the accessory device sends a signal of authentication success to the smartphone.

[0105] Step 3: Prepare for prompts

[0106] Input: The smartphone receives a signal that authentication was successful.

[0107] How it works: The smartphone displays a prompt input screen for the generative AI.

[0108] Output: The user sees the prompt input screen and begins entering the prompt statement.

[0109] Step 4: Prompt Input

[0110] Input: The user inputs a prompt for the generative AI through a smartphone application.

[0111] How it works: The smartphone receives the input prompt text and formats it internally.

[0112] Output: The prepared prompt is ready to be sent to the generative AI model.

[0113] Step 5: Send to generative AI model

[0114] Input: The smartphone sends a prepared prompt to the generative AI model.

[0115] Operation: The server runs the generative AI model based on the received prompt sentence and creates a generated result.

[0116] Output: The generated results are sent from the server to the smartphone.

[0117] Step 6: Viewing the generation results

[0118] Input: The smartphone receives the generated results from the server.

[0119] How it works: The smartphone displays the generated results it receives to the user.

[0120] Output: The user checks the displayed generated results and can save or share them as needed.

[0121] (Application example 1)

[0122] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0123] To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and insufficient security for user authentication, which creates the risk of operational errors and unauthorized access. This can lead to reduced productivity and safety in factories.

[0124] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0125] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a means for generating instructions for managing and operating a robot in a factory, and a means for transmitting the instructions to the robot by wireless communication means. This makes it possible to efficiently and safely manage and operate the robot in the factory.

[0126] An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

[0127] "Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

[0128] "Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

[0129] "User authentication means" is a function for confirming and authenticating the identity of a user.

[0130] A "fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

[0131] The "operation means" is an interface that allows the user to input prompts to the generative AI.

[0132] The "one-click operation means" is a function that allows the user to input prompts to the generative AI with a single click.

[0133] "Means for generating instructions for managing and operating robots in a factory" refers to a function for generating specific work instructions for robots in a factory.

[0134] "Wireless communication means" refers to a function for transmitting and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

[0135] The "means for transmitting to the robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

[0136] This invention relates to a system for efficiently and safely managing and operating robots in a factory using an accessory device connectable to a smartphone. A specific embodiment of this system will be described below.

[0137] System Configuration

[0138] The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

[0139] Hardware

[0140] Smartphone: Functions as a user interface and is connected to accessory devices via wireless communication means.

[0141] Accessory device: Connected to a smartphone, the accessory device includes a fingerprint authentication means and an operation means.

[0142] Fingerprint authentication method: Scans the user's fingerprint to authenticate them.

[0143] Wireless communication means: Data is sent and received between the smartphone and the factory robot using Bluetooth, Wi-Fi, etc.

[0144] Factory robot: Performs actions according to instructions.

[0145] software

[0146] Generative AI model: Generates specific instructions for factory robots based on user input prompts.

[0147] Smartphone application: Provides the user interface and manages communication with the generative AI model.

[0148] Operating procedure

[0149] 1. User authentication: The user places a finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

[0150] 2. Prompt input: An authenticated user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[0151] 3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

[0152] 4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

[0153] 5. Robot operation: The factory robot performs the operation according to the instructions it receives.

[0154] Examples

[0155] For example, when a robot needs to perform a specific task in a factory, a manager can use a smartphone and accessory devices to give the robot instructions.

[0156] "Move to assembly line 3 and start welding."

[0157] "Move to the inspection area and perform quality checks."

[0158] "Go get part A and take it to assembly line 2."

[0159] This makes it possible to efficiently and safely manage and operate robots in factories. The system utilizes user authentication and generative AI to reduce the risk of incorrect operation and unauthorized access, improving factory productivity and safety.

[0160] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0161] Step 1:

[0162] The user places a finger on the accessory device and is authenticated by the fingerprint authentication means.

[0163] Input: User's fingerprint

[0164] Data processing: The fingerprint authentication means scans the fingerprint and compares it with registered fingerprint data.

[0165] Output: Authentication result (success or failure)

[0166] How it works: The fingerprint sensor scans the user's fingerprint and matches it against an internal database for authentication.

[0167] Step 2:

[0168] If the authentication is successful, the user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[0169] Input: Authentication result (success), user prompt

[0170] Data processing: If the authentication result is successful, a prompt sentence is sent to the generative AI model.

[0171] Output: prompt input to the generative AI model

[0172] Specific operation: When the user presses the button, the smartphone application sends a prompt sentence to the generative AI model.

[0173] Step 3:

[0174] Based on the prompts, a generative AI model generates specific instructions for factory robots.

[0175] Input: User prompt text

[0176] Data processing: A generative AI model analyzes the prompt and generates appropriate instructions.

[0177] Output: Instructions for the factory robots

[0178] Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

[0179] Step 4:

[0180] The generated instructions are transmitted to the factory robot using wireless communication means.

[0181] Input: Instructions from a generative AI model

[0182] Data processing: Instructions are sent to factory robots via wireless communication means.

[0183] Output: Sending instructions to factory robots

[0184] Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth and Wi-Fi.

[0185] Step 5:

[0186] The factory robot operates according to the instructions it receives.

[0187] Input: Instructions for the factory robots

[0188] Data processing: The factory robot analyzes the instructions and performs the corresponding actions.

[0189] Output: Robot movement

[0190] Specific actions: The factory robot performs actions according to instructions such as "move to assembly line 3 and start welding."

[0191] Example 2

[0192] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0193] In order to use conventional generative AI models, users had to operate their smart devices directly and input prompt text. This operation was tedious and inconvenient, especially when their hands were full or in situations where a quick response was required. Also, from a security standpoint, user authentication was sometimes insufficient, creating a risk of unauthorized use by third parties. There is a demand for a system that can solve these issues and make it easier and safer to use generative AI models.

[0194] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0195] In this invention, the server includes a user authentication means, an operation means for an authenticated user to operate, a means for transmitting a signal to the smart device using the operation means to launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without directly operating the smart device, and the user authentication means also improves security.

[0196] "User authentication means" means the means for verifying a user's identity and granting access to a system.

[0197] "Operation means" refers to an interface or device that allows a user to operate the system.

[0198] A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

[0199] The "means for transmitting a signal" refers to a means for transmitting specific instructions or data from the operating means to the smart device.

[0200] A "specific application" is a software program designed to provide a particular function or service.

[0201] A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

[0202] A "prompt" is a text input used to give instructions or ask a question to a generative AI model.

[0203] A "means for generating a response" is a means by which a generative AI model creates an appropriate response based on a prompt sentence.

[0204] The "means for sending a response" refers to a means for sending the generated response to the smart device.

[0205] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0206] The present invention is a system for enabling a user to utilize a generative AI model without directly operating a smart device. A specific embodiment of the system will be described below.

[0207] Hardware and Software Used

[0208] Hardware: Accessory devices (with fingerprint authentication sensor), smart devices (smartphones, etc.)

[0209] Software: Fingerprint authentication systems, smart device applications, generative AI models

[0210] System Configuration

[0211] 1. User authentication method:

[0212] The user places his or her finger on the fingerprint sensor on the accessory device.

[0213] The fingerprint authentication system captures fingerprint data and authenticates the user.

[0214] 2. Operating means:

[0215] If authentication is successful, the button on the accessory device is enabled.

[0216] The user presses an enabled button.

[0217] 3. Signal transmission means:

[0218] When the button is pressed, the accessory device sends a signal to the smart device.

[0219] The smart device receives the signal and launches a specific application.

[0220] 4. Prompt sending method:

[0221] The launched application sends a prompt to the generative AI model.

[0222] Prompts are pre-defined statements that contain the instructions or questions the user wants to ask the generative AI model.

[0223] 5. Response Generation Method:

[0224] The server uses a generative AI model to generate a response based on the received prompt.

[0225] 6. Response sending method:

[0226] The server generates a response and sends it to the smart device.

[0227] The smart device receives the response and displays it to the user.

[0228] Examples

[0229] When a user places their finger on the fingerprint authentication sensor of the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device to launch an application. The application launches and sends a prompt sentence, "Tell me about the weather today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can confirm the information, "Today's weather is sunny," on the screen.

[0230] Examples of prompt statements

[0231] "Tell me about the weather today."

[0232] "Check the schedule for the next meeting."

[0233] "Tell me the latest news."

[0234] This system allows users to easily use generative AI models without having to directly operate a smart device, and security is also improved through user authentication measures.

[0235] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0236] Program processing flow

[0237] Step 1:

[0238] The user places his or her finger on the fingerprint authentication sensor of the accessory device.

[0239] Input: User's fingerprint

[0240] Data processing: The fingerprint sensor captures the fingerprint data.

[0241] Output: The captured fingerprint data.

[0242] Step 2:

[0243] The device captures the fingerprint data and authenticates the user.

[0244] Input: Captured fingerprint data

[0245] Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

[0246] Output: Authentication result (success or failure)

[0247] Step 3:

[0248] If authentication is successful, the device's buttons are enabled.

[0249] Input: Authentication result (success)

[0250] Data processing: If authentication is successful, enable the device buttons.

[0251] Output: Enabled button

[0252] Step 4:

[0253] The user presses a button.

[0254] Input: Enabled button

[0255] Data processing: The terminal detects that a button has been pressed.

[0256] Output: Button press signal

[0257] Step 5:

[0258] The terminal sends a signal to the smart device to launch the application.

[0259] Input: Button press signal

[0260] Data processing: The terminal sends a signal to the smart device to launch a specific application.

[0261] Output: Launched application

[0262] Step 6:

[0263] The application sends a prompt to the generative AI model.

[0264] Input: The application that was launched

[0265] Data processing: The application sends pre-defined prompts to the generative AI model.

[0266] Output: The prompt sent to the generative AI model

[0267] Step 7:

[0268] The server generates a response using a generative AI model.

[0269] Input: The prompt sent to the generative AI model

[0270] Data computation: A generative AI model analyzes the prompt and generates an appropriate response.

[0271] Output: The generated response

[0272] Step 8:

[0273] The server sends the response to the smart device.

[0274] Input: The generated response

[0275] Data processing: The server generates a response and sends it to the smart device.

[0276] Output: Response sent to smart device

[0277] Step 9:

[0278] The smart device displays the response.

[0279] Input: Response sent to smart device

[0280] Data processing: The smart device displays the response received to the user.

[0281] Output: The response displayed to the user

[0282] In this way, users can utilize generative AI models without directly operating a smart device.

[0283] (Application example 2)

[0284] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0285] Conventional food delivery systems require users to use their smartphones to place orders, which makes operation complicated. There are also security risks due to insufficient user authentication. Furthermore, ordering systems using voice recognition are not widespread, which reduces user convenience. There is a need to provide a system that solves these problems and allows users to use food delivery services easily and safely.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0287] An accessory device connectable to a smart device includes a user authentication means for authenticating a user who operates the accessory device by fingerprint authentication, a button provided on the accessory device that is pressed by a user authenticated by the user authentication means to input instructions to the generative AI, and a voice recognition means for recognizing the user's voice in response to pressing of the button. The server includes a means for acquiring an output of the generative AI using a prompt sentence including voice data acquired by the voice recognition means and the generative AI. The prompt sentence instructs ordering through a food delivery service, and the acquiring means acquires the result of the order using a food delivery API. This allows a user to easily and safely use food delivery services using only voice, without operating a smartphone.

[0288] A "smart device" is a portable electronic device, such as a smartphone or tablet, that has Internet connectivity and can run applications.

[0289] An "accessory device" is additional hardware that is connected to a smart device to provide a specific function.

[0290] "Generative AI" is an artificial intelligence technique that generates text and other content based on instructions or prompts from a user.

[0291] "Instruction input" refers to commands or requests that a user inputs to a generative AI.

[0292] "User authentication means" refers to technology or devices for confirming and authenticating the identity of a user.

[0293] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and authenticates the user based on that information.

[0294] "Operation means" refers to the interface or buttons that allow the user to operate the accessory device or smart device.

[0295] A "voice recognition means" is a technology or device that analyzes a user's voice and converts it into text data.

[0296] "Voice data" refers to data that is a digital recording of a user's voice.

[0297] "Transmission means" refers to techniques or devices for transmitting data to other devices or systems.

[0298] The present invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. A specific embodiment of this system will be described below.

[0299] System Configuration

[0300] The system consists of the following main components:

[0301] 1. Smart device: A portable electronic device such as a smartphone or tablet that has internet connectivity and can run applications.

[0302] 2. Accessory device: Additional hardware used by connecting to a smart device, including fingerprint authentication means, operation means, and voice recognition means.

[0303] 3. Generative AI: Artificial intelligence techniques that generate text and other content based on instructions or prompts from the user.

[0304] Program Processing

[0305] Hardware

[0306] Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

[0307] Smart device: Functions as the execution environment for applications.

[0308] Microphone: Captures the user's voice as a means of voice recognition.

[0309] software

[0310] Fingerprint authentication library: A software module for performing fingerprint authentication.

[0311] Speech Recognition Library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

[0312] Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3 (registered trademark)).

[0313] Food Delivery API: Place orders using the API of food delivery services.

[0314] Processing flow

[0315] 1. User authentication: The user authenticates by placing his / her finger on the fingerprint authentication sensor of the accessory device. If authentication is successful, the next step is performed.

[0316] 2. Voice input: The user presses the operation means (one-click button) of the accessory device and speaks the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

[0317] 3. Prompt generation: Based on the acquired text data, a prompt sentence is generated for the generative AI.

[0318] 4. AI response generation: The generative AI model generates the optimal response based on the prompt.

[0319] 5. Order fulfillment: Based on the response generated, complete the order using the food delivery API.

[0320] Examples

[0321] When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model is given the prompt "User wants to order: pizza," the generative AI model suggests the best pizza restaurant and menu, and completes the order using a food delivery API.

[0322] Examples of prompt statements

[0323] "User wants to order: pizza"

[0324] In this way, users can use voice commands to easily and safely receive food delivery services without having to operate a smartphone.

[0325] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0326] Step 1:

[0327] The user places his or her finger on the fingerprint sensor.

[0328] Input: The user's fingerprint data.

[0329] Processing: The fingerprint authentication library analyzes the fingerprint data and matches it with registered fingerprint data.

[0330] Output: The authentication result (success or failure).

[0331] Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

[0332] Step 2:

[0333] The user presses the operating means (one-click button) of the accessory device.

[0334] Input: User action (button press).

[0335] Processing: Detects that an operating means has been pressed and activates the voice recognition mode.

[0336] Output: Activate speech recognition mode.

[0337] Specific operation: The sensor detects that a button has been pressed, and the speech recognition library begins preparing for voice input.

[0338] Step 3:

[0339] The user verbally communicates the order.

[0340] Input: The user's voice data.

[0341] Processing: The microphone picks up the voice data, and the speech recognition library converts it into text data.

[0342] Output: Text data (order details).

[0343] What happens: A microphone captures the user's voice, and a speech recognition library analyzes the speech and converts it into text.

[0344] Step 4:

[0345] The terminal generates a prompt sentence based on the acquired text data.

[0346] Input: Text data (order details).

[0347] Processing: The text data is converted into a prompt sentence of the form "The user would like to order: [order details]".

[0348] Output: The prompt statement.

[0349] Specific operation: Convert text data into prompt sentences according to the format.

[0350] Step 5:

[0351] The device sends a prompt to the generative AI model.

[0352] Input: prompt statement.

[0353] Processing: The generative AI model analyzes the prompt and generates the most appropriate response.

[0354] Output: AI response (optimal order content).

[0355] Specific operation: The prompt sentence is sent to the generative AI model, and the AI ​​generates a response.

[0356] Step 6:

[0357] The device uses the generated response to complete the order using the food delivery API.

[0358] Input: AI response (optimal order content).

[0359] Processing: Fulfilling the order using the food delivery API.

[0360] Output: Order confirmation (data indicating the order was successfully completed).

[0361] Specific behavior: Send the AI ​​response to the food delivery API and confirm the order.

[0362] The server may further include a means for recognizing the user's emotion using an emotion engine described later, and may obtain an output of the generative AI using the voice data obtained by the voice recognition means, a prompt sentence including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," a prompt sentence "User wants to order: pizza, user's emotion: sadness" is generated, and the generative AI model proposes an optimal pizza restaurant and menu considering the user's emotion, and completes the order using a food delivery API.

[0363] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0364] "Example 1"

[0365] As one embodiment of the present invention, an accessory device connectable to a smartphone is provided. The device includes a means for enabling prompt input to the generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI. Furthermore, the device includes an emotion engine that recognizes the user's emotions. Specifically, the device recognizes emotions from the user's tone of voice and facial expressions, and feeds back the information to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

[0366] "Example 2"

[0367] As another embodiment of the present invention, a system is provided in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone in the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotions from the tone of the voice data and feeds back the results to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

[0368] The process flow of each embodiment will be described below.

[0369] "Example 1"

[0370] Step 1: A user operates an accessory device that can be connected to a smartphone.

[0371] Step 2: The user authentication means of the device authenticates the user. One possible authentication method is fingerprint authentication.

[0372] Step 3: The authenticated user prompts the generative AI.

[0373] Step 4: The device's emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[0374] Step 5: The emotion engine feeds that information back to the generative AI.

[0375] Step 6: The generative AI generates an appropriate response based on the user's emotions.

[0376] "Example 2"

[0377] Step 1: The user talks to the accessory device.

[0378] Step 2: The microphone in the device captures the user's voice and sends the voice data to the emotion engine.

[0379] Step 3: The emotion engine analyzes the user's emotion from the tone of the voice data.

[0380] Step 4: The emotion engine feeds its results back to the generative AI.

[0381] Step 5: The generative AI generates an appropriate response based on the user's emotions.

[0382] Example 1

[0383] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0384] Conventional accessory devices that can be connected to smartphones allow prompt input to generative AI models, but they have the problem of being unable to generate responses that take into account the user's emotional state. This makes it difficult to obtain appropriate responses according to the user's intentions and emotions, resulting in a lack of improvement in the user experience.

[0385] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0386] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI model, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI model, an emotion recognition means for recognizing the emotion of the user, and a means for feeding back the emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the emotional state of the user.

[0387] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer.

[0388] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[0389] A "generative AI model" is an artificial intelligence system that generates output, such as text or speech, based on input prompts.

[0390] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[0391] "User Authentication Means" means a technical means for verifying a user's identity.

[0392] A "fingerprint authentication means" is a technical means that reads a user's fingerprint and performs authentication based on that information.

[0393] The "operation means" is an interface that enables a user to operate an accessory device.

[0394] A "one-click operation means" is an interface that allows a user to complete an operation with a single click.

[0395] An "emotion recognition means" is a technical means for recognizing emotions by analyzing the user's tone of voice and facial expressions.

[0396] "Emotion information" is data relating to the user's emotional state obtained by emotion recognition means.

[0397] "Feedback means" refers to the technical means for transmitting acquired emotional information to the generative AI model.

[0398] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0399] The present invention is a system that uses an accessory device connectable to a smartphone to provide prompt input to a generative AI model and obtain a response that takes into account the emotional state of the user. A specific embodiment of the system is described below.

[0400] Hardware and Software Configuration

[0401] Accessory Equipment

[0402] The accessory device is connected to the smartphone using wireless communication means such as Bluetooth. The device includes the following main components:

[0403] Fingerprint authentication sensor: Reads the user's fingerprint for authentication.

[0404] Action button: An interface that allows the user to initiate prompt input.

[0405] Emotion recognition engine: Analyzes the user's tone of voice and facial expressions to obtain emotional information.

[0406] Smartphone

[0407] The smartphone runs a dedicated application that works in conjunction with the accessory device and provides the following functions:

[0408] Prompt input interface: A screen where users can input instructions or questions to a generative AI model.

[0409] Data transmission function: Sends prompts and emotion information to the server.

[0410] server

[0411] The server provides a backend service that communicates data between the smartphone and the generative AI model. The server has the following functions:

[0412] Data relay function: Transmits prompts and emotion information received from a smartphone to the generative AI model.

[0413] Response reception function: Receives responses from the generative AI model and transfers them to the smartphone.

[0414] Examples

[0415] Consider a scenario in which a user uses an accessory device connected to a smartphone to ask a generative AI model questions.

[0416] 1. User Authentication

[0417] The user places their finger on the accessory device's fingerprint authentication sensor to authenticate themselves. The terminal (smartphone) receives the fingerprint data from the accessory device and compares it with registered fingerprint data. If authentication is successful, the user can proceed to the next step.

[0418] 2. Prompt Input

[0419] The user opens a dedicated application on their smartphone and enters a prompt, for example, "What's the weather like today?" The device prepares to send this prompt to the generative AI model.

[0420] 3. Emotion recognition

[0421] The accessory device uses an emotion engine that senses the user's tone of voice and facial expressions to analyze the user's emotional state. The device prepares this emotion data for transmission to the generative AI model.

[0422] 4. Sending prompts to generative AI models

[0423] The device sends the prompt and emotion data entered by the user to the server, which then sends this data to the generative AI model, which then generates an appropriate response based on the prompt and emotion data.

[0424] 5. Receiving a response from the generative AI model

[0425] The server receives the response from the generative AI model and forwards it to the device, which prepares the response to be displayed to the user.

[0426] 6. Viewing the response

[0427] The device displays the response received from the generative AI model to the user, for example, "It's sunny today. Have a nice day." The user can review this response and enter further prompts if necessary.

[0428] Examples of prompt statements

[0429] "How's the weather today?"

[0430] "Please tell me when the next meeting is scheduled."

[0431] "Please tell me the latest news."

[0432] In this way, users can naturally interact with generative AI models through accessory devices.

[0433] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0434] Step 1: User authentication

[0435] The user places his / her finger on the fingerprint authentication sensor of the accessory device. The terminal receives the fingerprint data from the accessory device and compares it with the registered fingerprint data. The input is the user's fingerprint data and the output is the authentication result. The terminal analyzes the fingerprint data and checks whether it matches the registered data. If the authentication is successful, the terminal displays "Authentication successful."

[0436] Step 2: Connect your accessory device to your smartphone

[0437] The terminal establishes a connection with the accessory device via Bluetooth communication. The input is a Bluetooth connection request, and the output is a connection status. The terminal receives a connection request from the accessory device and establishes a connection. The server monitors this connection and relays data as necessary. If the connection is successful, the terminal displays "Connection successful."

[0438] Step 3: Prompt Input

[0439] The user opens a dedicated application on their smartphone and inputs a prompt. For example, they input "What's the weather like today?" The input is the user's prompt sentence, and the output is the prompt data. The device prepares to send this prompt to the generative AI model. When the user has completed the input, the device displays "Prompt input completed."

[0440] Step 4: Emotion Recognition

[0441] The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions. The input is the user's tone of voice and facial expression data, and the output is emotion data. The device prepares this emotion data to send to the generative AI model. Once emotion data is acquired, the device displays "Emotion data acquisition successful."

[0442] Step 5: Sending prompts to the generative AI model

[0443] The terminal sends the prompt and emotion data entered by the user to the server. The input is the prompt data and emotion data, and the output is the result of sending it to the server. The server sends this data to the generative AI model. If the data transmission is successful, the terminal displays "Data transmission successful."

[0444] Step 6: Receiving a response from the generative AI model

[0445] The server receives the response from the generative AI model and transfers it to the terminal. The input is the response data from the generative AI model, and the output is the transfer result to the terminal. The terminal prepares to display this response to the user. When the response is received, the terminal displays "Response received successfully."

[0446] Step 7: View the response

[0447] The terminal displays the response received from the generative AI model to the user. For example, it displays "It's sunny today. Have a nice day." The input is the response data from the generative AI model, and the output is the display result to the user. The user can check this response and enter further prompts if necessary.

[0448] (Application example 1)

[0449] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0450] Conventional food delivery services have difficulty making customized suggestions based on users' emotions and situations. In addition, there is a lack of means for users to use generative AI to simplify the ordering process. This can lead to poor user experience and reduced frequency of service use.

[0451] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0452] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, an emotion recognition means for recognizing the emotion of the user, and a means for feeding back the emotion information recognized by the emotion recognition means to the generative AI. This makes it possible to propose food delivery customized according to the emotion of the user.

[0453] An "accessory device connectable to a smartphone" is an external device that is connected to a smartphone using wireless communication means and provides a specific function.

[0454] "Generative AI" is an artificial intelligence system that generates responses or suggestions based on prompting input from a user.

[0455] "Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

[0456] "User authentication means" refers to technology for verifying the identity of a user and ensuring that only specific users can operate the system.

[0457] A "fingerprint authentication means" is a technology that scans a user's fingerprint and authenticates the user based on that information.

[0458] An "operation means" is a physical or software interface that allows a user to operate the system.

[0459] A "one-click operation means" is an interface that allows a user to execute a specific function with a single click.

[0460] "Emotion recognition means" is a technology that analyzes the user's tone of voice and facial expressions to recognize their emotions.

[0461] A "means for feeding back emotional information to a generative AI" is a function that sends recognized emotional information to a generative AI, enabling the generative AI to generate responses or suggestions based on that information.

[0462] This invention is a system that uses an accessory device connectable to a smartphone to simplify the ordering process when a user uses a food delivery service through generative AI and makes customized suggestions based on the user's emotions.

[0463] System Configuration

[0464] The system consists of the following main components:

[0465] 1. Accessory devices that can be connected to smartphones

[0466] Wireless communication means: Connects to a smartphone using Bluetooth or similar.

[0467] User authentication method: A fingerprint sensor is used to authenticate the user.

[0468] Control: A one-click control is used by the user to prompt the generative AI.

[0469] Emotion recognition: Analyzes the user's tone of voice and facial expressions to recognize emotions.

[0470] 2. Generative AI

[0471] Prompt input: Receives instructions or questions from the user and generates responses or suggestions.

[0472] Emotional feedback: Generate customized responses and suggestions based on recognized emotional information.

[0473] Program Processing

[0474] The server does the following:

[0475] 1. Bluetooth connection: Connect your smartphone and accessory device using Bluetooth.

[0476] 2. User authentication: The fingerprint sensor scans the user's fingerprint to authenticate them.

[0477] 3. Emotion Recognition: Recognize emotions by analyzing the user’s tone of voice and facial expressions.

[0478] 4. Prompt generation: Generate prompts for the generative AI based on the recognized emotions.

[0479] 5. AI Response Generation: Use generative AI to generate responses and suggestions to prompts.

[0480] Hardware and Software Used

[0481] Hardware: Bluetooth module, fingerprint sensor, camera (for face recognition), microphone (for voice recognition)

[0482] Software: Bluetooth connection library, fingerprint authentication library, emotion recognition engine, generative AI model

[0483] Examples

[0484] If the user is recognized as tired, the generative AI will suggest "food delivery options recommended for when you're tired." For example, the following prompts may be generated:

[0485] User is feeling tired, please suggest food delivery options.

[0486] In this way, it becomes possible to provide customized food delivery suggestions based on the user's emotions.

[0487] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0488] Step 1:

[0489] The server connects the smartphone and the accessory device using a Bluetooth module. The input is pairing information between the smartphone and the accessory device, and the output is the establishment of a connection. Specifically, the server uses a Bluetooth connection library to establish communication between the smartphone and the accessory device.

[0490] Step 2:

[0491] The device uses a fingerprint authentication sensor to scan the user's fingerprint and perform user authentication. The input is the user's fingerprint data, and the output is the authentication result. Specifically, the fingerprint authentication library is used to compare the scanned fingerprint data with registered data to determine whether authentication was successful.

[0492] Step 3:

[0493] The device uses a microphone and a camera to capture the user's tone of voice and facial expressions to perform emotion recognition. The input is voice data and image data, and the output is recognized emotion information. Specifically, an emotion recognition engine is used to analyze the voice data and image data to identify the user's emotion.

[0494] Step 4:

[0495] The server generates a prompt for the generative AI based on the recognized emotion information. The input is emotion information, and the output is a prompt. Specifically, the server converts the emotion information into text format and creates a prompt suitable for the generative AI.

[0496] Step 5:

[0497] The server uses a generative AI model to generate responses and suggestions to prompts. The input is the prompt, and the output is the generated response or suggestion. Specifically, the prompt is input to the generative AI model, and the AI ​​generates the response or suggestion.

[0498] Step 6:

[0499] The terminal presents the generated response or suggestion to the user. The input is the generated response or suggestion, and the output is the display to the user. Specifically, the response or suggestion is displayed on the smartphone display so that the user can check it.

[0500] Example 2

[0501] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0502] In conventional systems using generative AI models, users had to operate their smartphones, making them difficult to use. In addition, they were unable to generate appropriate responses according to the user's emotions, limiting the user experience. This made it difficult for users to effectively utilize generative AI models.

[0503] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0504] In this invention, the server includes a user authentication means, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI model, an emotion recognition means for capturing the user's voice data and analyzing the emotion, and a means for feeding back the emotion information analyzed by the emotion recognition means to the generative AI model. This allows the user to easily use the generative AI model without operating a smartphone, and further enables an appropriate response according to the user's emotion to be obtained.

[0505] "User authentication means" means a means for verifying a user's identity, and includes biometric authentication technologies such as fingerprint authentication and facial recognition.

[0506] "Operation means" refers to the means by which a user inputs prompts to the generated AI model, and includes interfaces such as buttons and touch screens.

[0507] An "emotion recognition means" is a means for capturing a user's voice data and analyzing the user's emotions from the tone and content of the data.

[0508] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on prompts from a user.

[0509] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[0510] The "feedback means" refers to a means for transmitting the emotion information analyzed by the emotion recognition means to the generative AI model.

[0511] The present invention provides a system including a user authentication means, an operation means, an emotion recognition means, and a feedback means. A specific embodiment of the system will be described below.

[0512] First, the user places his or her finger on the accessory device's fingerprint sensor. The fingerprint sensor captures the user's fingerprint data and sends it to the device. The device then sends this data to the server, which matches the fingerprint data with an existing database to authenticate the user. If authentication is successful, the device's buttons are enabled.

[0513] Next, when the user presses the enabled button, the device launches a dedicated application on the smartphone. This application has the function of inputting a prompt sentence to the generative AI model. For example, the prompt sentence "Please tell me about the weather today" is input to the generative AI model.

[0514] Furthermore, when the user speaks to the accessory device, the microphone in the terminal captures voice data. This voice data is sent to the emotion recognition means, which analyzes the tone of the voice data to analyze the user's emotion. The analysis result is sent to the generative AI model through the feedback means. The generative AI model generates an appropriate response based on this emotion information. For example, in response to the prompt "The user seems a little tired. Please generate an encouraging message," the generative AI model generates a response such as "Let's take a break today and refresh ourselves."

[0515] This system allows users to easily use generative AI models without operating a smartphone, and improves the user experience by providing appropriate responses according to the user's emotions.

[0516] As a concrete example, when a user places his / her finger on the fingerprint authentication sensor of an accessory device and presses a button after authentication is successful, an application on the smartphone is launched and the following prompt sentence is input to the generative AI model:

[0517] "Tell me about the weather today."

[0518] As another specific example, when a user speaks to the accessory device saying, "How are you feeling today?", the emotion recognition means recognizes the emotion from the tone of the user's voice and inputs the following prompt sentence into the generative AI model:

[0519] "The user seems a bit tired. Generate an uplifting message."

[0520] This allows users to easily utilize generative AI without having to operate a smartphone.

[0521] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0522] Step 1:

[0523] The user places his or her finger on the fingerprint sensor.

[0524] Input: User's fingerprint

[0525] Specific operation: When a user places his or her finger on the fingerprint sensor on the accessory device, the sensor glows blue.

[0526] Output: The captured fingerprint data.

[0527] Step 2:

[0528] The device captures fingerprint data and sends it to the server.

[0529] Input: Captured fingerprint data

[0530] Specific operation: The device captures fingerprint data from the fingerprint authentication sensor, and when the data is sent to the server, the device screen displays "Authentication...".

[0531] Output: Fingerprint data sent to the server

[0532] Step 3:

[0533] The server authenticates the fingerprint data.

[0534] Input: Fingerprint data sent to the server

[0535] Specific operation: The server compares the received fingerprint data with an existing database and authenticates the user.

[0536] Output: Authentication result (success or failure)

[0537] Step 4:

[0538] If the authentication is successful, the terminal button is enabled.

[0539] Input: Authentication result (success)

[0540] Specific operation: If the server succeeds in authentication, it sends a signal to the terminal to activate the button. The button on the terminal turns green.

[0541] Output: Enabled buttons

[0542] Step 5:

[0543] The user presses a button.

[0544] Input: Enabled button

[0545] Specific Action: The user presses an enabled button, which triggers the next step.

[0546] Output: Button press signal

[0547] Step 6:

[0548] The terminal launches an application on the smartphone.

[0549] Input: Button press signal

[0550] Specific operation: The device launches a dedicated application on the smartphone. The application logo appears on the smartphone screen.

[0551] Output: Launched application

[0552] Step 7:

[0553] The application prompts input to the generative AI model.

[0554] Input: The launched application

[0555] Specific operation: The application inputs a prompt sentence to the generative AI model. The screen displays "Sending prompt...". For example, the prompt sentence is "Tell me about the weather today."

[0556] Output: The prompt sent to the generative AI model

[0557] Step 8:

[0558] The user speaks to the accessory device.

[0559] Input: User's voice

[0560] Specific Actions: The user speaks to the accessory device, for example, "How are you feeling today?" The microphone on the device lights up red to indicate that it is capturing voice data.

[0561] Output: Captured audio data

[0562] Step 9:

[0563] The device captures voice data and sends it to the emotion engine.

[0564] Input: Captured audio data

[0565] Specific operation: The device uses the built-in microphone to capture voice data and transmits the data to the emotion engine.

[0566] Output: Audio data sent to the emotion engine

[0567] Step 10:

[0568] The emotion engine analyzes the user's emotions from the tone of the voice data.

[0569] Input: Audio data sent to the emotion engine

[0570] Specific operation: The emotion engine analyzes the tone of the voice data and analyzes the user's emotion. The device screen displays "Analyzing emotion...".

[0571] Output: Analyzed sentiment information

[0572] Step 11:

[0573] The emotion engine feeds the analysis results back into the generative AI model.

[0574] Input: Analyzed sentiment information

[0575] How it works: The emotion engine feeds back the analysis results to the generative AI model, for example sending information like "The user seems a bit tired."

[0576] Output: Emotion information sent to the generative AI model

[0577] Step 12:

[0578] The generative AI model generates responses based on the user's emotions.

[0579] Input: Emotion information sent to the generative AI model

[0580] Specific operation: The generative AI model generates an appropriate response based on the fed-back emotional information. For example, in response to the prompt "Generate an uplifting message," the model generates the response "Take a break today and refresh yourself." The response is displayed on the smartphone screen.

[0581] Output: The generated response

[0582] (Application example 2)

[0583] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0584] In conventional food delivery services, when users use their smartphones to place orders, the operation is complicated and appropriate suggestions based on the user's emotions are not given. In particular, there is a problem that the user experience is not improved for tired or stressed users because there are not enough suggestions for relaxing meals.

[0585] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0586] In this invention, the server includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a voice capture means for capturing the user's voice and recognizing emotions, an emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the user's emotion, and a means for the generative AI to generate an appropriate response based on the emotion recognized by the emotion recognition means. This enables the user to receive appropriate food delivery suggestions through voice and emotion recognition without operating a smartphone.

[0587] "Generative AI" is an artificial intelligence system that generates appropriate responses and suggestions based on user input.

[0588] "Prompt input" refers to the act of inputting instructions or questions to a generative AI.

[0589] An "accessory device" is an external device that can be connected to a smartphone and provides a specific function.

[0590] "User authentication means" means a means for verifying a user's identity and ensuring that only specific users can operate the system.

[0591] A "fingerprint authentication means" is a means for reading a user's fingerprint and authenticating him or her.

[0592] "Operation means" refers to means such as an interface or buttons that allow the user to operate the system.

[0593] A "one-click operation means" is a means that allows a user to complete an operation with a single click.

[0594] "Audio capture means" means for recording or capturing a user's voice.

[0595] An "emotion recognition means" is a means for analyzing captured voice data and recognizing a user's emotion.

[0596] "Response" refers to the reply or suggestion that a generative AI generates in response to user input.

[0597] A system for implementing this invention is configured as follows. First, a user uses an accessory device connected to a smartphone. This accessory device is equipped with a fingerprint authentication means, and authentication is performed by the user placing his / her finger on the accessory device. If authentication is successful, the operation means of the accessory device is enabled, and the user can use the one-click operation means to input a prompt to the generative AI.

[0598] Next, the voice capture means captures the user's voice. This voice data is sent to the emotion recognition means, which analyzes the user's emotions. The emotion recognition means recognizes the user's emotions from the tone and content of the voice data, and feeds the results back to the generative AI. The generative AI generates an appropriate response based on the recognized emotions.

[0599] Specifically, the following hardware and software are used: A fingerprint sensor is used as the fingerprint authentication means. A microphone and voice recognition software (e.g., speech_recognition library) are used as the voice capture means. An emotion engine (e.g., EmotionEngine) is used as the emotion recognition means. An AI generation model (e.g., AIGenerator) is used as the generative AI.

[0600] For example, if a user says, "I'm tired today," the voice capture means captures the voice, and the emotion recognition means detects "fatigue." Based on this emotion, the generative AI generates a response suggesting a "relaxing meal." In this way, appropriate food delivery suggestions are made according to the user's emotions.

[0601] Examples of prompts to input to a generative AI model include:

[0602] User Emotion: Fatigue

[0603] Prompt for generative AI: "Please suggest relaxing meals for a user who is tired."

[0604] This system allows users to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate a smartphone.

[0605] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0606] Step 1:

[0607] The user places his / her finger on the fingerprint authentication sensor of the accessory device connected to the smartphone. The input is the user's fingerprint data, and the output is the authentication result. The accessory device reads the fingerprint data and performs authentication by comparing it with its internal database. If authentication is successful, it proceeds to the next step.

[0608] Step 2:

[0609] If the authentication is successful, the operation means of the accessory device is enabled. The user uses the one-click operation means to input a prompt to the generative AI. The input is the user's click operation, and the output is the trigger for the prompt input. The accessory device detects the click operation and starts inputting a prompt to the generative AI.

[0610] Step 3:

[0611] The terminal's voice capture means captures the user's voice. The input is the user's voice data and the output is the captured voice data. The terminal's microphone records the user's voice and the voice recognition software converts the data into a digital format.

[0612] Step 4:

[0613] The captured voice data is sent to an emotion recognition means. The input is the captured voice data and the output is the recognized emotion data. The emotion engine analyzes the voice data and recognizes the user's emotion from tone and content.

[0614] Step 5:

[0615] The recognized emotion data is fed back to the generative AI. The input is the recognized emotion data, and the output is a prompt to the generative AI. The generative AI creates a prompt to generate an appropriate response based on the emotion data.

[0616] Step 6:

[0617] The generative AI generates an appropriate response based on the prompt. The input is the prompt and the output is the generated response. The generative AI analyzes the prompt and generates food delivery suggestions based on the user's sentiment.

[0618] Step 7:

[0619] The generated response is provided to the user. The input is the generated response, and the output is a suggestion to the user. Appropriate food delivery suggestions are displayed or audibly conveyed to the user through the display or audio output device of the terminal.

[0620] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0621] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generative AIs. Data generation model 58 is

[0622] , obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0623] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0624] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0625] [Second embodiment]

[0626] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0627] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0628] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0629] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0630] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0631] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0632] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0633] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0634] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0635] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0636] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0637] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0638] "Example 1"

[0639] As one embodiment of the present invention, an accessory device connectable to a smartphone is provided. This device includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI. Specifically, the accessory device is connected to a smartphone using a wireless communication means such as Bluetooth, and the prompt input to the generative AI is performed through an application running on the smartphone. As the user authentication means, a fingerprint authentication sensor is built into the device, and authentication is performed by matching the fingerprint information of a registered user. As the operation means, a prompt input to the generative AI is performed by a user pressing a button provided on the device.

[0640] "Example 2"

[0641] As a specific example of use, a user authenticates themselves by placing their finger on the fingerprint authentication sensor of the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is made to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

[0642] The process flow of each embodiment will be described below.

[0643] "Example 1"

[0644] Step 1: A user connects an accessory device connectable to a smartphone to the smartphone using a wireless communication means such as Bluetooth.

[0645] Step 2: The user places his or her finger on the fingerprint authentication sensor of the accessory device, and the fingerprint authentication sensor authenticates the user by matching the fingerprint information with that of the registered user.

[0646] Step 3: If the authentication is successful, the device's buttons are enabled.

[0647] Step 4: When the user presses a button on the device, an application on the smartphone is launched and a prompt input is sent to the generative AI.

[0648] "Example 2"

[0649] Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

[0650] Step 2: If authentication is successful, the device's buttons are enabled.

[0651] Step 3: When the user presses the button, an application on the smartphone is launched and prompts the generative AI to input the information.

[0652] Step 4: This will enable users to easily utilize generative AI without having to operate a smartphone.

[0653] Example 1

[0654] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0655] In conventional systems using generative AI models, users had to go through complicated operations and multi-step authentication when entering a prompt, which resulted in poor user experience. In addition, there was a lack of efficient means to quickly and accurately obtain the generated results. This reduced the convenience of users when using generative AI models.

[0656] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0657] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a means for running an application for inputting a prompt to the generative AI on a smartphone, and a means for sending a prompt sentence to the generative AI model and receiving a generation result. This allows a user to input a prompt sentence to the generative AI model with a simple operation and quickly and accurately obtain a generation result.

[0658] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer, and can have a variety of functions installed on it.

[0659] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[0660] "Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompt sentences.

[0661] "Prompt input" is the act of inputting instructions or questions to a generative AI.

[0662] "User authentication means" refers to technology or devices used to verify a user's identity.

[0663] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and verifies the user's identity based on that information.

[0664] The "operation means" refers to an interface or device that allows the user to operate the accessory device or smartphone.

[0665] A "one-click operation means" is an interface or device that allows a user to execute a specific function with a single click.

[0666] An "application" is software that runs on a smartphone and provides specific functions or services.

[0667] "Generated results" are output such as text and images that the generative AI generates based on the prompt text.

[0668] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0669] The present invention is a system for easily inputting prompts to a generative AI model using an accessory device connectable to a smartphone. A specific embodiment of the system will be described below.

[0670] Hardware and Software Configuration

[0671] Accessory equipment:

[0672] An accessory device is an external device that is connected to a smartphone and has the following main functions:

[0673] User authentication method: Equipped with a fingerprint authentication sensor to read the user's fingerprint information.

[0674] Operation: Equipped with a one-click button, users can easily operate it.

[0675] Smartphone:

[0676] The smartphone is connected to the accessory device using a wireless communication means such as Bluetooth. An application for inputting prompts to the generative AI model is installed on the smartphone.

[0677] Generative AI models:

[0678] The generative AI model runs on a server and generates text and images based on prompts sent from the smartphone.

[0679] System Operation

[0680] User authentication:

[0681] The user places his / her finger on the fingerprint authentication sensor installed in the accessory device. The accessory device reads the fingerprint information and compares it with the fingerprint information stored in the internal database. If the authentication is successful, the accessory device sends a signal of authentication success to the smartphone.

[0682] Prompt Input:

[0683] After successful authentication, the user presses a button on the accessory device. The smartphone application starts and a prompt input screen is displayed. The user inputs a prompt sentence for the generative AI model.

[0684] Send to generative AI model:

[0685] The smartphone sends the input prompt sentence to the generative AI model. The server operates the generative AI model based on the received prompt sentence and creates the generated result.

[0686] View the generated results:

[0687] The server sends the generated results to the smartphone. The smartphone displays the received generated results to the user. The user checks the displayed generated results and saves or shares them as necessary.

[0688] Examples

[0689] The specific steps a user takes if they want a generative AI to create a new poem are as follows:

[0690] 1. User Authentication:

[0691] The user places his or her finger on the fingerprint authentication sensor of the accessory device.

[0692] The accessory device reads the fingerprint and performs authentication.

[0693] If the authentication is successful, a signal indicating successful authentication is sent to the smartphone.

[0694] 2. Prompt input:

[0695] The user presses a button on the accessory device.

[0696] The smartphone application will start and a prompt input screen will be displayed.

[0697] The user enters the prompt text "Create a new poem."

[0698] 3. Send to generative AI model:

[0699] The smartphone sends the entered prompt text to the generative AI model.

[0700] The server runs a generative AI model based on the prompt text to generate a poem.

[0701] 4. View the generated results:

[0702] The server sends the generated poem to the smartphone.

[0703] The smartphone displays the received poem to the user.

[0704] The user can review the displayed poem and save or share it as desired.

[0705] In this way, a user can use the accessory device and a smartphone to provide prompt input to the generative AI model and obtain generative results.

[0706] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0707] Step 1: Connect accessory devices

[0708] Input: A user opens the Bluetooth settings on their smartphone and selects an accessory device.

[0709] How it works: A smartphone establishes a connection with an accessory device using Bluetooth.

[0710] Output: The smartphone will inform you that the connection was successful.

[0711] Step 2: User authentication

[0712] Input: The user places their finger on the fingerprint sensor on the accessory device.

[0713] How it works: The accessory device reads your fingerprint and matches it with fingerprint information stored in an internal database.

[0714] Output: If authentication is successful, the accessory device sends a signal of authentication success to the smartphone.

[0715] Step 3: Prepare for prompts

[0716] Input: The smartphone receives a signal that authentication was successful.

[0717] How it works: The smartphone displays a prompt input screen for the generative AI.

[0718] Output: The user sees the prompt input screen and begins entering the prompt statement.

[0719] Step 4: Prompt Input

[0720] Input: The user inputs a prompt for the generative AI through a smartphone application.

[0721] How it works: The smartphone receives the input prompt text and formats it internally.

[0722] Output: The prepared prompt is ready to be sent to the generative AI model.

[0723] Step 5: Send to generative AI model

[0724] Input: The smartphone sends a prepared prompt to the generative AI model.

[0725] Operation: The server runs the generative AI model based on the received prompt sentence and creates a generated result.

[0726] Output: The generated results are sent from the server to the smartphone.

[0727] Step 6: Viewing the generation results

[0728] Input: The smartphone receives the generated results from the server.

[0729] How it works: The smartphone displays the generated results it receives to the user.

[0730] Output: The user checks the displayed generated results and can save or share them as needed.

[0731] (Application example 1)

[0732] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0733] To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and insufficient security for user authentication, which creates the risk of operational errors and unauthorized access. This can lead to reduced productivity and safety in factories.

[0734] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0735] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a means for generating instructions for managing and operating a robot in a factory, and a means for transmitting the instructions to the robot by wireless communication means. This makes it possible to efficiently and safely manage and operate the robot in the factory.

[0736] An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

[0737] "Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

[0738] "Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

[0739] "User authentication means" is a function for confirming and authenticating the identity of a user.

[0740] A "fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

[0741] The "operation means" is an interface that allows the user to input prompts to the generative AI.

[0742] The "one-click operation means" is a function that allows the user to input prompts to the generative AI with a single click.

[0743] "Means for generating instructions for managing and operating robots in a factory" refers to a function for generating specific work instructions for robots in a factory.

[0744] "Wireless communication means" refers to a function for transmitting and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

[0745] The "means for transmitting to the robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

[0746] This invention relates to a system for efficiently and safely managing and operating robots in a factory using an accessory device connectable to a smartphone. A specific embodiment of this system will be described below.

[0747] System Configuration

[0748] The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

[0749] Hardware

[0750] Smartphone: Functions as a user interface and is connected to accessory devices via wireless communication means.

[0751] Accessory device: Connected to a smartphone, the accessory device includes a fingerprint authentication means and an operation means.

[0752] Fingerprint authentication method: Scans the user's fingerprint to authenticate them.

[0753] Wireless communication means: Data is sent and received between the smartphone and the factory robot using Bluetooth, Wi-Fi, etc.

[0754] Factory robot: Performs actions according to instructions.

[0755] software

[0756] Generative AI model: Generates specific instructions for factory robots based on user input prompts.

[0757] Smartphone application: Provides the user interface and manages communication with the generative AI model.

[0758] Operating procedure

[0759] 1. User authentication: The user places a finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

[0760] 2. Prompt input: An authenticated user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[0761] 3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

[0762] 4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

[0763] 5. Robot operation: The factory robot performs the operation according to the instructions it receives.

[0764] Examples

[0765] For example, when a robot needs to perform a specific task in a factory, a manager can use a smartphone and accessory devices to give the robot instructions.

[0766] "Move to assembly line 3 and start welding."

[0767] "Move to the inspection area and perform quality checks."

[0768] "Go get part A and take it to assembly line 2."

[0769] This makes it possible to efficiently and safely manage and operate robots in factories. The system utilizes user authentication and generative AI to reduce the risk of incorrect operation and unauthorized access, improving factory productivity and safety.

[0770] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0771] Step 1:

[0772] The user places a finger on the accessory device and is authenticated by the fingerprint authentication means.

[0773] Input: User's fingerprint

[0774] Data processing: The fingerprint authentication means scans the fingerprint and compares it with registered fingerprint data.

[0775] Output: Authentication result (success or failure)

[0776] How it works: The fingerprint sensor scans the user's fingerprint and matches it against an internal database for authentication.

[0777] Step 2:

[0778] If the authentication is successful, the user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[0779] Input: Authentication result (success), user prompt

[0780] Data processing: If the authentication result is successful, a prompt sentence is sent to the generative AI model.

[0781] Output: prompt input to the generative AI model

[0782] Specific operation: When the user presses the button, the smartphone application sends a prompt sentence to the generative AI model.

[0783] Step 3:

[0784] Based on the prompts, a generative AI model generates specific instructions for factory robots.

[0785] Input: User prompt text

[0786] Data processing: A generative AI model analyzes the prompt and generates appropriate instructions.

[0787] Output: Instructions for the factory robots

[0788] Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

[0789] Step 4:

[0790] The generated instructions are transmitted to the factory robot using wireless communication means.

[0791] Input: Instructions from a generative AI model

[0792] Data processing: Instructions are sent to factory robots via wireless communication means.

[0793] Output: Sending instructions to factory robots

[0794] Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth and Wi-Fi.

[0795] Step 5:

[0796] The factory robot operates according to the instructions it receives.

[0797] Input: Instructions for the factory robots

[0798] Data processing: The factory robot analyzes the instructions and performs the corresponding actions.

[0799] Output: Robot movement

[0800] Specific actions: The factory robot performs actions according to instructions such as "move to assembly line 3 and start welding."

[0801] Example 2

[0802] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0803] In order to use conventional generative AI models, users had to operate their smart devices directly and input prompt text. This operation was tedious and inconvenient, especially when their hands were full or in situations where a quick response was required. Also, from a security standpoint, user authentication was sometimes insufficient, creating a risk of unauthorized use by third parties. There is a demand for a system that can solve these issues and make it easier and safer to use generative AI models.

[0804] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0805] In this invention, the server includes a user authentication means, an operation means for an authenticated user to operate, a means for transmitting a signal to the smart device using the operation means to launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without directly operating the smart device, and the user authentication means also improves security.

[0806] "User authentication means" means the means for verifying a user's identity and granting access to a system.

[0807] "Operation means" refers to an interface or device that allows a user to operate the system.

[0808] A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

[0809] The "means for transmitting a signal" refers to a means for transmitting specific instructions or data from the operating means to the smart device.

[0810] A "specific application" is a software program designed to provide a particular function or service.

[0811] A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

[0812] A "prompt" is a text input used to give instructions or ask a question to a generative AI model.

[0813] A "means for generating a response" is a means by which a generative AI model creates an appropriate response based on a prompt sentence.

[0814] The "means for sending a response" refers to a means for sending the generated response to the smart device.

[0815] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0816] The present invention is a system for enabling a user to utilize a generative AI model without directly operating a smart device. A specific embodiment of the system will be described below.

[0817] Hardware and Software Used

[0818] Hardware: Accessory devices (with fingerprint authentication sensor), smart devices (smartphones, etc.)

[0819] Software: Fingerprint authentication systems, smart device applications, generative AI models

[0820] System Configuration

[0821] 1. User authentication method:

[0822] The user places his or her finger on the fingerprint sensor on the accessory device.

[0823] The fingerprint authentication system captures fingerprint data and authenticates the user.

[0824] 2. Operating means:

[0825] If authentication is successful, the button on the accessory device is enabled.

[0826] The user presses an enabled button.

[0827] 3. Signal transmission means:

[0828] When the button is pressed, the accessory device sends a signal to the smart device.

[0829] The smart device receives the signal and launches a specific application.

[0830] 4. Prompt sending method:

[0831] The launched application sends a prompt to the generative AI model.

[0832] Prompts are pre-defined statements that contain the instructions or questions the user wants to ask the generative AI model.

[0833] 5. Response Generation Method:

[0834] The server uses a generative AI model to generate a response based on the received prompt.

[0835] 6. Response sending method:

[0836] The server generates a response and sends it to the smart device.

[0837] The smart device receives the response and displays it to the user.

[0838] Examples

[0839] When a user places their finger on the fingerprint authentication sensor of the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device to launch an application. The application launches and sends a prompt sentence, "Tell me about the weather today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can confirm the information, "Today's weather is sunny," on the screen.

[0840] Examples of prompt statements

[0841] "Tell me about the weather today."

[0842] "Check the schedule for the next meeting."

[0843] "Tell me the latest news."

[0844] This system allows users to easily use generative AI models without having to directly operate a smart device, and security is also improved through user authentication measures.

[0845] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0846] Program processing flow

[0847] Step 1:

[0848] The user places his or her finger on the fingerprint authentication sensor of the accessory device.

[0849] Input: User's fingerprint

[0850] Data processing: The fingerprint sensor captures the fingerprint data.

[0851] Output: The captured fingerprint data.

[0852] Step 2:

[0853] The device captures the fingerprint data and authenticates the user.

[0854] Input: Captured fingerprint data

[0855] Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

[0856] Output: Authentication result (success or failure)

[0857] Step 3:

[0858] If authentication is successful, the device's buttons are enabled.

[0859] Input: Authentication result (success)

[0860] Data processing: If authentication is successful, enable the device buttons.

[0861] Output: Enabled button

[0862] Step 4:

[0863] The user presses a button.

[0864] Input: Enabled button

[0865] Data processing: The terminal detects that a button has been pressed.

[0866] Output: Button press signal

[0867] Step 5:

[0868] The terminal sends a signal to the smart device to launch the application.

[0869] Input: Button press signal

[0870] Data processing: The terminal sends a signal to the smart device to launch a specific application.

[0871] Output: Launched application

[0872] Step 6:

[0873] The application sends a prompt to the generative AI model.

[0874] Input: The application that was launched

[0875] Data processing: The application sends pre-defined prompts to the generative AI model.

[0876] Output: The prompt sent to the generative AI model

[0877] Step 7:

[0878] The server generates a response using a generative AI model.

[0879] Input: The prompt sent to the generative AI model

[0880] Data computation: A generative AI model analyzes the prompt and generates an appropriate response.

[0881] Output: The generated response

[0882] Step 8:

[0883] The server sends the response to the smart device.

[0884] Input: The generated response

[0885] Data processing: The server generates a response and sends it to the smart device.

[0886] Output: Response sent to smart device

[0887] Step 9:

[0888] The smart device displays the response.

[0889] Input: Response sent to smart device

[0890] Data processing: The smart device displays the response received to the user.

[0891] Output: The response displayed to the user

[0892] In this way, users can utilize generative AI models without directly operating a smart device.

[0893] (Application example 2)

[0894] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0895] Conventional food delivery systems require users to use their smartphones to place orders, which makes operation complicated. There are also security risks due to insufficient user authentication. Furthermore, ordering systems using voice recognition are not widespread, which reduces user convenience. There is a need to provide a system that solves these problems and allows users to use food delivery services easily and safely.

[0896] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0897] An accessory device connectable to a smart device includes a user authentication means for authenticating a user who operates the accessory device by fingerprint authentication, a button provided on the accessory device that is pressed by a user authenticated by the user authentication means to input instructions to the generative AI, and a voice recognition means for recognizing the user's voice in response to pressing of the button. The server includes a means for acquiring an output of the generative AI using a prompt sentence including voice data acquired by the voice recognition means and the generative AI. The prompt sentence instructs ordering through a food delivery service, and the acquiring means acquires the result of the order using a food delivery API. This allows a user to easily and safely use food delivery services using only voice, without operating a smartphone.

[0898] A "smart device" is a portable electronic device, such as a smartphone or tablet, that has Internet connectivity and can run applications.

[0899] An "accessory device" is additional hardware that is connected to a smart device to provide a specific function.

[0900] "Generative AI" is an artificial intelligence technique that generates text and other content based on instructions or prompts from a user.

[0901] "Instruction input" refers to commands or requests that a user inputs to a generative AI.

[0902] "User authentication means" refers to technology or devices for confirming and authenticating the identity of a user.

[0903] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and authenticates the user based on that information.

[0904] "Operation means" refers to the interface or buttons that allow the user to operate the accessory device or smart device.

[0905] A "voice recognition means" is a technology or device that analyzes a user's voice and converts it into text data.

[0906] "Voice data" refers to data that is a digital recording of a user's voice.

[0907] "Transmission means" refers to techniques or devices for transmitting data to other devices or systems.

[0908] The present invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. A specific embodiment of this system will be described below.

[0909] System Configuration

[0910] The system consists of the following main components:

[0911] 1. Smart device: A portable electronic device such as a smartphone or tablet that has internet connectivity and can run applications.

[0912] 2. Accessory device: Additional hardware used by connecting to a smart device, including fingerprint authentication means, operation means, and voice recognition means.

[0913] 3. Generative AI: Artificial intelligence techniques that generate text and other content based on instructions or prompts from the user.

[0914] Program Processing

[0915] Hardware

[0916] Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

[0917] Smart device: Functions as the execution environment for applications.

[0918] Microphone: Captures the user's voice as a means of voice recognition.

[0919] software

[0920] Fingerprint authentication library: A software module for performing fingerprint authentication.

[0921] Speech Recognition Library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

[0922] Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3 (registered trademark)).

[0923] Food Delivery API: Place orders using the API of food delivery services.

[0924] Processing flow

[0925] 1. User authentication: The user authenticates by placing his / her finger on the fingerprint authentication sensor of the accessory device. If authentication is successful, the next step is performed.

[0926] 2. Voice input: The user presses the operation means (one-click button) of the accessory device and speaks the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

[0927] 3. Prompt generation: Based on the acquired text data, a prompt sentence is generated for the generative AI.

[0928] 4. AI response generation: The generative AI model generates the optimal response based on the prompt.

[0929] 5. Order fulfillment: Based on the response generated, complete the order using the food delivery API.

[0930] Examples

[0931] When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model is given the prompt "User wants to order: pizza," the generative AI model suggests the best pizza restaurant and menu, and completes the order using a food delivery API.

[0932] Examples of prompt statements

[0933] "User wants to order: pizza"

[0934] In this way, users can use voice commands to easily and safely receive food delivery services without having to operate a smartphone.

[0935] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0936] Step 1:

[0937] The user places his or her finger on the fingerprint sensor.

[0938] Input: The user's fingerprint data.

[0939] Processing: The fingerprint authentication library analyzes the fingerprint data and matches it with registered fingerprint data.

[0940] Output: The authentication result (success or failure).

[0941] Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

[0942] Step 2:

[0943] The user presses the operating means (one-click button) of the accessory device.

[0944] Input: User action (button press).

[0945] Processing: Detects that an operating means has been pressed and activates the voice recognition mode.

[0946] Output: Activate speech recognition mode.

[0947] Specific operation: The sensor detects that a button has been pressed, and the speech recognition library begins preparing for voice input.

[0948] Step 3:

[0949] The user verbally communicates the order.

[0950] Input: The user's voice data.

[0951] Processing: The microphone picks up the voice data, and the speech recognition library converts it into text data.

[0952] Output: Text data (order details).

[0953] What happens: A microphone captures the user's voice, and a speech recognition library analyzes the speech and converts it into text.

[0954] Step 4:

[0955] The terminal generates a prompt sentence based on the acquired text data.

[0956] Input: Text data (order details).

[0957] Processing: The text data is converted into a prompt sentence of the form "The user would like to order: [order details]".

[0958] Output: The prompt statement.

[0959] Specific operation: Convert text data into prompt sentences according to the format.

[0960] Step 5:

[0961] The device sends a prompt to the generative AI model.

[0962] Input: prompt statement.

[0963] Processing: The generative AI model analyzes the prompt and generates the most appropriate response.

[0964] Output: AI response (optimal order content).

[0965] Specific operation: The prompt sentence is sent to the generative AI model, and the AI ​​generates a response.

[0966] Step 6:

[0967] The device uses the generated response to complete the order using the food delivery API.

[0968] Input: AI response (optimal order content).

[0969] Processing: Fulfilling the order using the food delivery API.

[0970] Output: Order confirmation (data indicating the order was successfully completed).

[0971] Specific behavior: Send the AI ​​response to the food delivery API and confirm the order.

[0972] The server may further include a means for recognizing the user's emotion using an emotion engine described later, and may obtain an output of the generative AI using the voice data obtained by the voice recognition means, a prompt sentence including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," a prompt sentence "User wants to order: pizza, user's emotion: sadness" is generated, and the generative AI model proposes an optimal pizza restaurant and menu considering the user's emotion, and completes the order using a food delivery API.

[0973] In addition, an emotion engine that estimates the emotion of the user may be further combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0974] "Example 1"

[0975] As one embodiment of the present invention, an accessory device connectable to a smartphone is provided. The device includes a means for enabling prompt input to the generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI. Furthermore, the device includes an emotion engine that recognizes the user's emotions. Specifically, the device recognizes emotions from the user's tone of voice and facial expressions, and feeds back the information to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

[0976] "Example 2"

[0977] As another embodiment of the present invention, a system is provided in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone in the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotions from the tone of the voice data and feeds back the results to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

[0978] The process flow of each embodiment will be described below.

[0979] "Example 1"

[0980] Step 1: A user operates an accessory device that can be connected to a smartphone.

[0981] Step 2: The user authentication means of the device authenticates the user. One possible authentication method is fingerprint authentication.

[0982] Step 3: The authenticated user prompts the generative AI.

[0983] Step 4: The device's emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[0984] Step 5: The emotion engine feeds that information back to the generative AI.

[0985] Step 6: The generative AI generates an appropriate response based on the user's emotions.

[0986] "Example 2"

[0987] Step 1: The user talks to the accessory device.

[0988] Step 2: The microphone in the device captures the user's voice and sends the voice data to the emotion engine.

[0989] Step 3: The emotion engine analyzes the user's emotion from the tone of the voice data.

[0990] Step 4: The emotion engine feeds its results back to the generative AI.

[0991] Step 5: The generative AI generates an appropriate response based on the user's emotions.

[0992] Example 1

[0993] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0994] Conventional accessory devices that can be connected to smartphones allow prompt input to generative AI models, but they have the problem of being unable to generate responses that take into account the user's emotional state. This makes it difficult to obtain appropriate responses according to the user's intentions and emotions, resulting in a lack of improvement in the user experience.

[0995] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0996] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI model, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI model, an emotion recognition means for recognizing the emotion of the user, and a means for feeding back the emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the emotional state of the user.

[0997] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer.

[0998] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[0999] A "generative AI model" is an artificial intelligence system that generates output, such as text or speech, based on input prompts.

[1000] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[1001] "User Authentication Means" means a technical means for verifying a user's identity.

[1002] A "fingerprint authentication means" is a technical means that reads a user's fingerprint and performs authentication based on that information.

[1003] The "operation means" is an interface that enables a user to operate an accessory device.

[1004] A "one-click operation means" is an interface that allows a user to complete an operation with a single click.

[1005] An "emotion recognition means" is a technical means for recognizing emotions by analyzing the user's tone of voice and facial expressions.

[1006] "Emotion information" is data relating to the user's emotional state obtained by emotion recognition means.

[1007] "Feedback means" refers to the technical means for transmitting acquired emotional information to the generative AI model.

[1008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1009] The present invention is a system that uses an accessory device connectable to a smartphone to provide prompt input to a generative AI model and obtain a response that takes into account the emotional state of the user. A specific embodiment of the system is described below.

[1010] Hardware and Software Configuration

[1011] Accessory Equipment

[1012] The accessory device is connected to the smartphone using wireless communication means such as Bluetooth. The device includes the following main components:

[1013] Fingerprint authentication sensor: Reads the user's fingerprint for authentication.

[1014] Action button: An interface that allows the user to initiate prompt input.

[1015] Emotion recognition engine: Analyzes the user's tone of voice and facial expressions to obtain emotional information.

[1016] Smartphone

[1017] The smartphone runs a dedicated application that works in conjunction with the accessory device and provides the following functions:

[1018] Prompt input interface: A screen where users can input instructions or questions to a generative AI model.

[1019] Data transmission function: Sends prompts and emotion information to the server.

[1020] server

[1021] The server provides a backend service that communicates data between the smartphone and the generative AI model. The server has the following functions:

[1022] Data relay function: Transmits prompts and emotion information received from a smartphone to the generative AI model.

[1023] Response reception function: Receives responses from the generative AI model and transfers them to the smartphone.

[1024] Examples

[1025] Consider a scenario in which a user uses an accessory device connected to a smartphone to ask a generative AI model questions.

[1026] 1. User Authentication

[1027] The user places their finger on the accessory device's fingerprint authentication sensor to authenticate themselves. The terminal (smartphone) receives the fingerprint data from the accessory device and compares it with registered fingerprint data. If authentication is successful, the user can proceed to the next step.

[1028] 2. Prompt Input

[1029] The user opens a dedicated application on their smartphone and enters a prompt, for example, "What's the weather like today?" The device prepares to send this prompt to the generative AI model.

[1030] 3. Emotion recognition

[1031] The accessory device uses an emotion engine that senses the user's tone of voice and facial expressions to analyze the user's emotional state. The device prepares this emotion data for transmission to the generative AI model.

[1032] 4. Sending prompts to generative AI models

[1033] The device sends the prompt and emotion data entered by the user to the server, which then sends this data to the generative AI model, which then generates an appropriate response based on the prompt and emotion data.

[1034] 5. Receiving a response from the generative AI model

[1035] The server receives the response from the generative AI model and forwards it to the device, which prepares the response to be displayed to the user.

[1036] 6. Viewing the response

[1037] The device displays the response received from the generative AI model to the user, for example, "It's sunny today. Have a nice day." The user can review this response and enter further prompts if necessary.

[1038] Examples of prompt statements

[1039] "How's the weather today?"

[1040] "Please tell me when the next meeting is scheduled."

[1041] "Please tell me the latest news."

[1042] In this way, users can naturally interact with generative AI models through accessory devices.

[1043] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1044] Step 1: User authentication

[1045] The user places his / her finger on the fingerprint authentication sensor of the accessory device. The terminal receives the fingerprint data from the accessory device and compares it with the registered fingerprint data. The input is the user's fingerprint data and the output is the authentication result. The terminal analyzes the fingerprint data and checks whether it matches the registered data. If the authentication is successful, the terminal displays "Authentication successful."

[1046] Step 2: Connect your accessory device to your smartphone

[1047] The terminal establishes a connection with the accessory device via Bluetooth communication. The input is a Bluetooth connection request, and the output is a connection status. The terminal receives a connection request from the accessory device and establishes a connection. The server monitors this connection and relays data as necessary. If the connection is successful, the terminal displays "Connection successful."

[1048] Step 3: Prompt Input

[1049] The user opens a dedicated application on their smartphone and inputs a prompt. For example, they input "What's the weather like today?" The input is the user's prompt sentence, and the output is the prompt data. The device prepares to send this prompt to the generative AI model. When the user has completed the input, the device displays "Prompt input completed."

[1050] Step 4: Emotion Recognition

[1051] The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions. The input is the user's tone of voice and facial expression data, and the output is emotion data. The device prepares this emotion data to send to the generative AI model. Once emotion data is acquired, the device displays "Emotion data acquisition successful."

[1052] Step 5: Sending prompts to the generative AI model

[1053] The terminal sends the prompt and emotion data entered by the user to the server. The input is the prompt data and emotion data, and the output is the result of sending it to the server. The server sends this data to the generative AI model. If the data transmission is successful, the terminal displays "Data transmission successful."

[1054] Step 6: Receiving a response from the generative AI model

[1055] The server receives the response from the generative AI model and transfers it to the terminal. The input is the response data from the generative AI model, and the output is the transfer result to the terminal. The terminal prepares to display this response to the user. When the response is received, the terminal displays "Response received successfully."

[1056] Step 7: View the response

[1057] The terminal displays the response received from the generative AI model to the user. For example, it displays "It's sunny today. Have a nice day." The input is the response data from the generative AI model, and the output is the display result to the user. The user can check this response and enter further prompts if necessary.

[1058] (Application example 1)

[1059] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1060] Conventional food delivery services have difficulty making customized suggestions based on users' emotions and situations. In addition, there is a lack of means for users to use generative AI to simplify the ordering process. This can lead to poor user experience and reduced frequency of service use.

[1061] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1062] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, an emotion recognition means for recognizing the emotion of the user, and a means for feeding back the emotion information recognized by the emotion recognition means to the generative AI. This makes it possible to propose food delivery customized according to the emotion of the user.

[1063] An "accessory device connectable to a smartphone" is an external device that is connected to a smartphone using wireless communication means and provides a specific function.

[1064] "Generative AI" is an artificial intelligence system that generates responses or suggestions based on prompting input from a user.

[1065] "Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

[1066] "User authentication means" refers to technology for verifying the identity of a user and ensuring that only specific users can operate the system.

[1067] A "fingerprint authentication means" is a technology that scans a user's fingerprint and authenticates the user based on that information.

[1068] An "operation means" is a physical or software interface that allows a user to operate the system.

[1069] A "one-click operation means" is an interface that allows a user to execute a specific function with a single click.

[1070] "Emotion recognition means" is a technology that analyzes the user's tone of voice and facial expressions to recognize their emotions.

[1071] A "means for feeding back emotional information to a generative AI" is a function that sends recognized emotional information to a generative AI, enabling the generative AI to generate responses or suggestions based on that information.

[1072] This invention is a system that uses an accessory device connectable to a smartphone to simplify the ordering process when a user uses a food delivery service through generative AI and makes customized suggestions based on the user's emotions.

[1073] System Configuration

[1074] The system consists of the following main components:

[1075] 1. Accessory devices that can be connected to smartphones

[1076] Wireless communication means: Connects to a smartphone using Bluetooth or similar.

[1077] User authentication method: A fingerprint sensor is used to authenticate the user.

[1078] Control: A one-click control is used by the user to prompt the generative AI.

[1079] Emotion recognition: Analyzes the user's tone of voice and facial expressions to recognize emotions.

[1080] 2. Generative AI

[1081] Prompt input: Receives instructions or questions from the user and generates responses or suggestions.

[1082] Emotional feedback: Generate customized responses and suggestions based on recognized emotional information.

[1083] Program Processing

[1084] The server does the following:

[1085] 1. Bluetooth connection: Connect your smartphone and accessory device using Bluetooth.

[1086] 2. User authentication: The fingerprint sensor scans the user's fingerprint to authenticate them.

[1087] 3. Emotion Recognition: Recognize emotions by analyzing the user’s tone of voice and facial expressions.

[1088] 4. Prompt generation: Generate prompts for the generative AI based on the recognized emotions.

[1089] 5. AI Response Generation: Use generative AI to generate responses and suggestions to prompts.

[1090] Hardware and Software Used

[1091] Hardware: Bluetooth module, fingerprint sensor, camera (for face recognition), microphone (for voice recognition)

[1092] Software: Bluetooth connection library, fingerprint authentication library, emotion recognition engine, generative AI model

[1093] Examples

[1094] If the user is recognized as tired, the generative AI will suggest "food delivery options recommended for when you're tired." For example, the following prompts may be generated:

[1095] User is feeling tired, please suggest food delivery options.

[1096] In this way, it becomes possible to provide customized food delivery suggestions based on the user's emotions.

[1097] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1098] Step 1:

[1099] The server connects the smartphone and the accessory device using a Bluetooth module. The input is pairing information between the smartphone and the accessory device, and the output is the establishment of a connection. Specifically, the server uses a Bluetooth connection library to establish communication between the smartphone and the accessory device.

[1100] Step 2:

[1101] The device uses a fingerprint authentication sensor to scan the user's fingerprint and perform user authentication. The input is the user's fingerprint data, and the output is the authentication result. Specifically, the fingerprint authentication library is used to compare the scanned fingerprint data with registered data to determine whether authentication was successful.

[1102] Step 3:

[1103] The device uses a microphone and a camera to capture the user's tone of voice and facial expressions to perform emotion recognition. The input is voice data and image data, and the output is recognized emotion information. Specifically, an emotion recognition engine is used to analyze the voice data and image data to identify the user's emotion.

[1104] Step 4:

[1105] The server generates a prompt for the generative AI based on the recognized emotion information. The input is emotion information, and the output is a prompt. Specifically, the server converts the emotion information into text format and creates a prompt suitable for the generative AI.

[1106] Step 5:

[1107] The server uses a generative AI model to generate responses and suggestions to prompts. The input is the prompt, and the output is the generated response or suggestion. Specifically, the prompt is input to the generative AI model, and the AI ​​generates the response or suggestion.

[1108] Step 6:

[1109] The terminal presents the generated response or suggestion to the user. The input is the generated response or suggestion, and the output is the display to the user. Specifically, the response or suggestion is displayed on the smartphone display so that the user can check it.

[1110] Example 2

[1111] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1112] In conventional systems using generative AI models, users had to operate their smartphones, making them difficult to use. In addition, they were unable to generate appropriate responses according to the user's emotions, limiting the user experience. This made it difficult for users to effectively utilize generative AI models.

[1113] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1114] In this invention, the server includes a user authentication means, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI model, an emotion recognition means for capturing the user's voice data and analyzing the emotion, and a means for feeding back the emotion information analyzed by the emotion recognition means to the generative AI model. This allows the user to easily use the generative AI model without operating a smartphone, and further enables an appropriate response according to the user's emotion to be obtained.

[1115] "User authentication means" means a means for verifying a user's identity, and includes biometric authentication technologies such as fingerprint authentication and facial recognition.

[1116] "Operation means" refers to the means by which a user inputs prompts to the generated AI model, and includes interfaces such as buttons and touch screens.

[1117] An "emotion recognition means" is a means for capturing a user's voice data and analyzing the user's emotions from the tone and content of the data.

[1118] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on prompts from a user.

[1119] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[1120] The "feedback means" refers to a means for transmitting the emotion information analyzed by the emotion recognition means to the generative AI model.

[1121] The present invention provides a system including a user authentication means, an operation means, an emotion recognition means, and a feedback means. A specific embodiment of the system will be described below.

[1122] First, the user places his or her finger on the accessory device's fingerprint sensor. The fingerprint sensor captures the user's fingerprint data and sends it to the device. The device then sends this data to the server, which matches the fingerprint data with an existing database to authenticate the user. If authentication is successful, the device's buttons are enabled.

[1123] Next, when the user presses the enabled button, the device launches a dedicated application on the smartphone. This application has the function of inputting a prompt sentence to the generative AI model. For example, the prompt sentence "Please tell me about the weather today" is input to the generative AI model.

[1124] Furthermore, when the user speaks to the accessory device, the microphone in the terminal captures voice data. This voice data is sent to the emotion recognition means, which analyzes the tone of the voice data to analyze the user's emotion. The analysis result is sent to the generative AI model through the feedback means. The generative AI model generates an appropriate response based on this emotion information. For example, in response to the prompt "The user seems a little tired. Please generate an encouraging message," the generative AI model generates a response such as "Let's take a break today and refresh ourselves."

[1125] This system allows users to easily use generative AI models without operating a smartphone, and improves the user experience by providing appropriate responses according to the user's emotions.

[1126] As a concrete example, when a user places his / her finger on the fingerprint authentication sensor of an accessory device and presses a button after authentication is successful, an application on the smartphone is launched and the following prompt sentence is input to the generative AI model:

[1127] "Tell me about the weather today."

[1128] As another specific example, when a user speaks to the accessory device saying, "How are you feeling today?", the emotion recognition means recognizes the emotion from the tone of the user's voice and inputs the following prompt sentence into the generative AI model:

[1129] "The user seems a bit tired. Generate an uplifting message."

[1130] This allows users to easily utilize generative AI without having to operate a smartphone.

[1131] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1132] Step 1:

[1133] The user places his or her finger on the fingerprint sensor.

[1134] Input: User's fingerprint

[1135] Specific operation: When a user places his or her finger on the fingerprint sensor on the accessory device, the sensor glows blue.

[1136] Output: The captured fingerprint data.

[1137] Step 2:

[1138] The device captures fingerprint data and sends it to the server.

[1139] Input: Captured fingerprint data

[1140] Specific operation: The device captures fingerprint data from the fingerprint authentication sensor, and when the data is sent to the server, the device screen displays "Authentication...".

[1141] Output: Fingerprint data sent to the server

[1142] Step 3:

[1143] The server authenticates the fingerprint data.

[1144] Input: Fingerprint data sent to the server

[1145] Specific operation: The server compares the received fingerprint data with an existing database and authenticates the user.

[1146] Output: Authentication result (success or failure)

[1147] Step 4:

[1148] If the authentication is successful, the terminal button is enabled.

[1149] Input: Authentication result (success)

[1150] Specific operation: If the server succeeds in authentication, it sends a signal to the terminal to activate the button. The button on the terminal turns green.

[1151] Output: Enabled buttons

[1152] Step 5:

[1153] The user presses a button.

[1154] Input: Enabled button

[1155] Specific Action: The user presses an enabled button, which triggers the next step.

[1156] Output: Button press signal

[1157] Step 6:

[1158] The terminal launches an application on the smartphone.

[1159] Input: Button press signal

[1160] Specific operation: The device launches a dedicated application on the smartphone. The application logo appears on the smartphone screen.

[1161] Output: Launched application

[1162] Step 7:

[1163] The application prompts input to the generative AI model.

[1164] Input: The launched application

[1165] Specific operation: The application inputs a prompt sentence to the generative AI model. The screen displays "Sending prompt...". For example, the prompt sentence is "Tell me about the weather today."

[1166] Output: The prompt sent to the generative AI model

[1167] Step 8:

[1168] The user speaks to the accessory device.

[1169] Input: User's voice

[1170] Specific Actions: The user speaks to the accessory device, for example, "How are you feeling today?" The microphone on the device lights up red to indicate that it is capturing voice data.

[1171] Output: Captured audio data

[1172] Step 9:

[1173] The device captures voice data and sends it to the emotion engine.

[1174] Input: Captured audio data

[1175] Specific operation: The device uses the built-in microphone to capture voice data and transmits the data to the emotion engine.

[1176] Output: Audio data sent to the emotion engine

[1177] Step 10:

[1178] The emotion engine analyzes the user's emotions from the tone of the voice data.

[1179] Input: Audio data sent to the emotion engine

[1180] Specific operation: The emotion engine analyzes the tone of the voice data and analyzes the user's emotion. The device screen displays "Analyzing emotion...".

[1181] Output: Analyzed sentiment information

[1182] Step 11:

[1183] The emotion engine feeds the analysis results back into the generative AI model.

[1184] Input: Analyzed sentiment information

[1185] How it works: The emotion engine feeds back the analysis results to the generative AI model, for example sending information like "The user seems a bit tired."

[1186] Output: Emotion information sent to the generative AI model

[1187] Step 12:

[1188] The generative AI model generates responses based on the user's emotions.

[1189] Input: Emotion information sent to the generative AI model

[1190] Specific operation: The generative AI model generates an appropriate response based on the fed-back emotional information. For example, in response to the prompt "Generate an uplifting message," the model generates the response "Take a break today and refresh yourself." The response is displayed on the smartphone screen.

[1191] Output: The generated response

[1192] (Application example 2)

[1193] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1194] In conventional food delivery services, when users use their smartphones to place orders, the operation is complicated and appropriate suggestions based on the user's emotions are not given. In particular, there is a problem that the user experience is not improved for tired or stressed users because there are not enough suggestions for relaxing meals.

[1195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1196] In this invention, the server includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a voice capture means for capturing the user's voice and recognizing emotions, an emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the user's emotion, and a means for the generative AI to generate an appropriate response based on the emotion recognized by the emotion recognition means. This enables the user to receive appropriate food delivery suggestions through voice and emotion recognition without operating a smartphone.

[1197] "Generative AI" is an artificial intelligence system that generates appropriate responses and suggestions based on user input.

[1198] "Prompt input" refers to the act of inputting instructions or questions to a generative AI.

[1199] An "accessory device" is an external device that can be connected to a smartphone and provides a specific function.

[1200] "User authentication means" means a means for verifying a user's identity and ensuring that only specific users can operate the system.

[1201] A "fingerprint authentication means" is a means for reading a user's fingerprint and authenticating him or her.

[1202] "Operation means" refers to means such as an interface or buttons that allow the user to operate the system.

[1203] A "one-click operation means" is a means that allows a user to complete an operation with a single click.

[1204] "Audio capture means" means for recording or capturing a user's voice.

[1205] An "emotion recognition means" is a means for analyzing captured voice data and recognizing a user's emotion.

[1206] "Response" refers to the reply or suggestion that a generative AI generates in response to user input.

[1207] A system for implementing this invention is configured as follows. First, a user uses an accessory device connected to a smartphone. This accessory device is equipped with a fingerprint authentication means, and authentication is performed by the user placing his / her finger on the accessory device. If authentication is successful, the operation means of the accessory device is enabled, and the user can use the one-click operation means to input a prompt to the generative AI.

[1208] Next, the voice capture means captures the user's voice. This voice data is sent to the emotion recognition means, which analyzes the user's emotions. The emotion recognition means recognizes the user's emotions from the tone and content of the voice data, and feeds the results back to the generative AI. The generative AI generates an appropriate response based on the recognized emotions.

[1209] Specifically, the following hardware and software are used: A fingerprint sensor is used as the fingerprint authentication means. A microphone and voice recognition software (e.g., speech_recognition library) are used as the voice capture means. An emotion engine (e.g., EmotionEngine) is used as the emotion recognition means. An AI generation model (e.g., AIGenerator) is used as the generative AI.

[1210] For example, if a user says, "I'm tired today," the voice capture means captures the voice, and the emotion recognition means detects "fatigue." Based on this emotion, the generative AI generates a response suggesting a "relaxing meal." In this way, appropriate food delivery suggestions are made according to the user's emotions.

[1211] Examples of prompts to input to a generative AI model include:

[1212] User Emotion: Fatigue

[1213] Prompt for generative AI: "Please suggest relaxing meals for a user who is tired."

[1214] This system allows users to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate a smartphone.

[1215] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1216] Step 1:

[1217] The user places his / her finger on the fingerprint authentication sensor of the accessory device connected to the smartphone. The input is the user's fingerprint data, and the output is the authentication result. The accessory device reads the fingerprint data and performs authentication by comparing it with its internal database. If authentication is successful, it proceeds to the next step.

[1218] Step 2:

[1219] If the authentication is successful, the operation means of the accessory device is enabled. The user uses the one-click operation means to input a prompt to the generative AI. The input is the user's click operation, and the output is the trigger for the prompt input. The accessory device detects the click operation and starts inputting a prompt to the generative AI.

[1220] Step 3:

[1221] The terminal's voice capture means captures the user's voice. The input is the user's voice data and the output is the captured voice data. The terminal's microphone records the user's voice and the voice recognition software converts the data into a digital format.

[1222] Step 4:

[1223] The captured voice data is sent to an emotion recognition means. The input is the captured voice data and the output is the recognized emotion data. The emotion engine analyzes the voice data and recognizes the user's emotion from tone and content.

[1224] Step 5:

[1225] The recognized emotion data is fed back to the generative AI. The input is the recognized emotion data, and the output is a prompt to the generative AI. The generative AI creates a prompt to generate an appropriate response based on the emotion data.

[1226] Step 6:

[1227] The generative AI generates an appropriate response based on the prompt. The input is the prompt and the output is the generated response. The generative AI analyzes the prompt and generates food delivery suggestions based on the user's sentiment.

[1228] Step 7:

[1229] The generated response is provided to the user. The input is the generated response, and the output is a suggestion to the user. Appropriate food delivery suggestions are displayed or audibly conveyed to the user through the display or audio output device of the terminal.

[1230] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1231] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1232] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1233] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1234] [Third embodiment]

[1235] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1236] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1237] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1238] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1239] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1240] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1241] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1242] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1243] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1244] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1245] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1246] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1247] "Example 1"

[1248] As one embodiment of the present invention, an accessory device connectable to a smartphone is provided. This device includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI. Specifically, the accessory device is connected to a smartphone using a wireless communication means such as Bluetooth, and the prompt input to the generative AI is performed through an application running on the smartphone. As the user authentication means, a fingerprint authentication sensor is built into the device, and authentication is performed by matching the fingerprint information of a registered user. As the operation means, a prompt input to the generative AI is performed by a user pressing a button provided on the device.

[1249] "Example 2"

[1250] As a specific example of use, a user authenticates themselves by placing their finger on the fingerprint authentication sensor of the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is made to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

[1251] The process flow of each embodiment will be described below.

[1252] "Example 1"

[1253] Step 1: A user connects an accessory device connectable to a smartphone to the smartphone using a wireless communication means such as Bluetooth.

[1254] Step 2: The user places his or her finger on the fingerprint authentication sensor of the accessory device, and the fingerprint authentication sensor authenticates the user by matching the fingerprint information with that of the registered user.

[1255] Step 3: If the authentication is successful, the device's buttons are enabled.

[1256] Step 4: When the user presses a button on the device, an application on the smartphone is launched and a prompt input is sent to the generative AI.

[1257] "Example 2"

[1258] Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

[1259] Step 2: If authentication is successful, the device's buttons are enabled.

[1260] Step 3: When the user presses the button, an application on the smartphone is launched and prompts the generative AI to input the information.

[1261] Step 4: This will enable users to easily utilize generative AI without having to operate a smartphone.

[1262] Example 1

[1263] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1264] In conventional systems using generative AI models, users had to go through complicated operations and multi-step authentication when entering a prompt, which resulted in poor user experience. In addition, there was a lack of efficient means to quickly and accurately obtain the generated results. This reduced the convenience of users when using generative AI models.

[1265] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1266] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a means for running an application for inputting a prompt to the generative AI on a smartphone, and a means for sending a prompt sentence to the generative AI model and receiving a generation result. This allows a user to input a prompt sentence to the generative AI model with a simple operation and quickly and accurately obtain a generation result.

[1267] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer, and can have a variety of functions installed on it.

[1268] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[1269] "Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompt sentences.

[1270] "Prompt input" is the act of inputting instructions or questions to a generative AI.

[1271] "User authentication means" refers to technology or devices used to verify a user's identity.

[1272] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and verifies the user's identity based on that information.

[1273] The "operation means" refers to an interface or device that allows the user to operate the accessory device or smartphone.

[1274] A "one-click operation means" is an interface or device that allows a user to execute a specific function with a single click.

[1275] An "application" is software that runs on a smartphone and provides specific functions or services.

[1276] "Generated results" are output such as text and images that the generative AI generates based on the prompt text.

[1277] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1278] The present invention is a system for easily inputting prompts to a generative AI model using an accessory device connectable to a smartphone. A specific embodiment of the system will be described below.

[1279] Hardware and Software Configuration

[1280] Accessory equipment:

[1281] An accessory device is an external device that is connected to a smartphone and has the following main functions:

[1282] User authentication method: Equipped with a fingerprint authentication sensor to read the user's fingerprint information.

[1283] Operation: Equipped with a one-click button, users can easily operate it.

[1284] Smartphone:

[1285] The smartphone is connected to the accessory device using a wireless communication means such as Bluetooth. An application for inputting prompts to the generative AI model is installed on the smartphone.

[1286] Generative AI models:

[1287] The generative AI model runs on a server and generates text and images based on prompts sent from the smartphone.

[1288] System Operation

[1289] User authentication:

[1290] The user places his / her finger on the fingerprint authentication sensor installed in the accessory device. The accessory device reads the fingerprint information and compares it with the fingerprint information stored in the internal database. If the authentication is successful, the accessory device sends a signal of authentication success to the smartphone.

[1291] Prompt Input:

[1292] After successful authentication, the user presses a button on the accessory device. The smartphone application starts and a prompt input screen is displayed. The user inputs a prompt sentence for the generative AI model.

[1293] Send to generative AI model:

[1294] The smartphone sends the input prompt sentence to the generative AI model. The server operates the generative AI model based on the received prompt sentence and creates the generated result.

[1295] View the generated results:

[1296] The server sends the generated results to the smartphone. The smartphone displays the received generated results to the user. The user checks the displayed generated results and saves or shares them as necessary.

[1297] Examples

[1298] The specific steps a user takes if they want a generative AI to create a new poem are as follows:

[1299] 1. User Authentication:

[1300] The user places his or her finger on the fingerprint authentication sensor of the accessory device.

[1301] The accessory device reads the fingerprint and performs authentication.

[1302] If the authentication is successful, a signal indicating successful authentication is sent to the smartphone.

[1303] 2. Prompt input:

[1304] The user presses a button on the accessory device.

[1305] The smartphone application will start and a prompt input screen will be displayed.

[1306] The user enters the prompt text "Create a new poem."

[1307] 3. Send to generative AI model:

[1308] The smartphone sends the entered prompt text to the generative AI model.

[1309] The server runs a generative AI model based on the prompt text to generate a poem.

[1310] 4. View the generated results:

[1311] The server sends the generated poem to the smartphone.

[1312] The smartphone displays the received poem to the user.

[1313] The user can review the displayed poem and save or share it as desired.

[1314] In this way, a user can use the accessory device and a smartphone to provide prompt input to the generative AI model and obtain generative results.

[1315] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1316] Step 1: Connect accessory devices

[1317] Input: A user opens the Bluetooth settings on their smartphone and selects an accessory device.

[1318] How it works: A smartphone establishes a connection with an accessory device using Bluetooth.

[1319] Output: The smartphone will inform you that the connection was successful.

[1320] Step 2: User authentication

[1321] Input: The user places their finger on the fingerprint sensor on the accessory device.

[1322] How it works: The accessory device reads your fingerprint and matches it with fingerprint information stored in an internal database.

[1323] Output: If authentication is successful, the accessory device sends a signal of authentication success to the smartphone.

[1324] Step 3: Prepare for prompts

[1325] Input: The smartphone receives a signal that authentication was successful.

[1326] How it works: The smartphone displays a prompt input screen for the generative AI.

[1327] Output: The user sees the prompt input screen and begins entering the prompt statement.

[1328] Step 4: Prompt Input

[1329] Input: The user inputs a prompt for the generative AI through a smartphone application.

[1330] How it works: The smartphone receives the input prompt text and formats it internally.

[1331] Output: The prepared prompt is ready to be sent to the generative AI model.

[1332] Step 5: Send to generative AI model

[1333] Input: The smartphone sends a prepared prompt to the generative AI model.

[1334] Operation: The server runs the generative AI model based on the received prompt sentence and creates a generated result.

[1335] Output: The generated results are sent from the server to the smartphone.

[1336] Step 6: Viewing the generation results

[1337] Input: The smartphone receives the generated results from the server.

[1338] How it works: The smartphone displays the generated results it receives to the user.

[1339] Output: The user checks the displayed generated results and can save or share them as needed.

[1340] (Application example 1)

[1341] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1342] To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and insufficient security for user authentication, which creates the risk of operational errors and unauthorized access. This can lead to reduced productivity and safety in factories.

[1343] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1344] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a means for generating instructions for managing and operating a robot in a factory, and a means for transmitting the instructions to the robot by wireless communication means. This makes it possible to efficiently and safely manage and operate the robot in the factory.

[1345] An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

[1346] "Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

[1347] "Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

[1348] "User authentication means" is a function for confirming and authenticating the identity of a user.

[1349] A "fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

[1350] The "operation means" is an interface that allows the user to input prompts to the generative AI.

[1351] The "one-click operation means" is a function that allows the user to input prompts to the generative AI with a single click.

[1352] "Means for generating instructions for managing and operating robots in a factory" refers to a function for generating specific work instructions for robots in a factory.

[1353] "Wireless communication means" refers to a function for transmitting and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

[1354] The "means for transmitting to the robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

[1355] This invention relates to a system for efficiently and safely managing and operating robots in a factory using an accessory device connectable to a smartphone. A specific embodiment of this system will be described below.

[1356] System Configuration

[1357] The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

[1358] Hardware

[1359] Smartphone: Functions as a user interface and is connected to accessory devices via wireless communication means.

[1360] Accessory device: Connected to a smartphone, the accessory device includes a fingerprint authentication means and an operation means.

[1361] Fingerprint authentication method: Scans the user's fingerprint to authenticate them.

[1362] Wireless communication means: Data is sent and received between the smartphone and the factory robot using Bluetooth, Wi-Fi, etc.

[1363] Factory robot: Performs actions according to instructions.

[1364] software

[1365] Generative AI model: Generates specific instructions for factory robots based on user input prompts.

[1366] Smartphone application: Provides the user interface and manages communication with the generative AI model.

[1367] Operating procedure

[1368] 1. User authentication: The user places a finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

[1369] 2. Prompt input: An authenticated user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[1370] 3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

[1371] 4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

[1372] 5. Robot operation: The factory robot performs the operation according to the instructions it receives.

[1373] Examples

[1374] For example, when a robot needs to perform a specific task in a factory, a manager can use a smartphone and accessory devices to give the robot instructions.

[1375] "Move to assembly line 3 and start welding."

[1376] "Move to the inspection area and perform quality checks."

[1377] "Go get part A and take it to assembly line 2."

[1378] This makes it possible to efficiently and safely manage and operate robots in factories. The system utilizes user authentication and generative AI to reduce the risk of incorrect operation and unauthorized access, improving factory productivity and safety.

[1379] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1380] Step 1:

[1381] The user places a finger on the accessory device and is authenticated by the fingerprint authentication means.

[1382] Input: User's fingerprint

[1383] Data processing: The fingerprint authentication means scans the fingerprint and compares it with registered fingerprint data.

[1384] Output: Authentication result (success or failure)

[1385] How it works: The fingerprint sensor scans the user's fingerprint and matches it against an internal database for authentication.

[1386] Step 2:

[1387] If the authentication is successful, the user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[1388] Input: Authentication result (success), user prompt

[1389] Data processing: If the authentication result is successful, a prompt sentence is sent to the generative AI model.

[1390] Output: prompt input to the generative AI model

[1391] Specific operation: When the user presses the button, the smartphone application sends a prompt sentence to the generative AI model.

[1392] Step 3:

[1393] Based on the prompts, a generative AI model generates specific instructions for factory robots.

[1394] Input: User prompt text

[1395] Data processing: A generative AI model analyzes the prompt and generates appropriate instructions.

[1396] Output: Instructions for the factory robots

[1397] Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

[1398] Step 4:

[1399] The generated instructions are transmitted to the factory robot using wireless communication means.

[1400] Input: Instructions from a generative AI model

[1401] Data processing: Instructions are sent to factory robots via wireless communication means.

[1402] Output: Sending instructions to factory robots

[1403] Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth and Wi-Fi.

[1404] Step 5:

[1405] The factory robot operates according to the instructions it receives.

[1406] Input: Instructions for the factory robots

[1407] Data processing: The factory robot analyzes the instructions and performs the corresponding actions.

[1408] Output: Robot movement

[1409] Specific actions: The factory robot performs actions according to instructions such as "move to assembly line 3 and start welding."

[1410] Example 2

[1411] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1412] In order to use conventional generative AI models, users had to operate their smart devices directly and input prompt text. This operation was tedious and inconvenient, especially when their hands were full or in situations where a quick response was required. Also, from a security standpoint, user authentication was sometimes insufficient, creating a risk of unauthorized use by third parties. There is a demand for a system that can solve these issues and make it easier and safer to use generative AI models.

[1413] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1414] In this invention, the server includes a user authentication means, an operation means for an authenticated user to operate, a means for transmitting a signal to the smart device using the operation means to launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without directly operating the smart device, and the user authentication means also improves security.

[1415] "User authentication means" means the means for verifying a user's identity and granting access to a system.

[1416] "Operation means" refers to an interface or device that allows a user to operate the system.

[1417] A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

[1418] The "means for transmitting a signal" refers to a means for transmitting specific instructions or data from the operating means to the smart device.

[1419] A "specific application" is a software program designed to provide a particular function or service.

[1420] A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

[1421] A "prompt" is a text input used to give instructions or ask a question to a generative AI model.

[1422] A "means for generating a response" is a means by which a generative AI model creates an appropriate response based on a prompt sentence.

[1423] The "means for sending a response" refers to a means for sending the generated response to the smart device.

[1424] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1425] The present invention is a system for enabling a user to utilize a generative AI model without directly operating a smart device. A specific embodiment of the system will be described below.

[1426] Hardware and Software Used

[1427] Hardware: Accessory devices (with fingerprint authentication sensor), smart devices (smartphones, etc.)

[1428] Software: Fingerprint authentication systems, smart device applications, generative AI models

[1429] System Configuration

[1430] 1. User authentication method:

[1431] The user places his or her finger on the fingerprint sensor on the accessory device.

[1432] The fingerprint authentication system captures fingerprint data and authenticates the user.

[1433] 2. Operating means:

[1434] If authentication is successful, the button on the accessory device is enabled.

[1435] The user presses an enabled button.

[1436] 3. Signal transmission means:

[1437] When the button is pressed, the accessory device sends a signal to the smart device.

[1438] The smart device receives the signal and launches a specific application.

[1439] 4. Prompt sending method:

[1440] The launched application sends a prompt to the generative AI model.

[1441] Prompts are pre-defined statements that contain the instructions or questions the user wants to ask the generative AI model.

[1442] 5. Response Generation Method:

[1443] The server uses a generative AI model to generate a response based on the received prompt.

[1444] 6. Response sending method:

[1445] The server generates a response and sends it to the smart device.

[1446] The smart device receives the response and displays it to the user.

[1447] Examples

[1448] When a user places their finger on the fingerprint authentication sensor of the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device to launch an application. The application launches and sends a prompt sentence, "Tell me about the weather today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can confirm the information, "Today's weather is sunny," on the screen.

[1449] Examples of prompt statements

[1450] "Tell me about the weather today."

[1451] "Check the schedule for the next meeting."

[1452] "Tell me the latest news."

[1453] This system allows users to easily use generative AI models without having to directly operate a smart device, and security is also improved through user authentication measures.

[1454] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1455] Program processing flow

[1456] Step 1:

[1457] The user places his or her finger on the fingerprint authentication sensor of the accessory device.

[1458] Input: User's fingerprint

[1459] Data processing: The fingerprint sensor captures the fingerprint data.

[1460] Output: The captured fingerprint data.

[1461] Step 2:

[1462] The device captures the fingerprint data and authenticates the user.

[1463] Input: Captured fingerprint data

[1464] Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

[1465] Output: Authentication result (success or failure)

[1466] Step 3:

[1467] If authentication is successful, the device's buttons are enabled.

[1468] Input: Authentication result (success)

[1469] Data processing: If authentication is successful, enable the device buttons.

[1470] Output: Enabled button

[1471] Step 4:

[1472] The user presses a button.

[1473] Input: Enabled button

[1474] Data processing: The terminal detects that a button has been pressed.

[1475] Output: Button press signal

[1476] Step 5:

[1477] The terminal sends a signal to the smart device to launch the application.

[1478] Input: Button press signal

[1479] Data processing: The terminal sends a signal to the smart device to launch a specific application.

[1480] Output: Launched application

[1481] Step 6:

[1482] The application sends a prompt to the generative AI model.

[1483] Input: The application that was launched

[1484] Data processing: The application sends pre-defined prompts to the generative AI model.

[1485] Output: The prompt sent to the generative AI model

[1486] Step 7:

[1487] The server generates a response using a generative AI model.

[1488] Input: The prompt sent to the generative AI model

[1489] Data computation: A generative AI model analyzes the prompt and generates an appropriate response.

[1490] Output: The generated response

[1491] Step 8:

[1492] The server sends the response to the smart device.

[1493] Input: The generated response

[1494] Data processing: The server generates a response and sends it to the smart device.

[1495] Output: Response sent to smart device

[1496] Step 9:

[1497] The smart device displays the response.

[1498] Input: Response sent to smart device

[1499] Data processing: The smart device displays the response received to the user.

[1500] Output: The response displayed to the user

[1501] In this way, users can utilize generative AI models without directly operating a smart device.

[1502] (Application example 2)

[1503] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1504] Conventional food delivery systems require users to use their smartphones to place orders, which makes operation complicated. There are also security risks due to insufficient user authentication. Furthermore, ordering systems using voice recognition are not widespread, which reduces user convenience. There is a need to provide a system that solves these problems and allows users to use food delivery services easily and safely.

[1505] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1506] An accessory device connectable to a smart device includes a user authentication means for authenticating a user who operates the accessory device by fingerprint authentication, a button provided on the accessory device that is pressed by a user authenticated by the user authentication means to input instructions to the generative AI, and a voice recognition means for recognizing the user's voice in response to pressing of the button. The server includes a means for acquiring an output of the generative AI using a prompt sentence including voice data acquired by the voice recognition means and the generative AI. The prompt sentence instructs ordering through a food delivery service, and the acquiring means acquires the result of the order using a food delivery API. This allows a user to easily and safely use food delivery services using only voice, without operating a smartphone.

[1507] A "smart device" is a portable electronic device, such as a smartphone or tablet, that has Internet connectivity and can run applications.

[1508] An "accessory device" is additional hardware that is connected to a smart device to provide a specific function.

[1509] "Generative AI" is an artificial intelligence technique that generates text and other content based on instructions or prompts from a user.

[1510] "Instruction input" refers to commands or requests that a user inputs to a generative AI.

[1511] "User authentication means" refers to technology or devices for confirming and authenticating the identity of a user.

[1512] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and authenticates the user based on that information.

[1513] "Operation means" refers to the interface or buttons that allow the user to operate the accessory device or smart device.

[1514] A "voice recognition means" is a technology or device that analyzes a user's voice and converts it into text data.

[1515] "Voice data" refers to data that is a digital recording of a user's voice.

[1516] "Transmission means" refers to techniques or devices for transmitting data to other devices or systems.

[1517] The present invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. A specific embodiment of this system will be described below.

[1518] System Configuration

[1519] The system consists of the following main components:

[1520] 1. Smart device: A portable electronic device such as a smartphone or tablet that has internet connectivity and can run applications.

[1521] 2. Accessory device: Additional hardware used by connecting to a smart device, including fingerprint authentication means, operation means, and voice recognition means.

[1522] 3. Generative AI: Artificial intelligence techniques that generate text and other content based on instructions or prompts from the user.

[1523] Program Processing

[1524] Hardware

[1525] Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

[1526] Smart device: Functions as the execution environment for applications.

[1527] Microphone: Captures the user's voice as a means of voice recognition.

[1528] software

[1529] Fingerprint authentication library: A software module for performing fingerprint authentication.

[1530] Speech Recognition Library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

[1531] Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3 (registered trademark)).

[1532] Food Delivery API: Place orders using the API of food delivery services.

[1533] Processing flow

[1534] 1. User authentication: The user authenticates by placing his / her finger on the fingerprint authentication sensor of the accessory device. If authentication is successful, the next step is performed.

[1535] 2. Voice input: The user presses the operation means (one-click button) of the accessory device and speaks the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

[1536] 3. Prompt generation: Based on the acquired text data, a prompt sentence is generated for the generative AI.

[1537] 4. AI response generation: The generative AI model generates the optimal response based on the prompt.

[1538] 5. Order fulfillment: Based on the response generated, complete the order using the food delivery API.

[1539] Examples

[1540] When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model is given the prompt "User wants to order: pizza," the generative AI model suggests the best pizza restaurant and menu, and completes the order using a food delivery API.

[1541] Examples of prompt statements

[1542] "User wants to order: pizza"

[1543] In this way, users can use voice commands to easily and safely receive food delivery services without having to operate a smartphone.

[1544] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1545] Step 1:

[1546] The user places his or her finger on the fingerprint sensor.

[1547] Input: The user's fingerprint data.

[1548] Processing: The fingerprint authentication library analyzes the fingerprint data and matches it with registered fingerprint data.

[1549] Output: The authentication result (success or failure).

[1550] Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

[1551] Step 2:

[1552] The user presses the operating means (one-click button) of the accessory device.

[1553] Input: User action (button press).

[1554] Processing: Detects that an operating means has been pressed and activates the voice recognition mode.

[1555] Output: Activate speech recognition mode.

[1556] Specific operation: The sensor detects that a button has been pressed, and the speech recognition library begins preparing for voice input.

[1557] Step 3:

[1558] The user verbally communicates the order.

[1559] Input: The user's voice data.

[1560] Processing: The microphone picks up the voice data, and the speech recognition library converts it into text data.

[1561] Output: Text data (order details).

[1562] What happens: A microphone captures the user's voice, and a speech recognition library analyzes the speech and converts it into text.

[1563] Step 4:

[1564] The terminal generates a prompt sentence based on the acquired text data.

[1565] Input: Text data (order details).

[1566] Processing: The text data is converted into a prompt sentence of the form "The user would like to order: [order details]".

[1567] Output: The prompt statement.

[1568] Specific operation: Convert text data into prompt sentences according to the format.

[1569] Step 5:

[1570] The device sends a prompt to the generative AI model.

[1571] Input: prompt statement.

[1572] Processing: The generative AI model analyzes the prompt and generates the most appropriate response.

[1573] Output: AI response (optimal order content).

[1574] Specific operation: The prompt sentence is sent to the generative AI model, and the AI ​​generates a response.

[1575] Step 6:

[1576] The device uses the generated response to complete the order using the food delivery API.

[1577] Input: AI response (optimal order content).

[1578] Processing: Fulfilling the order using the food delivery API.

[1579] Output: Order confirmation (data indicating the order was successfully completed).

[1580] Specific behavior: Send the AI ​​response to the food delivery API and confirm the order.

[1581] The server may further include a means for recognizing the user's emotion using an emotion engine described later, and may obtain an output of the generative AI using the voice data obtained by the voice recognition means, a prompt sentence including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," a prompt sentence "User wants to order: pizza, user's emotion: sadness" is generated, and the generative AI model proposes an optimal pizza restaurant and menu considering the user's emotion, and completes the order using a food delivery API.

[1582] In addition, an emotion engine that estimates the emotion of the user may be further combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[1583] "Example 1"

[1584] As one embodiment of the present invention, an accessory device connectable to a smartphone is provided. The device includes a means for enabling prompt input to the generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI. Furthermore, the device includes an emotion engine that recognizes the user's emotions. Specifically, the device recognizes emotions from the user's tone of voice and facial expressions, and feeds back the information to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

[1585] "Example 2"

[1586] As another embodiment of the present invention, a system is provided in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone in the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotions from the tone of the voice data and feeds back the results to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

[1587] The process flow of each embodiment will be described below.

[1588] "Example 1"

[1589] Step 1: A user operates an accessory device that can be connected to a smartphone.

[1590] Step 2: The user authentication means of the device authenticates the user. One possible authentication method is fingerprint authentication.

[1591] Step 3: The authenticated user prompts the generative AI.

[1592] Step 4: The device's emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[1593] Step 5: The emotion engine feeds that information back to the generative AI.

[1594] Step 6: The generative AI generates an appropriate response based on the user's emotions.

[1595] "Example 2"

[1596] Step 1: The user talks to the accessory device.

[1597] Step 2: The microphone in the device captures the user's voice and sends the voice data to the emotion engine.

[1598] Step 3: The emotion engine analyzes the user's emotion from the tone of the voice data.

[1599] Step 4: The emotion engine feeds its results back to the generative AI.

[1600] Step 5: The generative AI generates an appropriate response based on the user's emotions.

[1601] Example 1

[1602] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1603] Conventional accessory devices that can be connected to smartphones allow prompt input to generative AI models, but they have the problem of being unable to generate responses that take into account the user's emotional state. This makes it difficult to obtain appropriate responses according to the user's intentions and emotions, resulting in a lack of improvement in the user experience.

[1604] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1605] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI model, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI model, an emotion recognition means for recognizing the emotion of the user, and a means for feeding back the emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the emotional state of the user.

[1606] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer.

[1607] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[1608] A "generative AI model" is an artificial intelligence system that generates output, such as text or speech, based on input prompts.

[1609] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[1610] "User Authentication Means" means a technical means for verifying a user's identity.

[1611] A "fingerprint authentication means" is a technical means that reads a user's fingerprint and performs authentication based on that information.

[1612] The "operation means" is an interface that enables a user to operate an accessory device.

[1613] A "one-click operation means" is an interface that allows a user to complete an operation with a single click.

[1614] An "emotion recognition means" is a technical means for recognizing emotions by analyzing the user's tone of voice and facial expressions.

[1615] "Emotion information" is data relating to the user's emotional state obtained by emotion recognition means.

[1616] "Feedback means" refers to the technical means for transmitting acquired emotional information to the generative AI model.

[1617] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1618] The present invention is a system that uses an accessory device connectable to a smartphone to provide prompt input to a generative AI model and obtain a response that takes into account the emotional state of the user. A specific embodiment of the system is described below.

[1619] Hardware and Software Configuration

[1620] Accessory Equipment

[1621] The accessory device is connected to the smartphone using wireless communication means such as Bluetooth. The device includes the following main components:

[1622] Fingerprint authentication sensor: Reads the user's fingerprint for authentication.

[1623] Action button: An interface that allows the user to initiate prompt input.

[1624] Emotion recognition engine: Analyzes the user's tone of voice and facial expressions to obtain emotional information.

[1625] Smartphone

[1626] The smartphone runs a dedicated application that works in conjunction with the accessory device and provides the following functions:

[1627] Prompt input interface: A screen where users can input instructions or questions to a generative AI model.

[1628] Data transmission function: Sends prompts and emotion information to the server.

[1629] server

[1630] The server provides a backend service that communicates data between the smartphone and the generative AI model. The server has the following functions:

[1631] Data relay function: Transmits prompts and emotion information received from a smartphone to the generative AI model.

[1632] Response reception function: Receives responses from the generative AI model and transfers them to the smartphone.

[1633] Examples

[1634] Consider a scenario in which a user uses an accessory device connected to a smartphone to ask a generative AI model questions.

[1635] 1. User Authentication

[1636] The user places their finger on the accessory device's fingerprint authentication sensor to authenticate themselves. The terminal (smartphone) receives the fingerprint data from the accessory device and compares it with registered fingerprint data. If authentication is successful, the user can proceed to the next step.

[1637] 2. Prompt Input

[1638] The user opens a dedicated application on their smartphone and enters a prompt, for example, "What's the weather like today?" The device prepares to send this prompt to the generative AI model.

[1639] 3. Emotion recognition

[1640] The accessory device uses an emotion engine that senses the user's tone of voice and facial expressions to analyze the user's emotional state. The device prepares this emotion data for transmission to the generative AI model.

[1641] 4. Sending prompts to generative AI models

[1642] The device sends the prompt and emotion data entered by the user to the server, which then sends this data to the generative AI model, which then generates an appropriate response based on the prompt and emotion data.

[1643] 5. Receiving a response from the generative AI model

[1644] The server receives the response from the generative AI model and forwards it to the device, which prepares the response to be displayed to the user.

[1645] 6. Viewing the response

[1646] The device displays the response received from the generative AI model to the user, for example, "It's sunny today. Have a nice day." The user can review this response and enter further prompts if necessary.

[1647] Examples of prompt statements

[1648] "How's the weather today?"

[1649] "Please tell me when the next meeting is scheduled."

[1650] "Please tell me the latest news."

[1651] In this way, users can naturally interact with generative AI models through accessory devices.

[1652] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1653] Step 1: User authentication

[1654] The user places his / her finger on the fingerprint authentication sensor of the accessory device. The terminal receives the fingerprint data from the accessory device and compares it with the registered fingerprint data. The input is the user's fingerprint data and the output is the authentication result. The terminal analyzes the fingerprint data and checks whether it matches the registered data. If the authentication is successful, the terminal displays "Authentication successful."

[1655] Step 2: Connect your accessory device to your smartphone

[1656] The terminal establishes a connection with the accessory device via Bluetooth communication. The input is a Bluetooth connection request, and the output is a connection status. The terminal receives a connection request from the accessory device and establishes a connection. The server monitors this connection and relays data as necessary. If the connection is successful, the terminal displays "Connection successful."

[1657] Step 3: Prompt Input

[1658] The user opens a dedicated application on their smartphone and inputs a prompt. For example, they input "What's the weather like today?" The input is the user's prompt sentence, and the output is the prompt data. The device prepares to send this prompt to the generative AI model. When the user has completed the input, the device displays "Prompt input completed."

[1659] Step 4: Emotion Recognition

[1660] The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions. The input is the user's tone of voice and facial expression data, and the output is emotion data. The device prepares this emotion data to send to the generative AI model. Once emotion data is acquired, the device displays "Emotion data acquisition successful."

[1661] Step 5: Sending prompts to the generative AI model

[1662] The terminal sends the prompt and emotion data entered by the user to the server. The input is the prompt data and emotion data, and the output is the result of sending it to the server. The server sends this data to the generative AI model. If the data transmission is successful, the terminal displays "Data transmission successful."

[1663] Step 6: Receiving a response from the generative AI model

[1664] The server receives the response from the generative AI model and transfers it to the terminal. The input is the response data from the generative AI model, and the output is the transfer result to the terminal. The terminal prepares to display this response to the user. When the response is received, the terminal displays "Response received successfully."

[1665] Step 7: View the response

[1666] The terminal displays the response received from the generative AI model to the user. For example, it displays "It's sunny today. Have a nice day." The input is the response data from the generative AI model, and the output is the display result to the user. The user can check this response and enter further prompts if necessary.

[1667] (Application example 1)

[1668] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1669] Conventional food delivery services have difficulty making customized suggestions based on users' emotions and situations. In addition, there is a lack of means for users to use generative AI to simplify the ordering process. This can lead to poor user experience and reduced frequency of service use.

[1670] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1671] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, an emotion recognition means for recognizing the emotion of the user, and a means for feeding back the emotion information recognized by the emotion recognition means to the generative AI. This makes it possible to propose food delivery customized according to the emotion of the user.

[1672] An "accessory device connectable to a smartphone" is an external device that is connected to a smartphone using wireless communication means and provides a specific function.

[1673] "Generative AI" is an artificial intelligence system that generates responses or suggestions based on prompting input from a user.

[1674] "Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

[1675] "User authentication means" refers to technology for verifying the identity of a user and ensuring that only specific users can operate the system.

[1676] A "fingerprint authentication means" is a technology that scans a user's fingerprint and authenticates the user based on that information.

[1677] An "operation means" is a physical or software interface that allows a user to operate the system.

[1678] A "one-click operation means" is an interface that allows a user to execute a specific function with a single click.

[1679] "Emotion recognition means" is a technology that analyzes the user's tone of voice and facial expressions to recognize their emotions.

[1680] A "means for feeding back emotional information to a generative AI" is a function that sends recognized emotional information to a generative AI, enabling the generative AI to generate responses or suggestions based on that information.

[1681] This invention is a system that uses an accessory device connectable to a smartphone to simplify the ordering process when a user uses a food delivery service through generative AI and makes customized suggestions based on the user's emotions.

[1682] System Configuration

[1683] The system consists of the following main components:

[1684] 1. Accessory devices that can be connected to smartphones

[1685] Wireless communication means: Connects to a smartphone using Bluetooth or similar.

[1686] User authentication method: A fingerprint sensor is used to authenticate the user.

[1687] Control: A one-click control is used by the user to prompt the generative AI.

[1688] Emotion recognition: Analyzes the user's tone of voice and facial expressions to recognize emotions.

[1689] 2. Generative AI

[1690] Prompt input: Receives instructions or questions from the user and generates responses or suggestions.

[1691] Emotional feedback: Generate customized responses and suggestions based on recognized emotional information.

[1692] Program Processing

[1693] The server does the following:

[1694] 1. Bluetooth connection: Connect your smartphone and accessory device using Bluetooth.

[1695] 2. User authentication: The fingerprint sensor scans the user's fingerprint to authenticate them.

[1696] 3. Emotion Recognition: Recognize emotions by analyzing the user’s tone of voice and facial expressions.

[1697] 4. Prompt generation: Generate prompts for the generative AI based on the recognized emotions.

[1698] 5. AI Response Generation: Use generative AI to generate responses and suggestions to prompts.

[1699] Hardware and Software Used

[1700] Hardware: Bluetooth module, fingerprint sensor, camera (for face recognition), microphone (for voice recognition)

[1701] Software: Bluetooth connection library, fingerprint authentication library, emotion recognition engine, generative AI model

[1702] Examples

[1703] If the user is recognized as tired, the generative AI will suggest "food delivery options recommended for when you're tired." For example, the following prompts may be generated:

[1704] User is feeling tired, please suggest food delivery options.

[1705] In this way, it becomes possible to provide customized food delivery suggestions based on the user's emotions.

[1706] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1707] Step 1:

[1708] The server connects the smartphone and the accessory device using a Bluetooth module. The input is pairing information between the smartphone and the accessory device, and the output is the establishment of a connection. Specifically, the server uses a Bluetooth connection library to establish communication between the smartphone and the accessory device.

[1709] Step 2:

[1710] The device uses a fingerprint authentication sensor to scan the user's fingerprint and perform user authentication. The input is the user's fingerprint data, and the output is the authentication result. Specifically, the fingerprint authentication library is used to compare the scanned fingerprint data with registered data to determine whether authentication was successful.

[1711] Step 3:

[1712] The device uses a microphone and a camera to capture the user's tone of voice and facial expressions to perform emotion recognition. The input is voice data and image data, and the output is recognized emotion information. Specifically, an emotion recognition engine is used to analyze the voice data and image data to identify the user's emotion.

[1713] Step 4:

[1714] The server generates a prompt for the generative AI based on the recognized emotion information. The input is emotion information, and the output is a prompt. Specifically, the server converts the emotion information into text format and creates a prompt suitable for the generative AI.

[1715] Step 5:

[1716] The server uses a generative AI model to generate responses and suggestions to prompts. The input is the prompt, and the output is the generated response or suggestion. Specifically, the prompt is input to the generative AI model, and the AI ​​generates the response or suggestion.

[1717] Step 6:

[1718] The terminal presents the generated response or suggestion to the user. The input is the generated response or suggestion, and the output is the display to the user. Specifically, the response or suggestion is displayed on the smartphone display so that the user can check it.

[1719] Example 2

[1720] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1721] In conventional systems using generative AI models, users had to operate their smartphones, making them difficult to use. In addition, they were unable to generate appropriate responses according to the user's emotions, limiting the user experience. This made it difficult for users to effectively utilize generative AI models.

[1722] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1723] In this invention, the server includes a user authentication means, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI model, an emotion recognition means for capturing the user's voice data and analyzing the emotion, and a means for feeding back the emotion information analyzed by the emotion recognition means to the generative AI model. This allows the user to easily use the generative AI model without operating a smartphone, and further enables an appropriate response according to the user's emotion to be obtained.

[1724] "User authentication means" means a means for verifying a user's identity, and includes biometric authentication technologies such as fingerprint authentication and facial recognition.

[1725] "Operation means" refers to the means by which a user inputs prompts to the generated AI model, and includes interfaces such as buttons and touch screens.

[1726] An "emotion recognition means" is a means for capturing a user's voice data and analyzing the user's emotions from the tone and content of the data.

[1727] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on prompts from a user.

[1728] "Prompt input" is the act of a user inputting instructions or questions to a generative AI model.

[1729] The "feedback means" refers to a means for transmitting the emotion information analyzed by the emotion recognition means to the generative AI model.

[1730] The present invention provides a system including a user authentication means, an operation means, an emotion recognition means, and a feedback means. A specific embodiment of the system will be described below.

[1731] First, the user places his or her finger on the accessory device's fingerprint sensor. The fingerprint sensor captures the user's fingerprint data and sends it to the device. The device then sends this data to the server, which matches the fingerprint data with an existing database to authenticate the user. If authentication is successful, the device's buttons are enabled.

[1732] Next, when the user presses the enabled button, the device launches a dedicated application on the smartphone. This application has the function of inputting a prompt sentence to the generative AI model. For example, the prompt sentence "Please tell me about the weather today" is input to the generative AI model.

[1733] Furthermore, when the user speaks to the accessory device, the microphone in the terminal captures voice data. This voice data is sent to the emotion recognition means, which analyzes the tone of the voice data to analyze the user's emotion. The analysis result is sent to the generative AI model through the feedback means. The generative AI model generates an appropriate response based on this emotion information. For example, in response to the prompt "The user seems a little tired. Please generate an encouraging message," the generative AI model generates a response such as "Let's take a break today and refresh ourselves."

[1734] This system allows users to easily use generative AI models without operating a smartphone, and improves the user experience by providing appropriate responses according to the user's emotions.

[1735] As a concrete example, when a user places his / her finger on the fingerprint authentication sensor of an accessory device and presses a button after authentication is successful, an application on the smartphone is launched and the following prompt sentence is input to the generative AI model:

[1736] "Tell me about the weather today."

[1737] As another specific example, when a user speaks to the accessory device saying, "How are you feeling today?", the emotion recognition means recognizes the emotion from the tone of the user's voice and inputs the following prompt sentence into the generative AI model:

[1738] "The user seems a bit tired. Generate an uplifting message."

[1739] This allows users to easily utilize generative AI without having to operate a smartphone.

[1740] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1741] Step 1:

[1742] The user places his or her finger on the fingerprint sensor.

[1743] Input: User's fingerprint

[1744] Specific operation: When a user places his or her finger on the fingerprint sensor on the accessory device, the sensor glows blue.

[1745] Output: The captured fingerprint data.

[1746] Step 2:

[1747] The device captures fingerprint data and sends it to the server.

[1748] Input: Captured fingerprint data

[1749] Specific operation: The device captures fingerprint data from the fingerprint authentication sensor, and when the data is sent to the server, the device screen displays "Authentication...".

[1750] Output: Fingerprint data sent to the server

[1751] Step 3:

[1752] The server authenticates the fingerprint data.

[1753] Input: Fingerprint data sent to the server

[1754] Specific operation: The server compares the received fingerprint data with an existing database and authenticates the user.

[1755] Output: Authentication result (success or failure)

[1756] Step 4:

[1757] If the authentication is successful, the terminal button is enabled.

[1758] Input: Authentication result (success)

[1759] Specific operation: If the server succeeds in authentication, it sends a signal to the terminal to activate the button. The button on the terminal turns green.

[1760] Output: Enabled buttons

[1761] Step 5:

[1762] The user presses a button.

[1763] Input: Enabled button

[1764] Specific Action: The user presses an enabled button, which triggers the next step.

[1765] Output: Button press signal

[1766] Step 6:

[1767] The terminal launches an application on the smartphone.

[1768] Input: Button press signal

[1769] Specific operation: The device launches a dedicated application on the smartphone. The application logo appears on the smartphone screen.

[1770] Output: Launched application

[1771] Step 7:

[1772] The application prompts input to the generative AI model.

[1773] Input: The launched application

[1774] Specific operation: The application inputs a prompt sentence to the generative AI model. The screen displays "Sending prompt...". For example, the prompt sentence is "Tell me about the weather today."

[1775] Output: The prompt sent to the generative AI model

[1776] Step 8:

[1777] The user speaks to the accessory device.

[1778] Input: User's voice

[1779] Specific Actions: The user speaks to the accessory device, for example, "How are you feeling today?" The microphone on the device lights up red to indicate that it is capturing voice data.

[1780] Output: Captured audio data

[1781] Step 9:

[1782] The device captures voice data and sends it to the emotion engine.

[1783] Input: Captured audio data

[1784] Specific operation: The device uses the built-in microphone to capture voice data and transmits the data to the emotion engine.

[1785] Output: Audio data sent to the emotion engine

[1786] Step 10:

[1787] The emotion engine analyzes the user's emotions from the tone of the voice data.

[1788] Input: Audio data sent to the emotion engine

[1789] Specific operation: The emotion engine analyzes the tone of the voice data and analyzes the user's emotion. The device screen displays "Analyzing emotion...".

[1790] Output: Analyzed sentiment information

[1791] Step 11:

[1792] The emotion engine feeds the analysis results back into the generative AI model.

[1793] Input: Analyzed sentiment information

[1794] How it works: The emotion engine feeds back the analysis results to the generative AI model, for example sending information like "The user seems a bit tired."

[1795] Output: Emotion information sent to the generative AI model

[1796] Step 12:

[1797] The generative AI model generates responses based on the user's emotions.

[1798] Input: Emotion information sent to the generative AI model

[1799] Specific operation: The generative AI model generates an appropriate response based on the fed-back emotional information. For example, in response to the prompt "Generate an uplifting message," the model generates the response "Take a break today and refresh yourself." The response is displayed on the smartphone screen.

[1800] Output: The generated response

[1801] (Application example 2)

[1802] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1803] In conventional food delivery services, when users use their smartphones to place orders, the operation is complicated and appropriate suggestions based on the user's emotions are not given. In particular, there is a problem that the user experience is not improved for tired or stressed users because there are not enough suggestions for relaxing meals.

[1804] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1805] In this invention, the server includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a voice capture means for capturing the user's voice and recognizing emotions, an emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the user's emotion, and a means for the generative AI to generate an appropriate response based on the emotion recognized by the emotion recognition means. This enables the user to receive appropriate food delivery suggestions through voice and emotion recognition without operating a smartphone.

[1806] "Generative AI" is an artificial intelligence system that generates appropriate responses and suggestions based on user input.

[1807] "Prompt input" refers to the act of inputting instructions or questions to a generative AI.

[1808] An "accessory device" is an external device that can be connected to a smartphone and provides a specific function.

[1809] "User authentication means" means a means for verifying a user's identity and ensuring that only specific users can operate the system.

[1810] A "fingerprint authentication means" is a means for reading a user's fingerprint and authenticating him or her.

[1811] "Operation means" refers to means such as an interface or buttons that allow the user to operate the system.

[1812] A "one-click operation means" is a means that allows a user to complete an operation with a single click.

[1813] "Audio capture means" means for recording or capturing a user's voice.

[1814] An "emotion recognition means" is a means for analyzing captured voice data and recognizing a user's emotion.

[1815] "Response" refers to the reply or suggestion that a generative AI generates in response to user input.

[1816] A system for implementing this invention is configured as follows. First, a user uses an accessory device connected to a smartphone. This accessory device is equipped with a fingerprint authentication means, and authentication is performed by the user placing his / her finger on the accessory device. If authentication is successful, the operation means of the accessory device is enabled, and the user can use the one-click operation means to input a prompt to the generative AI.

[1817] Next, the voice capture means captures the user's voice. This voice data is sent to the emotion recognition means, which analyzes the user's emotions. The emotion recognition means recognizes the user's emotions from the tone and content of the voice data, and feeds the results back to the generative AI. The generative AI generates an appropriate response based on the recognized emotions.

[1818] Specifically, the following hardware and software are used: A fingerprint sensor is used as the fingerprint authentication means. A microphone and voice recognition software (e.g., speech_recognition library) are used as the voice capture means. An emotion engine (e.g., EmotionEngine) is used as the emotion recognition means. An AI generation model (e.g., AIGenerator) is used as the generative AI.

[1819] For example, if a user says, "I'm tired today," the voice capture means captures the voice, and the emotion recognition means detects "fatigue." Based on this emotion, the generative AI generates a response suggesting a "relaxing meal." In this way, appropriate food delivery suggestions are made according to the user's emotions.

[1820] Examples of prompts to input to a generative AI model include:

[1821] User Emotion: Fatigue

[1822] Prompt for generative AI: "Please suggest relaxing meals for a user who is tired."

[1823] This system allows users to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate a smartphone.

[1824] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1825] Step 1:

[1826] The user places his / her finger on the fingerprint authentication sensor of the accessory device connected to the smartphone. The input is the user's fingerprint data, and the output is the authentication result. The accessory device reads the fingerprint data and performs authentication by comparing it with its internal database. If authentication is successful, it proceeds to the next step.

[1827] Step 2:

[1828] If the authentication is successful, the operation means of the accessory device is enabled. The user uses the one-click operation means to input a prompt to the generative AI. The input is the user's click operation, and the output is the trigger for the prompt input. The accessory device detects the click operation and starts inputting a prompt to the generative AI.

[1829] Step 3:

[1830] The terminal's voice capture means captures the user's voice. The input is the user's voice data and the output is the captured voice data. The terminal's microphone records the user's voice and the voice recognition software converts the data into a digital format.

[1831] Step 4:

[1832] The captured voice data is sent to an emotion recognition means. The input is the captured voice data and the output is the recognized emotion data. The emotion engine analyzes the voice data and recognizes the user's emotion from tone and content.

[1833] Step 5:

[1834] The recognized emotion data is fed back to the generative AI. The input is the recognized emotion data, and the output is a prompt to the generative AI. The generative AI creates a prompt to generate an appropriate response based on the emotion data.

[1835] Step 6:

[1836] The generative AI generates an appropriate response based on the prompt. The input is the prompt and the output is the generated response. The generative AI analyzes the prompt and generates food delivery suggestions based on the user's sentiment.

[1837] Step 7:

[1838] The generated response is provided to the user. The input is the generated response, and the output is a suggestion to the user. Appropriate food delivery suggestions are displayed or audibly conveyed to the user through the display or audio output device of the terminal.

[1839] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1840] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generative AIs. Data generation model 58 is

[1841] , obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1842] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1843] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1844] [Fourth embodiment]

[1845] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1846] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1847] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1848] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1849] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1850] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1851] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1852] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1853] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1854] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1855] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1856] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1857] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1858] "Example 1"

[1859] As one embodiment of the present invention, an accessory device connectable to a smartphone is provided. This device includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI. Specifically, the accessory device is connected to a smartphone using a wireless communication means such as Bluetooth, and the prompt input to the generative AI is performed through an application running on the smartphone. As the user authentication means, a fingerprint authentication sensor is built into the device, and authentication is performed by matching the fingerprint information of a registered user. As the operation means, a prompt input to the generative AI is performed by a user pressing a button provided on the device.

[1860] "Example 2"

[1861] As a specific example of use, a user authenticates themselves by placing their finger on the fingerprint authentication sensor of the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is made to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

[1862] The process flow of each embodiment will be described below.

[1863] "Example 1"

[1864] Step 1: A user connects an accessory device connectable to a smartphone to the smartphone using a wireless communication means such as Bluetooth.

[1865] Step 2: The user places his or her finger on the fingerprint authentication sensor of the accessory device, and the fingerprint authentication sensor authenticates the user by matching the fingerprint information with that of the registered user.

[1866] Step 3: If the authentication is successful, the device's buttons are enabled.

[1867] Step 4: When the user presses a button on the device, an application on the smartphone is launched and a prompt input is sent to the generative AI.

[1868] "Example 2"

[1869] Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

[1870] Step 2: If authentication is successful, the device's buttons are enabled.

[1871] Step 3: When the user presses the button, an application on the smartphone is launched and prompts the generative AI to input the information.

[1872] Step 4: This will enable users to easily utilize generative AI without having to operate a smartphone.

[1873] Example 1

[1874] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1875] In conventional systems using generative AI models, users had to go through complicated operations and multi-step authentication when entering a prompt, which resulted in poor user experience. In addition, there was a lack of efficient means to quickly and accurately obtain the generated results. This reduced the convenience of users when using generative AI models.

[1876] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1877] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a means for running an application for inputting a prompt to the generative AI on a smartphone, and a means for sending a prompt sentence to the generative AI model and receiving a generation result. This allows a user to input a prompt sentence to the generative AI model with a simple operation and quickly and accurately obtain a generation result.

[1878] A "smartphone" is a portable information terminal that combines the functions of a mobile phone and a computer, and can have a variety of functions installed on it.

[1879] An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

[1880] "Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompt sentences.

[1881] "Prompt input" is the act of inputting instructions or questions to a generative AI.

[1882] "User authentication means" refers to technology or devices used to verify a user's identity.

[1883] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and verifies the user's identity based on that information.

[1884] The "operation means" refers to an interface or device that allows the user to operate the accessory device or smartphone.

[1885] A "one-click operation means" is an interface or device that allows a user to execute a specific function with a single click.

[1886] An "application" is software that runs on a smartphone and provides specific functions or services.

[1887] "Generated results" are output such as text and images that the generative AI generates based on the prompt text.

[1888] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1889] The present invention is a system for easily inputting prompts to a generative AI model using an accessory device connectable to a smartphone. A specific embodiment of the system will be described below.

[1890] Hardware and Software Configuration

[1891] Accessory equipment:

[1892] An accessory device is an external device that is connected to a smartphone and has the following main functions:

[1893] User authentication method: Equipped with a fingerprint authentication sensor to read the user's fingerprint information.

[1894] Operation: Equipped with a one-click button, users can easily operate it.

[1895] Smartphone:

[1896] The smartphone is connected to the accessory device using a wireless communication means such as Bluetooth. An application for inputting prompts to the generative AI model is installed on the smartphone.

[1897] Generative AI models:

[1898] The generative AI model runs on a server and generates text and images based on prompts sent from the smartphone.

[1899] System Operation

[1900] User authentication:

[1901] The user places his / her finger on the fingerprint authentication sensor installed in the accessory device. The accessory device reads the fingerprint information and compares it with the fingerprint information stored in the internal database. If the authentication is successful, the accessory device sends a signal of authentication success to the smartphone.

[1902] Prompt Input:

[1903] After successful authentication, the user presses a button on the accessory device. The smartphone application starts and a prompt input screen is displayed. The user inputs a prompt sentence for the generative AI model.

[1904] Send to generative AI model:

[1905] The smartphone sends the input prompt sentence to the generative AI model. The server operates the generative AI model based on the received prompt sentence and creates the generated result.

[1906] View the generated results:

[1907] The server sends the generated results to the smartphone. The smartphone displays the received generated results to the user. The user checks the displayed generated results and saves or shares them as necessary.

[1908] Examples

[1909] The specific steps a user takes if they want a generative AI to create a new poem are as follows:

[1910] 1. User Authentication:

[1911] The user places his or her finger on the fingerprint authentication sensor of the accessory device.

[1912] The accessory device reads the fingerprint and performs authentication.

[1913] If the authentication is successful, a signal indicating successful authentication is sent to the smartphone.

[1914] 2. Prompt input:

[1915] The user presses a button on the accessory device.

[1916] The smartphone application will start and a prompt input screen will be displayed.

[1917] The user enters the prompt text "Create a new poem."

[1918] 3. Send to generative AI model:

[1919] The smartphone sends the entered prompt text to the generative AI model.

[1920] The server runs a generative AI model based on the prompt text to generate a poem.

[1921] 4. View the generated results:

[1922] The server sends the generated poem to the smartphone.

[1923] The smartphone displays the received poem to the user.

[1924] The user can review the displayed poem and save or share it as desired.

[1925] In this way, a user can use the accessory device and a smartphone to provide prompt input to the generative AI model and obtain generative results.

[1926] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1927] Step 1: Connect accessory devices

[1928] Input: A user opens the Bluetooth settings on their smartphone and selects an accessory device.

[1929] How it works: A smartphone establishes a connection with an accessory device using Bluetooth.

[1930] Output: The smartphone will inform you that the connection was successful.

[1931] Step 2: User authentication

[1932] Input: The user places their finger on the fingerprint sensor on the accessory device.

[1933] How it works: The accessory device reads your fingerprint and matches it with fingerprint information stored in an internal database.

[1934] Output: If authentication is successful, the accessory device sends a signal of authentication success to the smartphone.

[1935] Step 3: Prepare for prompts

[1936] Input: The smartphone receives a signal that authentication was successful.

[1937] How it works: The smartphone displays a prompt input screen for the generative AI.

[1938] Output: The user sees the prompt input screen and begins entering the prompt statement.

[1939] Step 4: Prompt Input

[1940] Input: The user inputs a prompt for the generative AI through a smartphone application.

[1941] How it works: The smartphone receives the input prompt text and formats it internally.

[1942] Output: The prepared prompt is ready to be sent to the generative AI model.

[1943] Step 5: Send to generative AI model

[1944] Input: The smartphone sends a prepared prompt to the generative AI model.

[1945] Operation: The server runs the generative AI model based on the received prompt sentence and creates a generated result.

[1946] Output: The generated results are sent from the server to the smartphone.

[1947] Step 6: Viewing the generation results

[1948] Input: The smartphone receives the generated results from the server.

[1949] How it works: The smartphone displays the generated results it receives to the user.

[1950] Output: The user checks the displayed generated results and can save or share them as needed.

[1951] (Application example 1)

[1952] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1953] To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and insufficient security for user authentication, which creates the risk of operational errors and unauthorized access. This can lead to reduced productivity and safety in factories.

[1954] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1955] In this invention, the server is an accessory device connectable to a smartphone, and includes a means for enabling prompt input to the generative AI, a user authentication means for operating the accessory device, an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI, a means for generating instructions for managing and operating a robot in a factory, and a means for transmitting the instructions to the robot by wireless communication means. This makes it possible to efficiently and safely manage and operate the robot in the factory.

[1956] An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

[1957] "Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

[1958] "Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

[1959] "User authentication means" is a function for confirming and authenticating the identity of a user.

[1960] A "fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

[1961] The "operation means" is an interface that allows the user to input prompts to the generative AI.

[1962] The "one-click operation means" is a function that allows the user to input prompts to the generative AI with a single click.

[1963] "Means for generating instructions for managing and operating robots in a factory" refers to a function for generating specific work instructions for robots in a factory.

[1964] "Wireless communication means" refers to a function for transmitting and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

[1965] The "means for transmitting to the robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

[1966] This invention relates to a system for efficiently and safely managing and operating robots in a factory using an accessory device connectable to a smartphone. A specific embodiment of this system will be described below.

[1967] System Configuration

[1968] The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

[1969] Hardware

[1970] Smartphone: Functions as a user interface and is connected to accessory devices via wireless communication means.

[1971] Accessory device: Connected to a smartphone, the accessory device includes a fingerprint authentication means and an operation means.

[1972] Fingerprint authentication method: Scans the user's fingerprint to authenticate them.

[1973] Wireless communication means: Data is sent and received between the smartphone and the factory robot using Bluetooth, Wi-Fi, etc.

[1974] Factory robot: Performs actions according to instructions.

[1975] software

[1976] Generative AI model: Generates specific instructions for factory robots based on user input prompts.

[1977] Smartphone application: Provides the user interface and manages communication with the generative AI model.

[1978] Operating procedure

[1979] 1. User authentication: The user places a finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

[1980] 2. Prompt input: An authenticated user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[1981] 3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

[1982] 4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

[1983] 5. Robot operation: The factory robot performs the operation according to the instructions it receives.

[1984] Examples

[1985] For example, when a robot needs to perform a specific task in a factory, a manager can use a smartphone and accessory devices to give the robot instructions.

[1986] "Move to assembly line 3 and start welding."

[1987] "Move to the inspection area and perform quality checks."

[1988] "Go get part A and take it to assembly line 2."

[1989] This makes it possible to efficiently and safely manage and operate robots in factories. The system utilizes user authentication and generative AI to reduce the risk of incorrect operation and unauthorized access, improving factory productivity and safety.

[1990] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1991] Step 1:

[1992] The user places a finger on the accessory device and is authenticated by the fingerprint authentication means.

[1993] Input: User's fingerprint

[1994] Data processing: The fingerprint authentication means scans the fingerprint and compares it with registered fingerprint data.

[1995] Output: Authentication result (success or failure)

[1996] How it works: The fingerprint sensor scans the user's fingerprint and matches it against an internal database for authentication.

[1997] Step 2:

[1998] If the authentication is successful, the user uses an operation means (one-click operation means) to input a prompt to the generative AI model.

[1999] Input: Authentication result (success), user prompt

[2000] Data processing: If the authentication result is successful, a prompt sentence is sent to the generative AI model.

[2001] Output: prompt input to the generative AI model

[2002] Specific operation: When the user presses the button, the smartphone application sends a prompt sentence to the generative AI model.

[2003] Step 3:

[2004] Based on the prompts, a generative AI model generates specific instructions for factory robots.

[2005] Input: User prompt text

[2006] Data processing: A generative AI model analyzes the prompt and generates appropriate instructions.

[2007] Output: Instructions for the factory robots

[2008] Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

[2009] Step 4:

[2010] The generated instructions are transmitted to the factory robot using wireless communication means.

[2011] Input: Instructions from a generative AI model

[2012] Data processing: Instructions are sent to factory robots via wireless communication means.

[2013] Output: Sending instructions to factory robots

[2014] Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth and Wi-Fi.

[2015] Step 5:

[2016] The factory robot operates according to the instructions it receives.

[2017] Input: Instructions for the factory robots

[2018] Data processing: The factory robot analyzes the instructions and performs the corresponding actions.

[2019] Output: Robot movement

[2020] Specific actions: The factory robot performs actions according to instructions such as "move to assembly line 3 and start welding."

[2021] Example 2

[2022] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[2023] In order to use conventional generative AI models, users had to operate their smart devices directly and input prompt text. This operation was tedious and inconvenient, especially when their hands were full or in situations where a quick response was required. Also, from a security standpoint, user authentication was sometimes insufficient, creating a risk of unauthorized use by third parties. There is a demand for a system that can solve these issues and make it easier and safer to use generative AI models.

[2024] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2025] In this invention, the server includes a user authentication means, an operation means for an authenticated user to operate, a means for transmitting a signal to the smart device using the operation means to launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without directly operating the smart device, and the user authentication means also improves security.

[2026] "User authentication means" means the means for verifying a user's identity and granting access to a system.

[2027] "Operation means" refers to an interface or device that allows a user to operate the system.

[2028] A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

[2029] The "means for transmitting a signal" refers to a means for transmitting specific instructions or data from the operating means to the smart device.

[2030] A "specific application" is a software program designed to provide a particular function or service.

[2031] A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

[2032] A "prompt" is a text input used to give instructions or ask a question to a generative AI model.

[2033] A "means for generating a response" is a means by which a generative AI model creates an appropriate response based on a prompt sentence.

[2034] The "means for sending a response" refers to a means for sending the generated response to the smart device.

[2035] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[2036] The present invention is a system for enabling a user to utilize a generative AI model without directly operating a smart device. A specific embodiment of the system will be described below.

[2037] Hardware and Software Used

[2038] Hardware: Accessory devices (with fingerprint authentication sensor), smart devices (smartphones, etc.)

[2039] Software: Fingerprint authentication systems, smart device applications, generative AI models

[2040] System Configuration

[2041] 1. User authentication method:

[2042] The user places his or her finger on the fingerprint sensor on the accessory device.

[2043] The fingerprint authentication system captures fingerprint data and authenticates the user.

[2044] 2. Operating means:

[2045] If authentication is successful, the button on the accessory device is enabled.

[2046] The user presses an enabled button.

[2047] 3. Signal transmission means:

[2048] When the button is pressed, the accessory device sends a signal to the smart device.

[2049] The smart device receives the signal and launches a specific application.

[2050] 4. Prompt sending method:

[2051] The launched application sends a prompt to the generative AI model.

[2052] Prompts are pre-defined statements that contain the instructions or questions the user wants to ask the generative AI model.

[2053] 5. Response Generation Method:

[2054] The server uses a generative AI model to generate a response based on the received prompt.

[2055] 6. Response sending method:

[2056] The server generates a response and sends it to the smart device.

[2057] The smart device receives the response and displays it to the user.

[2058] Examples

[2059] When a user places their finger on the fingerprint authentication sensor of the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device to launch an application. The application launches and sends a prompt sentence, "Tell me about the weather today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can confirm the information, "Today's weather is sunny," on the screen.

[2060] Examples of prompt statements

[2061] "Tell me about the weather today."

[2062] "Check the schedule for the next meeting."

[2063] "Tell me the latest news."

[2064] This system allows users to easily use generative AI models without having to directly operate a smart device, and security is also improved through user authentication measures.

[2065] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2066] Program processing flow

[2067] Step 1:

[2068] The user places his or her finger on the fingerprint authentication sensor of the accessory device.

[2069] Input: User's fingerprint

[2070] Data processing: The fingerprint sensor captures the fingerprint data.

[2071] Output: The captured fingerprint data.

[2072] Step 2:

[2073] The device captures the fingerprint data and authenticates the user.

[2074] Input: Captured fingerprint data

[2075] Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

[2076] Output: Authentication result (success or failure)

[2077] Step 3:

[2078] If authentication is successful, the device's buttons are enabled.

[2079] Input: Authentication result (success)

[2080] Data processing: If authentication is successful, enable the device buttons.

[2081] Output: Enabled button

[2082] Step 4:

[2083] The user presses a button.

[2084] Input: Enabled button

[2085] Data processing: The terminal detects that a button has been pressed.

[2086] Output: Button press signal

[2087] Step 5:

[2088] The terminal sends a signal to the smart device to launch the application.

[2089] Input: Button press signal

[2090] Data processing: The terminal sends a signal to the smart device to launch a specific application.

[2091] Output: Launched application

[2092] Step 6:

[2093] The application sends a prompt to the generative AI model.

[2094] Input: The application that was launched

[2095] Data processing: The application sends pre-defined prompts to the generative AI model.

[2096] Output: The prompt sent to the generative AI model

[2097] Step 7:

[2098] The server generates a response using a generative AI model.

[2099] Input: The prompt sent to the generative AI model

[2100] Data computation: A generative AI model analyzes the prompt and generates an appropriate response.

[2101] Output: The generated response

[2102] Step 8:

[2103] The server sends the response to the smart device.

[2104] Input: The generated response

[2105] Data processing: The server generates a response and sends it to the smart device.

[2106] Output: Response sent to smart device

[2107] Step 9:

[2108] The smart device displays the response.

[2109] Input: Response sent to smart device

[2110] Data processing: The smart device displays the response received to the user.

[2111] Output: The response displayed to the user

[2112] In this way, users can utilize generative AI models without directly operating a smart device.

[2113] (Application example 2)

[2114] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[2115] Conventional food delivery systems require users to use their smartphones to place orders, which makes operation complicated. There are also security risks due to insufficient user authentication. Furthermore, ordering systems using voice recognition are not widespread, which reduces user convenience. There is a need to provide a system that solves these problems and allows users to use food delivery services easily and safely.

[2116] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2117] An accessory device connectable to a smart device includes a user authentication means for authenticating a user who operates the accessory device by fingerprint authentication, a button provided on the accessory device that is pressed by a user authenticated by the user authentication means to input instructions to the generative AI, and a voice recognition means for recognizing the user's voice in response to pressing of the button. The server includes a means for acquiring an output of the generative AI using a prompt sentence including voice data acquired by the voice recognition means and the generative AI. The prompt sentence instructs ordering through a food delivery service, and the acquiring means acquires the result of the order using a food delivery API. This allows a user to easily and safely use food delivery services using only voice, without operating a smartphone.

[2118] A "smart device" is a portable electronic device, such as a smartphone or tablet, that has Internet connectivity and can run applications.

[2119] An "accessory device" is additional hardware that is connected to a smart device to provide a specific function.

[2120] "Generative AI" is an artificial intelligence technique that generates text and other content based on instructions or prompts from a user.

[2121] "Instruction input" refers to commands or requests that a user inputs to a generative AI.

[2122] "User authentication means" refers to technology or devices for confirming and authenticating the identity of a user.

[2123] A "fingerprint authentication means" is a technology or device that reads a user's fingerprint and authenticates the user based on that information.

[2124] "Operation means" refers to the interface or buttons that allow the user to operate the accessory device or smart device.

[2125] A "voice recognition means" is a technology or device that analyzes a user's voice and converts it into text data.

[2126] "Voice data" refers to data that is a digital recording of a user's voice.

[2127] "Transmission means" refers to techniques or devices for transmitting data to other devices or systems.

[2128] The present invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. A specific embodiment of this system will be described below.

[2129] System Configuration

[2130] The system consists of the following main components:

[2131] 1. Smart device: A portable electronic device such as a smartphone or tablet that has internet connectivity and can run applications.

[2132] 2. Accessory device: Additional hardware used by connecting to a smart device, including fingerprint authentication means, operation means, and voice recognition means.

[2133] 3. Generative AI: Artificial intelligence techniques that generate text and other content based on instructions or prompts from the user.

[2134] Program Processing

[2135] Hardware

[2136] Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

[2137] Smart device: Functions as the execution environment for applications.

[2138] Microphone: Captures the user's voice as a means of voice recognition.

[2139] software

[2140] Fingerprint authentication library: A software module for performing fingerprint authentication.

[2141] Speech Recognition Library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

[2142] Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3 (registered trademark)).

[2143] Food Delivery API: Place orders using the API of food delivery services.

[2144] Processing flow

[2145] 1. User authentication: The user authenticates by placing his / her finger on the fingerprint authentication sensor of the accessory device. If authentication is successful, the next step is performed.

[2146] 2. Voice input: The user presses the operation means (one-click button) of the accessory device and speaks the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

[2147] 3. Prompt generation: Based on the acquired text data, a prompt sentence is generated for the generative AI.

[2148] 4. AI response generation: The generative AI model generates the optimal response based on the prompt.

[2149] 5. Order fulfillment: Based on the response generated, complete the order using the food delivery API.

[2150] Examples

[2151] When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model is given the prompt "User wants to order: pizza," the generative AI model suggests the best pizza restaurant and menu, and completes the order using a food delivery API.

[2152] Examples of prompt statements

[2153] "User wants to order: pizza"

[2154] In this way, users can use voice commands to easily and safely receive food delivery services without having to operate a smartphone.

[2155] The flow of the specific process in the application example 2 will be described with reference to FIG.

[2156] Step 1:

[2157] The user places his or her finger on the fingerprint sensor.

[2158] Input: The user's fingerprint data.

[2159] Processing: The fingerprint authentication library analyzes the fingerprint data and matches it with registered fingerprint data.

[2160] Output: The authentication result (success or failure).

[2161] Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

[2162] Step 2:

[2163] The user presses the operating means (one-click button) of the accessory device.

[2164] Input: User action (button press).

[2165] Processing: Detects that an operating means has been pressed and activates the voice recognition mode.

[2166] Output: Activate speech recognition mode.

[2167] Specific operation: The sensor detects that a button has been pressed, and the speech recognition library begins preparing for voice input.

[2168] Step 3:

[2169] The user verbally communicates the order.

[2170] Input: The user's voice data.

[2171] Processing: The microphone picks up the voice data, and the speech recognition library converts it into text data.

[2172] Output: Text data (order details).

[2173] What happens: A microphone captures the user's voice, and a speech recognition library analyzes the speech and converts it into text.

[2174] Step 4:

[2175] The terminal generates a prompt sentence based on the acquired text data.

[2176] Input: Text data (order details).

[2177] Processing: The text data is converted into a prompt sentence of the form "The user would like to order: [order details]".

[2178] Output: The prompt statement.

[2179] Specific operation: Convert text data into prompt sentences according to the format.

[2180] Step 5:

[2181] The device sends a prompt to the generative AI model.

[2182] Input: prompt statement.

[2183] Processing: The generative AI model analyzes the prompt and generates the most appropriate response.

[2184] Output: AI response (optimal order content).

[2185] Specific operation: The prompt sentence is sent to the generative AI model, and the AI ​​generates a response.

[2186] Step 6:

[2187] The device uses the generated response to complete the order using the food delivery API.

[2188] Input: AI response (optimal order content).

[2189] Processing: Fulfilling the order using the food delivery API.

[2190] Output: Order confirmation (data indicating the order was successfully completed).

[2191] Specific behavior: Send the AI ​​response to the food delivery API and confirm the order.

[2192] The server may further include a means for recognizing the user's emotion using an emotion engine described later, and may obtain an output of the generative AI using the voice data obtained by the voice recognition means, a prompt sentence including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," a prompt sentence "User wants to order: pizza, user's emotion: sadness" is generated, and the generative AI model proposes an optimal pizza restaurant and menu considering the user's emotion, and completes the order using a food delivery API.

[2193] In addition, an emotion engine that estimates the emotion of the user may be further combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[2194] "Example 1"

[2195] As one embodiment of the present invention, an accessory device connectable to a smartphone is provided. The device includes a means for enabling prompt input to the generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI. Furthermore, the device includes an emotion engine that recognizes the user's emotions. Specifically, the device recognizes emotions from the user's tone of voice and facial expressions, and feeds back the information to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

[2196] "Example 2"

[2197] As another embodiment of the present invention, a system is provided in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone in the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotions from the tone of the voice data and feeds back the results to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

[2198] The process flow of each embodiment will be described below.

[2199] "Example 1"

[2200] Step 1: A user operates an accessory device that can be connected to a smartphone.

[2201] Step 2: The user authentication means of the device authenticates the user. One possible authentication method is fingerprint authentication.

[2202] Step 3: The authenticated user prompts the generative AI.

[2203] Step 4: The device's emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[2204] Step 5: The emotion engine feeds that information back to the generative AI.

[2205] Step 6: The generative AI generates an appropriate response based on the user's emotions.

[2206] "Example 2"

[2207] Step 1: The user talks to the accessory device.

[2208] Step 2: The microphone in the device captures the user's voice and sends the voice data to the emotion engine.

[2209] Step 3: The emotion engine analyzes the user's emotion from the tone of the voice data.

[2210] Step 4: The emotion engine feeds its results back to the generative AI.

[2211] Step 5: The generative AI generates an appropriate response based on the user's emotions.

[2212] Example 1

[2213] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[2214] Conventional accessory devices that can be connected to smartphones allow prompt input to generative AI models, but they have the problem of being unable to generate responses that take into account the user's emotional state. This makes it difficult to obtain appropriate responses according to the user's intentions and emotions, resulting in a lack of improvement in the user experience.

[2215] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. ...

Claims

1. A system including an accessory device connectable to a smart device, a user authentication means for authenticating a user who operates the accessory device by fingerprint authentication; a button provided on the auxiliary device, the button being pressed by a user authenticated by the user authentication means to input an instruction to the generative AI; a voice recognition means for recognizing a user's voice in response to pressing of the button; a means for acquiring an output of the generative AI using a prompt sentence including the voice data acquired by the voice recognition means and the generative AI; A system including:

2. The prompt instructs the user to order a food delivery service, The system according to claim 1 , wherein the acquiring means acquires the results of an order using a food delivery API.

3. means for recognizing an emotion of a user using an emotion engine; The acquiring means includes: The system according to claim 1 , further comprising: a voice recognition means for acquiring voice data; a prompt sentence including the user's emotion; and a generative AI for acquiring an output of the generative AI.

Images