system

Abstract

We provide the system. [Solution] A means for generating initial setup templates for smart devices using generative artificial intelligence and distributing them to participants' devices, A means of generating operating procedures in real time using generative artificial intelligence and displaying them on the participants' devices, A means of receiving questions from participants, generating answers using generative artificial intelligence, and delivering them to the participants' devices, A means of monitoring the progress of participants' operations and providing additional support information as needed, A system that includes this.

Description

Technical Field

[0004] , , , ,

[0005] , , , , ,

[0003] , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Modern smart devices are multifunctional and often have complex initial settings and operation methods. Therefore, users may have difficulty using new devices. In particular, when lacking knowledge about initial setting procedures and daily operations, the problem is that the functions of the device cannot be fully utilized without appropriate assistance. The present invention aims to solve the difficulties of users regarding the setting and operation of such smart devices and further provide real-time assistance.

Means for Solving the Problems

[0005] This invention provides a system that generates initial setup templates for smart devices using generative artificial intelligence and delivers them to participants' devices. It also includes means for generating operating procedures in real time using generative artificial intelligence and displaying them on participants' devices. Furthermore, it provides means for receiving questions from participants, generating answers using generative artificial intelligence, and delivering them to participants' devices. This allows users to receive detailed guides on device setup and operation in real time, and to quickly resolve any questions. Additionally, the system includes means for monitoring participants' progress and providing additional support information as needed, thereby improving the user experience. In this way, this invention reduces the burden on users regarding smart device setup and operation, thereby promoting their use.

[0006] "Generative artificial intelligence" refers to artificial intelligence that automatically generates new information and content based on given data and instructions.

[0007] "Smart devices" is a general term referring to mobile terminals and wearable devices that have internet connectivity and advanced computing capabilities.

[0008] An "initial setup template" is a guide that summarizes the basic settings required when using a smart device for the first time (e.g., network settings, account settings, application installation procedures).

[0009] "Operating procedures" are step-by-step guidelines for performing a specific task.

[0010] "Real-time" refers to the time when responses and processing occur instantly with little to no delay.

[0011] A "participant" is a user of a smart device who receives support for its operation and settings.

[0012] "Terminal" refers to the smart devices used by participants.

[0013] "Receiving a question" refers to the action by which the server receives a question entered by the user.

[0014] "Generating answers" refers to the process where generative artificial intelligence creates appropriate information or instructions based on the received question.

[0015] "Monitoring operation progress" means that the server or system checks how well the user understands and is executing the operation procedures.

[0016] "Support information" refers to additional information and instructions provided to help users smoothly perform operations and settings. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10]Shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the attached drawings.

[0019] First, explanations will be given of the terms used in the following description.

[0020] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention relates to a system that utilizes generative artificial intelligence to assist with the initial setup and operation of smart devices. This system consists of a server, a terminal, and a user as its main components, each working together to provide operational assistance.

[0039] Server roles and processing

[0040] Generation and distribution of initial configuration templates

[0041] The server generates an initial setup template for smart devices. This template includes instructions for network settings, account settings, and application installation. This template is then distributed to the participants' devices.

[0042] As a concrete example, the server generates Wi-Fi setup instructions for a new smartphone and sends them to the device. This allows the user to easily complete the initial setup.

[0043] Real-time generation and distribution of operating procedures

[0044] The server generates operating instructions in real time and displays them on participants' devices. For example, it generates instructions for scanning QR codes (registered trademark) or installing apps and immediately delivers them to the devices.

[0045] Question reception and answer generation

[0046] The server receives user questions from terminals and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals.

[0047] For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[0048] Monitoring of operation progress and provision of additional support information

[0049] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[0050] Terminal roles and processing

[0051] Receiving and displaying initial setup information

[0052] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[0053] For example, a device can receive a Wi-Fi setup guide and display it on its screen, allowing the user to configure the settings by following the guide.

[0054] Demonstration

[0055] The terminal receives operation instructions sent from the server in real time and displays them clearly to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[0056] Submitting questions and viewing answers

[0057] The terminal sends the question entered by the user to the server, receives the answer from the server again, and displays it to the user.

[0058] User roles and operations

[0059] Execute initial setup

[0060] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[0061] Review and execute the demo procedure.

[0062] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[0063] Resolving doubts and questions

[0064] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks "How do I adjust the screen brightness?", the system adjusts the settings based on the answer from the server.

[0065] As described above, this invention provides a system that allows users to efficiently learn how to set up and operate smart devices and to quickly resolve any questions they may have. This enables users to make the most of the device's functions and promotes its use.

[0066] The following describes the processing flow.

[0067] Step 1:

[0068] The server generates an initial configuration template.

[0069] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices. These templates include network settings, account settings, and application installation instructions.

[0070] Step 2:

[0071] The server delivers the initial setup template to the terminal.

[0072] The server distributes the generated initial setup template to each participant's device. This allows the device to receive the setup guide.

[0073] Step 3:

[0074] The device displays initial setup information.

[0075] The device visually displays the received initial setup template to the user. The user proceeds with the initial setup while confirming the information displayed on the screen.

[0076] Step 4:

[0077] The user performs the initial setup.

[0078] The user configures the smart device by following the initial setup template displayed on the terminal. For example, they might enter Wi-Fi connection information to connect to the network.

[0079] Step 5:

[0080] The server generates operating procedures in real time.

[0081] The server uses generative artificial intelligence to generate the necessary operating procedures in real time, based on the user's progress and questions.

[0082] Step 6:

[0083] The server delivers the operating instructions to the terminal.

[0084] The server distributes the generated operating instructions to the user's terminal. This allows the terminal to receive the latest operating guide.

[0085] Step 7:

[0086] The device displays the operating instructions.

[0087] The terminal displays the received operating instructions in an easy-to-understand manner for the user. The user then performs the next operation based on this information.

[0088] Step 8:

[0089] The user performs the operating procedure.

[0090] The user follows the instructions displayed on the device to operate the smart device. For example, they might install an app as instructed.

[0091] Step 9:

[0092] The terminal sends the operation progress to the server.

[0093] The terminal sends user progress information to the server. This allows the server to track the user's progress.

[0094] Step 10:

[0095] User enters and submits question.

[0096] Users input questions and concerns about the operation and settings of their smart devices into their terminals and send them to the server.

[0097] Step 11:

[0098] The server receives the question and generates the answer.

[0099] The server analyzes the received question and generates an answer using generative artificial intelligence.

[0100] Step 12:

[0101] The server delivers the generated response to the terminal.

[0102] The server sends the generated response to the device. This allows the device to receive the latest response information.

[0103] Step 13:

[0104] The device displays the answer.

[0105] The device displays the received response to the user. The user reviews the response displayed on the screen and uses it as a reference for operations and settings.

[0106] Step 14:

[0107] The user completes the operation.

[0108] Based on the answers and guides received from the server, users ultimately complete the setup and operation of their smart devices.

[0109] In this way, a system is realized in which servers, terminals, and users work together to smoothly configure and operate smart devices.

[0110] (Example 1)

[0111] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0112] Initial setup and learning how to operate smart devices are time-consuming and difficult, especially for beginners. Furthermore, there is a lack of real-time solutions to questions and problems encountered during operation. As a result, users cannot fully utilize the device's capabilities, hindering its adoption. A system is needed to address these challenges and provide simple and efficient support for setting up and operating smart devices.

[0113] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0114] In this invention, the server includes means for generating an initial setup template for a smart device using generative artificial intelligence and delivering it to the user's terminal; means for generating operating procedures in real time using generative artificial intelligence and displaying them on the user's terminal; means for receiving questions from the user, generating answers using generative artificial intelligence and delivering them to the user's terminal; and means for monitoring the user's operation progress and providing additional support information as needed. As a result, the user can easily learn how to set up and operate the smart device and quickly resolve any questions or problems they may encounter.

[0115] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new information and content based on large amounts of data.

[0116] A "smart device" refers to an electronic device that can connect to the internet and perform various functions through user operation.

[0117] A "template" refers to a pre-created basic framework that includes a set of settings and configuration information tailored to a specific purpose.

[0118] A "server" refers to a computer system that provides various services and data to other computers and terminals via a network.

[0119] A "terminal" refers to an electronic device equipped with input devices and display devices that can be directly operated by the user.

[0120] A "question" refers to information entered by the user, either as text or voice, that expresses doubts or unclear points that need to be resolved.

[0121] "Answer" refers to the solutions or information that the server provides to the user's question using generative artificial intelligence.

[0122] "Operation progress" refers to the status of operations being performed by the user on their smart device and the degree to which each step has been completed.

[0123] "Support information" refers to additional instructions and guidelines to help users resolve problems and questions they may encounter while using the service.

[0124] A "prompt" refers to text used to instruct a generative artificial intelligence on what tasks to perform or what information to generate.

[0125] This invention is a system that utilizes generative artificial intelligence to assist with the initial setup and operation of smart devices. This system mainly consists of three elements: a server, a terminal, and a user, each playing the following roles.

[0126] Server roles and processing

[0127] Generation and distribution of initial configuration templates

[0128] The server generates initial setup templates for smart devices using generative artificial intelligence. These templates include network settings, account settings, and application installation instructions. Specifically, the server uses generative AI models such as GPT-3.5, which run on Google® Cloud AI Platform.

[0129] example:

[0130] Prompt: "Please generate an initial setup guide for a new smartphone. The guide should include Wi-Fi connection information, Google account setup, and instructions for installing major apps."

[0131] The generated template is sent to the terminal in JSON format using an HTTP POST request.

[0132] Real-time generation and distribution of operating procedures

[0133] The server generates operation procedures in real time in response to user actions. This is also done using a generative artificial intelligence model.

[0134] example:

[0135] Prompt: "Generate instructions for scanning the QR code with your smartphone."

[0136] The generated instructions are delivered to the terminal in text format, and the terminal displays them visually to the user.

[0137] Question reception and answer generation

[0138] The server receives questions from users and generates answers using generative artificial intelligence. This process also uses GPT-3.5 running on Google Cloud AI Platform.

[0139] example:

[0140] Prompt: "When a user asks 'How to install the app,' generate an answer that provides detailed instructions."

[0141] The server sends the generated response back to the terminal and presents it to the user.

[0142] Monitoring of operation progress and provision of additional support information

[0143] The server monitors operation progress information sent from the terminal, generates additional support information as needed, and delivers it to the terminal. For example, if a user reports a connection error, the server generates and provides troubleshooting steps.

[0144] Terminal roles and processing

[0145] Receiving and displaying initial setup information

[0146] The device receives initial setup information sent from the server and displays it visually to the user. For example, the device receives a Wi-Fi setup guide from the server using an HTTP GET request and displays it on the screen, allowing the user to configure the settings.

[0147] Demonstration

[0148] The terminal receives operation instructions sent from the server in real time and displays them to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[0149] Submitting questions and viewing answers

[0150] The terminal sends the question entered by the user to the server, receives the answer from the server, and displays it to the user.

[0151] User roles and operations

[0152] Execute initial setup

[0153] The user configures the device based on the initial setup template displayed on the device. For example, they enter Wi-Fi settings and establish a connection.

[0154] Review and execute the demo procedure.

[0155] The user reviews the demo instructions displayed on the device and performs the actions accordingly. For example, they might follow the app installation instructions to download and install the app.

[0156] Resolving doubts and questions

[0157] Users input questions and inquiries into their terminals as text, and the server provides answers to resolve the issues. For example, a user might ask how to adjust the screen brightness and then adjust the settings based on the server's response.

[0158] In this way, the coordinated operation of each element allows users to efficiently learn the initial setup and operation methods of smart devices, and to quickly resolve any questions or problems they may encounter.

[0159] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0160] Step 1:

[0161] Generating an initial configuration template

[0162] The server uses generative artificial intelligence to generate initial setup templates for smart devices.

[0163] Input: The server receives the user's smart device type and required settings (e.g., Wi-Fi settings, account settings, app installation instructions) as input.

[0164] Specific operation: The server sends this input information as a prompt to a generative artificial intelligence model (e.g., GPT-3.5) to generate a template.

[0165] Output: The server outputs the generated template in JSON format.

[0166] Step 2:

[0167] Distribution of initial setup templates

[0168] The server distributes the generated initial configuration template to the terminal.

[0169] Input: The server receives the generated template data.

[0170] Specific operation: The server sends template data to the terminal using an HTTP POST request.

[0171] Output: The terminal displays the received template data on the screen.

[0172] Step 3:

[0173] Real-time generation of operating procedures

[0174] The server generates real-time operation procedures based on user actions.

[0175] Input: The server receives user requests and information (e.g., requests for QR code scanning procedures).

[0176] Specific operation: The server sends the request as a prompt to a generative artificial intelligence model, which then generates a procedure.

[0177] Output: The server outputs the generated procedure in text format.

[0178] Step 4:

[0179] Real-time streaming of operating procedures

[0180] The server distributes the generated operating instructions to the terminal.

[0181] Input: The server receives the generated procedure data.

[0182] Specific operation: The server sends the procedure data to the terminal via an HTTP POST request.

[0183] Output: The terminal displays the received procedure data to the user.

[0184] Step 5:

[0185] Question submission

[0186] The server receives questions from users.

[0187] Input: The terminal receives the question text data entered by the user.

[0188] Specific action: The terminal sends the question text to the server via an HTTP POST request.

[0189] Output: The server receives the question data.

[0190] Step 6:

[0191] Answer generation

[0192] The server uses generative artificial intelligence to generate answers to the questions it receives.

[0193] Input: The server will use the received question text data as input.

[0194] Specific operation: The server sends the question text as a prompt to a generative artificial intelligence model, which then generates an answer.

[0195] Output: The server outputs the generated response text.

[0196] Step 7:

[0197] Delivery of responses

[0198] The server delivers the generated response to the terminal.

[0199] Input: The server receives the generated response text data.

[0200] Specific operation: The server sends the response text to the terminal via an HTTP POST request.

[0201] Output: The terminal displays the received response text to the user.

[0202] Step 8:

[0203] Monitoring operation progress

[0204] The server monitors the operation progress information sent from the terminal.

[0205] Input: The terminal receives progress data as the user's actions progress.

[0206] Specific operation: The terminal sends progress information to the server in real time.

[0207] Output: The server receives progress data and updates the monitoring status.

[0208] Step 9:

[0209] Providing additional support information

[0210] The server generates necessary support information based on the progress data and delivers it to the terminal.

[0211] Input: The server receives operation progress data.

[0212] Specific operation: The server analyzes progress data and generates necessary support information using a generative artificial intelligence model.

[0213] Output: The server sends the generated support information to the terminal. The terminal displays this to the user.

[0214] (Application Example 1)

[0215] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0216] Current autonomous vehicles' smart devices require numerous settings and operations, placing a significant burden on users. Some users may find initial setup and operation difficult due to their unfamiliarity with electronic devices and operating procedures. Furthermore, limited means of quickly resolving questions or concerns during operation can detract from the user experience. An effective support system is needed to improve this situation.

[0217] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0218] In this invention, the server includes means for generating initial setup templates for smart devices using generative artificial intelligence and distributing them to participants' terminals; means for generating operating procedures in real time using generative artificial intelligence and displaying them on participants' terminals; means for receiving questions from participants, generating answers using generative artificial intelligence and distributing them to participants' terminals; means for monitoring participants' operational progress and providing additional support information as needed; means for performing initial setup and operational support for the smart system of an autonomous vehicle; and means for enabling users to efficiently learn how to set up and operate smart devices and quickly resolve any questions. This enables users to quickly and easily perform complex smart device setup and operation, thereby improving the user experience of autonomous vehicles.

[0219] "Generative artificial intelligence" is an artificial intelligence technology that outputs content generated in natural language or other formats based on given data or prompts.

[0220] "Smart devices" refer to all electronic devices that have internet connectivity and application execution capabilities, including smartphones, tablets, and smartwatches.

[0221] An "initial setup template" is a guideline that outlines a series of settings and procedures required when starting to use a smart device, including network settings, account settings, and application installation procedures.

[0222] "Real-time generation" refers to the process of instantly generating and delivering content and information in response to user requests and circumstances.

[0223] "Participants" refers to users of this system, including individuals and groups operating smart devices.

[0224] "Terminal" refers to a device operated by a user, and includes smartphones, computers, and other interactive electronic devices.

[0225] "Operation progress" refers to the progress of operations performed by the user on their smart device.

[0226] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously without driver intervention.

[0227] A "navigation system" is a system that uses map data and GPS technology to provide the vehicle's current location and the optimal route to its destination.

[0228] An "entertainment system" is a system that provides entertainment content such as music and videos inside a vehicle.

[0229] "Safety devices" refer to various systems and devices that ensure safety while driving a vehicle, and include brake assist and lane departure warning systems.

[0230] This invention relates to a system that utilizes generative artificial intelligence to perform initial setup and operational support for the smart system of an autonomous vehicle. The following describes specific embodiments of the system, with a server, terminal, and user as its main components.

[0231] Server roles and processing

[0232] The server is responsible for the following roles and processes.

[0233] 1. Generation and distribution of initial configuration templates

[0234] The server uses generative artificial intelligence to generate initial setup templates for the smart devices of autonomous vehicles. These templates include network settings, account settings, application installation procedures, and system configuration procedures related to autonomous vehicles. The generated templates are then delivered to the participants' devices.

[0235] 2. Real-time generation and distribution of operating procedures

[0236] The server uses generative artificial intelligence to generate operating instructions in real time and display them on participants' devices. For example, it generates instructions for setting up the navigation system, operating the entertainment system, and setting up safety devices, and immediately delivers them to the devices.

[0237] 3. Receiving questions and generating answers

[0238] The server receives user questions from terminals and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals. For example, in response to a question like, "How do I configure the navigation system in an autonomous vehicle?", the server generates and delivers a detailed guide.

[0239] 4. Monitoring of operation progress and provision of additional support information

[0240] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[0241] Terminal roles and processing

[0242] The terminal is responsible for the following roles and processes.

[0243] 1. Receiving and displaying initial setup information

[0244] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[0245] 2. Receiving and displaying operating instructions

[0246] The terminal receives operation instructions sent from the server in real time and displays them clearly to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[0247] 3. Submitting questions and viewing answers

[0248] The terminal sends the question entered by the user to the server, receives the answer from the server, and displays it to the user.

[0249] User roles and operations

[0250] The user will be responsible for the following roles and operations.

[0251] 1. Perform initial setup

[0252] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[0253] 2. Confirm and execute the demo procedure.

[0254] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[0255] 3. Resolving doubts and questions

[0256] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks, "How do I configure the navigation system of an autonomous vehicle?", the system will configure it based on the answer from the server.

[0257] Examples of specific cases and prompt statements

[0258] The following are examples of specific prompt messages.

[0259] "Please generate Wi-Fi setup instructions for autonomous vehicles. Include the following information: network selection, password entry, and connection confirmation."

[0260] "Please explain the procedure for installing an app into the entertainment system of an autonomous vehicle."

[0261] "Please generate the configuration procedure for the Advanced Driver-Assistance Systems (ADAS) of autonomous vehicles."

[0262] This system allows users to quickly and easily configure and operate the complex smart devices in autonomous vehicles, improving the overall user experience.

[0263] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0264] Step 1:

[0265] Generating an initial configuration template

[0266] The server uses generative artificial intelligence to generate initial setup templates for smart devices and autonomous vehicles.

[0267] Input: Prompt message (Example: "Generate Wi-Fi setup instructions for an autonomous vehicle. Include the following information: network selection, password entry, connection confirmation").

[0268] Data processing: Based on the prompt text, the generative artificial intelligence generates the setup procedure.

[0269] Output: The generated initial configuration template.

[0270] Specific operation: The server sends a prompt message to the AI ​​model and delivers the received configuration template to the terminal.

[0271] Step 2:

[0272] Distribution of initial setup templates

[0273] The server distributes the generated initial setup template to the participants' devices.

[0274] Input: The generated initial configuration template.

[0275] Data processing: The generated template is sent directly to the device.

[0276] Output: The template is delivered to the terminal.

[0277] Specific operation: The server transfers the generated template via a link to the terminal, which receives it and displays it to the user.

[0278] Step 3:

[0279] Real-time generation of operating procedures

[0280] The server uses generative artificial intelligence to generate specific operation procedures in real time.

[0281] Input: Prompt sentence for operation procedures (e.g., "Please generate the setup procedures for the navigation system of an autonomous vehicle.").

[0282] Data processing: Based on the prompt sentence, the generative artificial intelligence generates operation procedures.

[0283] Output: Generated operation procedures.

[0284] Specific operation: The server sends the prompt sentence to the generative AI model and receives the generated operation procedures.

[0285] Step 4:

[0286] Distribution and display of operation procedures

[0287] The server distributes the generated operation procedures to the participants' terminals, and the terminals display them.

[0288] Input: Generated operation procedures.

[0289] [[ID=3 6]]Data processing: Send the generated operation procedures to the terminal as they are.

[0290] Output: The operation procedures are displayed on the terminal.

[0291] Specific operation: The server sends the operation procedures to the terminal, and the terminal visually displays the received procedures to the user.

[0292] Step 5:

[0293] Receiving questions and generating answers

[0294] The server receives the user's questions from the terminal and uses generative artificial intelligence to generate answers.

[0295] Input: Prompt text for the question (e.g., "Please teach me how to set up the navigation system of an autonomous vehicle.")

[0296] Data processing: Based on the prompt text, the generative artificial intelligence generates an answer.

[0297] Output: The generated answer.

[0298] Specific operation: The server sends the question to the generative AI model and receives the generated answer.

[0299] Step 6:

[0300] Distribution and display of the answer

[0301] The server distributes the generated answer to the participants' terminals, and the terminals display it.

[0302] Input: The generated answer.

[0303] Data processing: Send the generated answer to the terminal as it is.

[0304] Output: The answer is displayed on the terminal.

[0305] Specific operation: The server sends the answer to the terminal, and the terminal visually displays the received answer to the user.

[0306] Step 7:

[0307] Monitoring of operation progress and provision of additional support information

[0308] The server monitors the operation progress of the participants and provides additional support information as needed.

[0309] Input: Data on operation progress.

[0310] Data processing: Generate and provide additional support information based on the operation progress.

[0311] Output: Additional support information.

[0312] Specific operation: The server monitors the progress of operations sent from the terminal in real time, generates necessary support information, and delivers it to the terminal.

[0313] The above processing steps enable users to quickly and easily configure and operate the smart devices of autonomous vehicles.

[0314] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0315] This invention relates to a system that combines generative artificial intelligence and an emotion engine to provide initial setup and operational support for a user's smart device. This system provides operational support through the cooperation of a server, a terminal, and the user.

[0316] Server roles and processing

[0317] Generation and distribution of initial configuration templates

[0318] The server uses generative artificial intelligence to generate an initial setup template for a specific smart device. This template includes instructions for network settings, account settings, and application installation. This template is then distributed to the participants' devices.

[0319] As a concrete example, a server generates Wi-Fi setup instructions for a new smartphone and sends them to each device. This allows users to easily complete the initial setup.

[0320] Real-time generation and distribution of operating procedures

[0321] The server generates operating instructions in real time and displays them on participants' devices. For example, it generates instructions for scanning QR codes or installing apps and delivers them to the devices immediately.

[0322] Question reception and answer generation

[0323] The server receives user questions from the terminal and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals.

[0324] For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[0325] Monitoring of operation progress and provision of additional support information

[0326] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[0327] Terminal roles and processing

[0328] Receiving and displaying initial setup information

[0329] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[0330] For example, a device can receive a Wi-Fi setup guide and display it on its screen, allowing the user to configure the settings by following the guide.

[0331] Demonstration

[0332] The terminal receives operation instructions sent from the server in real time and displays them to the user in an easy-to-understand manner. Once the user completes the operation, the terminal sends progress information to the server.

[0333] Submitting questions and viewing answers

[0334] The terminal sends the question entered by the user to the server, receives the answer from the server again, and displays it to the user.

[0335] The role of the emotional engine

[0336] Recognizing the user's emotional state

[0337] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state.

[0338] For example, if a user is irritated, the emotion engine recognizes this and notifies the server.

[0339] Adjusting operating procedures based on emotions

[0340] Based on user emotion information obtained from the emotion engine, the server adjusts operating procedures and support information as needed. For example, if the user is frustrated, the server may simplify the operating procedures or add encouraging or supportive messages.

[0341] Providing an emotional response guide

[0342] The server-generated instructions and responses are customized to take the user's emotional state into consideration. This allows users to proceed more smoothly and reduce frustration.

[0343] User roles and operations

[0344] Execute initial setup

[0345] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[0346] Review and execute the demo procedure.

[0347] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[0348] Resolving doubts and questions

[0349] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks "How do I adjust the screen brightness?", the system adjusts the settings based on the answer from the server.

[0350] As described above, this invention provides a system that allows users to efficiently learn how to set up and operate smart devices and to quickly resolve any questions they may have. Furthermore, by incorporating an emotion engine, it can provide appropriate support tailored to the user's emotional state, thereby improving the user experience. In this way, users can make the most of the device's functions, leading to increased usage.

[0351] The following describes the processing flow.

[0352] Step 1:

[0353] The server generates an initial configuration template.

[0354] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices. These templates include network settings, account settings, and application installation instructions.

[0355] Step 2:

[0356] The server delivers the initial setup template to the terminal.

[0357] The server distributes the generated initial setup template to each participant's device. This allows the device to receive the setup guide.

[0358] Step 3:

[0359] The device displays initial setup information.

[0360] The device visually displays the received initial setup template to the user. The user proceeds with the initial setup while confirming the information displayed on the screen.

[0361] Step 4:

[0362] The user performs the initial setup.

[0363] The user configures the smart device by following the initial setup template displayed on the terminal. For example, they might enter Wi-Fi connection information to connect to the network.

[0364] Step 5:

[0365] The server generates operating procedures in real time.

[0366] The server uses generative artificial intelligence to generate the necessary operating procedures in real time, based on the user's progress and questions.

[0367] Step 6:

[0368] The server delivers the operating instructions to the terminal.

[0369] The server distributes the generated operating instructions to the user's terminal. This allows the terminal to receive the latest operating guide.

[0370] Step 7:

[0371] The device displays the operating instructions.

[0372] The terminal displays the received operating instructions in an easy-to-understand manner for the user. The user then performs the next operation based on this information.

[0373] Step 8:

[0374] The emotion engine recognizes the user's emotions.

[0375] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, and touch operation speed to determine their emotions.

[0376] Step 9:

[0377] Send the user's emotional state to the server.

[0378] The device sends the recognized user's emotional state to the server. This allows the server to understand the user's current emotional state.

[0379] Step 10:

[0380] The server adjusts the operating procedures based on emotional state.

[0381] The server adjusts operating procedures and support information as needed based on user emotion information obtained from the emotion engine. For example, if the user is frustrated, it may simplify the operating procedures or add encouraging or supportive messages.

[0382] Step 11:

[0383] The server delivers the coordinated operating instructions to the terminal.

[0384] The server sends emotionally-adjusted operating instructions to the device. This allows the device to receive updated operating guides.

[0385] Step 12:

[0386] The user performs the operating procedure.

[0387] The user follows the instructions displayed on the device to operate the smart device. For example, they might install an app as instructed.

[0388] Step 13:

[0389] The terminal sends the operation progress to the server.

[0390] The terminal sends user progress information to the server. This allows the server to track the user's progress.

[0391] Step 14:

[0392] User enters and submits question.

[0393] Users input questions and concerns about the operation and settings of their smart devices into their terminals and send them to the server.

[0394] Step 15:

[0395] The server receives the question and generates the answer.

[0396] The server analyzes the received question and generates an answer using generative artificial intelligence.

[0397] Step 16:

[0398] The server delivers the generated response to the terminal.

[0399] The server sends the generated response to the device. This allows the device to receive the latest response information.

[0400] Step 17:

[0401] The device displays the answer.

[0402] The device displays the received response to the user. The user reviews the response displayed on the screen and uses it as a reference for operations and settings.

[0403] Step 18:

[0404] The user completes the operation.

[0405] Based on the answers and guides received from the server, users ultimately complete the setup and operation of their smart devices.

[0406] In this way, a system is realized in which the server, terminal, user, and emotion engine work together to smoothly configure and operate smart devices.

[0407] (Example 2)

[0408] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0409] Modern smart devices are multi-functional, making their initial setup and operation complex, often leaving many users confused. Traditional systems lacked standardized initial setup and operating procedures, hindering smooth device use. Furthermore, the lack of support that considered the user's emotional state could lead to frustration and stress. A system is needed to address these challenges.

[0410] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0411] In this invention, the server includes means for generating an initial setup template for a smart device using generative artificial intelligence and delivering it to the terminal; means for generating operation procedures in real time using generative artificial intelligence and displaying them on the terminal; means for receiving questions from the terminal, generating answers using generative artificial intelligence and delivering them to the terminal; means for monitoring the operation progress of the terminal and providing additional support information as needed; means for recognizing the user's emotional state using an emotion engine and notifying the server of the recognition results; and means for adjusting operation procedures and support information based on emotional information. As a result, users can smoothly proceed through complex initial setup and operation procedures, reducing frustration and stress.

[0412] "Generative artificial intelligence" is a type of artificial intelligence that has the ability to automatically generate new information or answers based on given prompts or data.

[0413] A "smart device" is an electronic device that can connect to the internet and perform multi-functional operations. Specific examples include smartphones, tablets, and smartwatches.

[0414] An "initial setup template" is a document containing guidelines and procedures for the initial setup of a smart device. This includes network settings, account settings, and application installation instructions.

[0415] "Operating procedures" refer to a series of steps or guidelines necessary to accomplish a specific task. These include specific operating methods and setup procedures.

[0416] "Real-time" refers to a state where data and information are processed and provided immediately. Immediacy and lack of delay are essential.

[0417] "Progress information" refers to data and status related to the progress of operations and settings. This allows you to understand what stage you have progressed to.

[0418] An "emotion engine" is a technology or system that detects and analyzes the user's emotional state. This involves using techniques such as facial recognition, voice tone analysis, and touch operation speed.

[0419] "Emotional information" refers to data about the user's emotional state detected by the emotion engine. This includes, for example, the user's level of frustration or satisfaction.

[0420] This invention relates to a system that utilizes generative artificial intelligence and an emotion engine to assist with the initial setup and operation of a user's smart device. This system provides operation assistance through the cooperation of a server, a terminal, and the user.

[0421] Server roles and processing

[0422] Generation and distribution of initial configuration templates

[0423] The server uses a generation AI model to generate an initial setup template for a smart device. This template includes instructions for network settings, account settings, and application installation. A prompt message such as "Generate Wi-Fi setup instructions for a new smartphone" can be used. This template is then delivered to the device.

[0424] Real-time generation and distribution of operating procedures

[0425] The server generates operating instructions in real time and displays them on the terminal. For example, it generates instructions for scanning a QR code or installing an app and delivers them to the terminal immediately. The prompt message used is "Please generate instructions for installing a specific app."

[0426] Question reception and answer generation

[0427] The server receives user questions from the device and generates answers using a generative AI model. These answers are then sent back to the device. For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[0428] Monitoring of operation progress and provision of additional support information

[0429] The server monitors the progress of the terminal's operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties during the process.

[0430] Processing and responding to emotional information

[0431] Based on user emotion information obtained from the emotion engine, the server adjusts operating procedures and support information. For example, if the user is frustrated, the operating procedures are simplified and encouraging messages are added.

[0432] Terminal roles and processing

[0433] Receiving and displaying initial setup information

[0434] The device receives initial setup information sent from the server and displays it visually to the user. For example, it can receive a Wi-Fi setup guide and display it on the screen, allowing the user to configure the settings according to the guide.

[0435] Display of operating procedures and submission of progress

[0436] The terminal receives operation instructions sent from the server in real time and displays them to the user in an easy-to-understand manner. Once the user completes the operation, the terminal sends progress information to the server.

[0437] Submitting questions and viewing answers

[0438] The terminal sends the user's entered question to the server, receives the answer from the server, and displays it. This allows the user to resolve their question immediately.

[0439] The role of the emotional engine

[0440] Recognition of emotional states

[0441] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state. For example, if the user is irritated, the emotion engine recognizes this and notifies the server.

[0442] Providing an emotional response guide

[0443] The server-generated instructions and responses are customized to take the user's emotional state into consideration. This allows users to proceed more smoothly and reduce frustration.

[0444] User roles and operations

[0445] Execute initial setup

[0446] The user configures the device based on the initial setup template displayed on the device. For example, they might enter Wi-Fi settings and establish a connection.

[0447] Confirm and execute the operating procedure.

[0448] The user checks the operating instructions displayed on the device and performs the operation accordingly. For example, they may download and install an app by following the app installation instructions.

[0449] Resolving doubts and questions

[0450] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks how to adjust the screen brightness, they will adjust the settings based on the answer from the server.

[0451] As described above, the system of the present invention combines generative artificial intelligence and an emotion engine, enabling users to efficiently configure and operate smart devices and quickly resolve any questions they may have. Furthermore, the incorporation of an emotion engine provides appropriate support tailored to the user's emotional state, improving the user experience. In this way, users can maximize the functionality of the device, leading to increased usage.

[0452] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0453] Step 1:

[0454] The server generates an initial configuration template.

[0455] Input: A prompt given to the server (e.g., "Generate Wi-Fi setup instructions for a new smartphone")

[0456] Data Processing / Calculation: The server uses a generation AI model to generate Wi-Fi setup instructions, network settings, account settings, and application installation instructions based on the prompt text.

[0457] Output: Generated initial setup template (e.g., a file in JSON format containing Wi-Fi setup instructions, etc.)

[0458] Step 2:

[0459] The server distributes the initial configuration template it generates to the terminal.

[0460] Input: Generated initial configuration template

[0461] Data processing / calculation: The server encodes the template into the appropriate format and delivers it to the terminal.

[0462] Output: Initial setup template delivered to the device

[0463] Step 3:

[0464] The device receives and displays the initial setup template.

[0465] Input: Initial configuration template delivered from the server

[0466] Data processing / calculation: The terminal decodes the template and displays it visually to the user.

[0467] Output: Initial setup guide displayed on the user screen (e.g., Wi-Fi setup guide)

[0468] Step 4:

[0469] The user performs the initial setup.

[0470] Input: Initial setup guide displayed on the device

[0471] Data Processing / Calculation: The user enters information according to the guide and configures the smart device. For example, they enter the Wi-Fi SSID and password.

[0472] Output: Completed initial setup (e.g., connected to Wi-Fi)

[0473] Step 5:

[0474] The server generates and distributes operating instructions in real time.

[0475] Input: User action request (e.g., "Generate the app download instructions")

[0476] Data processing / calculation: The server uses a generation AI model to generate app download instructions based on the prompt text.

[0477] Output: Generated operating instructions (e.g., step-by-step app download instructions)

[0478] Step 6:

[0479] The device receives and displays the operating instructions.

[0480] Input: Operating instructions sent from the server

[0481] Data processing / calculation: The terminal decodes the operation procedure data and displays it visually to the user.

[0482] Output: Operating instructions displayed on the user screen (e.g., app download guide)

[0483] Step 7:

[0484] The user performs the operation.

[0485] Input: Instructions displayed on the device

[0486] Data processing / calculation: The user performs operations according to the procedure. For example, downloading and installing an app.

[0487] Output: Completed operation (e.g., app installed)

[0488] Step 8:

[0489] The device sends progress information to the server.

[0490] Input: Status information of completed operations

[0491] Data processing / calculation: The terminal encodes the progress information and sends it to the server.

[0492] Output: Progress information sent to the server (e.g., a notification saying "The app was successfully installed")

[0493] Step 9:

[0494] The server receives user questions and generates answers.

[0495] Input: User question (e.g., "How do I adjust the screen brightness?")

[0496] Data processing / computation: The server uses a generative AI model to generate answers to questions.

[0497] Output: Generated answer (e.g., steps to adjust screen brightness)

[0498] Step 10:

[0499] The device receives and displays the response.

[0500] Input: Response sent from the server

[0501] Data processing / calculation: The terminal decodes the response and displays it visually to the user.

[0502] Output: Answers displayed on the user screen (e.g., brightness adjustment guide)

[0503] Step 11:

[0504] The server monitors the progress of the operation and provides additional support information.

[0505] Input: User operation progress information

[0506] Data processing / calculation: The server analyzes progress information and generates and distributes additional support information as needed.

[0507] Output: Additional support information (e.g., "The next step is XX")

[0508] Step 12:

[0509] The emotion engine recognizes the user's emotional state and notifies the server.

[0510] Input: User's facial expressions, voice tone, and touch operation data

[0511] Data processing / calculation: The emotion engine analyzes this data to determine the user's emotional state.

[0512] Output: Sentiment information from the analysis results (e.g., information indicating the user is irritated)

[0513] Step 13:

[0514] The server adjusts operating procedures and support based on emotional information.

[0515] Input: Sentiment information (e.g., the user is irritated)

[0516] Data processing / calculation: The server uses emotional information to simplify operating procedures and generate encouraging messages.

[0517] Output: Adjusted operating instructions and encouraging messages (e.g., "Please contact us if you have difficulty operating the device.")

[0518] (Application Example 2)

[0519] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0520] Conventional smart device initial setup and operation support systems have the problem of not being able to provide sufficient support when users encounter difficulties with setup or operation. Furthermore, they fail to optimize the user experience because they provide uniform support without considering the user's emotional state. The present invention aims to solve these problems and provide a system that allows users to set up and operate devices efficiently and comfortably.

[0521] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0522] In this invention, the server includes means for generating initial setup templates for smart devices using generative artificial intelligence and delivering them to the participant's terminal; means for generating operation procedures in real time using generative artificial intelligence and displaying them on the participant's terminal; means for receiving questions from the participant, generating answers using generative artificial intelligence and delivering them to the participant's terminal; means for monitoring the participant's operation progress and providing additional support information as needed; and means for recognizing the participant's emotional state using an emotion engine and adjusting support information and operation procedures based on the emotional state. This makes it possible to reduce barriers to user setup and operation and provide personalized support according to the emotional state.

[0523] "Generative artificial intelligence" is a type of artificial intelligence that has the ability to automatically generate content and information in response to user requests, and generates data in real time based on specific patterns and algorithms.

[0524] An "initial setup template" is a template that comprehensively summarizes the information and procedures necessary for the initial setup of a new smart device, including network settings, account settings, and application installation procedures.

[0525] "Participants" refer to users who receive instructions on setting up and operating their smart devices through the system, and are the individuals who receive support for operating their devices.

[0526] An "emotion engine" is software or an algorithm used to recognize a user's emotional state. It analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine their emotional state.

[0527] "Operating instructions" are a guide that details the steps required for a specific task, and are a collection of instructions provided to help users operate a device efficiently.

[0528] This invention is a system that combines generative artificial intelligence and an emotion engine to provide initial setup and operational support for a user's smart device. This system provides operational support through the cooperation of a server, a terminal, and the user.

[0529] Server roles and processing

[0530] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices and delivers them to the terminals. For example, the server can generate Wi-Fi setup instructions for a new smartphone and send them to each terminal, allowing users to easily complete the initial setup. The server also generates operating instructions in real time, such as instructions for scanning QR codes or installing applications, and delivers them immediately. Furthermore, the server receives user questions from the terminals and generates answers using generative artificial intelligence. For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on that procedure and sends it to the terminal. The server monitors the progress of participants and provides additional support information as needed.

[0531] Terminal roles and processing

[0532] The device receives initial setup information sent from the server and displays it visually to the user. This allows the user to proceed with the initial setup based on this information. For example, if the device receives a Wi-Fi setup guide and displays it on the screen, the user can follow the guide to complete the setup. Furthermore, it can receive operation instructions sent from the server in real time and display them to the user in an easy-to-understand manner. Once the user completes the operation, the device sends progress information to the server. The device also sends questions entered by the user to the server, receives the answers from the server again, and displays them to the user.

[0533] The role of the emotional engine

[0534] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state. For example, if the user is irritated, the emotion engine recognizes this and notifies the server. Based on the user's emotional information obtained from the emotion engine, the server adjusts the operation procedures and support information as needed. For example, if the user is irritated, the server may simplify the operation procedures or add encouraging or supportive messages. The operation procedures and responses generated by the server are customized considering the user's emotional state. This allows the user to proceed more smoothly and reduce frustration.

[0535] Program processing and required hardware and software

[0536] To implement this system, the following hardware and software are required:

[0537] Hardware: Smartphones, tablets

[0538] Software: Python, Emotion Recognition module, Generative AI Chatbot module (AIChatBot)

[0539] The server uses generative artificial intelligence to generate initial setup templates and processes HTTP requests for delivery to terminals. The server also generates operation procedures for specific tasks in real time, accepts user questions, and generates answers using a generative AI chatbot. The emotion engine analyzes the user's emotional state using an emotion recognition module and adjusts support accordingly.

[0540] As a concrete example, the following steps are shown for a user who wants to set up Wi-Fi during the initial setup:

[0541] (Example prompt): "Please generate Wi-Fi setup instructions for my new smartphone."

[0542] Based on this, the following Wi-Fi setup instructions were generated:

[0543] Open the home screen and tap the Settings app.

[0544] Select Wi-Fi and tap Add New Network.

[0545] Enter the network name and then the password.

[0546] Tap the connect button to complete the setup.

[0547] In this way, users can easily proceed with settings and operations, and support tailored to their emotional state is provided, thus improving the user experience.

[0548] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0549] Step 1:

[0550] The server uses a generative AI model to generate initial setup templates for specific smart devices. The input includes the device type and a prompt message. For example, based on the prompt message "Generate Wi-Fi setup instructions for a new smartphone," a template is generated that includes network settings, account settings, and application installation instructions.

[0551] Step 2:

[0552] The server delivers the generated initial configuration template to the terminal. The template data is taken as input and sent to the terminal using an HTTP request. The terminal receives this request and saves the configuration template locally.

[0553] Step 3:

[0554] The user begins the initial setup. The user follows the initial setup template displayed on the device screen. As input, the user follows the steps in the template to complete Wi-Fi and account settings. As output, once the setup is complete, the device sends progress information to the server.

[0555] Step 4:

[0556] The server uses generative artificial intelligence to generate operating procedures in real time, providing step-by-step guidance for specific tasks. For example, if a user requests to know "how to scan a QR code," the server will generate the procedure based on that request. The user's request is the input, and the real-time generated operating procedure is delivered to the terminal as output.

[0557] Step 5:

[0558] The user enters a question through their device. For example, they might ask, "How do I install a specific app?" The user inputs the question, and the device sends this question to the server.

[0559] Step 6:

[0560] The server uses generative artificial intelligence to generate answers to questions. The input consists of the user's question and the generative AI model, and the output is a detailed guide. The server then delivers this guide to the terminal.

[0561] Step 7:

[0562] The server monitors the progress of participants' operations and provides additional support information as needed. Progress information from the terminal is taken as input, and additional support information and advice are generated as output and sent to the terminal.

[0563] Step 8:

[0564] The emotion engine recognizes the user's emotional state. Inputs include the user's facial expressions, voice tone, and touch operation speed, and the emotion engine determines the user's current emotional state. For example, if the user is irritated, this information is sent to the server.

[0565] Step 9:

[0566] The server adjusts operating procedures and support information based on emotional information obtained from the emotion engine. Emotional data is the input, and support information and operating procedures tailored to the user's emotional state are generated as output. For example, if the user is frustrated, the server may simplify the operating procedures or add encouraging messages.

[0567] Step 10:

[0568] The terminal displays adjusted operating procedures and support information to the user. Adjusted data is sent from the server as input, and the terminal displays this information to further improve the user experience.

[0569] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0570] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0571] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0572] [Second Embodiment]

[0573] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0574] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0575] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0576] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0577] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0578] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0579] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0580] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0581] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0582] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0583] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0584] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0585] This invention relates to a system that utilizes generative artificial intelligence to assist with the initial setup and operation of smart devices. This system consists of a server, a terminal, and a user as its main components, each working together to provide operational assistance.

[0586] Server roles and processing

[0587] Generation and distribution of initial configuration templates

[0588] The server generates an initial setup template for smart devices. This template includes instructions for network settings, account settings, and application installation. This template is then distributed to the participants' devices.

[0589] As a concrete example, the server generates Wi-Fi setup instructions for a new smartphone and sends them to the device. This allows the user to easily complete the initial setup.

[0590] Real-time generation and distribution of operating procedures

[0591] The server generates operating instructions in real time and displays them on participants' devices. For example, it generates instructions for scanning QR codes or installing apps and immediately delivers them to the devices.

[0592] Question reception and answer generation

[0593] The server receives user questions from terminals and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals.

[0594] For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[0595] Monitoring of operation progress and provision of additional support information

[0596] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[0597] Terminal roles and processing

[0598] Receiving and displaying initial setup information

[0599] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[0600] For example, a device can receive a Wi-Fi setup guide and display it on its screen, allowing the user to configure the settings by following the guide.

[0601] Demonstration

[0602] The terminal receives operation instructions sent from the server in real time and displays them clearly to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[0603] Submitting questions and viewing answers

[0604] The terminal sends the question entered by the user to the server, receives the answer from the server again, and displays it to the user.

[0605] User roles and operations

[0606] Execute initial setup

[0607] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[0608] Review and execute the demo procedure.

[0609] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[0610] Resolving doubts and questions

[0611] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks "How do I adjust the screen brightness?", the system adjusts the settings based on the answer from the server.

[0612] As described above, this invention provides a system that allows users to efficiently learn how to set up and operate smart devices and to quickly resolve any questions they may have. This enables users to make the most of the device's functions and promotes its use.

[0613] The following describes the processing flow.

[0614] Step 1:

[0615] The server generates an initial configuration template.

[0616] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices. These templates include network settings, account settings, and application installation instructions.

[0617] Step 2:

[0618] The server delivers the initial setup template to the terminal.

[0619] The server distributes the generated initial setup template to each participant's device. This allows the device to receive the setup guide.

[0620] Step 3:

[0621] The device displays initial setup information.

[0622] The device visually displays the received initial setup template to the user. The user proceeds with the initial setup while confirming the information displayed on the screen.

[0623] Step 4:

[0624] The user performs the initial setup.

[0625] The user configures the smart device by following the initial setup template displayed on the terminal. For example, they might enter Wi-Fi connection information to connect to the network.

[0626] Step 5:

[0627] The server generates operating procedures in real time.

[0628] The server uses generative artificial intelligence to generate the necessary operating procedures in real time, based on the user's progress and questions.

[0629] Step 6:

[0630] The server delivers the operating instructions to the terminal.

[0631] The server distributes the generated operating instructions to the user's terminal. This allows the terminal to receive the latest operating guide.

[0632] Step 7:

[0633] The device displays the operating instructions.

[0634] The terminal displays the received operating instructions in an easy-to-understand manner for the user. The user then performs the next operation based on this information.

[0635] Step 8:

[0636] The user performs the operating procedure.

[0637] The user follows the instructions displayed on the device to operate the smart device. For example, they might install an app as instructed.

[0638] Step 9:

[0639] The terminal sends the operation progress to the server.

[0640] The terminal sends user progress information to the server. This allows the server to track the user's progress.

[0641] Step 10:

[0642] User enters and submits question.

[0643] Users input questions and concerns about the operation and settings of their smart devices into their terminals and send them to the server.

[0644] Step 11:

[0645] The server receives the question and generates the answer.

[0646] The server analyzes the received question and generates an answer using generative artificial intelligence.

[0647] Step 12:

[0648] The server delivers the generated response to the terminal.

[0649] The server sends the generated response to the device. This allows the device to receive the latest response information.

[0650] Step 13:

[0651] The device displays the answer.

[0652] The device displays the received response to the user. The user reviews the response displayed on the screen and uses it as a reference for operations and settings.

[0653] Step 14:

[0654] The user completes the operation.

[0655] Based on the answers and guides received from the server, users ultimately complete the setup and operation of their smart devices.

[0656] In this way, a system is realized in which servers, terminals, and users work together to smoothly configure and operate smart devices.

[0657] (Example 1)

[0658] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0659] Initial setup and learning how to operate smart devices are time-consuming and difficult, especially for beginners. Furthermore, there is a lack of real-time solutions to questions and problems encountered during operation. As a result, users cannot fully utilize the device's capabilities, hindering its adoption. A system is needed to address these challenges and provide simple and efficient support for setting up and operating smart devices.

[0660] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0661] In this invention, the server includes means for generating an initial setup template for a smart device using generative artificial intelligence and delivering it to the user's terminal; means for generating operating procedures in real time using generative artificial intelligence and displaying them on the user's terminal; means for receiving questions from the user, generating answers using generative artificial intelligence and delivering them to the user's terminal; and means for monitoring the user's operation progress and providing additional support information as needed. As a result, the user can easily learn how to set up and operate the smart device and quickly resolve any questions or problems they may encounter.

[0662] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new information and content based on large amounts of data.

[0663] A "smart device" refers to an electronic device that can connect to the internet and perform various functions through user operation.

[0664] A "template" refers to a pre-created basic framework that includes a set of settings and configuration information tailored to a specific purpose.

[0665] A "server" refers to a computer system that provides various services and data to other computers and terminals via a network.

[0666] A "terminal" refers to an electronic device equipped with input devices and display devices that can be directly operated by the user.

[0667] A "question" refers to information entered by the user, either as text or voice, that expresses doubts or unclear points that need to be resolved.

[0668] "Answer" refers to the solutions or information that the server provides to the user's question using generative artificial intelligence.

[0669] "Operation progress" refers to the status of operations being performed by the user on their smart device and the degree to which each step has been completed.

[0670] "Support information" refers to additional instructions and guidelines to help users resolve problems and questions they may encounter while using the service.

[0671] A "prompt" refers to text used to instruct a generative artificial intelligence on what tasks to perform or what information to generate.

[0672] This invention is a system that utilizes generative artificial intelligence to assist with the initial setup and operation of smart devices. This system mainly consists of three elements: a server, a terminal, and a user, each playing the following roles.

[0673] Server roles and processing

[0674] Generation and distribution of initial configuration templates

[0675] The server uses generative artificial intelligence to generate initial setup templates for smart devices. These templates include network settings, account settings, and application installation instructions. Specifically, the server uses generative AI models such as GPT-3.5 running on the Google Cloud AI Platform.

[0676] example:

[0677] Prompt: "Please generate an initial setup guide for a new smartphone. The guide should include Wi-Fi connection information, Google account setup, and instructions for installing major apps."

[0678] The generated template is sent to the terminal in JSON format using an HTTP POST request.

[0679] Real-time generation and distribution of operating procedures

[0680] The server generates operation procedures in real time in response to user actions. This is also done using a generative artificial intelligence model.

[0681] example:

[0682] Prompt: "Generate instructions for scanning the QR code with your smartphone."

[0683] The generated instructions are delivered to the terminal in text format, and the terminal displays them visually to the user.

[0684] Question reception and answer generation

[0685] The server receives questions from users and generates answers using generative artificial intelligence. This process also uses GPT-3.5 running on Google Cloud AI Platform.

[0686] example:

[0687] Prompt: "When a user asks 'How to install the app,' generate an answer that provides detailed instructions."

[0688] The server sends the generated response back to the terminal and presents it to the user.

[0689] Monitoring of operation progress and provision of additional support information

[0690] The server monitors operation progress information sent from the terminal, generates additional support information as needed, and delivers it to the terminal. For example, if a user reports a connection error, the server generates and provides troubleshooting steps.

[0691] Terminal roles and processing

[0692] Receiving and displaying initial setup information

[0693] The device receives initial setup information sent from the server and displays it visually to the user. For example, the device receives a Wi-Fi setup guide from the server using an HTTP GET request and displays it on the screen, allowing the user to configure the settings.

[0694] Demonstration

[0695] The terminal receives operation instructions sent from the server in real time and displays them to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[0696] Submitting questions and viewing answers

[0697] The terminal sends the question entered by the user to the server, receives the answer from the server, and displays it to the user.

[0698] User roles and operations

[0699] Execute initial setup

[0700] The user configures the device based on the initial setup template displayed on the device. For example, they enter Wi-Fi settings and establish a connection.

[0701] Review and execute the demo procedure.

[0702] The user reviews the demo instructions displayed on the device and performs the actions accordingly. For example, they might follow the app installation instructions to download and install the app.

[0703] Resolving doubts and questions

[0704] Users input questions and inquiries into their terminals as text, and the server provides answers to resolve the issues. For example, a user might ask how to adjust the screen brightness and then adjust the settings based on the server's response.

[0705] In this way, the coordinated operation of each element allows users to efficiently learn the initial setup and operation methods of smart devices, and to quickly resolve any questions or problems they may encounter.

[0706] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0707] Step 1:

[0708] Generating an initial configuration template

[0709] The server uses generative artificial intelligence to generate initial setup templates for smart devices.

[0710] Input: The server receives the user's smart device type and required settings (e.g., Wi-Fi settings, account settings, app installation instructions) as input.

[0711] Specific operation: The server sends this input information as a prompt to a generative artificial intelligence model (e.g., GPT-3.5) to generate a template.

[0712] Output: The server outputs the generated template in JSON format.

[0713] Step 2:

[0714] Distribution of initial setup templates

[0715] The server distributes the generated initial configuration template to the terminal.

[0716] Input: The server receives the generated template data.

[0717] Specific operation: The server sends template data to the terminal using an HTTP POST request.

[0718] Output: The terminal displays the received template data on the screen.

[0719] Step 3:

[0720] Real-time generation of operating procedures

[0721] The server generates real-time operation procedures based on user actions.

[0722] Input: The server receives user requests and information (e.g., requests for QR code scanning procedures).

[0723] Specific operation: The server sends the request as a prompt to a generative artificial intelligence model, which then generates a procedure.

[0724] Output: The server outputs the generated procedure in text format.

[0725] Step 4:

[0726] Real-time streaming of operating procedures

[0727] The server distributes the generated operating instructions to the terminal.

[0728] Input: The server receives the generated procedure data.

[0729] Specific operation: The server sends the procedure data to the terminal via an HTTP POST request.

[0730] Output: The terminal displays the received procedure data to the user.

[0731] Step 5:

[0732] Question submission

[0733] The server receives questions from users.

[0734] Input: The terminal receives the question text data entered by the user.

[0735] Specific action: The terminal sends the question text to the server via an HTTP POST request.

[0736] Output: The server receives the question data.

[0737] Step 6:

[0738] Answer generation

[0739] The server uses generative artificial intelligence to generate answers to the questions it receives.

[0740] Input: The server will use the received question text data as input.

[0741] Specific operation: The server sends the question text as a prompt to a generative artificial intelligence model, which then generates an answer.

[0742] Output: The server outputs the generated response text.

[0743] Step 7:

[0744] Delivery of responses

[0745] The server delivers the generated response to the terminal.

[0746] Input: The server receives the generated response text data.

[0747] Specific operation: The server sends the response text to the terminal via an HTTP POST request.

[0748] Output: The terminal displays the received response text to the user.

[0749] Step 8:

[0750] Monitoring operation progress

[0751] The server monitors the operation progress information sent from the terminal.

[0752] Input: The terminal receives progress data as the user's actions progress.

[0753] Specific operation: The terminal sends progress information to the server in real time.

[0754] Output: The server receives progress data and updates the monitoring status.

[0755] Step 9:

[0756] Providing additional support information

[0757] The server generates necessary support information based on the progress data and delivers it to the terminal.

[0758] Input: The server receives operation progress data.

[0759] Specific operation: The server analyzes progress data and generates necessary support information using a generative artificial intelligence model.

[0760] Output: The server sends the generated support information to the terminal. The terminal displays this to the user.

[0761] (Application Example 1)

[0762] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0763] Current autonomous vehicles' smart devices require numerous settings and operations, placing a significant burden on users. Some users may find initial setup and operation difficult due to their unfamiliarity with electronic devices and operating procedures. Furthermore, limited means of quickly resolving questions or concerns during operation can detract from the user experience. An effective support system is needed to improve this situation.

[0764] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0765] In this invention, the server includes means for generating initial setup templates for smart devices using generative artificial intelligence and distributing them to participants' terminals; means for generating operating procedures in real time using generative artificial intelligence and displaying them on participants' terminals; means for receiving questions from participants, generating answers using generative artificial intelligence and distributing them to participants' terminals; means for monitoring participants' operational progress and providing additional support information as needed; means for performing initial setup and operational support for the smart system of an autonomous vehicle; and means for enabling users to efficiently learn how to set up and operate smart devices and quickly resolve any questions. This enables users to quickly and easily perform complex smart device setup and operation, thereby improving the user experience of autonomous vehicles.

[0766] "Generative artificial intelligence" is an artificial intelligence technology that outputs content generated in natural language or other formats based on given data or prompts.

[0767] "Smart devices" refer to all electronic devices that have internet connectivity and application execution capabilities, including smartphones, tablets, and smartwatches.

[0768] An "initial setup template" is a guideline that outlines a series of settings and procedures required when starting to use a smart device, including network settings, account settings, and application installation procedures.

[0769] "Real-time generation" refers to the process of instantly generating and delivering content and information in response to user requests and circumstances.

[0770] "Participants" refers to users of this system, including individuals and groups operating smart devices.

[0771] "Terminal" refers to a device operated by a user, and includes smartphones, computers, and other interactive electronic devices.

[0772] "Operation progress" refers to the progress of operations performed by the user on their smart device.

[0773] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously without driver intervention.

[0774] A "navigation system" is a system that uses map data and GPS technology to provide the vehicle's current location and the optimal route to its destination.

[0775] An "entertainment system" is a system that provides entertainment content such as music and videos inside a vehicle.

[0776] "Safety devices" refer to various systems and devices that ensure safety while driving a vehicle, and include brake assist and lane departure warning systems.

[0777] This invention relates to a system that utilizes generative artificial intelligence to perform initial setup and operational support for the smart system of an autonomous vehicle. The following describes specific embodiments of the system, with a server, terminal, and user as its main components.

[0778] Server roles and processing

[0779] The server is responsible for the following roles and processes.

[0780] 1. Generation and distribution of initial configuration templates

[0781] The server uses generative artificial intelligence to generate initial setup templates for the smart devices of autonomous vehicles. These templates include network settings, account settings, application installation procedures, and system configuration procedures related to autonomous vehicles. The generated templates are then delivered to the participants' devices.

[0782] 2. Real-time generation and distribution of operating procedures

[0783] The server uses generative artificial intelligence to generate operating instructions in real time and display them on participants' devices. For example, it generates instructions for setting up the navigation system, operating the entertainment system, and setting up safety devices, and immediately delivers them to the devices.

[0784] 3. Receiving questions and generating answers

[0785] The server receives user questions from terminals and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals. For example, in response to a question like, "How do I configure the navigation system in an autonomous vehicle?", the server generates and delivers a detailed guide.

[0786] 4. Monitoring of operation progress and provision of additional support information

[0787] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[0788] Terminal roles and processing

[0789] The terminal is responsible for the following roles and processes.

[0790] 1. Receiving and displaying initial setup information

[0791] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[0792] 2. Receiving and displaying operating instructions

[0793] The terminal receives operation instructions sent from the server in real time and displays them clearly to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[0794] 3. Submitting questions and viewing answers

[0795] The terminal sends the question entered by the user to the server, receives the answer from the server, and displays it to the user.

[0796] User roles and operations

[0797] The user will be responsible for the following roles and operations.

[0798] 1. Perform initial setup

[0799] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[0800] 2. Confirm and execute the demo procedure.

[0801] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[0802] 3. Resolving doubts and questions

[0803] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks, "How do I configure the navigation system of an autonomous vehicle?", the system will configure it based on the answer from the server.

[0804] Examples of specific cases and prompt statements

[0805] The following are examples of specific prompt messages.

[0806] "Please generate Wi-Fi setup instructions for autonomous vehicles. Include the following information: network selection, password entry, and connection confirmation."

[0807] "Please explain the procedure for installing an app into the entertainment system of an autonomous vehicle."

[0808] "Please generate the configuration procedure for the Advanced Driver-Assistance Systems (ADAS) of autonomous vehicles."

[0809] This system allows users to quickly and easily configure and operate the complex smart devices in autonomous vehicles, improving the overall user experience.

[0810] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0811] Step 1:

[0812] Generating an initial configuration template

[0813] The server uses generative artificial intelligence to generate initial setup templates for smart devices and autonomous vehicles.

[0814] Input: Prompt message (Example: "Generate Wi-Fi setup instructions for an autonomous vehicle. Include the following information: network selection, password entry, connection confirmation").

[0815] Data processing: Based on the prompt text, the generative artificial intelligence generates the setup procedure.

[0816] Output: The generated initial configuration template.

[0817] Specific operation: The server sends a prompt message to the AI ​​model and delivers the received configuration template to the terminal.

[0818] Step 2:

[0819] Distribution of initial setup templates

[0820] The server distributes the generated initial setup template to the participants' devices.

[0821] Input: The generated initial configuration template.

[0822] Data processing: The generated template is sent directly to the device.

[0823] Output: The template is delivered to the terminal.

[0824] Specific operation: The server transfers the generated template via a link to the terminal, which receives it and displays it to the user.

[0825] Step 3:

[0826] Real-time generation of operating procedures

[0827] The server uses generative artificial intelligence to generate specific operating procedures in real time.

[0828] Input: Prompt text for the operation procedure (e.g., "Generate the setup procedure for the navigation system of an autonomous vehicle.").

[0829] Data processing: Based on the prompt text, the generative artificial intelligence generates the operation procedure.

[0830] Output: Generated operating procedure.

[0831] Specific operation: The server sends a prompt message to the AI ​​model that generates the operation procedure and receives the generated procedure.

[0832] Step 4:

[0833] Distribution and display of operating procedures

[0834] The server distributes the generated operating instructions to the participants' devices, and the devices display them.

[0835] Input: Generated operating procedure.

[0836] Data processing: The generated operating procedure is sent directly to the terminal.

[0837] Output: The operating instructions will be displayed on the terminal.

[0838] Specific operation: The server sends operating instructions to the terminal, and the terminal visually displays the received instructions to the user.

[0839] Step 5:

[0840] Question reception and answer generation

[0841] The server receives user questions from the terminal and generates answers using generative artificial intelligence.

[0842] Input: A prompt for your question (e.g., "How do I configure the navigation system in an autonomous vehicle?").

[0843] Data processing: Based on the prompt text, the generative artificial intelligence generates an answer.

[0844] Output: The generated answer.

[0845] Specific operation: The server sends a question to the AI ​​model that generates the question and receives the generated answer.

[0846] Step 6:

[0847] Delivery and display of responses

[0848] The server delivers the generated responses to the participants' devices, and the devices display them.

[0849] Input: Generated response.

[0850] Data processing: The generated response is sent directly to the device.

[0851] Output: The answer will be displayed on the terminal.

[0852] Specific operation: The server sends the response to the terminal, and the terminal visually displays the received response to the user.

[0853] Step 7:

[0854] Monitoring of operation progress and provision of additional support information

[0855] The server monitors the participants' progress and provides additional support information as needed.

[0856] Input: Operation progress data.

[0857] Data processing: Generate and provide additional support information based on the operation progress.

[0858] Output: Additional support information.

[0859] Specific operation: The server monitors the progress of operations sent from the terminal in real time, generates necessary support information, and delivers it to the terminal.

[0860] The above processing steps enable users to quickly and easily configure and operate the smart devices of autonomous vehicles.

[0861] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0862] This invention relates to a system that combines generative artificial intelligence and an emotion engine to provide initial setup and operational support for a user's smart device. This system provides operational support through the cooperation of a server, a terminal, and the user.

[0863] Server roles and processing

[0864] Generation and distribution of initial configuration templates

[0865] The server uses generative artificial intelligence to generate an initial setup template for a specific smart device. This template includes instructions for network settings, account settings, and application installation. This template is then distributed to the participants' devices.

[0866] As a concrete example, a server generates Wi-Fi setup instructions for a new smartphone and sends them to each device. This allows users to easily complete the initial setup.

[0867] Real-time generation and distribution of operating procedures

[0868] The server generates operating instructions in real time and displays them on participants' devices. For example, it generates instructions for scanning QR codes or installing apps and delivers them to the devices immediately.

[0869] Question reception and answer generation

[0870] The server receives user questions from the terminal and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals.

[0871] For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[0872] Monitoring of operation progress and provision of additional support information

[0873] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[0874] Terminal roles and processing

[0875] Receiving and displaying initial setup information

[0876] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[0877] For example, a device can receive a Wi-Fi setup guide and display it on its screen, allowing the user to configure the settings by following the guide.

[0878] Demonstration

[0879] The terminal receives operation instructions sent from the server in real time and displays them to the user in an easy-to-understand manner. Once the user completes the operation, the terminal sends progress information to the server.

[0880] Submitting questions and viewing answers

[0881] The terminal sends the question entered by the user to the server, receives the answer from the server again, and displays it to the user.

[0882] The role of the emotional engine

[0883] Recognizing the user's emotional state

[0884] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state.

[0885] For example, if a user is irritated, the emotion engine recognizes this and notifies the server.

[0886] Adjusting operating procedures based on emotions

[0887] Based on user emotion information obtained from the emotion engine, the server adjusts operating procedures and support information as needed. For example, if a user is frustrated, the server may simplify the operating procedures or add encouraging or supportive messages.

[0888] Providing an emotional response guide

[0889] The server-generated instructions and responses are customized to take the user's emotional state into consideration. This allows users to proceed more smoothly and reduce frustration.

[0890] User roles and operations

[0891] Execute initial setup

[0892] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[0893] Review and execute the demo procedure.

[0894] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[0895] Resolving doubts and questions

[0896] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks "How do I adjust the screen brightness?", the system adjusts the settings based on the answer from the server.

[0897] As described above, this invention provides a system that allows users to efficiently learn how to set up and operate smart devices and to quickly resolve any questions they may have. Furthermore, by incorporating an emotion engine, it can provide appropriate support tailored to the user's emotional state, thereby improving the user experience. In this way, users can make the most of the device's functions, leading to increased usage.

[0898] The following describes the processing flow.

[0899] Step 1:

[0900] The server generates an initial configuration template.

[0901] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices. These templates include network settings, account settings, and application installation instructions.

[0902] Step 2:

[0903] The server delivers the initial setup template to the terminal.

[0904] The server distributes the generated initial setup template to each participant's device. This allows the device to receive the setup guide.

[0905] Step 3:

[0906] The device displays initial setup information.

[0907] The device visually displays the received initial setup template to the user. The user proceeds with the initial setup while confirming the information displayed on the screen.

[0908] Step 4:

[0909] The user performs the initial setup.

[0910] The user configures the smart device by following the initial setup template displayed on the terminal. For example, they might enter Wi-Fi connection information to connect to the network.

[0911] Step 5:

[0912] The server generates operating procedures in real time.

[0913] The server uses generative artificial intelligence to generate the necessary operating procedures in real time, based on the user's progress and questions.

[0914] Step 6:

[0915] The server delivers the operating instructions to the terminal.

[0916] The server distributes the generated operating instructions to the user's terminal. This allows the terminal to receive the latest operating guide.

[0917] Step 7:

[0918] The device displays the operating instructions.

[0919] The terminal displays the received operating instructions in an easy-to-understand manner for the user. The user then performs the next operation based on this information.

[0920] Step 8:

[0921] The emotion engine recognizes the user's emotions.

[0922] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, and touch operation speed to determine their emotions.

[0923] Step 9:

[0924] Send the user's emotional state to the server.

[0925] The device sends the recognized user's emotional state to the server. This allows the server to understand the user's current emotional state.

[0926] Step 10:

[0927] The server adjusts the operating procedures based on emotional state.

[0928] Based on user emotion information obtained from the emotion engine, the server adjusts operating procedures and support information as needed. For example, if the user is frustrated, it may simplify the operating procedures or add encouraging or supportive messages.

[0929] Step 11:

[0930] The server delivers the coordinated operating instructions to the terminal.

[0931] The server sends emotionally-adjusted operating instructions to the device. This allows the device to receive updated operating guides.

[0932] Step 12:

[0933] The user performs the operating procedure.

[0934] The user follows the instructions displayed on the device to operate the smart device. For example, they might install an app as instructed.

[0935] Step 13:

[0936] The terminal sends the operation progress to the server.

[0937] The terminal sends user progress information to the server. This allows the server to track the user's progress.

[0938] Step 14:

[0939] User enters and submits question.

[0940] Users input questions and concerns about the operation and settings of their smart devices into their terminals and send them to the server.

[0941] Step 15:

[0942] The server receives the question and generates the answer.

[0943] The server analyzes the received question and generates an answer using generative artificial intelligence.

[0944] Step 16:

[0945] The server delivers the generated response to the terminal.

[0946] The server sends the generated response to the device. This allows the device to receive the latest response information.

[0947] Step 17:

[0948] The device displays the answer.

[0949] The device displays the received response to the user. The user reviews the response displayed on the screen and uses it as a reference for operations and settings.

[0950] Step 18:

[0951] The user completes the operation.

[0952] Based on the answers and guides received from the server, users ultimately complete the setup and operation of their smart devices.

[0953] In this way, a system is realized in which the server, terminal, user, and emotion engine work together to smoothly configure and operate smart devices.

[0954] (Example 2)

[0955] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0956] Modern smart devices are multi-functional, making their initial setup and operation complex, often leaving many users confused. Traditional systems lacked standardized initial setup and operating procedures, hindering smooth device use. Furthermore, the lack of support that considered the user's emotional state could lead to frustration and stress. A system is needed to address these challenges.

[0957] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0958] In this invention, the server includes means for generating an initial setup template for a smart device using generative artificial intelligence and delivering it to the terminal; means for generating operation procedures in real time using generative artificial intelligence and displaying them on the terminal; means for receiving questions from the terminal, generating answers using generative artificial intelligence and delivering them to the terminal; means for monitoring the operation progress of the terminal and providing additional support information as needed; means for recognizing the user's emotional state using an emotion engine and notifying the server of the recognition results; and means for adjusting operation procedures and support information based on emotional information. As a result, users can smoothly proceed through complex initial setup and operation procedures, reducing frustration and stress.

[0959] "Generative artificial intelligence" is a type of artificial intelligence that has the ability to automatically generate new information or answers based on given prompts or data.

[0960] A "smart device" is an electronic device that can connect to the internet and perform multiple functions. Specific examples include smartphones, tablets, and smartwatches.

[0961] An "initial setup template" is a document containing guidelines and procedures for the initial setup of a smart device. This includes network settings, account settings, and application installation instructions.

[0962] "Operating procedures" refer to a series of steps or guidelines necessary to accomplish a specific task. These include specific operating methods and setup procedures.

[0963] "Real-time" refers to a state where data and information are processed and provided immediately. Immediacy and lack of delay are essential.

[0964] "Progress information" refers to data and status related to the progress of operations and settings. This allows you to understand what stage you have progressed to.

[0965] An "emotion engine" is a technology or system that detects and analyzes the emotional state of a user. This involves using techniques such as facial recognition, voice tone analysis, and touch operation speed.

[0966] "Emotional information" refers to data about the user's emotional state detected by the emotion engine. This includes, for example, the user's level of frustration or satisfaction.

[0967] This invention relates to a system that utilizes generative artificial intelligence and an emotion engine to assist with the initial setup and operation of a user's smart device. This system provides operation assistance through the cooperation of a server, a terminal, and the user.

[0968] Server roles and processing

[0969] Generation and distribution of initial configuration templates

[0970] The server uses a generation AI model to generate an initial setup template for a smart device. This template includes instructions for network settings, account settings, and application installation. A prompt message such as "Generate Wi-Fi setup instructions for a new smartphone" can be used. This template is then delivered to the device.

[0971] Real-time generation and distribution of operating procedures

[0972] The server generates operating instructions in real time and displays them on the terminal. For example, it generates instructions for scanning a QR code or installing an app and delivers them to the terminal immediately. The prompt message used is "Please generate instructions for installing a specific app."

[0973] Question reception and answer generation

[0974] The server receives user questions from the device and generates answers using a generative AI model. These answers are then sent back to the device. For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[0975] Monitoring of operation progress and provision of additional support information

[0976] The server monitors the progress of the terminal's operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties during the process.

[0977] Processing and responding to emotional information

[0978] Based on user emotion information obtained from the emotion engine, the server adjusts operating procedures and support information. For example, if the user is frustrated, the operating procedures are simplified and encouraging messages are added.

[0979] Terminal roles and processing

[0980] Receiving and displaying initial setup information

[0981] The device receives initial setup information sent from the server and displays it visually to the user. For example, it can receive a Wi-Fi setup guide and display it on the screen, allowing the user to configure the settings according to the guide.

[0982] Display of operating procedures and submission of progress

[0983] The terminal receives operation instructions sent from the server in real time and displays them to the user in an easy-to-understand manner. Once the user completes the operation, the terminal sends progress information to the server.

[0984] Submitting questions and viewing answers

[0985] The terminal sends the user's entered question to the server, receives the answer from the server, and displays it. This allows the user to resolve their question immediately.

[0986] The role of the emotional engine

[0987] Recognition of emotional states

[0988] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state. For example, if the user is irritated, the emotion engine recognizes this and notifies the server.

[0989] Providing an emotional response guide

[0990] The server-generated instructions and responses are customized to take the user's emotional state into consideration. This allows users to proceed more smoothly and reduce frustration.

[0991] User roles and operations

[0992] Execute initial setup

[0993] The user configures the device based on the initial setup template displayed on the device. For example, they might enter Wi-Fi settings and establish a connection.

[0994] Confirm and execute the operating procedure.

[0995] The user checks the operating instructions displayed on the device and performs the actions accordingly. For example, they might follow the app installation instructions to download and install the app.

[0996] Resolving doubts and questions

[0997] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks how to adjust the screen brightness, they will adjust the settings based on the answer from the server.

[0998] As described above, the system of the present invention combines generative artificial intelligence and an emotion engine, enabling users to efficiently configure and operate smart devices and quickly resolve any questions they may have. Furthermore, by incorporating an emotion engine, appropriate support is provided according to the user's emotional state, improving the user experience. In this way, users can make the most of the device's functions, leading to increased usage.

[0999] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1000] Step 1:

[1001] The server generates an initial configuration template.

[1002] Input: A prompt given to the server (e.g., "Generate Wi-Fi setup instructions for a new smartphone")

[1003] Data Processing / Calculation: The server uses a generation AI model to generate Wi-Fi setup instructions, network settings, account settings, and application installation instructions based on the prompt text.

[1004] Output: Generated initial setup template (e.g., a file in JSON format containing Wi-Fi setup instructions, etc.)

[1005] Step 2:

[1006] The server distributes the initial configuration template it generates to the terminal.

[1007] Input: Generated initial configuration template

[1008] Data processing / calculation: The server encodes the template into the appropriate format and delivers it to the terminal.

[1009] Output: Initial setup template delivered to the device

[1010] Step 3:

[1011] The device receives and displays the initial setup template.

[1012] Input: Initial configuration template delivered from the server

[1013] Data processing / calculation: The terminal decodes the template and displays it visually to the user.

[1014] Output: Initial setup guide displayed on the user screen (e.g., Wi-Fi setup guide)

[1015] Step 4:

[1016] The user performs the initial setup.

[1017] Input: Initial setup guide displayed on the device

[1018] Data Processing / Calculation: The user enters information according to the guide and configures the smart device. For example, they enter the Wi-Fi SSID and password.

[1019] Output: Completed initial setup (e.g., connected to Wi-Fi)

[1020] Step 5:

[1021] The server generates and distributes operating instructions in real time.

[1022] Input: User action request (e.g., "Generate the app download instructions")

[1023] Data processing / calculation: The server uses a generation AI model to generate app download instructions based on the prompt text.

[1024] Output: Generated operating instructions (e.g., step-by-step app download instructions)

[1025] Step 6:

[1026] The device receives and displays the operating instructions.

[1027] Input: Operating instructions sent from the server

[1028] Data processing / calculation: The terminal decodes the operation procedure data and displays it visually to the user.

[1029] Output: Operating instructions displayed on the user screen (e.g., app download guide)

[1030] Step 7:

[1031] The user performs the operation.

[1032] Input: Instructions displayed on the device

[1033] Data processing / calculation: The user performs operations according to the procedure. For example, downloading and installing an app.

[1034] Output: Completed operation (e.g., app installed)

[1035] Step 8:

[1036] The device sends progress information to the server.

[1037] Input: Status information of completed operations

[1038] Data processing / calculation: The terminal encodes the progress information and sends it to the server.

[1039] Output: Progress information sent to the server (e.g., a notification saying "The app was successfully installed")

[1040] Step 9:

[1041] The server receives user questions and generates answers.

[1042] Input: User question (e.g., "How do I adjust the screen brightness?")

[1043] Data processing / computation: The server uses a generative AI model to generate answers to questions.

[1044] Output: Generated answer (e.g., steps to adjust screen brightness)

[1045] Step 10:

[1046] The device receives and displays the response.

[1047] Input: Response sent from the server

[1048] Data processing / calculation: The terminal decodes the response and displays it visually to the user.

[1049] Output: Answers displayed on the user screen (e.g., brightness adjustment guide)

[1050] Step 11:

[1051] The server monitors the progress of the operation and provides additional support information.

[1052] Input: User operation progress information

[1053] Data processing / calculation: The server analyzes progress information and generates and distributes additional support information as needed.

[1054] Output: Additional support information (e.g., "The next step is XX")

[1055] Step 12:

[1056] The emotion engine recognizes the user's emotional state and notifies the server.

[1057] Input: User's facial expressions, voice tone, and touch operation data

[1058] Data processing / calculation: The emotion engine analyzes this data to determine the user's emotional state.

[1059] Output: Sentiment information from the analysis results (e.g., information indicating the user is irritated)

[1060] Step 13:

[1061] The server adjusts operating procedures and support based on emotional information.

[1062] Input: Sentiment information (e.g., the user is irritated)

[1063] Data processing / calculation: The server uses emotional information to simplify operating procedures and generate encouraging messages.

[1064] Output: Adjusted operating instructions and encouraging messages (e.g., "Please contact us if you have difficulty operating the device.")

[1065] (Application Example 2)

[1066] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1067] Conventional smart device initial setup and operation support systems have the problem of not being able to provide sufficient support when users encounter difficulties with setup or operation. Furthermore, they fail to optimize the user experience because they provide uniform support without considering the user's emotional state. The present invention aims to solve these problems and provide a system that allows users to set up and operate devices efficiently and comfortably.

[1068] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1069] In this invention, the server includes means for generating initial setup templates for smart devices using generative artificial intelligence and delivering them to the participant's terminal; means for generating operation procedures in real time using generative artificial intelligence and displaying them on the participant's terminal; means for receiving questions from the participant, generating answers using generative artificial intelligence and delivering them to the participant's terminal; means for monitoring the participant's operation progress and providing additional support information as needed; and means for recognizing the participant's emotional state using an emotion engine and adjusting support information and operation procedures based on the emotional state. This makes it possible to reduce barriers to user setup and operation and provide personalized support according to the emotional state.

[1070] "Generative artificial intelligence" is a type of artificial intelligence that has the ability to automatically generate content and information in response to user requests, and generates data in real time based on specific patterns and algorithms.

[1071] An "initial setup template" is a template that comprehensively summarizes the information and procedures necessary for the initial setup of a new smart device, and includes network settings, account settings, application installation procedures, and more.

[1072] "Participants" refer to users who receive instructions on setting up and operating their smart devices through the system, and are the individuals who receive support for operating their devices.

[1073] An "emotion engine" is software or an algorithm used to recognize a user's emotional state. It analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine their emotional state.

[1074] "Operating instructions" are a guide that details the steps required for a specific task, and are a collection of instructions provided to help users operate a device efficiently.

[1075] This invention is a system that combines generative artificial intelligence and an emotion engine to provide initial setup and operational support for a user's smart device. This system provides operational support through the cooperation of a server, a terminal, and the user.

[1076] Server roles and processing

[1077] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices and delivers them to the terminals. For example, the server can generate Wi-Fi setup instructions for a new smartphone and send them to each terminal, allowing users to easily complete the initial setup. The server also generates operating instructions in real time, such as instructions for scanning QR codes or installing applications, and delivers them immediately. Furthermore, the server receives user questions from the terminals and generates answers using generative artificial intelligence. For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on that procedure and sends it to the terminal. The server monitors the progress of participants and provides additional support information as needed.

[1078] Terminal roles and processing

[1079] The device receives initial setup information sent from the server and displays it visually to the user. This allows the user to proceed with the initial setup based on this information. For example, if the device receives a Wi-Fi setup guide and displays it on the screen, the user can follow the guide to complete the setup. Furthermore, it can receive operation instructions sent from the server in real time and display them to the user in an easy-to-understand manner. Once the user completes the operation, the device sends progress information to the server. The device also sends questions entered by the user to the server, receives the answers from the server again, and displays them to the user.

[1080] The role of the emotional engine

[1081] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state. For example, if the user is irritated, the emotion engine recognizes this and notifies the server. Based on the user's emotional information obtained from the emotion engine, the server adjusts the operation procedures and support information as needed. For example, if the user is irritated, the server may simplify the operation procedures or add encouraging or supportive messages. The operation procedures and responses generated by the server are customized considering the user's emotional state. This allows the user to proceed more smoothly and reduce frustration.

[1082] Program processing and required hardware and software

[1083] To implement this system, the following hardware and software are required:

[1084] Hardware: Smartphones, tablets

[1085] Software: Python, Emotion Recognition module, Generative AI Chatbot module (AIChatBot)

[1086] The server uses generative artificial intelligence to generate initial setup templates and processes HTTP requests for delivery to terminals. The server also generates operation procedures for specific tasks in real time, accepts user questions, and generates answers using a generative AI chatbot. The emotion engine analyzes the user's emotional state using an emotion recognition module and adjusts support accordingly.

[1087] As a concrete example, the following steps are shown for a user who wants to set up Wi-Fi during the initial setup:

[1088] (Example prompt): "Please generate Wi-Fi setup instructions for my new smartphone."

[1089] Based on this, the following Wi-Fi setup instructions were generated:

[1090] Open the home screen and tap the Settings app.

[1091] Select Wi-Fi and tap Add New Network.

[1092] Enter the network name and then the password.

[1093] Tap the connect button to complete the setup.

[1094] In this way, users can easily proceed with settings and operations, and support tailored to their emotional state is provided, thus improving the user experience.

[1095] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1096] Step 1:

[1097] The server uses a generative AI model to generate initial setup templates for specific smart devices. The input includes the device type and a prompt message. For example, based on the prompt message "Generate Wi-Fi setup instructions for a new smartphone," a template is generated that includes network settings, account settings, and application installation instructions.

[1098] Step 2:

[1099] The server delivers the generated initial configuration template to the terminal. The template data is taken as input and sent to the terminal using an HTTP request. The terminal receives this request and saves the configuration template locally.

[1100] Step 3:

[1101] The user begins the initial setup. The user follows the initial setup template displayed on the device screen. As input, the user follows the steps in the template to complete Wi-Fi and account settings. As output, once the setup is complete, the device sends progress information to the server.

[1102] Step 4:

[1103] The server uses generative artificial intelligence to generate operating procedures in real time, providing step-by-step guidance for specific tasks. For example, if a user requests to know "how to scan a QR code," the server will generate the procedure based on that request. The user's request is the input, and the real-time generated operating procedure is delivered to the terminal as output.

[1104] Step 5:

[1105] The user enters a question through their device. For example, they might ask, "How do I install a specific app?" The user inputs the question, and the device sends this question to the server.

[1106] Step 6:

[1107] The server uses generative artificial intelligence to generate answers to questions. The input consists of the user's question and the generative AI model, and the output is a detailed guide. The server then delivers this guide to the terminal.

[1108] Step 7:

[1109] The server monitors the participants' progress and provides additional support information as needed. Progress information from the terminal is taken as input, and additional support information and advice are generated as output and sent to the terminal.

[1110] Step 8:

[1111] The emotion engine recognizes the user's emotional state. Inputs include the user's facial expressions, voice tone, and touch operation speed, and the emotion engine determines the current emotional state. For example, if the user is irritated, this information is sent to the server.

[1112] Step 9:

[1113] The server adjusts operating procedures and support information based on emotional information obtained from the emotion engine. Emotional data is the input, and support information and operating procedures tailored to the user's emotional state are generated as output. For example, if the user is frustrated, the server may simplify the operating procedures or add encouraging messages.

[1114] Step 10:

[1115] The terminal displays adjusted operating procedures and support information to the user. Adjusted data is sent from the server as input, and the terminal displays this information to further improve the user experience.

[1116] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1117] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1118] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1119] [Third Embodiment]

[1120] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1121] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1122] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1123] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1124] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1125] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1126] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1127] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1128] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1129] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1130] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1131] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1132] This invention relates to a system that utilizes generative artificial intelligence to assist with the initial setup and operation of smart devices. This system consists of a server, a terminal, and a user as its main components, each working together to provide operational assistance.

[1133] Server roles and processing

[1134] Generation and distribution of initial configuration templates

[1135] The server generates an initial setup template for smart devices. This template includes instructions for network settings, account settings, and application installation. This template is then distributed to the participants' devices.

[1136] As a concrete example, the server generates Wi-Fi setup instructions for a new smartphone and sends them to the device. This allows the user to easily complete the initial setup.

[1137] Real-time generation and distribution of operating procedures

[1138] The server generates operating instructions in real time and displays them on participants' devices. For example, it generates instructions for scanning QR codes or installing apps and immediately delivers them to the devices.

[1139] Question reception and answer generation

[1140] The server receives user questions from terminals and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals.

[1141] For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[1142] Monitoring of operation progress and provision of additional support information

[1143] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[1144] Terminal roles and processing

[1145] Receiving and displaying initial setup information

[1146] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[1147] For example, a device can receive a Wi-Fi setup guide and display it on its screen, allowing the user to configure the settings by following the guide.

[1148] Demonstration

[1149] The terminal receives operation instructions sent from the server in real time and displays them clearly to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[1150] Submitting questions and viewing answers

[1151] The terminal sends the question entered by the user to the server, receives the answer from the server again, and displays it to the user.

[1152] User roles and operations

[1153] Execute initial setup

[1154] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[1155] Review and execute the demo procedure.

[1156] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[1157] Resolving doubts and questions

[1158] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks "How do I adjust the screen brightness?", the system adjusts the settings based on the answer from the server.

[1159] As described above, this invention provides a system that allows users to efficiently learn how to set up and operate smart devices and to quickly resolve any questions they may have. This enables users to make the most of the device's functions and promotes its use.

[1160] The following describes the processing flow.

[1161] Step 1:

[1162] The server generates an initial configuration template.

[1163] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices. These templates include network settings, account settings, and application installation instructions.

[1164] Step 2:

[1165] The server delivers the initial setup template to the terminal.

[1166] The server distributes the generated initial setup template to each participant's device. This allows the device to receive the setup guide.

[1167] Step 3:

[1168] The device displays initial setup information.

[1169] The device visually displays the received initial setup template to the user. The user proceeds with the initial setup while confirming the information displayed on the screen.

[1170] Step 4:

[1171] The user performs the initial setup.

[1172] The user configures the smart device by following the initial setup template displayed on the terminal. For example, they might enter Wi-Fi connection information to connect to the network.

[1173] Step 5:

[1174] The server generates operating procedures in real time.

[1175] The server uses generative artificial intelligence to generate the necessary operating procedures in real time, based on the user's progress and questions.

[1176] Step 6:

[1177] The server delivers the operating instructions to the terminal.

[1178] The server distributes the generated operating instructions to the user's terminal. This allows the terminal to receive the latest operating guide.

[1179] Step 7:

[1180] The device displays the operating instructions.

[1181] The terminal displays the received operating instructions in an easy-to-understand manner for the user. The user then performs the next operation based on this information.

[1182] Step 8:

[1183] The user performs the operating procedure.

[1184] The user follows the instructions displayed on the device to operate the smart device. For example, they might install an app as instructed.

[1185] Step 9:

[1186] The terminal sends the operation progress to the server.

[1187] The terminal sends user progress information to the server. This allows the server to track the user's progress.

[1188] Step 10:

[1189] User enters and submits question.

[1190] Users input questions and concerns about the operation and settings of their smart devices into their terminals and send them to the server.

[1191] Step 11:

[1192] The server receives the question and generates the answer.

[1193] The server analyzes the received question and generates an answer using generative artificial intelligence.

[1194] Step 12:

[1195] The server delivers the generated response to the terminal.

[1196] The server sends the generated response to the device. This allows the device to receive the latest response information.

[1197] Step 13:

[1198] The device displays the answer.

[1199] The device displays the received response to the user. The user reviews the response displayed on the screen and uses it as a reference for operations and settings.

[1200] Step 14:

[1201] The user completes the operation.

[1202] Based on the answers and guides received from the server, users ultimately complete the setup and operation of their smart devices.

[1203] In this way, a system is realized in which servers, terminals, and users work together to smoothly configure and operate smart devices.

[1204] (Example 1)

[1205] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1206] Initial setup and learning how to operate smart devices are time-consuming and difficult, especially for beginners. Furthermore, there is a lack of real-time solutions to questions and problems encountered during operation. As a result, users cannot fully utilize the device's capabilities, hindering its adoption. A system is needed to address these challenges and provide simple and efficient support for setting up and operating smart devices.

[1207] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1208] In this invention, the server includes means for generating an initial setup template for a smart device using generative artificial intelligence and delivering it to the user's terminal; means for generating operating procedures in real time using generative artificial intelligence and displaying them on the user's terminal; means for receiving questions from the user, generating answers using generative artificial intelligence and delivering them to the user's terminal; and means for monitoring the user's operation progress and providing additional support information as needed. As a result, the user can easily learn how to set up and operate the smart device and quickly resolve any questions or problems they may encounter.

[1209] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new information and content based on large amounts of data.

[1210] A "smart device" refers to an electronic device that can connect to the internet and perform various functions through user operation.

[1211] A "template" refers to a pre-created basic framework that includes a set of settings and configuration information tailored to a specific purpose.

[1212] A "server" refers to a computer system that provides various services and data to other computers and terminals via a network.

[1213] A "terminal" refers to an electronic device equipped with input devices and display devices that can be directly operated by the user.

[1214] A "question" refers to information entered by the user, either as text or voice, that expresses doubts or unclear points that need to be resolved.

[1215] "Answer" refers to the solutions or information that the server provides to the user's question using generative artificial intelligence.

[1216] "Operation progress" refers to the status of operations being performed by the user on their smart device and the degree to which each step has been completed.

[1217] "Support information" refers to additional instructions and guidelines to help users resolve problems and questions they may encounter while using the service.

[1218] A "prompt" refers to text used to instruct a generative artificial intelligence on what tasks to perform or what information to generate.

[1219] This invention is a system that utilizes generative artificial intelligence to assist with the initial setup and operation of smart devices. This system mainly consists of three elements: a server, a terminal, and a user, each playing the following roles.

[1220] Server roles and processing

[1221] Generation and distribution of initial configuration templates

[1222] The server uses generative artificial intelligence to generate initial setup templates for smart devices. These templates include network settings, account settings, and application installation instructions. Specifically, the server uses generative AI models such as GPT-3.5 running on the Google Cloud AI Platform.

[1223] example:

[1224] Prompt: "Please generate an initial setup guide for a new smartphone. The guide should include Wi-Fi connection information, Google account setup, and instructions for installing major apps."

[1225] The generated template is sent to the terminal in JSON format using an HTTP POST request.

[1226] Real-time generation and distribution of operating procedures

[1227] The server generates operation procedures in real time in response to user actions. This is also done using a generative artificial intelligence model.

[1228] example:

[1229] Prompt: "Generate instructions for scanning the QR code with your smartphone."

[1230] The generated instructions are delivered to the terminal in text format, and the terminal displays them visually to the user.

[1231] Question reception and answer generation

[1232] The server receives questions from users and generates answers using generative artificial intelligence. This process also uses GPT-3.5 running on Google Cloud AI Platform.

[1233] example:

[1234] Prompt: "When a user asks 'How to install the app,' generate an answer that provides detailed instructions."

[1235] The server sends the generated response back to the terminal and presents it to the user.

[1236] Monitoring of operation progress and provision of additional support information

[1237] The server monitors operation progress information sent from the terminal, generates additional support information as needed, and delivers it to the terminal. For example, if a user reports a connection error, the server generates and provides troubleshooting steps.

[1238] Terminal roles and processing

[1239] Receiving and displaying initial setup information

[1240] The device receives initial setup information sent from the server and displays it visually to the user. For example, the device receives a Wi-Fi setup guide from the server using an HTTP GET request and displays it on the screen, allowing the user to configure the settings.

[1241] Demonstration

[1242] The terminal receives operation instructions sent from the server in real time and displays them to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[1243] Submitting questions and viewing answers

[1244] The terminal sends the question entered by the user to the server, receives the answer from the server, and displays it to the user.

[1245] User roles and operations

[1246] Execute initial setup

[1247] The user configures the device based on the initial setup template displayed on the device. For example, they enter Wi-Fi settings and establish a connection.

[1248] Review and execute the demo procedure.

[1249] The user reviews the demo instructions displayed on the device and performs the actions accordingly. For example, they might follow the app installation instructions to download and install the app.

[1250] Resolving doubts and questions

[1251] Users input questions and inquiries into their terminals as text, and the server provides answers to resolve the issues. For example, a user might ask how to adjust the screen brightness and then adjust the settings based on the server's response.

[1252] In this way, the coordinated operation of each element allows users to efficiently learn the initial setup and operation methods of smart devices, and to quickly resolve any questions or problems they may encounter.

[1253] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1254] Step 1:

[1255] Generating an initial configuration template

[1256] The server uses generative artificial intelligence to generate initial setup templates for smart devices.

[1257] Input: The server receives the user's smart device type and required settings (e.g., Wi-Fi settings, account settings, app installation instructions) as input.

[1258] Specific operation: The server sends this input information as a prompt to a generative artificial intelligence model (e.g., GPT-3.5) to generate a template.

[1259] Output: The server outputs the generated template in JSON format.

[1260] Step 2:

[1261] Distribution of initial setup templates

[1262] The server distributes the generated initial configuration template to the terminal.

[1263] Input: The server receives the generated template data.

[1264] Specific operation: The server sends template data to the terminal using an HTTP POST request.

[1265] Output: The terminal displays the received template data on the screen.

[1266] Step 3:

[1267] Real-time generation of operating procedures

[1268] The server generates real-time operation procedures based on user actions.

[1269] Input: The server receives user requests and information (e.g., requests for QR code scanning procedures).

[1270] Specific operation: The server sends the request as a prompt to a generative artificial intelligence model, which then generates a procedure.

[1271] Output: The server outputs the generated procedure in text format.

[1272] Step 4:

[1273] Real-time streaming of operating procedures

[1274] The server distributes the generated operating instructions to the terminal.

[1275] Input: The server receives the generated procedure data.

[1276] Specific operation: The server sends the procedure data to the terminal via an HTTP POST request.

[1277] Output: The terminal displays the received procedure data to the user.

[1278] Step 5:

[1279] Question submission

[1280] The server receives questions from users.

[1281] Input: The terminal receives the question text data entered by the user.

[1282] Specific action: The terminal sends the question text to the server via an HTTP POST request.

[1283] Output: The server receives the question data.

[1284] Step 6:

[1285] Answer generation

[1286] The server uses generative artificial intelligence to generate answers to the questions it receives.

[1287] Input: The server will use the received question text data as input.

[1288] Specific operation: The server sends the question text as a prompt to a generative artificial intelligence model, which then generates an answer.

[1289] Output: The server outputs the generated response text.

[1290] Step 7:

[1291] Delivery of responses

[1292] The server delivers the generated response to the terminal.

[1293] Input: The server receives the generated response text data.

[1294] Specific operation: The server sends the response text to the terminal via an HTTP POST request.

[1295] Output: The terminal displays the received response text to the user.

[1296] Step 8:

[1297] Monitoring operation progress

[1298] The server monitors the operation progress information sent from the terminal.

[1299] Input: The terminal receives progress data as the user's actions progress.

[1300] Specific operation: The terminal sends progress information to the server in real time.

[1301] Output: The server receives progress data and updates the monitoring status.

[1302] Step 9:

[1303] Providing additional support information

[1304] The server generates necessary support information based on the progress data and delivers it to the terminal.

[1305] Input: The server receives operation progress data.

[1306] Specific operation: The server analyzes progress data and generates necessary support information using a generative artificial intelligence model.

[1307] Output: The server sends the generated support information to the terminal. The terminal displays this to the user.

[1308] (Application Example 1)

[1309] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1310] Current autonomous vehicles' smart devices require numerous settings and operations, placing a significant burden on users. Some users may find initial setup and operation difficult due to their unfamiliarity with electronic devices and operating procedures. Furthermore, limited means of quickly resolving questions or concerns during operation can detract from the user experience. An effective support system is needed to improve this situation.

[1311] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1312] In this invention, the server includes means for generating initial setup templates for smart devices using generative artificial intelligence and distributing them to participants' terminals; means for generating operating procedures in real time using generative artificial intelligence and displaying them on participants' terminals; means for receiving questions from participants, generating answers using generative artificial intelligence and distributing them to participants' terminals; means for monitoring participants' operational progress and providing additional support information as needed; means for performing initial setup and operational support for the smart system of an autonomous vehicle; and means for enabling users to efficiently learn how to set up and operate smart devices and quickly resolve any questions. This enables users to quickly and easily perform complex smart device setup and operation, thereby improving the user experience of autonomous vehicles.

[1313] "Generative artificial intelligence" is an artificial intelligence technology that outputs content generated in natural language or other formats based on given data or prompts.

[1314] "Smart devices" refer to all electronic devices that have internet connectivity and application execution capabilities, including smartphones, tablets, and smartwatches.

[1315] An "initial setup template" is a guideline that outlines a series of settings and procedures required when starting to use a smart device, including network settings, account settings, and application installation procedures.

[1316] "Real-time generation" refers to the process of instantly generating and delivering content and information in response to user requests and circumstances.

[1317] "Participants" refers to users of this system, including individuals and groups operating smart devices.

[1318] "Terminal" refers to a device operated by a user, and includes smartphones, computers, and other interactive electronic devices.

[1319] "Operation progress" refers to the progress of operations performed by the user on their smart device.

[1320] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously without driver intervention.

[1321] A "navigation system" is a system that uses map data and GPS technology to provide the vehicle's current location and the optimal route to its destination.

[1322] An "entertainment system" is a system that provides entertainment content such as music and videos inside a vehicle.

[1323] "Safety devices" refer to various systems and devices that ensure safety while driving a vehicle, and include brake assist and lane departure warning systems.

[1324] This invention relates to a system that utilizes generative artificial intelligence to perform initial setup and operational support for the smart system of an autonomous vehicle. The following describes specific embodiments of the system, with a server, terminal, and user as its main components.

[1325] Server roles and processing

[1326] The server is responsible for the following roles and processes.

[1327] 1. Generation and distribution of initial configuration templates

[1328] The server uses generative artificial intelligence to generate initial setup templates for the smart devices of autonomous vehicles. These templates include network settings, account settings, application installation procedures, and system configuration procedures related to autonomous vehicles. The generated templates are then delivered to the participants' devices.

[1329] 2. Real-time generation and distribution of operating procedures

[1330] The server uses generative artificial intelligence to generate operating instructions in real time and display them on participants' devices. For example, it generates instructions for setting up the navigation system, operating the entertainment system, and setting up safety devices, and immediately delivers them to the devices.

[1331] 3. Receiving questions and generating answers

[1332] The server receives user questions from terminals and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals. For example, in response to a question like, "How do I configure the navigation system in an autonomous vehicle?", the server generates and delivers a detailed guide.

[1333] 4. Monitoring of operation progress and provision of additional support information

[1334] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[1335] Terminal roles and processing

[1336] The terminal is responsible for the following roles and processes.

[1337] 1. Receiving and displaying initial setup information

[1338] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[1339] 2. Receiving and displaying operating instructions

[1340] The terminal receives operation instructions sent from the server in real time and displays them clearly to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[1341] 3. Submitting questions and viewing answers

[1342] The terminal sends the question entered by the user to the server, receives the answer from the server, and displays it to the user.

[1343] User roles and operations

[1344] The user will be responsible for the following roles and operations.

[1345] 1. Perform initial setup

[1346] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[1347] 2. Confirm and execute the demo procedure.

[1348] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[1349] 3. Resolving doubts and questions

[1350] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks, "How do I configure the navigation system of an autonomous vehicle?", the system will configure it based on the answer from the server.

[1351] Examples of specific cases and prompt statements

[1352] The following are examples of specific prompt messages.

[1353] "Please generate Wi-Fi setup instructions for autonomous vehicles. Include the following information: network selection, password entry, and connection confirmation."

[1354] "Please explain the procedure for installing an app into the entertainment system of an autonomous vehicle."

[1355] "Please generate the configuration procedure for the Advanced Driver-Assistance Systems (ADAS) of autonomous vehicles."

[1356] This system allows users to quickly and easily configure and operate the complex smart devices in autonomous vehicles, improving the overall user experience.

[1357] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1358] Step 1:

[1359] Generating an initial configuration template

[1360] The server uses generative artificial intelligence to generate initial setup templates for smart devices and autonomous vehicles.

[1361] Input: Prompt message (Example: "Generate Wi-Fi setup instructions for an autonomous vehicle. Include the following information: network selection, password entry, connection confirmation").

[1362] Data processing: Based on the prompt text, the generative artificial intelligence generates the setup procedure.

[1363] Output: The generated initial configuration template.

[1364] Specific operation: The server sends a prompt message to the AI ​​model and delivers the received configuration template to the terminal.

[1365] Step 2:

[1366] Distribution of initial setup templates

[1367] The server distributes the generated initial setup template to the participants' devices.

[1368] Input: The generated initial configuration template.

[1369] Data processing: The generated template is sent directly to the device.

[1370] Output: The template is delivered to the terminal.

[1371] Specific operation: The server transfers the generated template via a link to the terminal, which receives it and displays it to the user.

[1372] Step 3:

[1373] Real-time generation of operating procedures

[1374] The server uses generative artificial intelligence to generate specific operating procedures in real time.

[1375] Input: Prompt text for the operation procedure (e.g., "Generate the setup procedure for the navigation system of an autonomous vehicle.").

[1376] Data processing: Based on the prompt text, the generative artificial intelligence generates the operation procedure.

[1377] Output: Generated operating procedure.

[1378] Specific operation: The server sends a prompt message to the AI ​​model that generates the operation procedure and receives the generated procedure.

[1379] Step 4:

[1380] Distribution and display of operating procedures

[1381] The server distributes the generated operating instructions to the participants' devices, and the devices display them.

[1382] Input: Generated operating procedure.

[1383] Data processing: The generated operating procedure is sent directly to the terminal.

[1384] Output: The operating instructions will be displayed on the terminal.

[1385] Specific operation: The server sends operating instructions to the terminal, and the terminal visually displays the received instructions to the user.

[1386] Step 5:

[1387] Question reception and answer generation

[1388] The server receives user questions from the terminal and generates answers using generative artificial intelligence.

[1389] Input: A prompt for your question (e.g., "How do I configure the navigation system in an autonomous vehicle?").

[1390] Data processing: Based on the prompt text, the generative artificial intelligence generates an answer.

[1391] Output: The generated answer.

[1392] Specific operation: The server sends a question to the AI ​​model that generates the question and receives the generated answer.

[1393] Step 6:

[1394] Delivery and display of responses

[1395] The server delivers the generated responses to the participants' devices, and the devices display them.

[1396] Input: Generated answer.

[1397] Data processing: The generated response is sent directly to the device.

[1398] Output: The answer will be displayed on the terminal.

[1399] Specific operation: The server sends the response to the terminal, and the terminal visually displays the received response to the user.

[1400] Step 7:

[1401] Monitoring of operation progress and provision of additional support information

[1402] The server monitors the participants' progress and provides additional support information as needed.

[1403] Input: Operation progress data.

[1404] Data processing: Generate and provide additional support information based on the operation progress.

[1405] Output: Additional support information.

[1406] Specific operation: The server monitors the progress of operations sent from the terminal in real time, generates necessary support information, and delivers it to the terminal.

[1407] The above processing steps enable users to quickly and easily configure and operate the smart devices of autonomous vehicles.

[1408] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1409] This invention relates to a system that combines generative artificial intelligence and an emotion engine to provide initial setup and operational support for a user's smart device. This system provides operational support through the cooperation of a server, a terminal, and the user.

[1410] Server roles and processing

[1411] Generation and distribution of initial configuration templates

[1412] The server uses generative artificial intelligence to generate an initial setup template for a specific smart device. This template includes instructions for network settings, account settings, and application installation. This template is then distributed to the participants' devices.

[1413] As a concrete example, a server generates Wi-Fi setup instructions for a new smartphone and sends them to each device. This allows users to easily complete the initial setup.

[1414] Real-time generation and distribution of operating procedures

[1415] The server generates operating instructions in real time and displays them on participants' devices. For example, it generates instructions for scanning QR codes or installing apps and delivers them to the devices immediately.

[1416] Question reception and answer generation

[1417] The server receives user questions from the terminal and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals.

[1418] For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[1419] Monitoring of operation progress and provision of additional support information

[1420] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[1421] Terminal roles and processing

[1422] Receiving and displaying initial setup information

[1423] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[1424] For example, a device can receive a Wi-Fi setup guide and display it on its screen, allowing the user to configure the settings by following the guide.

[1425] Demonstration

[1426] The terminal receives operation instructions sent from the server in real time and displays them to the user in an easy-to-understand manner. Once the user completes the operation, the terminal sends progress information to the server.

[1427] Submitting questions and viewing answers

[1428] The terminal sends the question entered by the user to the server, receives the answer from the server again, and displays it to the user.

[1429] The role of the emotional engine

[1430] Recognizing the user's emotional state

[1431] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state.

[1432] For example, if a user is irritated, the emotion engine recognizes this and notifies the server.

[1433] Adjusting operating procedures based on emotions

[1434] Based on user emotion information obtained from the emotion engine, the server adjusts operating procedures and support information as needed. For example, if the user is frustrated, the server may simplify the operating procedures or add encouraging or supportive messages.

[1435] Providing an emotional response guide

[1436] The server-generated instructions and responses are customized to take the user's emotional state into consideration. This allows users to proceed more smoothly and reduce frustration.

[1437] User roles and operations

[1438] Execute initial setup

[1439] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[1440] Review and execute the demo procedure.

[1441] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[1442] Resolving doubts and questions

[1443] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks "How do I adjust the screen brightness?", the system adjusts the settings based on the answer from the server.

[1444] As described above, this invention provides a system that allows users to efficiently learn how to set up and operate smart devices and to quickly resolve any questions they may have. Furthermore, by incorporating an emotion engine, it can provide appropriate support tailored to the user's emotional state, thereby improving the user experience. In this way, users can make the most of the device's functions, leading to increased usage.

[1445] The following describes the processing flow.

[1446] Step 1:

[1447] The server generates an initial configuration template.

[1448] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices. These templates include network settings, account settings, and application installation instructions.

[1449] Step 2:

[1450] The server delivers the initial setup template to the terminal.

[1451] The server distributes the generated initial setup template to each participant's device. This allows the device to receive the setup guide.

[1452] Step 3:

[1453] The device displays initial setup information.

[1454] The device visually displays the received initial setup template to the user. The user proceeds with the initial setup while confirming the information displayed on the screen.

[1455] Step 4:

[1456] The user performs the initial setup.

[1457] The user configures the smart device by following the initial setup template displayed on the terminal. For example, they might enter Wi-Fi connection information to connect to the network.

[1458] Step 5:

[1459] The server generates operating procedures in real time.

[1460] The server uses generative artificial intelligence to generate the necessary operating procedures in real time, based on the user's progress and questions.

[1461] Step 6:

[1462] The server delivers the operating instructions to the terminal.

[1463] The server distributes the generated operating instructions to the user's terminal. This allows the terminal to receive the latest operating guide.

[1464] Step 7:

[1465] The device displays the operating instructions.

[1466] The terminal displays the received operating instructions in an easy-to-understand manner for the user. The user then performs the next operation based on this information.

[1467] Step 8:

[1468] The emotion engine recognizes the user's emotions.

[1469] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, and touch operation speed to determine their emotions.

[1470] Step 9:

[1471] Send the user's emotional state to the server.

[1472] The device sends the recognized user's emotional state to the server. This allows the server to understand the user's current emotional state.

[1473] Step 10:

[1474] The server adjusts the operating procedures based on emotional state.

[1475] The server adjusts operating procedures and support information as needed based on user emotion information obtained from the emotion engine. For example, if the user is frustrated, it may simplify the operating procedures or add encouraging or supportive messages.

[1476] Step 11:

[1477] The server delivers the coordinated operating instructions to the terminal.

[1478] The server sends emotionally-adjusted operating instructions to the device. This allows the device to receive updated operating guides.

[1479] Step 12:

[1480] The user performs the operating procedure.

[1481] The user follows the instructions displayed on the device to operate the smart device. For example, they might install an app as instructed.

[1482] Step 13:

[1483] The terminal sends the operation progress to the server.

[1484] The terminal sends user progress information to the server. This allows the server to track the user's progress.

[1485] Step 14:

[1486] User enters and submits question.

[1487] Users input questions and concerns about the operation and settings of their smart devices into their terminals and send them to the server.

[1488] Step 15:

[1489] The server receives the question and generates the answer.

[1490] The server analyzes the received question and generates an answer using generative artificial intelligence.

[1491] Step 16:

[1492] The server delivers the generated response to the terminal.

[1493] The server sends the generated response to the device. This allows the device to receive the latest response information.

[1494] Step 17:

[1495] The device displays the answer.

[1496] The device displays the received response to the user. The user reviews the response displayed on the screen and uses it as a reference for operations and settings.

[1497] Step 18:

[1498] The user completes the operation.

[1499] Based on the answers and guides received from the server, users ultimately complete the setup and operation of their smart devices.

[1500] In this way, a system is realized in which the server, terminal, user, and emotion engine work together to smoothly configure and operate smart devices.

[1501] (Example 2)

[1502] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1503] Modern smart devices are multi-functional, making their initial setup and operation complex, often leaving many users confused. Traditional systems lacked standardized initial setup and operating procedures, hindering smooth device use. Furthermore, the lack of support that considered the user's emotional state could lead to frustration and stress. A system is needed to address these challenges.

[1504] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1505] In this invention, the server includes means for generating an initial setup template for a smart device using generative artificial intelligence and delivering it to the terminal; means for generating operation procedures in real time using generative artificial intelligence and displaying them on the terminal; means for receiving questions from the terminal, generating answers using generative artificial intelligence and delivering them to the terminal; means for monitoring the operation progress of the terminal and providing additional support information as needed; means for recognizing the user's emotional state using an emotion engine and notifying the server of the recognition results; and means for adjusting operation procedures and support information based on emotional information. As a result, users can smoothly proceed through complex initial setup and operation procedures, reducing frustration and stress.

[1506] "Generative artificial intelligence" is a type of artificial intelligence that has the ability to automatically generate new information or answers based on given prompts or data.

[1507] A "smart device" is an electronic device that can connect to the internet and perform multi-functional operations. Specific examples include smartphones, tablets, and smartwatches.

[1508] An "initial setup template" is a document containing guidelines and procedures for the initial setup of a smart device. This includes network settings, account settings, and application installation instructions.

[1509] "Operating procedures" refer to a series of steps or guidelines necessary to accomplish a specific task. These include specific operating methods and setup procedures.

[1510] "Real-time" refers to a state where data and information are processed and provided immediately. Immediacy and lack of delay are essential.

[1511] "Progress information" refers to data and status related to the progress of operations and settings. This allows you to understand what stage you have progressed to.

[1512] An "emotion engine" is a technology or system that detects and analyzes the user's emotional state. This involves using techniques such as facial recognition, voice tone analysis, and touch operation speed.

[1513] "Emotional information" refers to data about the user's emotional state detected by the emotion engine. This includes, for example, the user's level of frustration or satisfaction.

[1514] This invention relates to a system that utilizes generative artificial intelligence and an emotion engine to assist with the initial setup and operation of a user's smart device. This system provides operation assistance through the cooperation of a server, a terminal, and the user.

[1515] Server roles and processing

[1516] Generation and distribution of initial configuration templates

[1517] The server uses a generation AI model to generate an initial setup template for a smart device. This template includes instructions for network settings, account settings, and application installation. A prompt message such as "Generate Wi-Fi setup instructions for a new smartphone" can be used. This template is then delivered to the device.

[1518] Real-time generation and distribution of operating procedures

[1519] The server generates operating instructions in real time and displays them on the terminal. For example, it generates instructions for scanning a QR code or installing an app and delivers them to the terminal immediately. The prompt message used is "Please generate instructions for installing a specific app."

[1520] Question reception and answer generation

[1521] The server receives user questions from the device and generates answers using a generative AI model. These answers are then sent back to the device. For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[1522] Monitoring of operation progress and provision of additional support information

[1523] The server monitors the progress of the terminal's operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties during the process.

[1524] Processing and responding to emotional information

[1525] Based on user emotion information obtained from the emotion engine, the server adjusts operating procedures and support information. For example, if the user is frustrated, the operating procedures are simplified and encouraging messages are added.

[1526] Terminal roles and processing

[1527] Receiving and displaying initial setup information

[1528] The device receives initial setup information sent from the server and displays it visually to the user. For example, it can receive a Wi-Fi setup guide and display it on the screen, allowing the user to configure the settings according to the guide.

[1529] Display of operating procedures and submission of progress

[1530] The terminal receives operation instructions sent from the server in real time and displays them to the user in an easy-to-understand manner. Once the user completes the operation, the terminal sends progress information to the server.

[1531] Submitting questions and viewing answers

[1532] The terminal sends the user's entered question to the server, receives the answer from the server, and displays it. This allows the user to resolve their question immediately.

[1533] The role of the emotional engine

[1534] Recognition of emotional states

[1535] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state. For example, if the user is irritated, the emotion engine recognizes this and notifies the server.

[1536] Providing an emotional response guide

[1537] The server-generated instructions and responses are customized to take the user's emotional state into consideration. This allows users to proceed more smoothly and reduce frustration.

[1538] User roles and operations

[1539] Execute initial setup

[1540] The user configures the device based on the initial setup template displayed on the device. For example, they might enter Wi-Fi settings and establish a connection.

[1541] Confirm and execute the operating procedure.

[1542] The user checks the operating instructions displayed on the device and performs the operation accordingly. For example, they may download and install an app by following the app installation instructions.

[1543] Resolving doubts and questions

[1544] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks how to adjust the screen brightness, they will adjust the settings based on the answer from the server.

[1545] As described above, the system of the present invention combines generative artificial intelligence and an emotion engine, enabling users to efficiently configure and operate smart devices and quickly resolve any questions they may have. Furthermore, the incorporation of an emotion engine provides appropriate support tailored to the user's emotional state, improving the user experience. In this way, users can maximize the functionality of the device, leading to increased usage.

[1546] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1547] Step 1:

[1548] The server generates an initial configuration template.

[1549] Input: A prompt given to the server (e.g., "Generate Wi-Fi setup instructions for a new smartphone")

[1550] Data Processing / Calculation: The server uses a generation AI model to generate Wi-Fi setup instructions, network settings, account settings, and application installation instructions based on the prompt text.

[1551] Output: Generated initial setup template (e.g., a file in JSON format containing Wi-Fi setup instructions, etc.)

[1552] Step 2:

[1553] The server distributes the initial configuration template it generates to the terminal.

[1554] Input: Generated initial configuration template

[1555] Data processing / calculation: The server encodes the template into the appropriate format and delivers it to the terminal.

[1556] Output: Initial setup template delivered to the device

[1557] Step 3:

[1558] The device receives and displays the initial setup template.

[1559] Input: Initial configuration template delivered from the server

[1560] Data processing / calculation: The terminal decodes the template and displays it visually to the user.

[1561] Output: Initial setup guide displayed on the user screen (e.g., Wi-Fi setup guide)

[1562] Step 4:

[1563] The user performs the initial setup.

[1564] Input: Initial setup guide displayed on the device

[1565] Data Processing / Calculation: The user enters information according to the guide and configures the smart device. For example, they enter the Wi-Fi SSID and password.

[1566] Output: Completed initial setup (e.g., connected to Wi-Fi)

[1567] Step 5:

[1568] The server generates and distributes operating instructions in real time.

[1569] Input: User action request (e.g., "Generate the app download instructions")

[1570] Data processing / calculation: The server uses a generation AI model to generate app download instructions based on the prompt text.

[1571] Output: Generated operating instructions (e.g., step-by-step app download instructions)

[1572] Step 6:

[1573] The device receives and displays the operating instructions.

[1574] Input: Operating instructions sent from the server

[1575] Data processing / calculation: The terminal decodes the operation procedure data and displays it visually to the user.

[1576] Output: Operating instructions displayed on the user screen (e.g., app download guide)

[1577] Step 7:

[1578] The user performs the operation.

[1579] Input: Instructions displayed on the device

[1580] Data processing / calculation: The user performs operations according to the procedure. For example, downloading and installing an app.

[1581] Output: Completed operation (e.g., app installed)

[1582] Step 8:

[1583] The device sends progress information to the server.

[1584] Input: Status information of completed operations

[1585] Data processing / calculation: The terminal encodes the progress information and sends it to the server.

[1586] Output: Progress information sent to the server (e.g., a notification saying "The app was successfully installed")

[1587] Step 9:

[1588] The server receives user questions and generates answers.

[1589] Input: User question (e.g., "How do I adjust the screen brightness?")

[1590] Data processing / computation: The server uses a generative AI model to generate answers to questions.

[1591] Output: Generated answer (e.g., steps to adjust screen brightness)

[1592] Step 10:

[1593] The device receives and displays the response.

[1594] Input: Response sent from the server

[1595] Data processing / calculation: The terminal decodes the response and displays it visually to the user.

[1596] Output: Answers displayed on the user screen (e.g., brightness adjustment guide)

[1597] Step 11:

[1598] The server monitors the progress of the operation and provides additional support information.

[1599] Input: User operation progress information

[1600] Data processing / calculation: The server analyzes progress information and generates and distributes additional support information as needed.

[1601] Output: Additional support information (e.g., "The next step is XX")

[1602] Step 12:

[1603] The emotion engine recognizes the user's emotional state and notifies the server.

[1604] Input: User's facial expressions, voice tone, and touch operation data

[1605] Data processing / calculation: The emotion engine analyzes this data to determine the user's emotional state.

[1606] Output: Sentiment information from the analysis results (e.g., information indicating the user is irritated)

[1607] Step 13:

[1608] The server adjusts operating procedures and support based on emotional information.

[1609] Input: Sentiment information (e.g., the user is irritated)

[1610] Data processing / calculation: The server uses emotional information to simplify operating procedures and generate encouraging messages.

[1611] Output: Adjusted operating instructions and encouraging messages (e.g., "Please contact us if you have difficulty operating the device.")

[1612] (Application Example 2)

[1613] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1614] Conventional smart device initial setup and operation support systems have the problem of not being able to provide sufficient support when users encounter difficulties with setup or operation. Furthermore, they fail to optimize the user experience because they provide uniform support without considering the user's emotional state. The present invention aims to solve these problems and provide a system that allows users to set up and operate devices efficiently and comfortably.

[1615] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1616] In this invention, the server includes means for generating initial setup templates for smart devices using generative artificial intelligence and delivering them to the participant's terminal; means for generating operation procedures in real time using generative artificial intelligence and displaying them on the participant's terminal; means for receiving questions from the participant, generating answers using generative artificial intelligence and delivering them to the participant's terminal; means for monitoring the participant's operation progress and providing additional support information as needed; and means for recognizing the participant's emotional state using an emotion engine and adjusting support information and operation procedures based on the emotional state. This makes it possible to reduce barriers to user setup and operation and provide personalized support according to the emotional state.

[1617] "Generative artificial intelligence" is a type of artificial intelligence that has the ability to automatically generate content and information in response to user requests, and generates data in real time based on specific patterns and algorithms.

[1618] An "initial setup template" is a template that comprehensively summarizes the information and procedures necessary for the initial setup of a new smart device, including network settings, account settings, and application installation procedures.

[1619] "Participants" refer to users who receive instructions on setting up and operating their smart devices through the system, and are the individuals who receive support for operating their devices.

[1620] An "emotion engine" is software or an algorithm used to recognize a user's emotional state. It analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine their emotional state.

[1621] "Operating instructions" are a guide that details the steps required for a specific task, and are a collection of instructions provided to help users operate a device efficiently.

[1622] This invention is a system that combines generative artificial intelligence and an emotion engine to provide initial setup and operational support for a user's smart device. This system provides operational support through the cooperation of a server, a terminal, and the user.

[1623] Server roles and processing

[1624] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices and delivers them to the terminals. For example, the server can generate Wi-Fi setup instructions for a new smartphone and send them to each terminal, allowing users to easily complete the initial setup. The server also generates operating instructions in real time, such as instructions for scanning QR codes or installing applications, and delivers them immediately. Furthermore, the server receives user questions from the terminals and generates answers using generative artificial intelligence. For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on that procedure and sends it to the terminal. The server monitors the progress of participants and provides additional support information as needed.

[1625] Terminal roles and processing

[1626] The device receives initial setup information sent from the server and displays it visually to the user. This allows the user to proceed with the initial setup based on this information. For example, if the device receives a Wi-Fi setup guide and displays it on the screen, the user can follow the guide to complete the setup. Furthermore, it can receive operation instructions sent from the server in real time and display them to the user in an easy-to-understand manner. Once the user completes the operation, the device sends progress information to the server. The device also sends questions entered by the user to the server, receives the answers from the server again, and displays them to the user.

[1627] The role of the emotional engine

[1628] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state. For example, if the user is irritated, the emotion engine recognizes this and notifies the server. Based on the user's emotional information obtained from the emotion engine, the server adjusts the operation procedures and support information as needed. For example, if the user is irritated, the server may simplify the operation procedures or add encouraging or supportive messages. The operation procedures and responses generated by the server are customized considering the user's emotional state. This allows the user to proceed more smoothly and reduce frustration.

[1629] Program processing and required hardware and software

[1630] To implement this system, the following hardware and software are required:

[1631] Hardware: Smartphones, tablets

[1632] Software: Python, Emotion Recognition module, Generative AI Chatbot module (AIChatBot)

[1633] The server uses generative artificial intelligence to generate initial setup templates and processes HTTP requests for delivery to terminals. The server also generates operation procedures for specific tasks in real time, accepts user questions, and generates answers using a generative AI chatbot. The emotion engine analyzes the user's emotional state using an emotion recognition module and adjusts support accordingly.

[1634] As a concrete example, the following steps are shown for a user who wants to set up Wi-Fi during the initial setup:

[1635] (Example prompt): "Please generate Wi-Fi setup instructions for my new smartphone."

[1636] Based on this, the following Wi-Fi setup instructions were generated:

[1637] Open the home screen and tap the Settings app.

[1638] Select Wi-Fi and tap Add New Network.

[1639] Enter the network name and then the password.

[1640] Tap the connect button to complete the setup.

[1641] In this way, users can easily proceed with settings and operations, and support tailored to their emotional state is provided, thus improving the user experience.

[1642] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1643] Step 1:

[1644] The server uses a generative AI model to generate initial setup templates for specific smart devices. The input includes the device type and a prompt message. For example, based on the prompt message "Generate Wi-Fi setup instructions for a new smartphone," a template is generated that includes network settings, account settings, and application installation instructions.

[1645] Step 2:

[1646] The server delivers the generated initial configuration template to the terminal. The template data is taken as input and sent to the terminal using an HTTP request. The terminal receives this request and saves the configuration template locally.

[1647] Step 3:

[1648] The user begins the initial setup. The user follows the initial setup template displayed on the device screen. As input, the user follows the steps in the template to complete Wi-Fi and account settings. As output, once the setup is complete, the device sends progress information to the server.

[1649] Step 4:

[1650] The server uses generative artificial intelligence to generate operating procedures in real time, providing step-by-step guidance for specific tasks. For example, if a user requests to know "how to scan a QR code," the server will generate the procedure based on that request. The user's request is the input, and the real-time generated operating procedure is delivered to the terminal as output.

[1651] Step 5:

[1652] The user enters a question through their device. For example, they might ask, "How do I install a specific app?" The user inputs the question, and the device sends this question to the server.

[1653] Step 6:

[1654] The server uses generative artificial intelligence to generate answers to questions. The input consists of the user's question and the generative AI model, and the output is a detailed guide. The server then delivers this guide to the terminal.

[1655] Step 7:

[1656] The server monitors the progress of participants' operations and provides additional support information as needed. Progress information from the terminal is taken as input, and additional support information and advice are generated as output and sent to the terminal.

[1657] Step 8:

[1658] The emotion engine recognizes the user's emotional state. Inputs include the user's facial expressions, voice tone, and touch operation speed, and the emotion engine determines the user's current emotional state. For example, if the user is irritated, this information is sent to the server.

[1659] Step 9:

[1660] The server adjusts operating procedures and support information based on emotional information obtained from the emotion engine. Emotional data is the input, and support information and operating procedures tailored to the user's emotional state are generated as output. For example, if the user is frustrated, the server may simplify the operating procedures or add encouraging messages.

[1661] Step 10:

[1662] The terminal displays adjusted operating procedures and support information to the user. Adjusted data is sent from the server as input, and the terminal displays this information to further improve the user experience.

[1663] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1664] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1665] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1666] [Fourth Embodiment]

[1667] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1668] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1669] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1670] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1671] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1672] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1673] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1674] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1675] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1676] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1677] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1678] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1679] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1680] This invention relates to a system that utilizes generative artificial intelligence to assist with the initial setup and operation of smart devices. This system consists of a server, a terminal, and a user as its main components, each working together to provide operational assistance.

[1681] Server roles and processing

[1682] Generation and distribution of initial configuration templates

[1683] The server generates an initial setup template for smart devices. This template includes instructions for network settings, account settings, and application installation. This template is then distributed to the participants' devices.

[1684] As a concrete example, the server generates Wi-Fi setup instructions for a new smartphone and sends them to the device. This allows the user to easily complete the initial setup.

[1685] Real-time generation and distribution of operating procedures

[1686] The server generates operating instructions in real time and displays them on participants' devices. For example, it generates instructions for scanning QR codes or installing apps and immediately delivers them to the devices.

[1687] Question reception and answer generation

[1688] The server receives user questions from terminals and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals.

[1689] For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[1690] Monitoring of operation progress and provision of additional support information

[1691] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[1692] Terminal roles and processing

[1693] Receiving and displaying initial setup information

[1694] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[1695] For example, a device can receive a Wi-Fi setup guide and display it on its screen, allowing the user to configure the settings by following the guide.

[1696] Demonstration

[1697] The terminal receives operation instructions sent from the server in real time and displays them clearly to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[1698] Submitting questions and viewing answers

[1699] The terminal sends the question entered by the user to the server, receives the answer from the server again, and displays it to the user.

[1700] User roles and operations

[1701] Execute initial setup

[1702] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[1703] Review and execute the demo procedure.

[1704] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[1705] Resolving doubts and questions

[1706] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks "How do I adjust the screen brightness?", the system adjusts the settings based on the answer from the server.

[1707] As described above, this invention provides a system that allows users to efficiently learn how to set up and operate smart devices and to quickly resolve any questions they may have. This enables users to make the most of the device's functions and promotes its use.

[1708] The following describes the processing flow.

[1709] Step 1:

[1710] The server generates an initial configuration template.

[1711] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices. These templates include network settings, account settings, and application installation instructions.

[1712] Step 2:

[1713] The server delivers the initial setup template to the terminal.

[1714] The server distributes the generated initial setup template to each participant's device. This allows the device to receive the setup guide.

[1715] Step 3:

[1716] The device displays initial setup information.

[1717] The device visually displays the received initial setup template to the user. The user proceeds with the initial setup while confirming the information displayed on the screen.

[1718] Step 4:

[1719] The user performs the initial setup.

[1720] The user configures the smart device by following the initial setup template displayed on the terminal. For example, they might enter Wi-Fi connection information to connect to the network.

[1721] Step 5:

[1722] The server generates operating procedures in real time.

[1723] The server uses generative artificial intelligence to generate the necessary operating procedures in real time, based on the user's progress and questions.

[1724] Step 6:

[1725] The server delivers the operating instructions to the terminal.

[1726] The server distributes the generated operating instructions to the user's terminal. This allows the terminal to receive the latest operating guide.

[1727] Step 7:

[1728] The device displays the operating instructions.

[1729] The terminal displays the received operating instructions in an easy-to-understand manner for the user. The user then performs the next operation based on this information.

[1730] Step 8:

[1731] The user performs the operating procedure.

[1732] The user follows the instructions displayed on the device to operate the smart device. For example, they might install an app as instructed.

[1733] Step 9:

[1734] The terminal sends the operation progress to the server.

[1735] The terminal sends user progress information to the server. This allows the server to track the user's progress.

[1736] Step 10:

[1737] User enters and submits question.

[1738] Users input questions and concerns about the operation and settings of their smart devices into their terminals and send them to the server.

[1739] Step 11:

[1740] The server receives the question and generates the answer.

[1741] The server analyzes the received question and generates an answer using generative artificial intelligence.

[1742] Step 12:

[1743] The server delivers the generated response to the terminal.

[1744] The server sends the generated response to the device. This allows the device to receive the latest response information.

[1745] Step 13:

[1746] The device displays the answer.

[1747] The device displays the received response to the user. The user reviews the response displayed on the screen and uses it as a reference for operations and settings.

[1748] Step 14:

[1749] The user completes the operation.

[1750] Based on the answers and guides received from the server, users ultimately complete the setup and operation of their smart devices.

[1751] In this way, a system is realized in which servers, terminals, and users work together to smoothly configure and operate smart devices.

[1752] (Example 1)

[1753] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1754] Initial setup and learning how to operate smart devices are time-consuming and difficult, especially for beginners. Furthermore, there is a lack of real-time solutions to questions and problems encountered during operation. As a result, users cannot fully utilize the device's capabilities, hindering its adoption. A system is needed to address these challenges and provide simple and efficient support for setting up and operating smart devices.

[1755] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1756] In this invention, the server includes means for generating an initial setup template for a smart device using generative artificial intelligence and delivering it to the user's terminal; means for generating operating procedures in real time using generative artificial intelligence and displaying them on the user's terminal; means for receiving questions from the user, generating answers using generative artificial intelligence and delivering them to the user's terminal; and means for monitoring the user's operation progress and providing additional support information as needed. As a result, the user can easily learn how to set up and operate the smart device and quickly resolve any questions or problems they may encounter.

[1757] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new information and content based on large amounts of data.

[1758] A "smart device" refers to an electronic device that can connect to the internet and perform various functions through user operation.

[1759] A "template" refers to a pre-created basic framework that includes a set of settings and configuration information tailored to a specific purpose.

[1760] A "server" refers to a computer system that provides various services and data to other computers and terminals via a network.

[1761] A "terminal" refers to an electronic device equipped with input devices and display devices that can be directly operated by the user.

[1762] A "question" refers to information entered by the user, either as text or voice, that expresses doubts or unclear points that need to be resolved.

[1763] "Answer" refers to the solutions or information that the server provides to the user's question using generative artificial intelligence.

[1764] "Operation progress" refers to the status of operations being performed by the user on their smart device and the degree to which each step has been completed.

[1765] "Support information" refers to additional instructions and guidelines to help users resolve problems and questions they may encounter while using the service.

[1766] A "prompt" refers to text used to instruct a generative artificial intelligence on what tasks to perform or what information to generate.

[1767] This invention is a system that utilizes generative artificial intelligence to assist with the initial setup and operation of smart devices. This system mainly consists of three elements: a server, a terminal, and a user, each playing the following roles.

[1768] Server roles and processing

[1769] Generation and distribution of initial configuration templates

[1770] The server uses generative artificial intelligence to generate initial setup templates for smart devices. These templates include network settings, account settings, and application installation instructions. Specifically, the server uses generative AI models such as GPT-3.5 running on the Google Cloud AI Platform.

[1771] example:

[1772] Prompt: "Please generate an initial setup guide for a new smartphone. The guide should include Wi-Fi connection information, Google account setup, and instructions for installing major apps."

[1773] The generated template is sent to the terminal in JSON format using an HTTP POST request.

[1774] Real-time generation and distribution of operating procedures

[1775] The server generates operation procedures in real time in response to user actions. This is also done using a generative artificial intelligence model.

[1776] example:

[1777] Prompt: "Generate instructions for scanning the QR code with your smartphone."

[1778] The generated instructions are delivered to the terminal in text format, and the terminal displays them visually to the user.

[1779] Question reception and answer generation

[1780] The server receives questions from users and generates answers using generative artificial intelligence. This process also uses GPT-3.5 running on Google Cloud AI Platform.

[1781] example:

[1782] Prompt: "When a user asks 'How to install the app,' generate an answer that provides detailed instructions."

[1783] The server sends the generated response back to the terminal and presents it to the user.

[1784] Monitoring of operation progress and provision of additional support information

[1785] The server monitors operation progress information sent from the terminal, generates additional support information as needed, and delivers it to the terminal. For example, if a user reports a connection error, the server generates and provides troubleshooting steps.

[1786] Terminal roles and processing

[1787] Receiving and displaying initial setup information

[1788] The device receives initial setup information sent from the server and displays it visually to the user. For example, the device receives a Wi-Fi setup guide from the server using an HTTP GET request and displays it on the screen, allowing the user to configure the settings.

[1789] Demonstration

[1790] The terminal receives operation instructions sent from the server in real time and displays them to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[1791] Submitting questions and viewing answers

[1792] The terminal sends the question entered by the user to the server, receives the answer from the server, and displays it to the user.

[1793] User roles and operations

[1794] Execute initial setup

[1795] The user configures the device based on the initial setup template displayed on the device. For example, they enter Wi-Fi settings and establish a connection.

[1796] Review and execute the demo procedure.

[1797] The user reviews the demo instructions displayed on the device and performs the actions accordingly. For example, they might follow the app installation instructions to download and install the app.

[1798] Resolving doubts and questions

[1799] Users input questions and inquiries into their terminals as text, and the server provides answers to resolve the issues. For example, a user might ask how to adjust the screen brightness and then adjust the settings based on the server's response.

[1800] In this way, the coordinated operation of each element allows users to efficiently learn the initial setup and operation methods of smart devices, and to quickly resolve any questions or problems they may encounter.

[1801] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1802] Step 1:

[1803] Generating an initial configuration template

[1804] The server uses generative artificial intelligence to generate initial setup templates for smart devices.

[1805] Input: The server receives the user's smart device type and required settings (e.g., Wi-Fi settings, account settings, app installation instructions) as input.

[1806] Specific operation: The server sends this input information as a prompt to a generative artificial intelligence model (e.g., GPT-3.5) to generate a template.

[1807] Output: The server outputs the generated template in JSON format.

[1808] Step 2:

[1809] Distribution of initial setup templates

[1810] The server distributes the generated initial configuration template to the terminal.

[1811] Input: The server receives the generated template data.

[1812] Specific operation: The server sends template data to the terminal using an HTTP POST request.

[1813] Output: The terminal displays the received template data on the screen.

[1814] Step 3:

[1815] Real-time generation of operating procedures

[1816] The server generates real-time operation procedures based on user actions.

[1817] Input: The server receives user requests and information (e.g., requests for QR code scanning procedures).

[1818] Specific operation: The server sends the request as a prompt to a generative artificial intelligence model, which then generates a procedure.

[1819] Output: The server outputs the generated procedure in text format.

[1820] Step 4:

[1821] Real-time streaming of operating procedures

[1822] The server distributes the generated operating instructions to the terminal.

[1823] Input: The server receives the generated procedure data.

[1824] Specific operation: The server sends the procedure data to the terminal via an HTTP POST request.

[1825] Output: The terminal displays the received procedure data to the user.

[1826] Step 5:

[1827] Question submission

[1828] The server receives questions from users.

[1829] Input: The terminal receives the question text data entered by the user.

[1830] Specific action: The terminal sends the question text to the server via an HTTP POST request.

[1831] Output: The server receives the question data.

[1832] Step 6:

[1833] Generating an answer

[1834] The server uses generative artificial intelligence to generate answers to the questions it receives.

[1835] Input: The server will use the received question text data as input.

[1836] Specific operation: The server sends the question text as a prompt to a generative artificial intelligence model, which then generates an answer.

[1837] Output: The server outputs the generated response text.

[1838] Step 7:

[1839] Delivery of responses

[1840] The server delivers the generated response to the terminal.

[1841] Input: The server receives the generated response text data.

[1842] Specific operation: The server sends the response text to the terminal via an HTTP POST request.

[1843] Output: The terminal displays the received response text to the user.

[1844] Step 8:

[1845] Monitoring operation progress

[1846] The server monitors the operation progress information sent from the terminal.

[1847] Input: The terminal receives progress data as the user's actions progress.

[1848] Specific operation: The terminal sends progress information to the server in real time.

[1849] Output: The server receives progress data and updates the monitoring status.

[1850] Step 9:

[1851] Providing additional support information

[1852] The server generates necessary support information based on the progress data and delivers it to the terminal.

[1853] Input: The server receives operation progress data.

[1854] Specific operation: The server analyzes progress data and generates necessary support information using a generative artificial intelligence model.

[1855] Output: The server sends the generated support information to the terminal. The terminal displays this to the user.

[1856] (Application Example 1)

[1857] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1858] Current autonomous vehicles' smart devices require numerous settings and operations, placing a significant burden on users. Some users may find initial setup and operation difficult due to their unfamiliarity with electronic devices and operating procedures. Furthermore, limited means of quickly resolving questions or concerns during operation can detract from the user experience. An effective support system is needed to improve this situation.

[1859] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1860] In this invention, the server includes means for generating initial setup templates for smart devices using generative artificial intelligence and distributing them to participants' terminals; means for generating operating procedures in real time using generative artificial intelligence and displaying them on participants' terminals; means for receiving questions from participants, generating answers using generative artificial intelligence and distributing them to participants' terminals; means for monitoring participants' operational progress and providing additional support information as needed; means for performing initial setup and operational support for the smart system of an autonomous vehicle; and means for enabling users to efficiently learn how to set up and operate smart devices and quickly resolve any questions. This enables users to quickly and easily perform complex smart device setup and operation, thereby improving the user experience of autonomous vehicles.

[1861] "Generative artificial intelligence" is an artificial intelligence technology that outputs content generated in natural language or other formats based on given data or prompts.

[1862] "Smart devices" refer to all electronic devices that have internet connectivity and application execution capabilities, including smartphones, tablets, and smartwatches.

[1863] An "initial setup template" is a guideline that outlines a series of settings and procedures required when starting to use a smart device, including network settings, account settings, and application installation procedures.

[1864] "Real-time generation" refers to the process of instantly generating and delivering content and information in response to user requests and circumstances.

[1865] "Participants" refers to users of this system, including individuals and groups operating smart devices.

[1866] "Terminal" refers to a device operated by a user, and includes smartphones, computers, and other interactive electronic devices.

[1867] "Operation progress" refers to the progress of operations performed by the user on their smart device.

[1868] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously without driver intervention.

[1869] A "navigation system" is a system that uses map data and GPS technology to provide the vehicle's current location and the optimal route to its destination.

[1870] An "entertainment system" is a system that provides entertainment content such as music and videos inside a vehicle.

[1871] "Safety devices" refer to various systems and devices that ensure safety while driving a vehicle, and include brake assist and lane departure warning systems.

[1872] This invention relates to a system that utilizes generative artificial intelligence to perform initial setup and operational support for the smart system of an autonomous vehicle. The following describes specific embodiments of the system, with a server, terminal, and user as its main components.

[1873] Server roles and processing

[1874] The server is responsible for the following roles and processes.

[1875] 1. Generation and distribution of initial configuration templates

[1876] The server uses generative artificial intelligence to generate initial setup templates for the smart devices of autonomous vehicles. These templates include network settings, account settings, application installation procedures, and system configuration procedures related to autonomous vehicles. The generated templates are then delivered to the participants' devices.

[1877] 2. Real-time generation and distribution of operating procedures

[1878] The server uses generative artificial intelligence to generate operating instructions in real time and display them on participants' devices. For example, it generates instructions for setting up the navigation system, operating the entertainment system, and setting up safety devices, and immediately delivers them to the devices.

[1879] 3. Receiving questions and generating answers

[1880] The server receives user questions from terminals and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals. For example, in response to a question like, "How do I configure the navigation system in an autonomous vehicle?", the server generates and delivers a detailed guide.

[1881] 4. Monitoring of operation progress and provision of additional support information

[1882] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[1883] Terminal roles and processing

[1884] The terminal is responsible for the following roles and processes.

[1885] 1. Receiving and displaying initial setup information

[1886] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[1887] 2. Receiving and displaying operating instructions

[1888] The terminal receives operation instructions sent from the server in real time and displays them clearly to the user. Once the user completes the operation, the terminal sends the progress information to the server.

[1889] 3. Submitting questions and viewing answers

[1890] The terminal sends the question entered by the user to the server, receives the answer from the server, and displays it to the user.

[1891] User roles and operations

[1892] The user will be responsible for the following roles and operations.

[1893] 1. Perform initial setup

[1894] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[1895] 2. Confirm and execute the demo procedure.

[1896] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[1897] 3. Resolving doubts and questions

[1898] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks, "How do I configure the navigation system of an autonomous vehicle?", the system will configure it based on the answer from the server.

[1899] Examples of specific cases and prompt statements

[1900] The following are examples of specific prompt messages.

[1901] "Please generate Wi-Fi setup instructions for autonomous vehicles. Include the following information: network selection, password entry, and connection confirmation."

[1902] "Please explain the procedure for installing an app into the entertainment system of an autonomous vehicle."

[1903] "Please generate the configuration procedure for the Advanced Driver-Assistance Systems (ADAS) of autonomous vehicles."

[1904] This system allows users to quickly and easily configure and operate the complex smart devices in autonomous vehicles, improving the overall user experience.

[1905] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1906] Step 1:

[1907] Generating an initial configuration template

[1908] The server uses generative artificial intelligence to generate initial setup templates for smart devices and autonomous vehicles.

[1909] Input: Prompt message (Example: "Generate Wi-Fi setup instructions for an autonomous vehicle. Include the following information: network selection, password entry, connection confirmation").

[1910] Data processing: Based on the prompt text, the generative artificial intelligence generates the setup procedure.

[1911] Output: The generated initial configuration template.

[1912] Specific operation: The server sends a prompt message to the AI ​​model and delivers the received configuration template to the terminal.

[1913] Step 2:

[1914] Distribution of initial setup templates

[1915] The server distributes the generated initial setup template to the participants' devices.

[1916] Input: The generated initial configuration template.

[1917] Data processing: The generated template is sent directly to the device.

[1918] Output: The template is delivered to the terminal.

[1919] Specific operation: The server transfers the generated template via a link to the terminal, which receives it and displays it to the user.

[1920] Step 3:

[1921] Real-time generation of operating procedures

[1922] The server uses generative artificial intelligence to generate specific operating procedures in real time.

[1923] Input: Prompt text for the operation procedure (e.g., "Generate the setup procedure for the navigation system of an autonomous vehicle.").

[1924] Data processing: Based on the prompt text, the generative artificial intelligence generates the operation procedure.

[1925] Output: Generated operating procedure.

[1926] Specific operation: The server sends a prompt message to the AI ​​model that generates the operation procedure and receives the generated procedure.

[1927] Step 4:

[1928] Distribution and display of operating procedures

[1929] The server distributes the generated operating instructions to the participants' devices, and the devices display them.

[1930] Input: Generated operating procedure.

[1931] Data processing: The generated operating procedure is sent directly to the terminal.

[1932] Output: The operating instructions will be displayed on the terminal.

[1933] Specific operation: The server sends operating instructions to the terminal, and the terminal visually displays the received instructions to the user.

[1934] Step 5:

[1935] Question reception and answer generation

[1936] The server receives user questions from the terminal and generates answers using generative artificial intelligence.

[1937] Input: A prompt for your question (e.g., "How do I configure the navigation system in an autonomous vehicle?").

[1938] Data processing: Based on the prompt text, the generative artificial intelligence generates an answer.

[1939] Output: The generated answer.

[1940] Specific operation: The server sends a question to the AI ​​model that generates the question and receives the generated answer.

[1941] Step 6:

[1942] Delivery and display of responses

[1943] The server delivers the generated responses to the participants' devices, and the devices display them.

[1944] Input: Generated answer.

[1945] Data processing: The generated response is sent directly to the device.

[1946] Output: The answer will be displayed on the terminal.

[1947] Specific operation: The server sends the response to the terminal, and the terminal visually displays the received response to the user.

[1948] Step 7:

[1949] Monitoring of operation progress and provision of additional support information

[1950] The server monitors the participants' progress and provides additional support information as needed.

[1951] Input: Operation progress data.

[1952] Data processing: Generate and provide additional support information based on the operation progress.

[1953] Output: Additional support information.

[1954] Specific operation: The server monitors the progress of operations sent from the terminal in real time, generates necessary support information, and delivers it to the terminal.

[1955] The above processing steps enable users to quickly and easily configure and operate the smart devices of autonomous vehicles.

[1956] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1957] This invention relates to a system that combines generative artificial intelligence and an emotion engine to provide initial setup and operational support for a user's smart device. This system provides operational support through the cooperation of a server, a terminal, and the user.

[1958] Server roles and processing

[1959] Generation and distribution of initial configuration templates

[1960] The server uses generative artificial intelligence to generate an initial setup template for a specific smart device. This template includes instructions for network settings, account settings, and application installation. This template is then distributed to the participants' devices.

[1961] As a concrete example, a server generates Wi-Fi setup instructions for a new smartphone and sends them to each device. This allows users to easily complete the initial setup.

[1962] Real-time generation and distribution of operating procedures

[1963] The server generates operating instructions in real time and displays them on participants' devices. For example, it generates instructions for scanning QR codes or installing apps and delivers them to the devices immediately.

[1964] Question reception and answer generation

[1965] The server receives user questions from the terminal and generates answers using generative artificial intelligence. These answers are then delivered back to the participants' terminals.

[1966] For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[1967] Monitoring of operation progress and provision of additional support information

[1968] The server monitors the progress of participants' operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties.

[1969] Terminal roles and processing

[1970] Receiving and displaying initial setup information

[1971] The terminal receives initial setup information sent from the server and displays it visually to the user. The user then proceeds with the initial setup based on this information.

[1972] For example, a device can receive a Wi-Fi setup guide and display it on its screen, allowing the user to configure the settings by following the guide.

[1973] Demonstration

[1974] The terminal receives operation instructions sent from the server in real time and displays them to the user in an easy-to-understand manner. Once the user completes the operation, the terminal sends progress information to the server.

[1975] Submitting questions and viewing answers

[1976] The terminal sends the question entered by the user to the server, receives the answer from the server again, and displays it to the user.

[1977] The role of the emotional engine

[1978] Recognizing the user's emotional state

[1979] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state.

[1980] For example, if a user is irritated, the emotion engine recognizes this and notifies the server.

[1981] Adjusting operating procedures based on emotions

[1982] Based on user emotion information obtained from the emotion engine, the server adjusts operating procedures and support information as needed. For example, if the user is frustrated, the server may simplify the operating procedures or add encouraging or supportive messages.

[1983] Providing an emotional response guide

[1984] The server-generated instructions and responses are customized to take the user's emotional state into consideration. This allows users to proceed more smoothly and reduce frustration.

[1985] User roles and operations

[1986] Execute initial setup

[1987] The user configures the device based on the initial setup template displayed on the device. For example, the user enters Wi-Fi settings and establishes a connection.

[1988] Review and execute the demo procedure.

[1989] The user checks the demo instructions displayed on the device and performs the actions on their own smart device accordingly. For example, they might follow the app installation instructions to download and install the app.

[1990] Resolving doubts and questions

[1991] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks "How do I adjust the screen brightness?", the system adjusts the settings based on the answer from the server.

[1992] As described above, this invention provides a system that allows users to efficiently learn how to set up and operate smart devices and to quickly resolve any questions they may have. Furthermore, by incorporating an emotion engine, it can provide appropriate support tailored to the user's emotional state, thereby improving the user experience. In this way, users can make the most of the device's functions, leading to increased usage.

[1993] The following describes the processing flow.

[1994] Step 1:

[1995] The server generates an initial configuration template.

[1996] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices. These templates include network settings, account settings, and application installation instructions.

[1997] Step 2:

[1998] The server delivers the initial setup template to the terminal.

[1999] The server distributes the generated initial setup template to each participant's device. This allows the device to receive the setup guide.

[2000] Step 3:

[2001] The device displays initial setup information.

[2002] The device visually displays the received initial setup template to the user. The user proceeds with the initial setup while confirming the information displayed on the screen.

[2003] Step 4:

[2004] The user performs the initial setup.

[2005] The user configures the smart device by following the initial setup template displayed on the terminal. For example, they might enter Wi-Fi connection information to connect to the network.

[2006] Step 5:

[2007] The server generates operating procedures in real time.

[2008] The server uses generative artificial intelligence to generate the necessary operating procedures in real time, based on the user's progress and questions.

[2009] Step 6:

[2010] The server delivers the operating instructions to the terminal.

[2011] The server distributes the generated operating instructions to the user's terminal. This allows the terminal to receive the latest operating guide.

[2012] Step 7:

[2013] The device displays the operating instructions.

[2014] The terminal displays the received operating instructions in an easy-to-understand manner for the user. The user then performs the next operation based on this information.

[2015] Step 8:

[2016] The emotion engine recognizes the user's emotions.

[2017] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, and touch operation speed to determine their emotions.

[2018] Step 9:

[2019] Send the user's emotional state to the server.

[2020] The device sends the recognized user's emotional state to the server. This allows the server to understand the user's current emotional state.

[2021] Step 10:

[2022] The server adjusts the operating procedures based on emotional state.

[2023] The server adjusts operating procedures and support information as needed based on user emotion information obtained from the emotion engine. For example, if the user is frustrated, it may simplify the operating procedures or add encouraging or supportive messages.

[2024] Step 11:

[2025] The server delivers the coordinated operating instructions to the terminal.

[2026] The server sends emotionally-adjusted operating instructions to the device. This allows the device to receive updated operating guides.

[2027] Step 12:

[2028] The user performs the operating procedure.

[2029] The user follows the instructions displayed on the device to operate the smart device. For example, they might install an app as instructed.

[2030] Step 13:

[2031] The terminal sends the operation progress to the server.

[2032] The terminal sends user progress information to the server. This allows the server to track the user's progress.

[2033] Step 14:

[2034] User enters and submits question.

[2035] Users input questions and concerns about the operation and settings of their smart devices into their terminals and send them to the server.

[2036] Step 15:

[2037] The server receives the question and generates the answer.

[2038] The server analyzes the received question and generates an answer using generative artificial intelligence.

[2039] Step 16:

[2040] The server delivers the generated response to the terminal.

[2041] The server sends the generated response to the device. This allows the device to receive the latest response information.

[2042] Step 17:

[2043] The device displays the answer.

[2044] The device displays the received response to the user. The user reviews the response displayed on the screen and uses it as a reference for operations and settings.

[2045] Step 18:

[2046] The user completes the operation.

[2047] Based on the answers and guides received from the server, users ultimately complete the setup and operation of their smart devices.

[2048] In this way, a system is realized in which the server, terminal, user, and emotion engine work together to smoothly configure and operate smart devices.

[2049] (Example 2)

[2050] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2051] Modern smart devices are multi-functional, making their initial setup and operation complex, often leaving many users confused. Traditional systems lacked standardized initial setup and operating procedures, hindering smooth device use. Furthermore, the lack of support that considered the user's emotional state could lead to frustration and stress. A system is needed to address these challenges.

[2052] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[2053] In this invention, the server includes means for generating an initial setup template for a smart device using generative artificial intelligence and delivering it to the terminal; means for generating operation procedures in real time using generative artificial intelligence and displaying them on the terminal; means for receiving questions from the terminal, generating answers using generative artificial intelligence and delivering them to the terminal; means for monitoring the operation progress of the terminal and providing additional support information as needed; means for recognizing the user's emotional state using an emotion engine and notifying the server of the recognition results; and means for adjusting operation procedures and support information based on emotional information. As a result, users can smoothly proceed through complex initial setup and operation procedures, reducing frustration and stress.

[2054] "Generative artificial intelligence" is a type of artificial intelligence that has the ability to automatically generate new information or answers based on given prompts or data.

[2055] A "smart device" is an electronic device that can connect to the internet and perform multi-functional operations. Specific examples include smartphones, tablets, and smartwatches.

[2056] An "initial setup template" is a document containing guidelines and procedures for the initial setup of a smart device. This includes network settings, account settings, and application installation instructions.

[2057] "Operating procedures" refer to a series of steps or guidelines necessary to accomplish a specific task. These include specific operating methods and setup procedures.

[2058] "Real-time" refers to a state where data and information are processed and provided immediately. Immediacy and lack of delay are essential.

[2059] "Progress information" refers to data and status related to the progress of operations and settings. This allows you to understand what stage you have progressed to.

[2060] An "emotion engine" is a technology or system that detects and analyzes the user's emotional state. This involves using techniques such as facial recognition, voice tone analysis, and touch operation speed.

[2061] "Emotional information" refers to data about the user's emotional state detected by the emotion engine. This includes, for example, the user's level of frustration or satisfaction.

[2062] This invention relates to a system that utilizes generative artificial intelligence and an emotion engine to assist with the initial setup and operation of a user's smart device. This system provides operation assistance through the cooperation of a server, a terminal, and the user.

[2063] Server roles and processing

[2064] Generation and distribution of initial configuration templates

[2065] The server uses a generation AI model to generate an initial setup template for a smart device. This template includes instructions for network settings, account settings, and application installation. A prompt message such as "Generate Wi-Fi setup instructions for a new smartphone" can be used. This template is then delivered to the device.

[2066] Real-time generation and distribution of operating procedures

[2067] The server generates operating instructions in real time and displays them on the terminal. For example, it generates instructions for scanning a QR code or installing an app and delivers them to the terminal immediately. The prompt message used is "Please generate instructions for installing a specific app."

[2068] Question reception and answer generation

[2069] The server receives user questions from the device and generates answers using a generative AI model. These answers are then sent back to the device. For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on the procedure and sends it to the device.

[2070] Monitoring of operation progress and provision of additional support information

[2071] The server monitors the progress of the terminal's operations and provides additional support information as needed. This ensures that users receive appropriate assistance even if they encounter difficulties during the process.

[2072] Processing and responding to emotional information

[2073] Based on user emotion information obtained from the emotion engine, the server adjusts operating procedures and support information. For example, if the user is frustrated, the operating procedures are simplified and encouraging messages are added.

[2074] Terminal roles and processing

[2075] Receiving and displaying initial setup information

[2076] The device receives initial setup information sent from the server and displays it visually to the user. For example, it can receive a Wi-Fi setup guide and display it on the screen, allowing the user to configure the settings according to the guide.

[2077] Display of operating procedures and submission of progress

[2078] The terminal receives operation instructions sent from the server in real time and displays them to the user in an easy-to-understand manner. Once the user completes the operation, the terminal sends progress information to the server.

[2079] Submitting questions and viewing answers

[2080] The terminal sends the user's entered question to the server, receives the answer from the server, and displays it. This allows the user to resolve their question immediately.

[2081] The role of the emotional engine

[2082] Recognition of emotional states

[2083] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state. For example, if the user is irritated, the emotion engine recognizes this and notifies the server.

[2084] Providing an emotional response guide

[2085] The server-generated instructions and responses are customized to take the user's emotional state into consideration. This allows users to proceed more smoothly and reduce frustration.

[2086] User roles and operations

[2087] Execute initial setup

[2088] The user configures the device based on the initial setup template displayed on the device. For example, they might enter Wi-Fi settings and establish a connection.

[2089] Confirm and execute the operating procedure.

[2090] The user checks the operating instructions displayed on the device and performs the operation accordingly. For example, they may download and install an app by following the app installation instructions.

[2091] Resolving doubts and questions

[2092] Users send questions and inquiries to the server via their device, and the system solves problems based on the answers received. For example, if a user asks how to adjust the screen brightness, they will adjust the settings based on the answer from the server.

[2093] As described above, the system of the present invention combines generative artificial intelligence and an emotion engine, enabling users to efficiently configure and operate smart devices and quickly resolve any questions they may have. Furthermore, the incorporation of an emotion engine provides appropriate support tailored to the user's emotional state, improving the user experience. In this way, users can maximize the functionality of the device, leading to increased usage.

[2094] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2095] Step 1:

[2096] The server generates an initial configuration template.

[2097] Input: A prompt given to the server (e.g., "Generate Wi-Fi setup instructions for a new smartphone")

[2098] Data Processing / Calculation: The server uses a generation AI model to generate Wi-Fi setup instructions, network settings, account settings, and application installation instructions based on the prompt text.

[2099] Output: Generated initial setup template (e.g., a file in JSON format containing Wi-Fi setup instructions, etc.)

[2100] Step 2:

[2101] The server distributes the initial configuration template it generates to the terminal.

[2102] Input: Generated initial configuration template

[2103] Data processing / calculation: The server encodes the template into the appropriate format and delivers it to the terminal.

[2104] Output: Initial setup template delivered to the device

[2105] Step 3:

[2106] The device receives and displays the initial setup template.

[2107] Input: Initial configuration template delivered from the server

[2108] Data processing / calculation: The terminal decodes the template and displays it visually to the user.

[2109] Output: Initial setup guide displayed on the user screen (e.g., Wi-Fi setup guide)

[2110] Step 4:

[2111] The user performs the initial setup.

[2112] Input: Initial setup guide displayed on the device

[2113] Data Processing / Calculation: The user enters information according to the guide and configures the smart device. For example, they enter the Wi-Fi SSID and password.

[2114] Output: Completed initial setup (e.g., connected to Wi-Fi)

[2115] Step 5:

[2116] The server generates and distributes operating instructions in real time.

[2117] Input: User action request (e.g., "Generate the app download instructions")

[2118] Data processing / calculation: The server uses a generation AI model to generate app download instructions based on the prompt text.

[2119] Output: Generated operating instructions (e.g., step-by-step app download instructions)

[2120] Step 6:

[2121] The device receives and displays the operating instructions.

[2122] Input: Operating instructions sent from the server

[2123] Data processing / calculation: The terminal decodes the operation procedure data and displays it visually to the user.

[2124] Output: Operating instructions displayed on the user screen (e.g., app download guide)

[2125] Step 7:

[2126] The user performs the operation.

[2127] Input: Instructions displayed on the device

[2128] Data processing / calculation: The user performs operations according to the procedure. For example, downloading and installing an app.

[2129] Output: Completed operation (e.g., app installed)

[2130] Step 8:

[2131] The device sends progress information to the server.

[2132] Input: Status information of completed operations

[2133] Data processing / calculation: The terminal encodes the progress information and sends it to the server.

[2134] Output: Progress information sent to the server (e.g., a notification saying "The app was successfully installed")

[2135] Step 9:

[2136] The server receives user questions and generates answers.

[2137] Input: User question (e.g., "How do I adjust the screen brightness?")

[2138] Data processing / computation: The server uses a generative AI model to generate answers to questions.

[2139] Output: Generated answer (e.g., steps to adjust screen brightness)

[2140] Step 10:

[2141] The device receives and displays the response.

[2142] Input: Response sent from the server

[2143] Data processing / calculation: The terminal decodes the response and displays it visually to the user.

[2144] Output: Answers displayed on the user screen (e.g., brightness adjustment guide)

[2145] Step 11:

[2146] The server monitors the progress of the operation and provides additional support information.

[2147] Input: User operation progress information

[2148] Data processing / calculation: The server analyzes progress information and generates and distributes additional support information as needed.

[2149] Output: Additional support information (e.g., "The next step is XX")

[2150] Step 12:

[2151] The emotion engine recognizes the user's emotional state and notifies the server.

[2152] Input: User's facial expressions, voice tone, and touch operation data

[2153] Data processing / calculation: The emotion engine analyzes this data to determine the user's emotional state.

[2154] Output: Sentiment information from the analysis results (e.g., information indicating the user is irritated)

[2155] Step 13:

[2156] The server adjusts operating procedures and support based on emotional information.

[2157] Input: Sentiment information (e.g., the user is irritated)

[2158] Data processing / calculation: The server uses emotional information to simplify operating procedures and generate encouraging messages.

[2159] Output: Adjusted operating instructions and encouraging messages (e.g., "Please contact us if you have difficulty operating the device.")

[2160] (Application Example 2)

[2161] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2162] Conventional smart device initial setup and operation support systems have the problem of not being able to provide sufficient support when users encounter difficulties with setup or operation. Furthermore, they fail to optimize the user experience because they provide uniform support without considering the user's emotional state. The present invention aims to solve these problems and provide a system that allows users to set up and operate devices efficiently and comfortably.

[2163] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2164] In this invention, the server includes means for generating initial setup templates for smart devices using generative artificial intelligence and delivering them to the participant's terminal; means for generating operation procedures in real time using generative artificial intelligence and displaying them on the participant's terminal; means for receiving questions from the participant, generating answers using generative artificial intelligence and delivering them to the participant's terminal; means for monitoring the participant's operation progress and providing additional support information as needed; and means for recognizing the participant's emotional state using an emotion engine and adjusting support information and operation procedures based on the emotional state. This makes it possible to reduce barriers to user setup and operation and provide personalized support according to the emotional state.

[2165] "Generative artificial intelligence" is a type of artificial intelligence that has the ability to automatically generate content and information in response to user requests, and generates data in real time based on specific patterns and algorithms.

[2166] An "initial setup template" is a template that comprehensively summarizes the information and procedures necessary for the initial setup of a new smart device, including network settings, account settings, and application installation procedures.

[2167] "Participants" refer to users who receive instructions on setting up and operating their smart devices through the system, and are the individuals who receive support for operating their devices.

[2168] An "emotion engine" is software or an algorithm used to recognize a user's emotional state. It analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine their emotional state.

[2169] "Operating instructions" are a guide that details the steps required for a specific task, and are a collection of instructions provided to help users operate a device efficiently.

[2170] This invention is a system that combines generative artificial intelligence and an emotion engine to provide initial setup and operational support for a user's smart device. This system provides operational support through the cooperation of a server, a terminal, and the user.

[2171] Server roles and processing

[2172] The server uses generative artificial intelligence to generate initial setup templates for specific smart devices and delivers them to the terminals. For example, the server can generate Wi-Fi setup instructions for a new smartphone and send them to each terminal, allowing users to easily complete the initial setup. The server also generates operating instructions in real time, such as instructions for scanning QR codes or installing applications, and delivers them immediately. Furthermore, the server receives user questions from the terminals and generates answers using generative artificial intelligence. For example, if a user asks, "How do I install a specific app?", the server generates a detailed guide on that procedure and sends it to the terminal. The server monitors the progress of participants and provides additional support information as needed.

[2173] Terminal roles and processing

[2174] The device receives initial setup information sent from the server and displays it visually to the user. This allows the user to proceed with the initial setup based on this information. For example, if the device receives a Wi-Fi setup guide and displays it on the screen, the user can follow the guide to complete the setup. Furthermore, it can receive operation instructions sent from the server in real time and display them to the user in an easy-to-understand manner. Once the user completes the operation, the device sends progress information to the server. The device also sends questions entered by the user to the server, receives the answers from the server again, and displays them to the user.

[2175] The role of the emotional engine

[2176] The device uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions, voice tone, touch operation speed, etc., to determine the current emotional state. For example, if the user is irritated, the emotion engine recognizes this and notifies the server. Based on the user's emotional information obtained from the emotion engine, the server adjusts the operation procedures and support information as needed. For example, if the user is irritated, the server may simplify the operation procedures or add encouraging or supportive messages. The operation procedures and responses generated by the server are customized considering the user's emotional state. This allows the user to proceed more smoothly and reduce frustration.

[2177] Program processing and required hardware and software

[2178] To implement this system, the following hardware and software are required:

[2179] Hardware: Smartphones, tablets

[2180] Software: Python, Emotion Recognition module, Generative AI Chatbot module (AIChatBot)

[2181] The server uses generative artificial intelligence to generate initial setup templates and processes HTTP requests for delivery to terminals. The server also generates operation procedures for specific tasks in real time, accepts user questions, and generates answers using a generative AI chatbot. The emotion engine analyzes the user's emotional state using an emotion recognition module and adjusts support accordingly.

[2182] As a concrete example, the following steps are shown for a user who wants to set up Wi-Fi during the initial setup:

[2183] (Example prompt): "Please generate Wi-Fi setup instructions for my new smartphone."

[2184] Based on this, the following Wi-Fi setup instructions were generated:

[2185] Open the home screen and tap the Settings app.

[2186] Select Wi-Fi and tap Add New Network.

[2187] Enter the network name and then the password.

[2188] Tap the connect button to complete the setup.

[2189] In this way, users can easily proceed with settings and operations, and support tailored to their emotional state is provided, thus improving the user experience.

[2190] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2191] Step 1:

[2192] The server uses a generative AI model to generate initial setup templates for specific smart devices. The input includes the device type and a prompt message. For example, based ...

Claims

1. A method for generating initial setup templates for smart devices using generative artificial intelligence and distributing them to participants' devices, A means of generating operating procedures in real time using generative artificial intelligence and displaying them on the participants' devices, A means of receiving questions from participants, generating answers using generative artificial intelligence, and delivering them to the participants' devices, A means of monitoring the progress of participants' operations and providing additional support information as needed, A system that includes this.

2. The system according to claim 1, wherein the generated initial setup template includes network settings, account settings, and application installation procedures.

3. The system according to claim 1, wherein the generated operating procedure includes a step-by-step guide optimized for a specific task.

Images