System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving and storing user enrollment information in a database; means for receiving a challenge entered by a user and querying a generation and AI system; means for analyzing and processing an answer obtained from the generation and AI system; means for displaying the processed answer to the user; means for receiving and storing feedback from the user in a database; and means for analyzing the feedback and generating a new service offer.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern advanced artificial intelligence technology, especially generative AI, is convenient in many situations. However, its usage is complex, making it difficult for elderly users and users with low IT literacy. Another problem is the lack of effective feedback to improve the quality of answers provided by generative AI, and the lack of effective service improvements based on that feedback. The present invention aims to solve these problems by providing a system that effectively provides generative AI to a wider range of users and utilizes feedback. [Means for solving the problem]

[0005] The present invention provides a system including a means for receiving user registration information and storing it in a database, a means for receiving a task entered by a user and querying a generation AI system, a means for analyzing and processing an answer obtained from the generation AI system, a means for displaying the processed answer to the user, a means for receiving feedback from the user and storing it in a database, and a means for analyzing the feedback and generating a proposal for a new service. The present invention further includes a means for converting task data entered by a user into an appropriate API format and sending it to the generation AI system, and a means for adding warning messages and additional information to the answer from the generation AI system and processing it into a user-friendly format, thereby enabling users to easily use the generation AI.

[0006] "User registration information" refers to information required for authentication and identification to allow a user to access the system, and typically includes data such as name, email address, and password.

[0007] A "database" is a structured storage device and its management system for systematically managing information and efficiently storing and retrieving it.

[0008] "Issues" refer to problems, questions, or consultations that users want to solve, and are input data for analysis and answers by the generation AI.

[0009] A "generative AI system" is a system equipped with artificial intelligence technology that automatically generates text data and has the ability to provide optimal answers based on user input.

[0010] "Analysis" refers to the process of converting the data returned by a generative AI system into an understandable and usable format, and of formatting and extracting answers.

[0011] "Processing" refers to the process of further intentionally manipulating the analyzed data to convert it into a format that is easy for users to understand and use.

[0012] "Display" refers to the act of outputting data to a screen on a terminal so that the user can visually confirm it.

[0013] "Feedback" refers to data such as evaluations, impressions, and suggestions for improvement that users provide in response to answers obtained from a generative AI system.

[0014] "New service proposals" refers to proposing new ideas or features to improve existing services based on collected feedback and analysis results.

[0015] "API format" refers to the data format and structure required for communication with an application programming interface (API), which is a standardized communication protocol. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that uses generative AI to enable elderly people and users with low IT literacy to easily solve problems. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI about problems entered by the user, a part that analyzes and processes the generated answers and displays them to the user, and a part that collects and analyzes user feedback to propose new services.

[0038] System Programming and Processing

[0039] 1. User Registration

[0040] A user accesses the system and registers by entering information such as name, email address, and password, and then clicking the registration button.

[0041] The terminal sends this input information to the server.

[0042] The server stores the received user registration information in a database.

[0043] 2. Inputting the problem and querying the generation AI

[0044] The user enters a specific problem (e.g., "I want to know how to shop online") into an input form on the system.

[0045] The terminal sends this assignment data to the server.

[0046] The server converts the problem data into the appropriate API format and sends it to the generative AI system.

[0047] 3. Obtaining and displaying answers

[0048] The server receives the answer from the generative AI system.

[0049] The server parses this response and formats it in a user-friendly way, adding warnings and additional information as needed.

[0050] The terminal displays the processed answer to the user.

[0051] 4. Collect and use feedback

[0052] The user enters feedback about the answer from the generating AI (e.g., usefulness of the answer and areas for improvement).

[0053] The device sends this feedback to the server.

[0054] The server stores the received feedback in a database and analyzes it.

[0055] The server generates new service proposals and improvement plans based on the analysis results and reflects them in the system.

[0056] Specific examples

[0057] User registration: To use the service, elderly person A creates an account by entering their name, email address, and password. The device sends this information to the server, which then stores it in a database.

[0058] Task input: Person A inputs, "I want to know how to shop online." The device sends this task information to the server, which then queries the generation AI system.

[0059] Obtaining and displaying answers: The AI ​​generation system provides specific online shopping instructions, which are received by the server, analyzed, and processed. Any necessary warnings (e.g., warning against phishing scams) are attached and sent back to the device, where they are displayed to User A.

[0060] Collecting and utilizing feedback: User A uses the provided information to shop online and enters feedback about the ease of use. The device sends this to the server, which analyzes the feedback and uses it to improve the service next time.

[0061] The system of the present invention allows users to easily use the generated AI through these processing steps, and provides a mechanism for reflecting their usage experience as feedback in improving new services.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The user goes to the system access page and clicks the "New Registration" button.

[0065] Step 2:

[0066] The user enters their name, email address, and password and clicks the "Register" button.

[0067] Operation details: Information entered by the user is saved in an input form on the device.

[0068] Step 3:

[0069] The terminal sends the user's input information to the server.

[0070] What it does: An HTTP POST request containing the input data is generated and sent to the server.

[0071] Step 4:

[0072] The server stores the received user registration information in a database.

[0073] What it does: Validates user information, hashes passwords if necessary, and stores them securely in a database.

[0074] Step 5:

[0075] The server sends a message to the terminal indicating that the user registration was successful.

[0076] What it does: Generates and sends a response containing a success message and a link to the user's account page.

[0077] Step 6:

[0078] The terminal displays a registration completion message to the user.

[0079] Action details: A message will be displayed in the browser and a link will be provided to guide the user through the next steps.

[0080] Step 7:

[0081] The user goes to the "Enter Problem" page and enters the problem they want to solve (e.g., "I want to know how to shop online").

[0082] Step 8:

[0083] The user enters the assignment details and clicks the "Submit" button.

[0084] Operation details: The assignment content is saved in the input form.

[0085] Step 9:

[0086] The terminal sends the entered assignment data to the server.

[0087] What it does: Creates an HTTP POST request containing the issue data and sends it to the server.

[0088] Step 10:

[0089] The server analyzes the received task data and converts it into an API format suitable for the generative AI system.

[0090] Operational details: Through a data conversion routine, the task data is formatted into a format that can be understood by the generative AI system.

[0091] Step 11:

[0092] The server sends the converted task data to the generation AI system.

[0093] Operation details: Generates an API request and sends it to the generation AI system.

[0094] Step 12:

[0095] The generative AI system generates answers to the tasks and sends them back to the server.

[0096] Step 13:

[0097] The server receives the answer from the generative AI system.

[0098] Operation details: Receives the response as JSON format data.

[0099] Step 14:

[0100] The server parses the received response and processes it into a user-friendly format.

[0101] Operation details: Depending on the content of the answer, warning messages and additional information are added and the final display format is created.

[0102] Step 15:

[0103] The server then sends the processed response data to the terminal.

[0104] Operation details: Generates HTML content to display to the user and returns it as a response.

[0105] Step 16:

[0106] The terminal displays the response received from the server to the user.

[0107] Operation details: Displays HTML content in the browser so that the user can visually confirm it.

[0108] Step 17:

[0109] The user acts based on the answers and enters the results and impressions as feedback.

[0110] Step 18:

[0111] The user enters their feedback and clicks the "Submit" button.

[0112] Operation details: Feedback content is saved in the input form.

[0113] Step 19:

[0114] The device sends the feedback data to the server.

[0115] Operational details: An HTTP POST request containing the feedback data is generated and sent to the server.

[0116] Step 20:

[0117] The server stores the received feedback in a database.

[0118] Operation details: Analyzes feedback data and stores it in a database.

[0119] Step 21:

[0120] The server analyzes the collected feedback and generates new service proposals and improvement plans.

[0121] Operation details: Feedback data is processed using analytical tools to extract insights for service improvement.

[0122] Step 22:

[0123] The server reflects new service proposals and improvement ideas into the system.

[0124] How it works: We'll use your feedback to develop our next updates and new features.

[0125] Example 1

[0126] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0127] Currently, elderly people and users with low IT literacy face difficulties in solving the problems they encounter when using the Internet, and they often lack the ability to receive appropriate support. Therefore, there is a need for a system that allows these users to easily solve their problems.

[0128] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0129] In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving tasks entered by the user and querying the generative AI model, and means for analyzing and processing the answers obtained from the generative AI model. This allows appropriate answers to the tasks entered by the user to be provided quickly and in an easy-to-understand format, making it possible for even elderly people and users with low IT literacy to easily solve problems using the Internet.

[0130] "User registration information" refers to personal information such as name, email address, and password provided by a user to use the system.

[0131] A "database" is a system for systematically storing information and efficiently retrieving and managing it.

[0132] A "problem" is a specific problem or question that a user wants to solve.

[0133] A "generative AI model" is a system that uses artificial intelligence technology to generate answers to user problems.

[0134] "Analysis" is the process of examining and understanding the data and information obtained.

[0135] "Processing" refers to the process of converting the answers obtained into a form that is easy for the user to understand.

[0136] "Feedback" refers to information entered by a user regarding their impressions of the answers or services provided and suggestions for improvement.

[0137] An "API format" is a standardized data format used when exchanging data between different systems.

[0138] A "warning message" is a message that alerts the user.

[0139] "Additional information" refers to supplemental information added to the basic answer.

[0140] This invention is a system that uses a generative AI model to enable elderly people and users with low IT literacy to easily solve problems. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI model about problems entered by the user, a part that analyzes and processes the generated answers and displays them to the user, and a part that collects and analyzes user feedback to propose new services.

[0141] The specific implementation of the system is configured as follows depending on the conditions:

[0142] Hardware and Software Configuration

[0143] Server: Manages user registration information, task data, generative AI model queries, and feedback. Specifically, it includes a database management system (DBMS) and an API server.

[0144] Device: The device a user uses to access the system, enter information, and view responses, such as a smartphone, tablet, or personal computer.

[0145] Generative AI models: For example, advanced natural language processing models such as OpenAI's GPT-3 are used to analyze the challenges and generate answers.

[0146] Data processing and calculation

[0147] 1. Processing of User Registration Information

[0148] The terminal displays the user registration information (name, email address, password) in an input form, and the user enters it.

[0149] The device sends the entered registration information to the server as an HTTP POST request.

[0150] The server validates the received registration information and stores it in the database.

[0151] 2. Processing of issue data

[0152] A user enters a problem into an input form (e.g., "I want to know how to shop online").

[0153] The device sends this assignment data to the server as an HTTP POST request.

[0154] The server converts the problem data into the appropriate API format (e.g., JSON) and sends it to the generative AI model.

[0155] 3. Processing the generated answers

[0156] The server receives the answer from the generative AI model.

[0157] The server parses the received response and processes it in a user-friendly format, for example adding warnings or additional information.

[0158] The terminal displays the processed answer to the user.

[0159] 4. Gathering Feedback

[0160] The user provides feedback on the answer.

[0161] The device sends the feedback data to the server as an HTTP POST request.

[0162] The server stores the feedback data in a database, analyzes it, and generates new service proposals based on the analysis results, which are reflected in the system.

[0163] Specific examples

[0164] User registration: Elderly person A accesses the system and creates an account by entering his / her name, email address, and password. The device sends this information to the server, which stores it in a database.

[0165] Task input: Person A inputs "I want to know how to shop online." The device sends this task data to the server, which converts the data to query the generative AI model.

[0166] Obtaining and displaying answers: The generative AI model provides specific online shopping instructions, which the server analyzes and processes. The server then sends the instructions back to the device along with any necessary warnings (e.g., warning against phishing scams) and displays them to User A.

[0167] Feedback collection: User A makes an online purchase based on the provided information and enters feedback about the ease of use. The device sends this feedback to the server, which analyzes it and uses it to improve the service next time.

[0168] Prompt Sentence Examples

[0169] "I'm a senior citizen who isn't very tech-savvy. I'd like to start shopping online. Can you please tell me the specific steps?"

[0170] "I can't send emails properly. Please tell me the reason and how to solve it."

[0171] Through these detailed processing steps, this system makes it easier for elderly people and users with low IT literacy to use generated AI models to solve problems. Furthermore, by improving the service based on feedback, it is possible to provide a better user experience.

[0172] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0173] Step 1:

[0174] Input form display

[0175] When a user accesses the service, the terminal displays an input form for user registration on the screen.

[0176] Input: None

[0177] Output: A form for user registration

[0178] Step 2:

[0179] Enter user information

[0180] The user enters information such as name, email address, and password, and clicks the Register button.

[0181] Input: Name, Email Address, Password

[0182] Output: User registration information entered

[0183] Step 3:

[0184] Sending information

[0185] The device sends this input information to the server as an HTTP POST request.

[0186] Input: User registration information entered

[0187] Output: User registration information sent to the server

[0188] Step 4:

[0189] Database storage

[0190] The server performs validation checks on the user information received, for example, checking the format of the email address or the strength of the password.

[0191] The server saves the validated user information in the database.

[0192] Input: User registration information sent to the server

[0193] Output: User registration information stored in the database

[0194] Step 5:

[0195] Display assignment input form

[0196] A user logs in and accesses the assignment entry form.

[0197] The device will display the assignment entry form on the screen.

[0198] Input: None

[0199] Output: Assignment input form

[0200] Step 6:

[0201] Enter assignment details

[0202] The user enters the problem they want to solve (e.g., "I want to know how to shop online").

[0203] Input: Assignment details

[0204] Output: The input assignment

[0205] Step 7:

[0206] Submitting assignment data

[0207] The device sends this assignment data to the server as an HTTP POST request.

[0208] Input: The input assignment content

[0209] Output: Issue data sent to the server

[0210] Step 8:

[0211] API format conversion

[0212] The server converts the issue data into the appropriate API format (e.g., JSON).

[0213] The server sends the converted data to the generative AI model as an HTTP POST request.

[0214] Input: Issue data sent to the server

[0215] Output: Problem data converted into a format suitable for the generative AI model

[0216] Step 9:

[0217] Receiving a response

[0218] The server receives the answer as an HTTP response from the generative AI model.

[0219] Input: Approached issue data

[0220] Output: Answer from the generative AI model

[0221] Step 10:

[0222] Analysis and processing of responses

[0223] The server analyzes the received response and processes it in a more user-friendly way, for example by adding bullet points or warnings.

[0224] Input: Answer from a generative AI model

[0225] Output: A user-friendly, processed answer

[0226] Step 11:

[0227] Submitting and viewing answers

[0228] The server then sends the processed response to the terminal.

[0229] The device displays the received response on the screen.

[0230] Input: User-friendly edited answer

[0231] Output: The answer displayed on the user's terminal

[0232] Step 12:

[0233] Feedback input form display

[0234] The user accesses the feedback input form to provide feedback on the answers.

[0235] The device will display a feedback input form on the screen.

[0236] Input: None

[0237] Output: Feedback input form

[0238] Step 13:

[0239] Enter your feedback

[0240] Users provide feedback on the usefulness of the answer and how it can be improved.

[0241] Input: Feedback

[0242] Output: The input feedback

[0243] Step 14:

[0244] Sending feedback data

[0245] The device sends the feedback data to the server as an HTTP POST request.

[0246] Input: Feedback content entered

[0247] Output: Feedback data sent to the server

[0248] Step 15:

[0249] Feedback storage and analysis

[0250] The server stores the received feedback data in a database.

[0251] The server analyzes the saved feedback data and uses it to improve the service next time.

[0252] Input: Feedback data sent to the server

[0253] Output: Feedback data and analysis results stored in a database

[0254] (Application example 1)

[0255] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0256] Elderly people and users with low IT literacy face the challenge of finding products or asking questions about how to use them in physical stores. This often prevents them from fully enjoying their shopping experience and leaves them feeling stressed. The purpose of this invention is to provide these users with appropriate support using generative AI, making it easier for them to solve their problems.

[0257] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0258] In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving tasks entered by the user and querying the AI ​​generation system, and means for analyzing and processing the answers obtained from the AI ​​generation system. This enables users to easily enter tasks in a physical store using voice recognition and voice synthesis and receive appropriate answers in the form of voice and text.

[0259] "User registration information" refers to basic information such as name, email address, and password provided by the user to use the system.

[0260] "Database" means a digital storage system for storing and managing data such as user registration information and feedback.

[0261] A "generative AI system" is an artificial intelligence system that uses natural language processing to generate answers to questions entered by users.

[0262] "Speech recognition means" is a technology that converts a user's voice into text, thereby enabling tasks entered by voice to be sent to the generation AI system as text information.

[0263] "Speech synthesis means" refers to a technology that reproduces the answer obtained from the generative AI system as voice, and presents the answer in a way that is easy for the user to understand.

[0264] "Feedback" refers to opinions and impressions that users enter about the usefulness and areas for improvement of answers provided by the AI ​​generation.

[0265] An "API format" is a standardized data structure for sending data in an appropriate format to a generative AI system.

[0266] "Warnings and additional information" refers to notes and supplementary information that should be provided to users in addition to the answers obtained from the generative AI system.

[0267] A "user-friendly format" is a format in which information is processed and presented in a way that makes it easy for users to understand.

[0268] The present invention is a system for assisting users in smoothly searching for and using products in physical stores. This system is particularly useful for elderly people and users with low IT literacy, and is designed to enable them to easily obtain information. Specific embodiments for implementing the present invention are described below.

[0269] System configuration

[0270] 1. Hardware

[0271] Server: A central processing unit that manages the database and interacts with the generative AI system.

[0272] Smartphone (user's device): A device on which the user enters assignments and receives answers.

[0273] 2. Software

[0274] Firebase Authentication: Software for user authentication.

[0275] Firebase Firestore: A database for storing user registration information and feedback.

[0276] OpenAI's GPT-3 API: A generative AI system that generates answers in natural language to user challenges.

[0277] Google Cloud Speech-to-Text API: A speech recognition technology that converts user speech into text.

[0278] Google Cloud Text-to-Speech API: A speech synthesis technology that plays back answers obtained from generative AI systems.

[0279] Program processing

[0280] 1. User Registration

[0281] A user creates an account by entering basic information (name, email address, password) on a smartphone. The smartphone sends this information to the server, which authenticates it using Firebase Authentication and stores it in Firebase Firestore.

[0282] 2. Inputting the problem and querying the generation AI

[0283] The user uses the voice recognition function on their smartphone to input a specific question (e.g., "Where is the shelf for this product?"), which is then converted into text using the Google Cloud Speech-to-Text API and sent to the server.

[0284] The server converts this problem data into the appropriate API format and sends it to OpenAI's GPT-3 API to retrieve the answer.

[0285] 3. Obtaining and displaying answers

[0286] The server receives the answers from the AI ​​generation system, analyzes and processes them, and converts them into a user-friendly format, adding warnings and additional information if necessary.

[0287] The smartphone uses the Google Cloud Text-to-Speech API to present this information to the user in voice and text.

[0288] 4. Collect and use feedback

[0289] The user provides feedback on the generated AI's answers via voice or text, and the smartphone converts this feedback into text using the Google Cloud Speech-to-Text API and sends it to the server.

[0290] The server saves the feedback in Firebase Firestore and analyzes it. Based on the analysis results, new service proposals and feature improvements are made.

[0291] Specific examples

[0292] Prompt Sentence Examples

[0293] If the user is an elderly person using a smartphone for the first time and asks, "Where is this medicine cabinet?", the prompt to the generative AI would be:

[0294] User question: "Where is this medicine cabinet?"

[0295] Prompt for the AI ​​generator: "The exact location of the medicine shelves will vary depending on the layout of each store, but generally, medicines are often located towards the back or middle of the store."

[0296] Example of a GPT-3 response

[0297] GPT-3's answer: "Medicine shelves are typically located towards the back of the store, but this varies depending on the layout of a particular store, so please ask a store associate or refer to the store's signage for guidance."

[0298] Through these processing steps, the system of the present invention allows users to easily utilize the generated AI and improve their shopping experience.

[0299] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0300] Step 1:

[0301] A user creates an account by entering basic information (name, email address, password) on their smartphone. The smartphone sends this information to the server. The server authenticates using Firebase Authentication and stores the authenticated information in Firebase Firestore. The input of this step is user information, and the output is the stored user information.

[0302] Step 2:

[0303] The user uses the voice recognition function on their smartphone to input a specific task by voice. The smartphone converts this speech into text using the Google Cloud Speech-to-Text API and sends the generated text data to the server. The input of this step is voice data, and the output is text data.

[0304] Step 3:

[0305] The server receives the text data sent by the user and converts it into an API format suitable for the generative AI model. It then queries OpenAI's GPT-3 API. The input of this step is text data, and the output is the data converted into the API format.

[0306] Step 4:

[0307] The server receives the answer data obtained from the generative AI model (GPT-3). It analyzes the answer data and processes it into a user-friendly format with warnings and additional information. The input of this step is the answer data, and the output is the processed answer data.

[0308] Step 5:

[0309] The smartphone receives the processed answer data sent from the server. Using the Google Cloud Text-to-Speech API, this information is played back to the user aloud and presented in text form. The input for this step is the processed answer data, and the output is the answer presented in audio and text form.

[0310] Step 6:

[0311] The user provides feedback on the generated AI's answer via voice or text. The smartphone again uses the Google Cloud Speech-to-Text API to convert the user's voice feedback into text and send it to the server. The input for this step is voice feedback, and the output is text-formatted feedback.

[0312] Step 7:

[0313] The server receives user feedback data and stores it in Firebase Firestore. The received feedback is analyzed and new service suggestions and feature improvements are made based on the results. The input of this step is the feedback in text format, and the output is the analysis results and new service suggestions.

[0314] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0315] This invention is a system that uses generative AI to enable elderly people and users with low IT literacy to easily solve problems, and further provides a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI about the problem entered by the user, a part that analyzes and processes the generated answer and displays it to the user, a part that collects and analyzes user feedback and proposes new services, and a part that uses an emotion engine to recognize the user's emotions.

[0316] System Programming and Processing

[0317] 1. User Registration

[0318] A user accesses the system and registers by entering information such as name, email address, and password, and then clicking the registration button.

[0319] The terminal sends this input information to the server.

[0320] The server stores the received user registration information in a database.

[0321] 2. Inputting the problem and querying the generation AI

[0322] The user enters a specific problem (e.g., "I want to know how to shop online") into an input form on the system.

[0323] The terminal sends this assignment data to the server.

[0324] The server converts the problem data into the appropriate API format and sends it to the generative AI system.

[0325] 3. Emotion Recognition Using an Emotion Engine

[0326] The server uses an emotion engine based on the user's input tasks and feedback to recognize the user's emotions.

[0327] The server reflects the recognized emotion data in the generative AI system.

[0328] 4. Retrieving and displaying answers

[0329] The generative AI system generates answers to the tasks while taking into account the user's emotions and sends them back to the server.

[0330] The server receives the answer from the generative AI system.

[0331] The server parses this response and formats it in a user-friendly way, adding warnings and additional information as needed.

[0332] The terminal displays the processed answer to the user.

[0333] 5. Collect and use feedback

[0334] The user enters feedback about the answer from the generating AI (e.g., usefulness of the answer and areas for improvement).

[0335] The device sends this feedback to the server.

[0336] The server stores the received feedback in a database and analyzes it.

[0337] The server generates new service proposals and improvement plans based on the analysis results and reflects them in the system.

[0338] Specific examples

[0339] User registration: To use the service, elderly person A creates an account by entering their name, email address, and password. The device sends this information to the server, which then stores it in a database.

[0340] Task input: Person A inputs, "I want to know how to shop online." The device sends this task information to the server, which then queries the generation AI system.

[0341] Use of emotion recognition: The server uses an emotion engine based on Mr. A's input to recognize his emotion as "anxiety" and sends that data to the generative AI system.

[0342] Obtaining and displaying answers: The generative AI system provides specific online shopping procedures, which are received, analyzed, and processed by the server. The system then sends the instructions back to the device along with any necessary warnings (e.g., beware of phishing scams) and displays them to User A in a more reassuring manner.

[0343] Collecting and utilizing feedback: User A uses the provided information to shop online and enters feedback about the ease of use. The device sends this to the server, which analyzes the feedback and uses it to improve the service next time.

[0344] The system of the present invention allows users to easily use generative AI through these processing steps and utilizes an emotion engine to provide a more personalized experience. Collected feedback and emotion data are used to continuously improve the service, thereby increasing user satisfaction.

[0345] The processing flow will be explained below.

[0346] Step 1:

[0347] The user goes to the system access page and clicks the "New Registration" button.

[0348] Step 2:

[0349] The user enters their name, email address, and password and clicks the "Register" button.

[0350] Operation details: Information entered by the user is saved in an input form on the device.

[0351] Step 3:

[0352] The terminal sends the user's input information to the server.

[0353] What it does: An HTTP POST request containing the input data is generated and sent to the server.

[0354] Step 4:

[0355] The server stores the received user registration information in a database.

[0356] What it does: Validates user information, hashes passwords if necessary, and stores them securely in a database.

[0357] Step 5:

[0358] The server sends a message to the terminal indicating that the user registration was successful.

[0359] What it does: Generates and sends a response containing a success message and a link to the user's account page.

[0360] Step 6:

[0361] The terminal displays a registration completion message to the user.

[0362] Action details: A message will be displayed in the browser and a link will be provided to guide the user through the next steps.

[0363] Step 7:

[0364] The user goes to the "Enter Problem" page and enters the problem they want to solve (e.g., "I want to know how to shop online").

[0365] Step 8:

[0366] The user enters the assignment details and clicks the "Submit" button.

[0367] Operation details: The assignment content is saved in the input form.

[0368] Step 9:

[0369] The terminal sends the entered assignment data to the server.

[0370] What it does: Creates an HTTP POST request containing the issue data and sends it to the server.

[0371] Step 10:

[0372] The server analyzes the received task data and converts it into an API format suitable for the generative AI system.

[0373] Operational details: Through a data conversion routine, the task data is formatted into a format that can be understood by the generative AI system.

[0374] Step 11:

[0375] The server sends the converted task data to the generation AI system.

[0376] Operation details: Generates an API request and sends it to the generation AI system.

[0377] Step 12:

[0378] The generative AI system generates answers to the tasks and sends them back to the server.

[0379] Step 13:

[0380] The server receives the answer from the generative AI system.

[0381] Operation details: Receives the response as JSON format data.

[0382] Step 14:

[0383] The server parses the received response, adds warnings and additional information as needed, and processes it in a user-friendly format.

[0384] How it works: It uses an emotion engine to analyze the user's emotions and then formats the answer appropriately, taking into account the analysis results.

[0385] Step 15:

[0386] The server then sends the processed response data to the terminal.

[0387] Operation details: Generates HTML content to display to the user and returns it as a response.

[0388] Step 16:

[0389] The terminal displays the response received from the server to the user.

[0390] Operation details: Displays HTML content in the browser so that the user can visually confirm it.

[0391] Step 17:

[0392] The user acts based on the answers and enters the results and impressions as feedback.

[0393] Step 18:

[0394] The user enters their feedback and clicks the "Submit" button.

[0395] Operation details: Feedback content is saved in the input form.

[0396] Step 19:

[0397] The device sends the feedback data to the server.

[0398] Operational details: An HTTP POST request containing the feedback data is generated and sent to the server.

[0399] Step 20:

[0400] The server stores the received feedback in a database.

[0401] Operation details: Analyzes feedback data and stores it in a database.

[0402] Step 21:

[0403] The server analyzes the collected feedback and uses an emotion engine to generate new service proposals and improvement plans along with the user's emotional data.

[0404] How it works: Feedback and sentiment data is processed using analytics tools to extract insights for service improvement.

[0405] Step 22:

[0406] The server reflects new service proposals and improvement ideas into the system.

[0407] How it works: We use feedback and sentiment data to develop our next updates and new features.

[0408] Example 2

[0409] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0410] The purpose of this invention is to provide a system that enables elderly people and users with low IT literacy to easily solve problems. Another objective is to provide a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. This system reduces user stress and inconvenience and enables continuous improvement of services.

[0411] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving a task entered by the user and querying the generation AI system, means for analyzing and processing the answer obtained from the generation AI system, means for using an emotion engine that recognizes emotions from the user's input content and feedback, and means for reflecting the recognized emotion data in the generation AI system. This makes it possible to provide an appropriate response that takes emotions into consideration for the task entered by the user.

[0412] "User registration information" refers to basic information such as name, email address, and password entered by a user accessing the system for the first time.

[0413] The "database" is an information management system for storing received user registration information, assignment data, feedback, and the like.

[0414] A "problem" is a specific problem or question that a user wants to solve through the system.

[0415] A "generative AI system" is an artificial intelligence system that generates appropriate responses and information based on a task posed by the user.

[0416] An "emotion engine" is software that analyzes and recognizes emotions from user input and feedback.

[0417] "Analysis" is the process of breaking down received information, generated answers, and feedback to extract meaning and patterns.

[0418] "Processing" is the process of reconstructing analyzed information into a format that is easy for users to understand.

[0419] "Feedback" refers to the user's evaluation and opinion of the answer provided by the generative AI system.

[0420] "API format" means an appropriate data format used for data exchange with a generative AI system.

[0421] "Recognition" is the process of understanding the user's emotional state using an emotion engine.

[0422] An "appropriate response" is the optimal answer or suggestion provided by a generative AI system that is tailored to the user's challenges and emotions.

[0423] A "personalized experience" is the provision of customized services that respond to the user's individual feelings and needs.

[0424] This invention is a system that enables elderly users and users with low IT literacy to easily solve problems. This system provides a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. The system includes a component that receives and stores user registration information in a database, a component that queries a generation AI for the problem entered by the user, a component that analyzes and processes the generated answer and displays it to the user, a component that collects and analyzes user feedback to propose new services, and a component that uses an emotion engine to recognize the user's emotions.

[0425] In concrete terms, a user first accesses the system and registers. This requires information such as name, email address, and password. This information is sent from the terminal to the server and received by the server. The server verifies the received data and, if there are no problems, stores it in the database.

[0426] The user then enters the problem they want to solve into an input form on the system. For example, they might enter a problem like "I want to know how to shop online." This problem data is sent from the device to the server. The server converts the problem data into an appropriate API format and sends it to the generative AI system. In this case, OpenAI's GPT-3 or another model can be used as the generative AI model.

[0427] Furthermore, the server uses an emotion engine based on the user's input tasks and feedback to recognize the user's emotions. The emotion engine can be IBM's Watson Tone Analyzer, for example. The recognized emotion data is fed into a generative AI system, which generates answers that take the user's emotions into consideration.

[0428] The answer from the AI ​​generation system is sent back to the server. The server receives this answer data and analyzes and processes it. Specifically, it reconstructs the answer into a user-friendly format, adding, for example, a warning about phishing scams or additional information. This processed answer is then sent to the device and displayed to the user.

[0429] The user enters feedback about the answer provided. For example, the user may say, "The answer was helpful" or "I would like more detailed information." The feedback data is sent from the device to the server, which then stores it in a database. This stored feedback data is periodically analyzed and used to propose new services and improvements.

[0430] As a specific example, elderly person A accesses the system and creates an account by entering their name, email address, and password. When A enters, "I want to know how to shop online," the problem data is sent to the generative AI system. The server analyzes A's problem using an emotion engine and recognizes that A is feeling "anxious." The generative AI system's response, which takes emotion into consideration, is returned to the server and displayed to A with a warning message and additional information added. Finally, A enters feedback, which the server analyzes and uses to improve the service next time.

[0431] Prompt Sentence Examples

[0432] 1. "Please enter your name, email address, and password to register."

[0433] 2. "Please enter the problem you want to solve in the form below. Example: 'I want to know how to shop online.'"

[0434] 3. "Please provide feedback about the answer provided. For example, how helpful the answer was and how it could be improved."

[0435] Through these components and processing steps, the present invention makes generative AI easily accessible to users, and by utilizing an emotion engine, provides a more personalized experience. Collected feedback and emotion data are used to continuously improve the service.

[0436] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0437] Step 1: Receiving and storing user registration information

[0438] Input: The user enters their name, email address, and password.

[0439] process:

[0440] The user accesses the system's new registration screen, enters the required information, and clicks the "Register" button.

[0441] The terminal converts this input data into JSON format and sends the data to the server by sending an HTTP request.

[0442] The server receives the request and validates the data (e.g., checking the format of the email address, checking the strength of the password).

[0443] If the server passes the validation, it stores the user information in its database.

[0444] Output: Returns a user registration completion message to the user.

[0445] Step 2: Enter and submit your assignment

[0446] Input: The user enters the problem they want to solve (e.g., "I want to know how to shop online").

[0447] process:

[0448] The user enters the problem they want to solve into an input form on the system.

[0449] The device sends the entered assignment data to the server in JSON format.

[0450] The server converts the problem data into the appropriate API format (e.g., for OpenAI GPT-3).

[0451] Output: Issue data converted to API format is sent to the generation AI system.

[0452] Step 3: Performing emotion recognition

[0453] Input: Assignment and past feedback data entered by the user.

[0454] process:

[0455] The server sends the task input data and feedback data to the emotion engine.

[0456] The server analyzes the emotion data obtained from the emotion engine and understands the user's current emotions (e.g., "anxiety," "confusion," etc.).

[0457] Output: The recognized emotion data is fed back to the generative AI system.

[0458] Step 4: Generative AI system generates answers

[0459] Input: Issue data and sentiment data.

[0460] process:

[0461] The generative AI system generates answers to tasks that take into account the user's emotions based on task data and emotion data.

[0462] Output: The generated answer data is sent back to the server.

[0463] Step 5: Analyze, process and display responses

[0464] Input: Answer data obtained from the generative AI system.

[0465] process:

[0466] The server analyzes the response data received from the generation AI system and reconstructs it into a user-friendly format (e.g., adding specific examples and warnings).

[0467] The server then sends the processed response data to the terminal.

[0468] The terminal displays the received response data on the user interface.

[0469] Output: The answer information displayed to the user.

[0470] Step 6: Collect and analyze feedback

[0471] Input: User feedback data (e.g., "The answer was helpful," "I'd like more information").

[0472] process:

[0473] The user enters feedback about the answers provided.

[0474] The device sends the input feedback data in JSON format to the server.

[0475] The server stores the received feedback data in a database.

[0476] The server periodically analyzes the stored feedback data.

[0477] Output: Based on the analysis results, new service proposals and improvement plans are generated and reflected in the system.

[0478] Specific examples

[0479] As a specific example, elderly person A accesses the system and creates an account by entering their name, email address, and password. When A enters, "I want to know how to shop online," the problem data is sent to the generative AI system. The server analyzes A's problem using an emotion engine and recognizes that A is feeling "anxious." The generative AI system's response, which takes emotion into consideration, is returned to the server and displayed to A with a warning message and additional information added. Finally, A enters feedback, which the server analyzes and uses to improve the service next time.

[0480] (Application example 2)

[0481] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0482] There is a problem that elderly people and users with low IT literacy have difficulty shopping smoothly in physical stores. In addition, conventional systems lack personalized responses that take into account the user's emotions, making it difficult to improve user satisfaction.

[0483] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0484] In this invention, the server includes a means for receiving user registration information and storing it in a database, a means for receiving tasks entered by the user and querying the AI ​​system, and a means for analyzing and processing the answers obtained from the AI ​​system. This allows even elderly people and users with low IT literacy to smoothly shop in physical stores. Furthermore, by including a means for recognizing user emotions using an emotion recognition engine and reflecting that data in the AI ​​system, and a means for providing product location information and detailed information in physical stores, a more personalized experience can be provided.

[0485] "User registration information" refers to basic personal information such as the system user's name, email address, and password.

[0486] A "database" is a system for systematically storing and managing collected user information, feedback, emotional data, etc.

[0487] A "generative AI system" refers to an artificial intelligence that generates a response to a task entered by a user.

[0488] An "emotion recognition engine" is software or hardware that analyzes emotions from user input and actions and reflects that data in the system.

[0489] "API format" refers to the standard data format used when exchanging data between systems.

[0490] "Means for analysis and processing" refers to the technology used to convert the answers obtained from the generative AI system into a user-friendly format and add warnings or additional information as necessary.

[0491] A "warning" is a statement that provides information about points that users should be aware of and risks.

[0492] "Additional information" refers to useful supplemental information added to the answer from the generative AI system.

[0493] "Product location information in physical stores" refers to information indicating where products are located in stores.

[0494] "Detailed information" refers to information that allows users to understand the product in detail, such as how to use it, its features, price, and related precautions.

[0495] The system of this invention is designed to support elderly people and users with low IT literacy, with the aim of making the shopping experience in brick-and-mortar stores smoother and more personalized. The specific system configuration and program processing are described below.

[0496] System configuration

[0497] Hardware used:

[0498] Smartphone (Android / iOS)

[0499] Software used:

[0500] Development environment: Android Studio, Xcode

[0501] Emotion recognition engine: Microsoft Azure's "Emotion API"

[0502] Database: Firebase Realtime Database

[0503] Generation AI: OpenAI API

[0504] What the program does

[0505] The system consists of the following main processing steps:

[0506] User Registration:

[0507] When a user launches the app for the first time, they are prompted to create an account by entering their name, email address, and password, which is automatically stored in the Firebase Realtime Database.

[0508] Inputting the issue and querying the AI:

[0509] When a user enters a specific task in the app (e.g., "Where is the rice?"), this task data is first sent to Firebase, which then converts the data into the appropriate API format and queries the OpenAI API.

[0510] Emotion recognition:

[0511] The task data and feedback entered by the user are analyzed for emotions using Microsoft Azure's Emotion API. This emotional data is reflected in the generative AI and used to generate answers.

[0512] Get and display answers:

[0513] The answers obtained from the generative AI system are analyzed and processed on the server, and then displayed on the smartphone in a user-friendly format. Warning messages and additional information may be added as needed.

[0514] Specific example explanation

[0515] Example 1: User registration

[0516] A user enters their name, email address, and password in the in-app registration form and clicks the "Register" button. The registration information is stored in the Firebase Realtime Database.

[0517] Example 2: Entering an assignment

[0518] The user types in "Where is the rice?" and submits it. The server converts this task data into the appropriate API format and queries the OpenAI API.

[0519] Example 3: Emotion Recognition

[0520] The Emotion API analyzes the emotion of the task entered by the user and recognizes it as "anxiety." This emotion data is used as feedback for the generative AI's answer generation process.

[0521] Example 4: Displaying answers

[0522] The AI ​​system generates a response such as, "The rice is in aisle 3 of the food aisle. If you can't find it, please ask a member of staff." This is received by the server, analyzed and processed, and then displayed in a user-friendly format on the smartphone screen.

[0523] Prompt Sentence Examples

[0524] User prompt: "Where is the rice?"

[0525] Emotion: "Anxiety"

[0526] In this way, by combining generative AI and emotion recognition technology throughout the system, we can provide users with a more personalized shopping experience.

[0527] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0528] Step 1: User Registration

[0529] When the user first starts up the device, they enter their name, email address, and password. The device sends this information to the Firebase Realtime Database. The server then stores the received user registration information in the database. This saves the registered user's information and makes it available for later processing.

[0530] Step 2: Enter your assignment

[0531] A user uses an input form within the app to enter a specific task, such as "Where is the rice?", and clicks the submit button. The device then sends this task data to the server, which converts it into the appropriate API format and queries the OpenAI API. This process passes the user's task to the generative AI system.

[0532] Step 3: Emotion Recognition

[0533] The server receives the task data entered by the user and performs emotion analysis using Microsoft Azure's Emotion API. For example, the task data may be recognized as containing the emotion "anxiety." The server then reflects this emotion data in the generative AI system. This allows the user's emotions to be taken into account when generating answers.

[0534] Step 4: Generative AI generates answers

[0535] The server sends a request containing task data and emotion data to the OpenAI API and receives a response from the generative AI model. The generative AI system generates an answer that reflects the user's emotion. For example, the generated answer might be, "The rice is in aisle 3 of the food aisle. If you can't find it, please ask a staff member."

[0536] Step 5: Analyze and process the responses

[0537] The server receives the answers sent by the AI ​​generation system and processes them into a user-friendly format by adding warnings about phishing scams and other scams, as well as additional information. This processing involves analyzing the content of the answers and adding appropriate additional information. For example, a warning about phishing scams may be added.

[0538] Step 6: View your answers

[0539] The server then sends the processed response to the user's smartphone, which then displays the response to the user, including any necessary warnings or additional information. This step provides the information in a format that is easy for the user to understand.

[0540] Step 7: Gather feedback

[0541] Users can enter and submit feedback about the answers provided within the app. The device then sends the feedback data to the server. The server stores the received feedback in a database and analyzes it for use in improving the service next time. For example, feedback entered by a user as "easy to use" regarding an answer is stored and analyzed.

[0542] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0543] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0544] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0545] [Second embodiment]

[0546] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0547] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0548] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0549] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0550] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0551] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0552] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0553] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0554] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0555] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0556] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0557] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0558] This invention is a system that uses generative AI to enable elderly people and users with low IT literacy to easily solve problems. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI about problems entered by the user, a part that analyzes and processes the generated answers and displays them to the user, and a part that collects and analyzes user feedback to propose new services.

[0559] System Programming and Processing

[0560] 1. User Registration

[0561] A user accesses the system and registers by entering information such as name, email address, and password, and then clicking the registration button.

[0562] The terminal sends this input information to the server.

[0563] The server stores the received user registration information in a database.

[0564] 2. Inputting the problem and querying the generation AI

[0565] The user enters a specific problem (e.g., "I want to know how to shop online") into an input form on the system.

[0566] The terminal sends this assignment data to the server.

[0567] The server converts the problem data into the appropriate API format and sends it to the generative AI system.

[0568] 3. Obtaining and displaying answers

[0569] The server receives the answer from the generative AI system.

[0570] The server parses this response and formats it in a user-friendly way, adding warnings and additional information as needed.

[0571] The terminal displays the processed answer to the user.

[0572] 4. Collect and use feedback

[0573] The user enters feedback about the answer from the generating AI (e.g., usefulness of the answer and areas for improvement).

[0574] The device sends this feedback to the server.

[0575] The server stores the received feedback in a database and analyzes it.

[0576] The server generates new service proposals and improvement plans based on the analysis results and reflects them in the system.

[0577] Specific examples

[0578] User registration: To use the service, elderly person A creates an account by entering their name, email address, and password. The device sends this information to the server, which then stores it in a database.

[0579] Task input: Person A inputs, "I want to know how to shop online." The device sends this task information to the server, which then queries the generation AI system.

[0580] Obtaining and displaying answers: The AI ​​generation system provides specific online shopping instructions, which are received by the server, analyzed, and processed. Any necessary warnings (e.g., warning against phishing scams) are attached and sent back to the device, where they are displayed to User A.

[0581] Collecting and utilizing feedback: User A uses the provided information to shop online and enters feedback about the ease of use. The device sends this to the server, which analyzes the feedback and uses it to improve the service next time.

[0582] The system of the present invention allows users to easily use the generated AI through these processing steps, and provides a mechanism for reflecting their usage experience as feedback in improving new services.

[0583] The processing flow will be explained below.

[0584] Step 1:

[0585] The user goes to the system access page and clicks the "New Registration" button.

[0586] Step 2:

[0587] The user enters their name, email address, and password and clicks the "Register" button.

[0588] Operation details: Information entered by the user is saved in an input form on the device.

[0589] Step 3:

[0590] The terminal sends the user's input information to the server.

[0591] What it does: An HTTP POST request containing the input data is generated and sent to the server.

[0592] Step 4:

[0593] The server stores the received user registration information in a database.

[0594] What it does: Validates user information, hashes passwords if necessary, and stores them securely in a database.

[0595] Step 5:

[0596] The server sends a message to the terminal indicating that the user registration was successful.

[0597] What it does: Generates and sends a response containing a success message and a link to the user's account page.

[0598] Step 6:

[0599] The terminal displays a registration completion message to the user.

[0600] Action details: A message will be displayed in the browser and a link will be provided to guide the user through the next steps.

[0601] Step 7:

[0602] The user goes to the "Enter Problem" page and enters the problem they want to solve (e.g., "I want to know how to shop online").

[0603] Step 8:

[0604] The user enters the assignment details and clicks the "Submit" button.

[0605] Operation details: The assignment content is saved in the input form.

[0606] Step 9:

[0607] The terminal sends the entered assignment data to the server.

[0608] What it does: Creates an HTTP POST request containing the issue data and sends it to the server.

[0609] Step 10:

[0610] The server analyzes the received task data and converts it into an API format suitable for the generative AI system.

[0611] Operational details: Through a data conversion routine, the task data is formatted into a format that can be understood by the generative AI system.

[0612] Step 11:

[0613] The server sends the converted task data to the generation AI system.

[0614] Operation details: Generates an API request and sends it to the generation AI system.

[0615] Step 12:

[0616] The generative AI system generates answers to the tasks and sends them back to the server.

[0617] Step 13:

[0618] The server receives the answer from the generative AI system.

[0619] Operation details: Receives the response as JSON format data.

[0620] Step 14:

[0621] The server parses the received response and processes it into a user-friendly format.

[0622] Operation details: Depending on the content of the answer, warning messages and additional information are added and the final display format is created.

[0623] Step 15:

[0624] The server then sends the processed response data to the terminal.

[0625] Operation details: Generates HTML content to display to the user and returns it as a response.

[0626] Step 16:

[0627] The terminal displays the response received from the server to the user.

[0628] Operation details: Displays HTML content in the browser so that the user can visually confirm it.

[0629] Step 17:

[0630] The user acts based on the answers and enters the results and impressions as feedback.

[0631] Step 18:

[0632] The user enters their feedback and clicks the "Submit" button.

[0633] Operation details: Feedback content is saved in the input form.

[0634] Step 19:

[0635] The device sends the feedback data to the server.

[0636] Operational details: An HTTP POST request containing the feedback data is generated and sent to the server.

[0637] Step 20:

[0638] The server stores the received feedback in a database.

[0639] Operation details: Analyzes feedback data and stores it in a database.

[0640] Step 21:

[0641] The server analyzes the collected feedback and generates new service proposals and improvement plans.

[0642] Operation details: Feedback data is processed using analytical tools to extract insights for service improvement.

[0643] Step 22:

[0644] The server reflects new service proposals and improvement ideas into the system.

[0645] How it works: We'll use your feedback to develop our next updates and new features.

[0646] Example 1

[0647] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0648] Currently, elderly people and users with low IT literacy face difficulties in solving the problems they encounter when using the Internet, and they often lack the ability to receive appropriate support. Therefore, there is a need for a system that allows these users to easily solve their problems.

[0649] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0650] In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving tasks entered by the user and querying the generative AI model, and means for analyzing and processing the answers obtained from the generative AI model. This allows appropriate answers to the tasks entered by the user to be provided quickly and in an easy-to-understand format, making it possible for even elderly people and users with low IT literacy to easily solve problems using the Internet.

[0651] "User registration information" refers to personal information such as name, email address, and password provided by a user to use the system.

[0652] A "database" is a system for systematically storing information and efficiently retrieving and managing it.

[0653] A "problem" is a specific problem or question that a user wants to solve.

[0654] A "generative AI model" is a system that uses artificial intelligence technology to generate answers to user problems.

[0655] "Analysis" is the process of examining and understanding the data and information obtained.

[0656] "Processing" refers to the process of converting the answers obtained into a form that is easy for the user to understand.

[0657] "Feedback" refers to information entered by a user regarding their impressions of the answers or services provided and suggestions for improvement.

[0658] An "API format" is a standardized data format used when exchanging data between different systems.

[0659] A "warning message" is a message that alerts the user.

[0660] "Additional information" refers to supplemental information added to the basic answer.

[0661] This invention is a system that uses a generative AI model to enable elderly people and users with low IT literacy to easily solve problems. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI model about problems entered by the user, a part that analyzes and processes the generated answers and displays them to the user, and a part that collects and analyzes user feedback to propose new services.

[0662] The specific implementation of the system is configured as follows depending on the conditions:

[0663] Hardware and Software Configuration

[0664] Server: Manages user registration information, task data, generative AI model queries, and feedback. Specifically, it includes a database management system (DBMS) and an API server.

[0665] Device: The device a user uses to access the system, enter information, and view responses, such as a smartphone, tablet, or personal computer.

[0666] Generative AI models: For example, advanced natural language processing models such as OpenAI's GPT-3 are used to analyze the challenges and generate answers.

[0667] Data processing and calculation

[0668] 1. Processing of User Registration Information

[0669] The terminal displays the user registration information (name, email address, password) in an input form, and the user enters it.

[0670] The device sends the entered registration information to the server as an HTTP POST request.

[0671] The server validates the received registration information and stores it in the database.

[0672] 2. Processing of issue data

[0673] A user enters a problem into an input form (e.g., "I want to know how to shop online").

[0674] The device sends this assignment data to the server as an HTTP POST request.

[0675] The server converts the problem data into the appropriate API format (e.g., JSON) and sends it to the generative AI model.

[0676] 3. Processing the generated answers

[0677] The server receives the answer from the generative AI model.

[0678] The server parses the received response and processes it in a user-friendly format, for example adding warnings or additional information.

[0679] The terminal displays the processed answer to the user.

[0680] 4. Gathering Feedback

[0681] The user provides feedback on the answer.

[0682] The device sends the feedback data to the server as an HTTP POST request.

[0683] The server stores the feedback data in a database, analyzes it, and generates new service proposals based on the analysis results, which are reflected in the system.

[0684] Specific examples

[0685] User registration: Elderly person A accesses the system and creates an account by entering his / her name, email address, and password. The device sends this information to the server, which stores it in a database.

[0686] Task input: Person A inputs "I want to know how to shop online." The device sends this task data to the server, which converts the data to query the generative AI model.

[0687] Obtaining and displaying answers: The generative AI model provides specific online shopping instructions, which the server analyzes and processes. The server then sends the instructions back to the device along with any necessary warnings (e.g., warning against phishing scams) and displays them to User A.

[0688] Feedback collection: User A makes an online purchase based on the provided information and enters feedback about the ease of use. The device sends this feedback to the server, which analyzes it and uses it to improve the service next time.

[0689] Prompt Sentence Examples

[0690] "I'm a senior citizen who isn't very tech-savvy. I'd like to start shopping online. Can you please tell me the specific steps?"

[0691] "I can't send emails properly. Please tell me the reason and how to solve it."

[0692] Through these detailed processing steps, this system makes it easier for elderly people and users with low IT literacy to use generated AI models to solve problems. Furthermore, by improving the service based on feedback, it is possible to provide a better user experience.

[0693] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0694] Step 1:

[0695] Input form display

[0696] When a user accesses the service, the terminal displays an input form for user registration on the screen.

[0697] Input: None

[0698] Output: A form for user registration

[0699] Step 2:

[0700] Enter user information

[0701] The user enters information such as name, email address, and password, and clicks the Register button.

[0702] Input: Name, Email Address, Password

[0703] Output: User registration information entered

[0704] Step 3:

[0705] Sending information

[0706] The device sends this input information to the server as an HTTP POST request.

[0707] Input: User registration information entered

[0708] Output: User registration information sent to the server

[0709] Step 4:

[0710] Database storage

[0711] The server performs validation checks on the user information received, for example, checking the format of the email address or the strength of the password.

[0712] The server saves the validated user information in the database.

[0713] Input: User registration information sent to the server

[0714] Output: User registration information stored in the database

[0715] Step 5:

[0716] Display assignment input form

[0717] A user logs in and accesses the assignment entry form.

[0718] The device will display the assignment entry form on the screen.

[0719] Input: None

[0720] Output: Assignment input form

[0721] Step 6:

[0722] Enter assignment details

[0723] The user enters the problem they want to solve (e.g., "I want to know how to shop online").

[0724] Input: Assignment details

[0725] Output: The input assignment

[0726] Step 7:

[0727] Submitting assignment data

[0728] The device sends this assignment data to the server as an HTTP POST request.

[0729] Input: The input assignment content

[0730] Output: Issue data sent to the server

[0731] Step 8:

[0732] API format conversion

[0733] The server converts the issue data into the appropriate API format (e.g., JSON).

[0734] The server sends the converted data to the generative AI model as an HTTP POST request.

[0735] Input: Issue data sent to the server

[0736] Output: Problem data converted into a format suitable for the generative AI model

[0737] Step 9:

[0738] Receiving a response

[0739] The server receives the answer as an HTTP response from the generative AI model.

[0740] Input: Approached issue data

[0741] Output: Answer from the generative AI model

[0742] Step 10:

[0743] Analysis and processing of responses

[0744] The server analyzes the received response and processes it in a more user-friendly way, for example by adding bullet points or warnings.

[0745] Input: Answer from a generative AI model

[0746] Output: A user-friendly, processed answer

[0747] Step 11:

[0748] Submitting and viewing answers

[0749] The server then sends the processed response to the terminal.

[0750] The device displays the received response on the screen.

[0751] Input: User-friendly edited answer

[0752] Output: The answer displayed on the user's terminal

[0753] Step 12:

[0754] Feedback input form display

[0755] The user accesses the feedback input form to provide feedback on the answers.

[0756] The device will display a feedback input form on the screen.

[0757] Input: None

[0758] Output: Feedback input form

[0759] Step 13:

[0760] Enter your feedback

[0761] Users provide feedback on the usefulness of the answer and how it can be improved.

[0762] Input: Feedback

[0763] Output: The input feedback

[0764] Step 14:

[0765] Sending feedback data

[0766] The device sends the feedback data to the server as an HTTP POST request.

[0767] Input: Feedback content entered

[0768] Output: Feedback data sent to the server

[0769] Step 15:

[0770] Feedback storage and analysis

[0771] The server stores the received feedback data in a database.

[0772] The server analyzes the saved feedback data and uses it to improve the service next time.

[0773] Input: Feedback data sent to the server

[0774] Output: Feedback data and analysis results stored in a database

[0775] (Application example 1)

[0776] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0777] Elderly people and users with low IT literacy face the challenge of finding products or asking questions about how to use them in physical stores. This often prevents them from fully enjoying their shopping experience and leaves them feeling stressed. The purpose of this invention is to provide these users with appropriate support using generative AI, making it easier for them to solve their problems.

[0778] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0779] In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving tasks entered by the user and querying the AI ​​generation system, and means for analyzing and processing the answers obtained from the AI ​​generation system. This enables users to easily enter tasks in a physical store using voice recognition and voice synthesis and receive appropriate answers in the form of voice and text.

[0780] "User registration information" refers to basic information such as name, email address, and password provided by the user to use the system.

[0781] "Database" means a digital storage system for storing and managing data such as user registration information and feedback.

[0782] A "generative AI system" is an artificial intelligence system that uses natural language processing to generate answers to questions entered by users.

[0783] "Speech recognition means" is a technology that converts a user's voice into text, thereby enabling tasks entered by voice to be sent to the generation AI system as text information.

[0784] "Speech synthesis means" refers to a technology that reproduces the answer obtained from the generative AI system as voice, and presents the answer in a way that is easy for the user to understand.

[0785] "Feedback" refers to opinions and impressions that users enter about the usefulness and areas for improvement of answers provided by the AI ​​generation.

[0786] An "API format" is a standardized data structure for sending data in an appropriate format to a generative AI system.

[0787] "Warnings and additional information" refers to notes and supplementary information that should be provided to users in addition to the answers obtained from the generative AI system.

[0788] A "user-friendly format" is a format in which information is processed and presented in a way that makes it easy for users to understand.

[0789] The present invention is a system for assisting users in smoothly searching for and using products in physical stores. This system is particularly useful for elderly people and users with low IT literacy, and is designed to enable them to easily obtain information. Specific embodiments for implementing the present invention are described below.

[0790] System configuration

[0791] 1. Hardware

[0792] Server: A central processing unit that manages the database and interacts with the generative AI system.

[0793] Smartphone (user's device): A device on which the user enters assignments and receives answers.

[0794] 2. Software

[0795] Firebase Authentication: Software for user authentication.

[0796] Firebase Firestore: A database for storing user registration information and feedback.

[0797] OpenAI's GPT-3 API: A generative AI system that generates answers in natural language to user challenges.

[0798] Google Cloud Speech-to-Text API: A speech recognition technology that converts user speech into text.

[0799] Google Cloud Text-to-Speech API: A speech synthesis technology that plays back answers obtained from generative AI systems.

[0800] Program processing

[0801] 1. User Registration

[0802] A user creates an account by entering basic information (name, email address, password) on a smartphone. The smartphone sends this information to the server, which authenticates it using Firebase Authentication and stores it in Firebase Firestore.

[0803] 2. Inputting the problem and querying the generation AI

[0804] The user uses the voice recognition function on their smartphone to input a specific question (e.g., "Where is the shelf for this product?"), which is then converted into text using the Google Cloud Speech-to-Text API and sent to the server.

[0805] The server converts this problem data into the appropriate API format and sends it to OpenAI's GPT-3 API to retrieve the answer.

[0806] 3. Obtaining and displaying answers

[0807] The server receives the answers from the AI ​​generation system, analyzes and processes them, and converts them into a user-friendly format, adding warnings and additional information if necessary.

[0808] The smartphone uses the Google Cloud Text-to-Speech API to present this information to the user in voice and text.

[0809] 4. Collect and use feedback

[0810] The user provides feedback on the generated AI's answers via voice or text, and the smartphone converts this feedback into text using the Google Cloud Speech-to-Text API and sends it to the server.

[0811] The server saves the feedback in Firebase Firestore and analyzes it. Based on the analysis results, new service proposals and feature improvements are made.

[0812] Specific examples

[0813] Prompt Sentence Examples

[0814] If the user is an elderly person using a smartphone for the first time and asks, "Where is this medicine cabinet?", the prompt to the generative AI would be:

[0815] User question: "Where is this medicine cabinet?"

[0816] Prompt for the AI ​​generator: "The exact location of the medicine shelves will vary depending on the layout of each store, but generally, medicines are often located towards the back or middle of the store."

[0817] Example of a GPT-3 response

[0818] GPT-3's answer: "Medicine shelves are typically located towards the back of the store, but this varies depending on the layout of a particular store, so please ask a store associate or refer to the store's signage for guidance."

[0819] Through these processing steps, the system of the present invention allows users to easily utilize the generated AI and improve their shopping experience.

[0820] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0821] Step 1:

[0822] A user creates an account by entering basic information (name, email address, password) on their smartphone. The smartphone sends this information to the server. The server authenticates using Firebase Authentication and stores the authenticated information in Firebase Firestore. The input of this step is user information, and the output is the stored user information.

[0823] Step 2:

[0824] The user uses the voice recognition function on their smartphone to input a specific task by voice. The smartphone converts this speech into text using the Google Cloud Speech-to-Text API and sends the generated text data to the server. The input of this step is voice data, and the output is text data.

[0825] Step 3:

[0826] The server receives the text data sent by the user and converts it into an API format suitable for the generative AI model. It then queries OpenAI's GPT-3 API. The input of this step is text data, and the output is the data converted into the API format.

[0827] Step 4:

[0828] The server receives the answer data obtained from the generative AI model (GPT-3). It analyzes the answer data and processes it into a user-friendly format with warnings and additional information. The input of this step is the answer data, and the output is the processed answer data.

[0829] Step 5:

[0830] The smartphone receives the processed answer data sent from the server. Using the Google Cloud Text-to-Speech API, this information is played back to the user aloud and presented in text form. The input for this step is the processed answer data, and the output is the answer presented in audio and text form.

[0831] Step 6:

[0832] The user provides feedback on the generated AI's answer via voice or text. The smartphone again uses the Google Cloud Speech-to-Text API to convert the user's voice feedback into text and send it to the server. The input for this step is voice feedback, and the output is text-formatted feedback.

[0833] Step 7:

[0834] The server receives user feedback data and stores it in Firebase Firestore. The received feedback is analyzed and new service suggestions and feature improvements are made based on the results. The input of this step is the feedback in text format, and the output is the analysis results and new service suggestions.

[0835] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0836] This invention is a system that uses generative AI to enable elderly people and users with low IT literacy to easily solve problems, and further provides a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI about the problem entered by the user, a part that analyzes and processes the generated answer and displays it to the user, a part that collects and analyzes user feedback and proposes new services, and a part that uses an emotion engine to recognize the user's emotions.

[0837] System Programming and Processing

[0838] 1. User Registration

[0839] A user accesses the system and registers by entering information such as name, email address, and password, and then clicking the registration button.

[0840] The terminal sends this input information to the server.

[0841] The server stores the received user registration information in a database.

[0842] 2. Inputting the problem and querying the generation AI

[0843] The user enters a specific problem (e.g., "I want to know how to shop online") into an input form on the system.

[0844] The terminal sends this assignment data to the server.

[0845] The server converts the problem data into the appropriate API format and sends it to the generative AI system.

[0846] 3. Emotion Recognition Using an Emotion Engine

[0847] The server uses an emotion engine based on the user's input tasks and feedback to recognize the user's emotions.

[0848] The server reflects the recognized emotion data in the generative AI system.

[0849] 4. Retrieving and displaying answers

[0850] The generative AI system generates answers to the tasks while taking into account the user's emotions and sends them back to the server.

[0851] The server receives the answer from the generative AI system.

[0852] The server parses this response and formats it in a user-friendly way, adding warnings and additional information as needed.

[0853] The terminal displays the processed answer to the user.

[0854] 5. Collect and use feedback

[0855] The user enters feedback about the answer from the generating AI (e.g., usefulness of the answer and areas for improvement).

[0856] The device sends this feedback to the server.

[0857] The server stores the received feedback in a database and analyzes it.

[0858] The server generates new service proposals and improvement plans based on the analysis results and reflects them in the system.

[0859] Specific examples

[0860] User registration: To use the service, elderly person A creates an account by entering their name, email address, and password. The device sends this information to the server, which then stores it in a database.

[0861] Task input: Person A inputs, "I want to know how to shop online." The device sends this task information to the server, which then queries the generation AI system.

[0862] Use of emotion recognition: The server uses an emotion engine based on Mr. A's input to recognize his emotion as "anxiety" and sends that data to the generative AI system.

[0863] Obtaining and displaying answers: The generative AI system provides specific online shopping procedures, which are received, analyzed, and processed by the server. The system then sends the instructions back to the device along with any necessary warnings (e.g., beware of phishing scams) and displays them to User A in a more reassuring manner.

[0864] Collecting and utilizing feedback: User A uses the provided information to shop online and enters feedback about the ease of use. The device sends this to the server, which analyzes the feedback and uses it to improve the service next time.

[0865] The system of the present invention allows users to easily use generative AI through these processing steps and utilizes an emotion engine to provide a more personalized experience. Collected feedback and emotion data are used to continuously improve the service, thereby increasing user satisfaction.

[0866] The processing flow will be explained below.

[0867] Step 1:

[0868] The user goes to the system access page and clicks the "New Registration" button.

[0869] Step 2:

[0870] The user enters their name, email address, and password and clicks the "Register" button.

[0871] Operation details: Information entered by the user is saved in an input form on the device.

[0872] Step 3:

[0873] The terminal sends the user's input information to the server.

[0874] What it does: An HTTP POST request containing the input data is generated and sent to the server.

[0875] Step 4:

[0876] The server stores the received user registration information in a database.

[0877] What it does: Validates user information, hashes passwords if necessary, and stores them securely in a database.

[0878] Step 5:

[0879] The server sends a message to the terminal indicating that the user registration was successful.

[0880] What it does: Generates and sends a response containing a success message and a link to the user's account page.

[0881] Step 6:

[0882] The terminal displays a registration completion message to the user.

[0883] Action details: A message will be displayed in the browser and a link will be provided to guide the user through the next steps.

[0884] Step 7:

[0885] The user goes to the "Enter Problem" page and enters the problem they want to solve (e.g., "I want to know how to shop online").

[0886] Step 8:

[0887] The user enters the assignment details and clicks the "Submit" button.

[0888] Operation details: The assignment content is saved in the input form.

[0889] Step 9:

[0890] The terminal sends the entered assignment data to the server.

[0891] What it does: Creates an HTTP POST request containing the issue data and sends it to the server.

[0892] Step 10:

[0893] The server analyzes the received task data and converts it into an API format suitable for the generative AI system.

[0894] Operational details: Through a data conversion routine, the task data is formatted into a format that can be understood by the generative AI system.

[0895] Step 11:

[0896] The server sends the converted task data to the generation AI system.

[0897] Operation details: Generates an API request and sends it to the generation AI system.

[0898] Step 12:

[0899] The generative AI system generates answers to the tasks and sends them back to the server.

[0900] Step 13:

[0901] The server receives the answer from the generative AI system.

[0902] Operation details: Receives the response as JSON format data.

[0903] Step 14:

[0904] The server parses the received response, adds warnings and additional information as needed, and processes it in a user-friendly format.

[0905] How it works: It uses an emotion engine to analyze the user's emotions and then formats the answer appropriately, taking into account the analysis results.

[0906] Step 15:

[0907] The server then sends the processed response data to the terminal.

[0908] Operation details: Generates HTML content to display to the user and returns it as a response.

[0909] Step 16:

[0910] The terminal displays the response received from the server to the user.

[0911] Operation details: Displays HTML content in the browser so that the user can visually confirm it.

[0912] Step 17:

[0913] The user acts based on the answers and enters the results and impressions as feedback.

[0914] Step 18:

[0915] The user enters their feedback and clicks the "Submit" button.

[0916] Operation details: Feedback content is saved in the input form.

[0917] Step 19:

[0918] The device sends the feedback data to the server.

[0919] Operational details: An HTTP POST request containing the feedback data is generated and sent to the server.

[0920] Step 20:

[0921] The server stores the received feedback in a database.

[0922] Operation details: Analyzes feedback data and stores it in a database.

[0923] Step 21:

[0924] The server analyzes the collected feedback and uses an emotion engine to generate new service proposals and improvement plans along with the user's emotional data.

[0925] How it works: Feedback and sentiment data is processed using analytics tools to extract insights for service improvement.

[0926] Step 22:

[0927] The server reflects new service proposals and improvement ideas into the system.

[0928] How it works: We use feedback and sentiment data to develop our next updates and new features.

[0929] Example 2

[0930] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0931] The purpose of this invention is to provide a system that enables elderly people and users with low IT literacy to easily solve problems. Another objective is to provide a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. This system reduces user stress and inconvenience and enables continuous improvement of services.

[0932] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving a task entered by the user and querying the generation AI system, means for analyzing and processing the answer obtained from the generation AI system, means for using an emotion engine that recognizes emotions from the user's input content and feedback, and means for reflecting the recognized emotion data in the generation AI system. This makes it possible to provide an appropriate response that takes emotions into consideration for the task entered by the user.

[0933] "User registration information" refers to basic information such as name, email address, and password entered by a user accessing the system for the first time.

[0934] The "database" is an information management system for storing received user registration information, assignment data, feedback, and the like.

[0935] A "problem" is a specific problem or question that a user wants to solve through the system.

[0936] A "generative AI system" is an artificial intelligence system that generates appropriate responses and information based on a task posed by the user.

[0937] An "emotion engine" is software that analyzes and recognizes emotions from user input and feedback.

[0938] "Analysis" is the process of breaking down received information, generated answers, and feedback to extract meaning and patterns.

[0939] "Processing" is the process of reconstructing analyzed information into a format that is easy for users to understand.

[0940] "Feedback" refers to the user's evaluation and opinion of the answer provided by the generative AI system.

[0941] "API format" means an appropriate data format used for data exchange with a generative AI system.

[0942] "Recognition" is the process of understanding the user's emotional state using an emotion engine.

[0943] An "appropriate response" is the optimal answer or suggestion provided by a generative AI system that is tailored to the user's challenges and emotions.

[0944] A "personalized experience" is the provision of customized services that respond to the user's individual feelings and needs.

[0945] This invention is a system that enables elderly users and users with low IT literacy to easily solve problems. This system provides a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. The system includes a component that receives and stores user registration information in a database, a component that queries a generation AI for the problem entered by the user, a component that analyzes and processes the generated answer and displays it to the user, a component that collects and analyzes user feedback to propose new services, and a component that uses an emotion engine to recognize the user's emotions.

[0946] In concrete terms, a user first accesses the system and registers. This requires information such as name, email address, and password. This information is sent from the terminal to the server and received by the server. The server verifies the received data and, if there are no problems, stores it in the database.

[0947] The user then enters the problem they want to solve into an input form on the system. For example, they might enter a problem like "I want to know how to shop online." This problem data is sent from the device to the server. The server converts the problem data into an appropriate API format and sends it to the generative AI system. In this case, OpenAI's GPT-3 or another model can be used as the generative AI model.

[0948] Furthermore, the server uses an emotion engine based on the user's input tasks and feedback to recognize the user's emotions. The emotion engine can be IBM's Watson Tone Analyzer, for example. The recognized emotion data is fed into a generative AI system, which generates answers that take the user's emotions into consideration.

[0949] The answer from the AI ​​generation system is sent back to the server. The server receives this answer data and analyzes and processes it. Specifically, it reconstructs the answer into a user-friendly format, adding, for example, a warning about phishing scams or additional information. This processed answer is then sent to the device and displayed to the user.

[0950] The user enters feedback about the answer provided. For example, the user may say, "The answer was helpful" or "I would like more detailed information." The feedback data is sent from the device to the server, which then stores it in a database. This stored feedback data is periodically analyzed and used to propose new services and improvements.

[0951] As a specific example, elderly person A accesses the system and creates an account by entering their name, email address, and password. When A enters, "I want to know how to shop online," the problem data is sent to the generative AI system. The server analyzes A's problem using an emotion engine and recognizes that A is feeling "anxious." The generative AI system's response, which takes emotion into consideration, is returned to the server and displayed to A with a warning message and additional information added. Finally, A enters feedback, which the server analyzes and uses to improve the service next time.

[0952] Prompt Sentence Examples

[0953] 1. "Please enter your name, email address, and password to register."

[0954] 2. "Please enter the problem you want to solve in the form below. Example: 'I want to know how to shop online.'"

[0955] 3. "Please provide feedback about the answer provided. For example, how helpful the answer was and how it could be improved."

[0956] Through these components and processing steps, the present invention makes generative AI easily accessible to users, and by utilizing an emotion engine, provides a more personalized experience. Collected feedback and emotion data are used to continuously improve the service.

[0957] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0958] Step 1: Receiving and storing user registration information

[0959] Input: The user enters their name, email address, and password.

[0960] process:

[0961] The user accesses the system's new registration screen, enters the required information, and clicks the "Register" button.

[0962] The terminal converts this input data into JSON format and sends the data to the server by sending an HTTP request.

[0963] The server receives the request and validates the data (e.g., checking the format of the email address, checking the strength of the password).

[0964] If the server passes the validation, it stores the user information in its database.

[0965] Output: Returns a user registration completion message to the user.

[0966] Step 2: Enter and submit your assignment

[0967] Input: The user enters the problem they want to solve (e.g., "I want to know how to shop online").

[0968] process:

[0969] The user enters the problem they want to solve into an input form on the system.

[0970] The device sends the entered assignment data to the server in JSON format.

[0971] The server converts the problem data into the appropriate API format (e.g., for OpenAI GPT-3).

[0972] Output: Issue data converted to API format is sent to the generation AI system.

[0973] Step 3: Performing emotion recognition

[0974] Input: Assignment and past feedback data entered by the user.

[0975] process:

[0976] The server sends the task input data and feedback data to the emotion engine.

[0977] The server analyzes the emotion data obtained from the emotion engine and understands the user's current emotions (e.g., "anxiety," "confusion," etc.).

[0978] Output: The recognized emotion data is fed back to the generative AI system.

[0979] Step 4: Generative AI system generates answers

[0980] Input: Issue data and sentiment data.

[0981] process:

[0982] The generative AI system generates answers to tasks that take into account the user's emotions based on task data and emotion data.

[0983] Output: The generated answer data is sent back to the server.

[0984] Step 5: Analyze, process and display responses

[0985] Input: Answer data obtained from the generative AI system.

[0986] process:

[0987] The server analyzes the response data received from the generation AI system and reconstructs it into a user-friendly format (e.g., adding specific examples and warnings).

[0988] The server then sends the processed response data to the terminal.

[0989] The terminal displays the received response data on the user interface.

[0990] Output: The answer information displayed to the user.

[0991] Step 6: Collect and analyze feedback

[0992] Input: User feedback data (e.g., "The answer was helpful," "I'd like more information").

[0993] process:

[0994] The user enters feedback about the answers provided.

[0995] The device sends the input feedback data in JSON format to the server.

[0996] The server stores the received feedback data in a database.

[0997] The server periodically analyzes the stored feedback data.

[0998] Output: Based on the analysis results, new service proposals and improvement plans are generated and reflected in the system.

[0999] Specific examples

[1000] As a specific example, elderly person A accesses the system and creates an account by entering their name, email address, and password. When A enters, "I want to know how to shop online," the problem data is sent to the generative AI system. The server analyzes A's problem using an emotion engine and recognizes that A is feeling "anxious." The generative AI system's response, which takes emotion into consideration, is returned to the server and displayed to A with a warning message and additional information added. Finally, A enters feedback, which the server analyzes and uses to improve the service next time.

[1001] (Application example 2)

[1002] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1003] There is a problem that elderly people and users with low IT literacy have difficulty shopping smoothly in physical stores. In addition, conventional systems lack personalized responses that take into account the user's emotions, making it difficult to improve user satisfaction.

[1004] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1005] In this invention, the server includes a means for receiving user registration information and storing it in a database, a means for receiving tasks entered by the user and querying the AI ​​system, and a means for analyzing and processing the answers obtained from the AI ​​system. This allows even elderly people and users with low IT literacy to smoothly shop in physical stores. Furthermore, by including a means for recognizing user emotions using an emotion recognition engine and reflecting that data in the AI ​​system, and a means for providing product location information and detailed information in physical stores, a more personalized experience can be provided.

[1006] "User registration information" refers to basic personal information such as the system user's name, email address, and password.

[1007] A "database" is a system for systematically storing and managing collected user information, feedback, emotional data, etc.

[1008] A "generative AI system" refers to an artificial intelligence that generates a response to a task entered by a user.

[1009] An "emotion recognition engine" is software or hardware that analyzes emotions from user input and actions and reflects that data in the system.

[1010] "API format" refers to the standard data format used when exchanging data between systems.

[1011] "Means for analysis and processing" refers to the technology used to convert the answers obtained from the generative AI system into a user-friendly format and add warnings or additional information as necessary.

[1012] A "warning" is a statement that provides information about points that users should be aware of and risks.

[1013] "Additional information" refers to useful supplemental information added to the answer from the generative AI system.

[1014] "Product location information in physical stores" refers to information indicating where products are located in stores.

[1015] "Detailed information" refers to information that allows users to understand the product in detail, such as how to use it, its features, price, and related precautions.

[1016] The system of this invention is designed to support elderly people and users with low IT literacy, with the aim of making the shopping experience in brick-and-mortar stores smoother and more personalized. The specific system configuration and program processing are described below.

[1017] System configuration

[1018] Hardware used:

[1019] Smartphone (Android / iOS)

[1020] Software used:

[1021] Development environment: Android Studio, Xcode

[1022] Emotion recognition engine: Microsoft Azure's "Emotion API"

[1023] Database: Firebase Realtime Database

[1024] Generation AI: OpenAI API

[1025] What the program does

[1026] The system consists of the following main processing steps:

[1027] User Registration:

[1028] When a user launches the app for the first time, they are prompted to create an account by entering their name, email address, and password, which is automatically stored in the Firebase Realtime Database.

[1029] Inputting the issue and querying the AI:

[1030] When a user enters a specific task in the app (e.g., "Where is the rice?"), this task data is first sent to Firebase, which then converts the data into the appropriate API format and queries the OpenAI API.

[1031] Emotion recognition:

[1032] The task data and feedback entered by the user are analyzed for emotions using Microsoft Azure's Emotion API. This emotional data is reflected in the generative AI and used to generate answers.

[1033] Get and display answers:

[1034] The answers obtained from the generative AI system are analyzed and processed on the server, and then displayed on the smartphone in a user-friendly format. Warning messages and additional information may be added as needed.

[1035] Specific example explanation

[1036] Example 1: User registration

[1037] A user enters their name, email address, and password in the in-app registration form and clicks the "Register" button. The registration information is stored in the Firebase Realtime Database.

[1038] Example 2: Entering an assignment

[1039] The user types in "Where is the rice?" and submits it. The server converts this task data into the appropriate API format and queries the OpenAI API.

[1040] Example 3: Emotion Recognition

[1041] The Emotion API analyzes the emotion of the task entered by the user and recognizes it as "anxiety." This emotion data is used as feedback for the generative AI's answer generation process.

[1042] Example 4: Displaying answers

[1043] The AI ​​system generates a response such as, "The rice is in aisle 3 of the food aisle. If you can't find it, please ask a member of staff." This is received by the server, analyzed and processed, and then displayed in a user-friendly format on the smartphone screen.

[1044] Prompt Sentence Examples

[1045] User prompt: "Where is the rice?"

[1046] Emotion: "Anxiety"

[1047] In this way, by combining generative AI and emotion recognition technology throughout the system, we can provide users with a more personalized shopping experience.

[1048] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1049] Step 1: User Registration

[1050] When the user first starts up the device, they enter their name, email address, and password. The device sends this information to the Firebase Realtime Database. The server then stores the received user registration information in the database. This saves the registered user's information and makes it available for later processing.

[1051] Step 2: Enter your assignment

[1052] A user uses an input form within the app to enter a specific task, such as "Where is the rice?", and clicks the submit button. The device then sends this task data to the server, which converts it into the appropriate API format and queries the OpenAI API. This process passes the user's task to the generative AI system.

[1053] Step 3: Emotion Recognition

[1054] The server receives the task data entered by the user and performs emotion analysis using Microsoft Azure's Emotion API. For example, the task data may be recognized as containing the emotion "anxiety." The server then reflects this emotion data in the generative AI system. This allows the user's emotions to be taken into account when generating answers.

[1055] Step 4: Generative AI generates answers

[1056] The server sends a request containing task data and emotion data to the OpenAI API and receives a response from the generative AI model. The generative AI system generates an answer that reflects the user's emotion. For example, the generated answer might be, "The rice is in aisle 3 of the food aisle. If you can't find it, please ask a staff member."

[1057] Step 5: Analyze and process the responses

[1058] The server receives the answers sent by the AI ​​generation system and processes them into a user-friendly format by adding warnings about phishing scams and other scams, as well as additional information. This processing involves analyzing the content of the answers and adding appropriate additional information. For example, a warning about phishing scams may be added.

[1059] Step 6: View your answers

[1060] The server then sends the processed response to the user's smartphone, which then displays the response to the user, including any necessary warnings or additional information. This step provides the information in a format that is easy for the user to understand.

[1061] Step 7: Gather feedback

[1062] Users can enter and submit feedback about the answers provided within the app. The device then sends the feedback data to the server. The server stores the received feedback in a database and analyzes it for use in improving the service next time. For example, feedback entered by a user as "easy to use" regarding an answer is stored and analyzed.

[1063] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1064] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1065] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1066] [Third embodiment]

[1067] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1068] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1069] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1070] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1071] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1072] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1073] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1074] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1075] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1076] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1077] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1078] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1079] This invention is a system that uses generative AI to enable elderly people and users with low IT literacy to easily solve problems. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI about problems entered by the user, a part that analyzes and processes the generated answers and displays them to the user, and a part that collects and analyzes user feedback to propose new services.

[1080] System Programming and Processing

[1081] 1. User Registration

[1082] A user accesses the system and registers by entering information such as name, email address, and password, and then clicking the registration button.

[1083] The terminal sends this input information to the server.

[1084] The server stores the received user registration information in a database.

[1085] 2. Inputting the problem and querying the generation AI

[1086] The user enters a specific problem (e.g., "I want to know how to shop online") into an input form on the system.

[1087] The terminal sends this assignment data to the server.

[1088] The server converts the problem data into the appropriate API format and sends it to the generative AI system.

[1089] 3. Obtaining and displaying answers

[1090] The server receives the answer from the generative AI system.

[1091] The server parses this response and formats it in a user-friendly way, adding warnings and additional information as needed.

[1092] The terminal displays the processed answer to the user.

[1093] 4. Collect and use feedback

[1094] The user enters feedback about the answer from the generating AI (e.g., usefulness of the answer and areas for improvement).

[1095] The device sends this feedback to the server.

[1096] The server stores the received feedback in a database and analyzes it.

[1097] The server generates new service proposals and improvement plans based on the analysis results and reflects them in the system.

[1098] Specific examples

[1099] User registration: To use the service, elderly person A creates an account by entering their name, email address, and password. The device sends this information to the server, which then stores it in a database.

[1100] Task input: Person A inputs, "I want to know how to shop online." The device sends this task information to the server, which then queries the generation AI system.

[1101] Obtaining and displaying answers: The AI ​​generation system provides specific online shopping instructions, which are received by the server, analyzed, and processed. Any necessary warnings (e.g., warning against phishing scams) are attached and sent back to the device, where they are displayed to User A.

[1102] Collecting and utilizing feedback: User A uses the provided information to shop online and enters feedback about the ease of use. The device sends this to the server, which analyzes the feedback and uses it to improve the service next time.

[1103] The system of the present invention allows users to easily use the generated AI through these processing steps, and provides a mechanism for reflecting their usage experience as feedback in improving new services.

[1104] The processing flow will be explained below.

[1105] Step 1:

[1106] The user goes to the system access page and clicks the "New Registration" button.

[1107] Step 2:

[1108] The user enters their name, email address, and password and clicks the "Register" button.

[1109] Operation details: Information entered by the user is saved in an input form on the device.

[1110] Step 3:

[1111] The terminal sends the user's input information to the server.

[1112] What it does: An HTTP POST request containing the input data is generated and sent to the server.

[1113] Step 4:

[1114] The server stores the received user registration information in a database.

[1115] What it does: Validates user information, hashes passwords if necessary, and stores them securely in a database.

[1116] Step 5:

[1117] The server sends a message to the terminal indicating that the user registration was successful.

[1118] What it does: Generates and sends a response containing a success message and a link to the user's account page.

[1119] Step 6:

[1120] The terminal displays a registration completion message to the user.

[1121] Action details: A message will be displayed in the browser and a link will be provided to guide the user through the next steps.

[1122] Step 7:

[1123] The user goes to the "Enter Problem" page and enters the problem they want to solve (e.g., "I want to know how to shop online").

[1124] Step 8:

[1125] The user enters the assignment details and clicks the "Submit" button.

[1126] Operation details: The assignment content is saved in the input form.

[1127] Step 9:

[1128] The terminal sends the entered assignment data to the server.

[1129] What it does: Creates an HTTP POST request containing the issue data and sends it to the server.

[1130] Step 10:

[1131] The server analyzes the received task data and converts it into an API format suitable for the generative AI system.

[1132] Operational details: Through a data conversion routine, the task data is formatted into a format that can be understood by the generative AI system.

[1133] Step 11:

[1134] The server sends the converted task data to the generation AI system.

[1135] Operation details: Generates an API request and sends it to the generation AI system.

[1136] Step 12:

[1137] The generative AI system generates answers to the tasks and sends them back to the server.

[1138] Step 13:

[1139] The server receives the answer from the generative AI system.

[1140] Operation details: Receives the response as JSON format data.

[1141] Step 14:

[1142] The server parses the received response and processes it into a user-friendly format.

[1143] Operation details: Depending on the content of the answer, warning messages and additional information are added and the final display format is created.

[1144] Step 15:

[1145] The server then sends the processed response data to the terminal.

[1146] Operation details: Generates HTML content to display to the user and returns it as a response.

[1147] Step 16:

[1148] The terminal displays the response received from the server to the user.

[1149] Operation details: Displays HTML content in the browser so that the user can visually confirm it.

[1150] Step 17:

[1151] The user acts based on the answers and enters the results and impressions as feedback.

[1152] Step 18:

[1153] The user enters their feedback and clicks the "Submit" button.

[1154] Operation details: Feedback content is saved in the input form.

[1155] Step 19:

[1156] The device sends the feedback data to the server.

[1157] Operational details: An HTTP POST request containing the feedback data is generated and sent to the server.

[1158] Step 20:

[1159] The server stores the received feedback in a database.

[1160] Operation details: Analyzes feedback data and stores it in a database.

[1161] Step 21:

[1162] The server analyzes the collected feedback and generates new service proposals and improvement plans.

[1163] Operation details: Feedback data is processed using analytical tools to extract insights for service improvement.

[1164] Step 22:

[1165] The server reflects new service proposals and improvement ideas into the system.

[1166] How it works: We'll use your feedback to develop our next updates and new features.

[1167] Example 1

[1168] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1169] Currently, elderly people and users with low IT literacy face difficulties in solving the problems they encounter when using the Internet, and they often lack the ability to receive appropriate support. Therefore, there is a need for a system that allows these users to easily solve their problems.

[1170] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1171] In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving tasks entered by the user and querying the generative AI model, and means for analyzing and processing the answers obtained from the generative AI model. This allows appropriate answers to the tasks entered by the user to be provided quickly and in an easy-to-understand format, making it possible for even elderly people and users with low IT literacy to easily solve problems using the Internet.

[1172] "User registration information" refers to personal information such as name, email address, and password provided by a user to use the system.

[1173] A "database" is a system for systematically storing information and efficiently retrieving and managing it.

[1174] A "problem" is a specific problem or question that a user wants to solve.

[1175] A "generative AI model" is a system that uses artificial intelligence technology to generate answers to user problems.

[1176] "Analysis" is the process of examining and understanding the data and information obtained.

[1177] "Processing" refers to the process of converting the answers obtained into a form that is easy for the user to understand.

[1178] "Feedback" refers to information entered by a user regarding their impressions of the answers or services provided and suggestions for improvement.

[1179] An "API format" is a standardized data format used when exchanging data between different systems.

[1180] A "warning message" is a message that alerts the user.

[1181] "Additional information" refers to supplemental information added to the basic answer.

[1182] This invention is a system that uses a generative AI model to enable elderly people and users with low IT literacy to easily solve problems. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI model about problems entered by the user, a part that analyzes and processes the generated answers and displays them to the user, and a part that collects and analyzes user feedback to propose new services.

[1183] The specific implementation of the system is configured as follows depending on the conditions:

[1184] Hardware and Software Configuration

[1185] Server: Manages user registration information, task data, generative AI model queries, and feedback. Specifically, it includes a database management system (DBMS) and an API server.

[1186] Device: The device a user uses to access the system, enter information, and view responses, such as a smartphone, tablet, or personal computer.

[1187] Generative AI models: For example, advanced natural language processing models such as OpenAI's GPT-3 are used to analyze the challenges and generate answers.

[1188] Data processing and calculation

[1189] 1. Processing of User Registration Information

[1190] The terminal displays the user registration information (name, email address, password) in an input form, and the user enters it.

[1191] The device sends the entered registration information to the server as an HTTP POST request.

[1192] The server validates the received registration information and stores it in the database.

[1193] 2. Processing of issue data

[1194] A user enters a problem into an input form (e.g., "I want to know how to shop online").

[1195] The device sends this assignment data to the server as an HTTP POST request.

[1196] The server converts the problem data into the appropriate API format (e.g., JSON) and sends it to the generative AI model.

[1197] 3. Processing the generated answers

[1198] The server receives the answer from the generative AI model.

[1199] The server parses the received response and processes it in a user-friendly format, for example adding warnings or additional information.

[1200] The terminal displays the processed answer to the user.

[1201] 4. Gathering Feedback

[1202] The user provides feedback on the answer.

[1203] The device sends the feedback data to the server as an HTTP POST request.

[1204] The server stores the feedback data in a database, analyzes it, and generates new service proposals based on the analysis results, which are reflected in the system.

[1205] Specific examples

[1206] User registration: Elderly person A accesses the system and creates an account by entering his / her name, email address, and password. The device sends this information to the server, which stores it in a database.

[1207] Task input: Person A inputs "I want to know how to shop online." The device sends this task data to the server, which converts the data to query the generative AI model.

[1208] Obtaining and displaying answers: The generative AI model provides specific online shopping instructions, which the server analyzes and processes. The server then sends the instructions back to the device along with any necessary warnings (e.g., warning against phishing scams) and displays them to User A.

[1209] Feedback collection: User A makes an online purchase based on the provided information and enters feedback about the ease of use. The device sends this feedback to the server, which analyzes it and uses it to improve the service next time.

[1210] Prompt Sentence Examples

[1211] "I'm a senior citizen who isn't very tech-savvy. I'd like to start shopping online. Can you please tell me the specific steps?"

[1212] "I can't send emails properly. Please tell me the reason and how to solve it."

[1213] Through these detailed processing steps, this system makes it easier for elderly people and users with low IT literacy to use generated AI models to solve problems. Furthermore, by improving the service based on feedback, it is possible to provide a better user experience.

[1214] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1215] Step 1:

[1216] Input form display

[1217] When a user accesses the service, the terminal displays an input form for user registration on the screen.

[1218] Input: None

[1219] Output: A form for user registration

[1220] Step 2:

[1221] Enter user information

[1222] The user enters information such as name, email address, and password, and clicks the Register button.

[1223] Input: Name, Email Address, Password

[1224] Output: User registration information entered

[1225] Step 3:

[1226] Sending information

[1227] The device sends this input information to the server as an HTTP POST request.

[1228] Input: User registration information entered

[1229] Output: User registration information sent to the server

[1230] Step 4:

[1231] Database storage

[1232] The server performs validation checks on the user information received, for example, checking the format of the email address or the strength of the password.

[1233] The server saves the validated user information in the database.

[1234] Input: User registration information sent to the server

[1235] Output: User registration information stored in the database

[1236] Step 5:

[1237] Display assignment input form

[1238] A user logs in and accesses the assignment entry form.

[1239] The device will display the assignment entry form on the screen.

[1240] Input: None

[1241] Output: Assignment input form

[1242] Step 6:

[1243] Enter assignment details

[1244] The user enters the problem they want to solve (e.g., "I want to know how to shop online").

[1245] Input: Assignment details

[1246] Output: The input assignment

[1247] Step 7:

[1248] Submitting assignment data

[1249] The device sends this assignment data to the server as an HTTP POST request.

[1250] Input: The input assignment content

[1251] Output: Issue data sent to the server

[1252] Step 8:

[1253] API format conversion

[1254] The server converts the issue data into the appropriate API format (e.g., JSON).

[1255] The server sends the converted data to the generative AI model as an HTTP POST request.

[1256] Input: Issue data sent to the server

[1257] Output: Problem data converted into a format suitable for the generative AI model

[1258] Step 9:

[1259] Receiving a response

[1260] The server receives the answer as an HTTP response from the generative AI model.

[1261] Input: Approached issue data

[1262] Output: Answer from the generative AI model

[1263] Step 10:

[1264] Analysis and processing of responses

[1265] The server analyzes the received response and processes it in a more user-friendly way, for example by adding bullet points or warnings.

[1266] Input: Answer from a generative AI model

[1267] Output: A user-friendly, processed answer

[1268] Step 11:

[1269] Submitting and viewing answers

[1270] The server then sends the processed response to the terminal.

[1271] The device displays the received response on the screen.

[1272] Input: User-friendly edited answer

[1273] Output: The answer displayed on the user's terminal

[1274] Step 12:

[1275] Feedback input form display

[1276] The user accesses the feedback input form to provide feedback on the answers.

[1277] The device will display a feedback input form on the screen.

[1278] Input: None

[1279] Output: Feedback input form

[1280] Step 13:

[1281] Enter your feedback

[1282] Users provide feedback on the usefulness of the answer and how it can be improved.

[1283] Input: Feedback

[1284] Output: The input feedback

[1285] Step 14:

[1286] Sending feedback data

[1287] The device sends the feedback data to the server as an HTTP POST request.

[1288] Input: Feedback content entered

[1289] Output: Feedback data sent to the server

[1290] Step 15:

[1291] Feedback storage and analysis

[1292] The server stores the received feedback data in a database.

[1293] The server analyzes the saved feedback data and uses it to improve the service next time.

[1294] Input: Feedback data sent to the server

[1295] Output: Feedback data and analysis results stored in a database

[1296] (Application example 1)

[1297] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1298] Elderly people and users with low IT literacy face the challenge of finding products or asking questions about how to use them in physical stores. This often prevents them from fully enjoying their shopping experience and leaves them feeling stressed. The purpose of this invention is to provide these users with appropriate support using generative AI, making it easier for them to solve their problems.

[1299] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1300] In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving tasks entered by the user and querying the AI ​​generation system, and means for analyzing and processing the answers obtained from the AI ​​generation system. This enables users to easily enter tasks in a physical store using voice recognition and voice synthesis and receive appropriate answers in the form of voice and text.

[1301] "User registration information" refers to basic information such as name, email address, and password provided by the user to use the system.

[1302] "Database" means a digital storage system for storing and managing data such as user registration information and feedback.

[1303] A "generative AI system" is an artificial intelligence system that uses natural language processing to generate answers to questions entered by users.

[1304] "Speech recognition means" is a technology that converts a user's voice into text, thereby enabling tasks entered by voice to be sent to the generation AI system as text information.

[1305] "Speech synthesis means" refers to a technology that reproduces the answer obtained from the generative AI system as voice, and presents the answer in a way that is easy for the user to understand.

[1306] "Feedback" refers to opinions and impressions that users enter about the usefulness and areas for improvement of answers provided by the AI ​​generation.

[1307] An "API format" is a standardized data structure for sending data in an appropriate format to a generative AI system.

[1308] "Warnings and additional information" refers to notes and supplementary information that should be provided to users in addition to the answers obtained from the generative AI system.

[1309] A "user-friendly format" is a format in which information is processed and presented in a way that makes it easy for users to understand.

[1310] The present invention is a system for assisting users in smoothly searching for and using products in physical stores. This system is particularly useful for elderly people and users with low IT literacy, and is designed to enable them to easily obtain information. Specific embodiments for implementing the present invention are described below.

[1311] System configuration

[1312] 1. Hardware

[1313] Server: A central processing unit that manages the database and interacts with the generative AI system.

[1314] Smartphone (user's device): A device on which the user enters assignments and receives answers.

[1315] 2. Software

[1316] Firebase Authentication: Software for user authentication.

[1317] Firebase Firestore: A database for storing user registration information and feedback.

[1318] OpenAI's GPT-3 API: A generative AI system that generates answers in natural language to user challenges.

[1319] Google Cloud Speech-to-Text API: A speech recognition technology that converts user speech into text.

[1320] Google Cloud Text-to-Speech API: A speech synthesis technology that plays back answers obtained from generative AI systems.

[1321] Program processing

[1322] 1. User Registration

[1323] A user creates an account by entering basic information (name, email address, password) on a smartphone. The smartphone sends this information to the server, which authenticates it using Firebase Authentication and stores it in Firebase Firestore.

[1324] 2. Inputting the problem and querying the generation AI

[1325] The user uses the voice recognition function on their smartphone to input a specific question (e.g., "Where is the shelf for this product?"), which is then converted into text using the Google Cloud Speech-to-Text API and sent to the server.

[1326] The server converts this problem data into the appropriate API format and sends it to OpenAI's GPT-3 API to retrieve the answer.

[1327] 3. Obtaining and displaying answers

[1328] The server receives the answers from the AI ​​generation system, analyzes and processes them, and converts them into a user-friendly format, adding warnings and additional information if necessary.

[1329] The smartphone uses the Google Cloud Text-to-Speech API to present this information to the user in voice and text.

[1330] 4. Collect and use feedback

[1331] The user provides feedback on the generated AI's answers via voice or text, and the smartphone converts this feedback into text using the Google Cloud Speech-to-Text API and sends it to the server.

[1332] The server saves the feedback in Firebase Firestore and analyzes it. Based on the analysis results, new service proposals and feature improvements are made.

[1333] Specific examples

[1334] Prompt Sentence Examples

[1335] If the user is an elderly person using a smartphone for the first time and asks, "Where is this medicine cabinet?", the prompt to the generative AI would be:

[1336] User question: "Where is this medicine cabinet?"

[1337] Prompt for the AI ​​generator: "The exact location of the medicine shelves will vary depending on the layout of each store, but generally, medicines are often located towards the back or middle of the store."

[1338] Example of a GPT-3 response

[1339] GPT-3's answer: "Medicine shelves are typically located towards the back of the store, but this varies depending on the layout of a particular store, so please ask a store associate or refer to the store's signage for guidance."

[1340] Through these processing steps, the system of the present invention allows users to easily utilize the generated AI and improve their shopping experience.

[1341] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1342] Step 1:

[1343] A user creates an account by entering basic information (name, email address, password) on their smartphone. The smartphone sends this information to the server. The server authenticates using Firebase Authentication and stores the authenticated information in Firebase Firestore. The input of this step is user information, and the output is the stored user information.

[1344] Step 2:

[1345] The user uses the voice recognition function on their smartphone to input a specific task by voice. The smartphone converts this speech into text using the Google Cloud Speech-to-Text API and sends the generated text data to the server. The input of this step is voice data, and the output is text data.

[1346] Step 3:

[1347] The server receives the text data sent by the user and converts it into an API format suitable for the generative AI model. It then queries OpenAI's GPT-3 API. The input of this step is text data, and the output is the data converted into the API format.

[1348] Step 4:

[1349] The server receives the answer data obtained from the generative AI model (GPT-3). It analyzes the answer data and processes it into a user-friendly format with warnings and additional information. The input of this step is the answer data, and the output is the processed answer data.

[1350] Step 5:

[1351] The smartphone receives the processed answer data sent from the server. Using the Google Cloud Text-to-Speech API, this information is played back to the user aloud and presented in text form. The input for this step is the processed answer data, and the output is the answer presented in audio and text form.

[1352] Step 6:

[1353] The user provides feedback on the generated AI's answer via voice or text. The smartphone again uses the Google Cloud Speech-to-Text API to convert the user's voice feedback into text and send it to the server. The input for this step is voice feedback, and the output is text-formatted feedback.

[1354] Step 7:

[1355] The server receives user feedback data and stores it in Firebase Firestore. The received feedback is analyzed and new service suggestions and feature improvements are made based on the results. The input of this step is the feedback in text format, and the output is the analysis results and new service suggestions.

[1356] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1357] This invention is a system that uses generative AI to enable elderly people and users with low IT literacy to easily solve problems, and further provides a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI about the problem entered by the user, a part that analyzes and processes the generated answer and displays it to the user, a part that collects and analyzes user feedback and proposes new services, and a part that uses an emotion engine to recognize the user's emotions.

[1358] System Programming and Processing

[1359] 1. User Registration

[1360] A user accesses the system and registers by entering information such as name, email address, and password, and then clicking the registration button.

[1361] The terminal sends this input information to the server.

[1362] The server stores the received user registration information in a database.

[1363] 2. Inputting the problem and querying the generation AI

[1364] The user enters a specific problem (e.g., "I want to know how to shop online") into an input form on the system.

[1365] The terminal sends this assignment data to the server.

[1366] The server converts the problem data into the appropriate API format and sends it to the generative AI system.

[1367] 3. Emotion Recognition Using an Emotion Engine

[1368] The server uses an emotion engine based on the user's input tasks and feedback to recognize the user's emotions.

[1369] The server reflects the recognized emotion data in the generative AI system.

[1370] 4. Retrieving and displaying answers

[1371] The generative AI system generates answers to the tasks while taking into account the user's emotions and sends them back to the server.

[1372] The server receives the answer from the generative AI system.

[1373] The server parses this response and formats it in a user-friendly way, adding warnings and additional information as needed.

[1374] The terminal displays the processed answer to the user.

[1375] 5. Collect and use feedback

[1376] The user enters feedback about the answer from the generating AI (e.g., usefulness of the answer and areas for improvement).

[1377] The device sends this feedback to the server.

[1378] The server stores the received feedback in a database and analyzes it.

[1379] The server generates new service proposals and improvement plans based on the analysis results and reflects them in the system.

[1380] Specific examples

[1381] User registration: To use the service, elderly person A creates an account by entering their name, email address, and password. The device sends this information to the server, which then stores it in a database.

[1382] Task input: Person A inputs, "I want to know how to shop online." The device sends this task information to the server, which then queries the generation AI system.

[1383] Use of emotion recognition: The server uses an emotion engine based on Mr. A's input to recognize his emotion as "anxiety" and sends that data to the generative AI system.

[1384] Obtaining and displaying answers: The generative AI system provides specific online shopping procedures, which are received, analyzed, and processed by the server. The system then sends the instructions back to the device along with any necessary warnings (e.g., beware of phishing scams) and displays them to User A in a more reassuring manner.

[1385] Collecting and utilizing feedback: User A uses the provided information to shop online and enters feedback about the ease of use. The device sends this to the server, which analyzes the feedback and uses it to improve the service next time.

[1386] The system of the present invention allows users to easily use generative AI through these processing steps and utilizes an emotion engine to provide a more personalized experience. Collected feedback and emotion data are used to continuously improve the service, thereby increasing user satisfaction.

[1387] The processing flow will be explained below.

[1388] Step 1:

[1389] The user goes to the system access page and clicks the "New Registration" button.

[1390] Step 2:

[1391] The user enters their name, email address, and password and clicks the "Register" button.

[1392] Operation details: Information entered by the user is saved in an input form on the device.

[1393] Step 3:

[1394] The terminal sends the user's input information to the server.

[1395] What it does: An HTTP POST request containing the input data is generated and sent to the server.

[1396] Step 4:

[1397] The server stores the received user registration information in a database.

[1398] What it does: Validates user information, hashes passwords if necessary, and stores them securely in a database.

[1399] Step 5:

[1400] The server sends a message to the terminal indicating that the user registration was successful.

[1401] What it does: Generates and sends a response containing a success message and a link to the user's account page.

[1402] Step 6:

[1403] The terminal displays a registration completion message to the user.

[1404] Action details: A message will be displayed in the browser and a link will be provided to guide the user through the next steps.

[1405] Step 7:

[1406] The user goes to the "Enter Problem" page and enters the problem they want to solve (e.g., "I want to know how to shop online").

[1407] Step 8:

[1408] The user enters the assignment details and clicks the "Submit" button.

[1409] Operation details: The assignment content is saved in the input form.

[1410] Step 9:

[1411] The terminal sends the entered assignment data to the server.

[1412] What it does: Creates an HTTP POST request containing the issue data and sends it to the server.

[1413] Step 10:

[1414] The server analyzes the received task data and converts it into an API format suitable for the generative AI system.

[1415] Operational details: Through a data conversion routine, the task data is formatted into a format that can be understood by the generative AI system.

[1416] Step 11:

[1417] The server sends the converted task data to the generation AI system.

[1418] Operation details: Generates an API request and sends it to the generation AI system.

[1419] Step 12:

[1420] The generative AI system generates answers to the tasks and sends them back to the server.

[1421] Step 13:

[1422] The server receives the answer from the generative AI system.

[1423] Operation details: Receives the response as JSON format data.

[1424] Step 14:

[1425] The server parses the received response, adds warnings and additional information as needed, and processes it in a user-friendly format.

[1426] How it works: It uses an emotion engine to analyze the user's emotions and then formats the answer appropriately, taking into account the analysis results.

[1427] Step 15:

[1428] The server then sends the processed response data to the terminal.

[1429] Operation details: Generates HTML content to display to the user and returns it as a response.

[1430] Step 16:

[1431] The terminal displays the response received from the server to the user.

[1432] Operation details: Displays HTML content in the browser so that the user can visually confirm it.

[1433] Step 17:

[1434] The user acts based on the answers and enters the results and impressions as feedback.

[1435] Step 18:

[1436] The user enters their feedback and clicks the "Submit" button.

[1437] Operation details: Feedback content is saved in the input form.

[1438] Step 19:

[1439] The device sends the feedback data to the server.

[1440] Operational details: An HTTP POST request containing the feedback data is generated and sent to the server.

[1441] Step 20:

[1442] The server stores the received feedback in a database.

[1443] Operation details: Analyzes feedback data and stores it in a database.

[1444] Step 21:

[1445] The server analyzes the collected feedback and uses an emotion engine to generate new service proposals and improvement plans along with the user's emotional data.

[1446] How it works: Feedback and sentiment data is processed using analytics tools to extract insights for service improvement.

[1447] Step 22:

[1448] The server reflects new service proposals and improvement ideas into the system.

[1449] How it works: We use feedback and sentiment data to develop our next updates and new features.

[1450] Example 2

[1451] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1452] The purpose of this invention is to provide a system that enables elderly people and users with low IT literacy to easily solve problems. Another objective is to provide a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. This system reduces user stress and inconvenience and enables continuous improvement of services.

[1453] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving a task entered by the user and querying the generation AI system, means for analyzing and processing the answer obtained from the generation AI system, means for using an emotion engine that recognizes emotions from the user's input content and feedback, and means for reflecting the recognized emotion data in the generation AI system. This makes it possible to provide an appropriate response that takes emotions into consideration for the task entered by the user.

[1454] "User registration information" refers to basic information such as name, email address, and password entered by a user accessing the system for the first time.

[1455] The "database" is an information management system for storing received user registration information, assignment data, feedback, and the like.

[1456] A "problem" is a specific problem or question that a user wants to solve through the system.

[1457] A "generative AI system" is an artificial intelligence system that generates appropriate responses and information based on a task posed by the user.

[1458] An "emotion engine" is software that analyzes and recognizes emotions from user input and feedback.

[1459] "Analysis" is the process of breaking down received information, generated answers, and feedback to extract meaning and patterns.

[1460] "Processing" is the process of reconstructing analyzed information into a format that is easy for users to understand.

[1461] "Feedback" refers to the user's evaluation and opinion of the answer provided by the generative AI system.

[1462] "API format" means an appropriate data format used for data exchange with a generative AI system.

[1463] "Recognition" is the process of understanding the user's emotional state using an emotion engine.

[1464] An "appropriate response" is the optimal answer or suggestion provided by a generative AI system that is tailored to the user's challenges and emotions.

[1465] A "personalized experience" is the provision of customized services that respond to the user's individual feelings and needs.

[1466] This invention is a system that enables elderly users and users with low IT literacy to easily solve problems. This system provides a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. The system includes a component that receives and stores user registration information in a database, a component that queries a generation AI for the problem entered by the user, a component that analyzes and processes the generated answer and displays it to the user, a component that collects and analyzes user feedback to propose new services, and a component that uses an emotion engine to recognize the user's emotions.

[1467] In concrete terms, a user first accesses the system and registers. This requires information such as name, email address, and password. This information is sent from the terminal to the server and received by the server. The server verifies the received data and, if there are no problems, stores it in the database.

[1468] The user then enters the problem they want to solve into an input form on the system. For example, they might enter a problem like "I want to know how to shop online." This problem data is sent from the device to the server. The server converts the problem data into an appropriate API format and sends it to the generative AI system. In this case, OpenAI's GPT-3 or another model can be used as the generative AI model.

[1469] Furthermore, the server uses an emotion engine based on the user's input tasks and feedback to recognize the user's emotions. The emotion engine can be IBM's Watson Tone Analyzer, for example. The recognized emotion data is fed into a generative AI system, which generates answers that take the user's emotions into consideration.

[1470] The answer from the AI ​​generation system is sent back to the server. The server receives this answer data and analyzes and processes it. Specifically, it reconstructs the answer into a user-friendly format, adding, for example, a warning about phishing scams or additional information. This processed answer is then sent to the device and displayed to the user.

[1471] The user enters feedback about the answer provided. For example, the user may say, "The answer was helpful" or "I would like more detailed information." The feedback data is sent from the device to the server, which then stores it in a database. This stored feedback data is periodically analyzed and used to propose new services and improvements.

[1472] As a specific example, elderly person A accesses the system and creates an account by entering their name, email address, and password. When A enters, "I want to know how to shop online," the problem data is sent to the generative AI system. The server analyzes A's problem using an emotion engine and recognizes that A is feeling "anxious." The generative AI system's response, which takes emotion into consideration, is returned to the server and displayed to A with a warning message and additional information added. Finally, A enters feedback, which the server analyzes and uses to improve the service next time.

[1473] Prompt Sentence Examples

[1474] 1. "Please enter your name, email address, and password to register."

[1475] 2. "Please enter the problem you want to solve in the form below. Example: 'I want to know how to shop online.'"

[1476] 3. "Please provide feedback about the answer provided. For example, how helpful the answer was and how it could be improved."

[1477] Through these components and processing steps, the present invention makes generative AI easily accessible to users, and by utilizing an emotion engine, provides a more personalized experience. Collected feedback and emotion data are used to continuously improve the service.

[1478] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1479] Step 1: Receiving and storing user registration information

[1480] Input: The user enters their name, email address, and password.

[1481] process:

[1482] The user accesses the system's new registration screen, enters the required information, and clicks the "Register" button.

[1483] The terminal converts this input data into JSON format and sends the data to the server by sending an HTTP request.

[1484] The server receives the request and validates the data (e.g., checking the format of the email address, checking the strength of the password).

[1485] If the server passes the validation, it stores the user information in its database.

[1486] Output: Returns a user registration completion message to the user.

[1487] Step 2: Enter and submit your assignment

[1488] Input: The user enters the problem they want to solve (e.g., "I want to know how to shop online").

[1489] process:

[1490] The user enters the problem they want to solve into an input form on the system.

[1491] The device sends the entered assignment data to the server in JSON format.

[1492] The server converts the problem data into the appropriate API format (e.g., for OpenAI GPT-3).

[1493] Output: Issue data converted to API format is sent to the generation AI system.

[1494] Step 3: Performing emotion recognition

[1495] Input: Assignment and past feedback data entered by the user.

[1496] process:

[1497] The server sends the task input data and feedback data to the emotion engine.

[1498] The server analyzes the emotion data obtained from the emotion engine and understands the user's current emotions (e.g., "anxiety," "confusion," etc.).

[1499] Output: The recognized emotion data is fed back to the generative AI system.

[1500] Step 4: Generative AI system generates answers

[1501] Input: Issue data and sentiment data.

[1502] process:

[1503] The generative AI system generates answers to tasks that take into account the user's emotions based on task data and emotion data.

[1504] Output: The generated answer data is sent back to the server.

[1505] Step 5: Analyze, process and display responses

[1506] Input: Answer data obtained from the generative AI system.

[1507] process:

[1508] The server analyzes the response data received from the generation AI system and reconstructs it into a user-friendly format (e.g., adding specific examples and warnings).

[1509] The server then sends the processed response data to the terminal.

[1510] The terminal displays the received response data on the user interface.

[1511] Output: The answer information displayed to the user.

[1512] Step 6: Collect and analyze feedback

[1513] Input: User feedback data (e.g., "The answer was helpful," "I'd like more information").

[1514] process:

[1515] The user enters feedback about the answers provided.

[1516] The device sends the input feedback data in JSON format to the server.

[1517] The server stores the received feedback data in a database.

[1518] The server periodically analyzes the stored feedback data.

[1519] Output: Based on the analysis results, new service proposals and improvement plans are generated and reflected in the system.

[1520] Specific examples

[1521] As a specific example, elderly person A accesses the system and creates an account by entering their name, email address, and password. When A enters, "I want to know how to shop online," the problem data is sent to the generative AI system. The server analyzes A's problem using an emotion engine and recognizes that A is feeling "anxious." The generative AI system's response, which takes emotion into consideration, is returned to the server and displayed to A with a warning message and additional information added. Finally, A enters feedback, which the server analyzes and uses to improve the service next time.

[1522] (Application example 2)

[1523] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1524] There is a problem that elderly people and users with low IT literacy have difficulty shopping smoothly in physical stores. In addition, conventional systems lack personalized responses that take into account the user's emotions, making it difficult to improve user satisfaction.

[1525] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1526] In this invention, the server includes a means for receiving user registration information and storing it in a database, a means for receiving tasks entered by the user and querying the AI ​​system, and a means for analyzing and processing the answers obtained from the AI ​​system. This allows even elderly people and users with low IT literacy to smoothly shop in physical stores. Furthermore, by including a means for recognizing user emotions using an emotion recognition engine and reflecting that data in the AI ​​system, and a means for providing product location information and detailed information in physical stores, a more personalized experience can be provided.

[1527] "User registration information" refers to basic personal information such as the system user's name, email address, and password.

[1528] A "database" is a system for systematically storing and managing collected user information, feedback, emotional data, etc.

[1529] A "generative AI system" refers to an artificial intelligence that generates a response to a task entered by a user.

[1530] An "emotion recognition engine" is software or hardware that analyzes emotions from user input and actions and reflects that data in the system.

[1531] "API format" refers to the standard data format used when exchanging data between systems.

[1532] "Means for analysis and processing" refers to the technology used to convert the answers obtained from the generative AI system into a user-friendly format and add warnings or additional information as necessary.

[1533] A "warning" is a statement that provides information about points that users should be aware of and risks.

[1534] "Additional information" refers to useful supplemental information added to the answer from the generative AI system.

[1535] "Product location information in physical stores" refers to information indicating where products are located in stores.

[1536] "Detailed information" refers to information that allows users to understand the product in detail, such as how to use it, its features, price, and related precautions.

[1537] The system of this invention is designed to support elderly people and users with low IT literacy, with the aim of making the shopping experience in brick-and-mortar stores smoother and more personalized. The specific system configuration and program processing are described below.

[1538] System configuration

[1539] Hardware used:

[1540] Smartphone (Android / iOS)

[1541] Software used:

[1542] Development environment: Android Studio, Xcode

[1543] Emotion recognition engine: Microsoft Azure's "Emotion API"

[1544] Database: Firebase Realtime Database

[1545] Generation AI: OpenAI API

[1546] What the program does

[1547] The system consists of the following main processing steps:

[1548] User Registration:

[1549] When a user launches the app for the first time, they are prompted to create an account by entering their name, email address, and password, which is automatically stored in the Firebase Realtime Database.

[1550] Inputting the issue and querying the AI:

[1551] When a user enters a specific task in the app (e.g., "Where is the rice?"), this task data is first sent to Firebase, which then converts the data into the appropriate API format and queries the OpenAI API.

[1552] Emotion recognition:

[1553] The task data and feedback entered by the user are analyzed for emotions using Microsoft Azure's Emotion API. This emotional data is reflected in the generative AI and used to generate answers.

[1554] Get and display answers:

[1555] The answers obtained from the generative AI system are analyzed and processed on the server, and then displayed on the smartphone in a user-friendly format. Warning messages and additional information may be added as needed.

[1556] Specific example explanation

[1557] Example 1: User registration

[1558] A user enters their name, email address, and password in the in-app registration form and clicks the "Register" button. The registration information is stored in the Firebase Realtime Database.

[1559] Example 2: Entering an assignment

[1560] The user types in "Where is the rice?" and submits it. The server converts this task data into the appropriate API format and queries the OpenAI API.

[1561] Example 3: Emotion Recognition

[1562] The Emotion API analyzes the emotion of the task entered by the user and recognizes it as "anxiety." This emotion data is used as feedback for the generative AI's answer generation process.

[1563] Example 4: Displaying answers

[1564] The AI ​​system generates a response such as, "The rice is in aisle 3 of the food aisle. If you can't find it, please ask a member of staff." This is received by the server, analyzed and processed, and then displayed in a user-friendly format on the smartphone screen.

[1565] Prompt Sentence Examples

[1566] User prompt: "Where is the rice?"

[1567] Emotion: "Anxiety"

[1568] In this way, by combining generative AI and emotion recognition technology throughout the system, we can provide users with a more personalized shopping experience.

[1569] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1570] Step 1: User Registration

[1571] When the user first starts up the device, they enter their name, email address, and password. The device sends this information to the Firebase Realtime Database. The server then stores the received user registration information in the database. This saves the registered user's information and makes it available for later processing.

[1572] Step 2: Enter your assignment

[1573] A user uses an input form within the app to enter a specific task, such as "Where is the rice?", and clicks the submit button. The device then sends this task data to the server, which converts it into the appropriate API format and queries the OpenAI API. This process passes the user's task to the generative AI system.

[1574] Step 3: Emotion Recognition

[1575] The server receives the task data entered by the user and performs emotion analysis using Microsoft Azure's Emotion API. For example, the task data may be recognized as containing the emotion "anxiety." The server then reflects this emotion data in the generative AI system. This allows the user's emotions to be taken into account when generating answers.

[1576] Step 4: Generative AI generates answers

[1577] The server sends a request containing task data and emotion data to the OpenAI API and receives a response from the generative AI model. The generative AI system generates an answer that reflects the user's emotion. For example, the generated answer might be, "The rice is in aisle 3 of the food aisle. If you can't find it, please ask a staff member."

[1578] Step 5: Analyze and process the responses

[1579] The server receives the answers sent by the AI ​​generation system and processes them into a user-friendly format by adding warnings about phishing scams and other scams, as well as additional information. This processing involves analyzing the content of the answers and adding appropriate additional information. For example, a warning about phishing scams may be added.

[1580] Step 6: View your answers

[1581] The server then sends the processed response to the user's smartphone, which then displays the response to the user, including any necessary warnings or additional information. This step provides the information in a format that is easy for the user to understand.

[1582] Step 7: Gather feedback

[1583] Users can enter and submit feedback about the answers provided within the app. The device then sends the feedback data to the server. The server stores the received feedback in a database and analyzes it for use in improving the service next time. For example, feedback entered by a user as "easy to use" regarding an answer is stored and analyzed.

[1584] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1585] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1586] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1587] [Fourth embodiment]

[1588] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1589] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1590] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1591] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1592] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1593] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1594] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1595] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1596] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1597] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1598] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1599] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1600] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1601] This invention is a system that uses generative AI to enable elderly people and users with low IT literacy to easily solve problems. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI about problems entered by the user, a part that analyzes and processes the generated answers and displays them to the user, and a part that collects and analyzes user feedback to propose new services.

[1602] System Programming and Processing

[1603] 1. User Registration

[1604] A user accesses the system and registers by entering information such as name, email address, and password, and then clicking the registration button.

[1605] The terminal sends this input information to the server.

[1606] The server stores the received user registration information in a database.

[1607] 2. Inputting the problem and querying the generation AI

[1608] The user enters a specific problem (e.g., "I want to know how to shop online") into an input form on the system.

[1609] The terminal sends this assignment data to the server.

[1610] The server converts the problem data into the appropriate API format and sends it to the generative AI system.

[1611] 3. Obtaining and displaying answers

[1612] The server receives the answer from the generative AI system.

[1613] The server parses this response and formats it in a user-friendly way, adding warnings and additional information as needed.

[1614] The terminal displays the processed answer to the user.

[1615] 4. Collect and use feedback

[1616] The user enters feedback about the answer from the generating AI (e.g., usefulness of the answer and areas for improvement).

[1617] The device sends this feedback to the server.

[1618] The server stores the received feedback in a database and analyzes it.

[1619] The server generates new service proposals and improvement plans based on the analysis results and reflects them in the system.

[1620] Specific examples

[1621] User registration: To use the service, elderly person A creates an account by entering their name, email address, and password. The device sends this information to the server, which then stores it in a database.

[1622] Task input: Person A inputs, "I want to know how to shop online." The device sends this task information to the server, which then queries the generation AI system.

[1623] Obtaining and displaying answers: The AI ​​generation system provides specific online shopping instructions, which are received by the server, analyzed, and processed. Any necessary warnings (e.g., warning against phishing scams) are attached and sent back to the device, where they are displayed to User A.

[1624] Collecting and utilizing feedback: User A uses the provided information to shop online and enters feedback about the ease of use. The device sends this to the server, which analyzes the feedback and uses it to improve the service next time.

[1625] The system of the present invention allows users to easily use the generated AI through these processing steps, and provides a mechanism for reflecting their usage experience as feedback in improving new services.

[1626] The processing flow will be explained below.

[1627] Step 1:

[1628] The user goes to the system access page and clicks the "New Registration" button.

[1629] Step 2:

[1630] The user enters their name, email address, and password and clicks the "Register" button.

[1631] Operation details: Information entered by the user is saved in an input form on the device.

[1632] Step 3:

[1633] The terminal sends the user's input information to the server.

[1634] What it does: An HTTP POST request containing the input data is generated and sent to the server.

[1635] Step 4:

[1636] The server stores the received user registration information in a database.

[1637] What it does: Validates user information, hashes passwords if necessary, and stores them securely in a database.

[1638] Step 5:

[1639] The server sends a message to the terminal indicating that the user registration was successful.

[1640] What it does: Generates and sends a response containing a success message and a link to the user's account page.

[1641] Step 6:

[1642] The terminal displays a registration completion message to the user.

[1643] Action details: A message will be displayed in the browser and a link will be provided to guide the user through the next steps.

[1644] Step 7:

[1645] The user goes to the "Enter Problem" page and enters the problem they want to solve (e.g., "I want to know how to shop online").

[1646] Step 8:

[1647] The user enters the assignment details and clicks the "Submit" button.

[1648] Operation details: The assignment content is saved in the input form.

[1649] Step 9:

[1650] The terminal sends the entered assignment data to the server.

[1651] What it does: Creates an HTTP POST request containing the issue data and sends it to the server.

[1652] Step 10:

[1653] The server analyzes the received task data and converts it into an API format suitable for the generative AI system.

[1654] Operational details: Through a data conversion routine, the task data is formatted into a format that can be understood by the generative AI system.

[1655] Step 11:

[1656] The server sends the converted task data to the generation AI system.

[1657] Operation details: Generates an API request and sends it to the generation AI system.

[1658] Step 12:

[1659] The generative AI system generates answers to the tasks and sends them back to the server.

[1660] Step 13:

[1661] The server receives the answer from the generative AI system.

[1662] Operation details: Receives the response as JSON format data.

[1663] Step 14:

[1664] The server parses the received response and processes it into a user-friendly format.

[1665] Operation details: Depending on the content of the answer, warning messages and additional information are added and the final display format is created.

[1666] Step 15:

[1667] The server then sends the processed response data to the terminal.

[1668] Operation details: Generates HTML content to display to the user and returns it as a response.

[1669] Step 16:

[1670] The terminal displays the response received from the server to the user.

[1671] Operation details: Displays HTML content in the browser so that the user can visually confirm it.

[1672] Step 17:

[1673] The user acts based on the answers and enters the results and impressions as feedback.

[1674] Step 18:

[1675] The user enters their feedback and clicks the "Submit" button.

[1676] Operation details: Feedback content is saved in the input form.

[1677] Step 19:

[1678] The device sends the feedback data to the server.

[1679] Operational details: An HTTP POST request containing the feedback data is generated and sent to the server.

[1680] Step 20:

[1681] The server stores the received feedback in a database.

[1682] Operation details: Analyzes feedback data and stores it in a database.

[1683] Step 21:

[1684] The server analyzes the collected feedback and generates new service proposals and improvement plans.

[1685] Operation details: Feedback data is processed using analytical tools to extract insights for service improvement.

[1686] Step 22:

[1687] The server reflects new service proposals and improvement ideas into the system.

[1688] How it works: We'll use your feedback to develop our next updates and new features.

[1689] Example 1

[1690] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1691] Currently, elderly people and users with low IT literacy face difficulties in solving the problems they encounter when using the Internet, and they often lack the ability to receive appropriate support. Therefore, there is a need for a system that allows these users to easily solve their problems.

[1692] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1693] In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving tasks entered by the user and querying the generative AI model, and means for analyzing and processing the answers obtained from the generative AI model. This allows appropriate answers to the tasks entered by the user to be provided quickly and in an easy-to-understand format, making it possible for even elderly people and users with low IT literacy to easily solve problems using the Internet.

[1694] "User registration information" refers to personal information such as name, email address, and password provided by a user to use the system.

[1695] A "database" is a system for systematically storing information and efficiently retrieving and managing it.

[1696] A "problem" is a specific problem or question that a user wants to solve.

[1697] A "generative AI model" is a system that uses artificial intelligence technology to generate answers to user problems.

[1698] "Analysis" is the process of examining and understanding the data and information obtained.

[1699] "Processing" refers to the process of converting the answers obtained into a form that is easy for the user to understand.

[1700] "Feedback" refers to information entered by a user regarding their impressions of the answers or services provided and suggestions for improvement.

[1701] An "API format" is a standardized data format used when exchanging data between different systems.

[1702] A "warning message" is a message that alerts the user.

[1703] "Additional information" refers to supplemental information added to the basic answer.

[1704] This invention is a system that uses a generative AI model to enable elderly people and users with low IT literacy to easily solve problems. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI model about problems entered by the user, a part that analyzes and processes the generated answers and displays them to the user, and a part that collects and analyzes user feedback to propose new services.

[1705] The specific implementation of the system is configured as follows depending on the conditions:

[1706] Hardware and Software Configuration

[1707] Server: Manages user registration information, task data, generative AI model queries, and feedback. Specifically, it includes a database management system (DBMS) and an API server.

[1708] Device: The device a user uses to access the system, enter information, and view responses, such as a smartphone, tablet, or personal computer.

[1709] Generative AI models: For example, advanced natural language processing models such as OpenAI's GPT-3 are used to analyze the challenges and generate answers.

[1710] Data processing and calculation

[1711] 1. Processing of User Registration Information

[1712] The terminal displays the user registration information (name, email address, password) in an input form, and the user enters it.

[1713] The device sends the entered registration information to the server as an HTTP POST request.

[1714] The server validates the received registration information and stores it in the database.

[1715] 2. Processing of issue data

[1716] A user enters a problem into an input form (e.g., "I want to know how to shop online").

[1717] The device sends this assignment data to the server as an HTTP POST request.

[1718] The server converts the problem data into the appropriate API format (e.g., JSON) and sends it to the generative AI model.

[1719] 3. Processing the generated answers

[1720] The server receives the answer from the generative AI model.

[1721] The server parses the received response and processes it in a user-friendly format, for example adding warnings or additional information.

[1722] The terminal displays the processed answer to the user.

[1723] 4. Gathering Feedback

[1724] The user provides feedback on the answer.

[1725] The device sends the feedback data to the server as an HTTP POST request.

[1726] The server stores the feedback data in a database, analyzes it, and generates new service proposals based on the analysis results, which are reflected in the system.

[1727] Specific examples

[1728] User registration: Elderly person A accesses the system and creates an account by entering his / her name, email address, and password. The device sends this information to the server, which stores it in a database.

[1729] Task input: Person A inputs "I want to know how to shop online." The device sends this task data to the server, which converts the data to query the generative AI model.

[1730] Obtaining and displaying answers: The generative AI model provides specific online shopping instructions, which the server analyzes and processes. The server then sends the instructions back to the device along with any necessary warnings (e.g., warning against phishing scams) and displays them to User A.

[1731] Feedback collection: User A makes an online purchase based on the provided information and enters feedback about the ease of use. The device sends this feedback to the server, which analyzes it and uses it to improve the service next time.

[1732] Prompt Sentence Examples

[1733] "I'm a senior citizen who isn't very tech-savvy. I'd like to start shopping online. Can you please tell me the specific steps?"

[1734] "I can't send emails properly. Please tell me the reason and how to solve it."

[1735] Through these detailed processing steps, this system makes it easier for elderly people and users with low IT literacy to use generated AI models to solve problems. Furthermore, by improving the service based on feedback, it is possible to provide a better user experience.

[1736] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1737] Step 1:

[1738] Input form display

[1739] When a user accesses the service, the terminal displays an input form for user registration on the screen.

[1740] Input: None

[1741] Output: A form for user registration

[1742] Step 2:

[1743] Enter user information

[1744] The user enters information such as name, email address, and password, and clicks the Register button.

[1745] Input: Name, Email Address, Password

[1746] Output: User registration information entered

[1747] Step 3:

[1748] Sending information

[1749] The device sends this input information to the server as an HTTP POST request.

[1750] Input: User registration information entered

[1751] Output: User registration information sent to the server

[1752] Step 4:

[1753] Database storage

[1754] The server performs validation checks on the user information received, for example, checking the format of the email address or the strength of the password.

[1755] The server saves the validated user information in the database.

[1756] Input: User registration information sent to the server

[1757] Output: User registration information stored in the database

[1758] Step 5:

[1759] Display assignment input form

[1760] A user logs in and accesses the assignment entry form.

[1761] The device will display the assignment entry form on the screen.

[1762] Input: None

[1763] Output: Assignment input form

[1764] Step 6:

[1765] Enter assignment details

[1766] The user enters the problem they want to solve (e.g., "I want to know how to shop online").

[1767] Input: Assignment details

[1768] Output: The input assignment

[1769] Step 7:

[1770] Submitting assignment data

[1771] The device sends this assignment data to the server as an HTTP POST request.

[1772] Input: The input assignment content

[1773] Output: Issue data sent to the server

[1774] Step 8:

[1775] API format conversion

[1776] The server converts the issue data into the appropriate API format (e.g., JSON).

[1777] The server sends the converted data to the generative AI model as an HTTP POST request.

[1778] Input: Issue data sent to the server

[1779] Output: Problem data converted into a format suitable for the generative AI model

[1780] Step 9:

[1781] Receiving a response

[1782] The server receives the answer as an HTTP response from the generative AI model.

[1783] Input: Approached issue data

[1784] Output: Answer from the generative AI model

[1785] Step 10:

[1786] Analysis and processing of responses

[1787] The server analyzes the received response and processes it in a more user-friendly way, for example by adding bullet points or warnings.

[1788] Input: Answer from a generative AI model

[1789] Output: A user-friendly, processed answer

[1790] Step 11:

[1791] Submitting and viewing answers

[1792] The server then sends the processed response to the terminal.

[1793] The device displays the received response on the screen.

[1794] Input: User-friendly edited answer

[1795] Output: The answer displayed on the user's terminal

[1796] Step 12:

[1797] Feedback input form display

[1798] The user accesses the feedback input form to provide feedback on the answers.

[1799] The device will display a feedback input form on the screen.

[1800] Input: None

[1801] Output: Feedback input form

[1802] Step 13:

[1803] Enter your feedback

[1804] Users provide feedback on the usefulness of the answer and how it can be improved.

[1805] Input: Feedback

[1806] Output: The input feedback

[1807] Step 14:

[1808] Sending feedback data

[1809] The device sends the feedback data to the server as an HTTP POST request.

[1810] Input: Feedback content entered

[1811] Output: Feedback data sent to the server

[1812] Step 15:

[1813] Feedback storage and analysis

[1814] The server stores the received feedback data in a database.

[1815] The server analyzes the saved feedback data and uses it to improve the service next time.

[1816] Input: Feedback data sent to the server

[1817] Output: Feedback data and analysis results stored in a database

[1818] (Application example 1)

[1819] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1820] Elderly people and users with low IT literacy face the challenge of finding products or asking questions about how to use them in physical stores. This often prevents them from fully enjoying their shopping experience and leaves them feeling stressed. The purpose of this invention is to provide these users with appropriate support using generative AI, making it easier for them to solve their problems.

[1821] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1822] In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving tasks entered by the user and querying the AI ​​generation system, and means for analyzing and processing the answers obtained from the AI ​​generation system. This enables users to easily enter tasks in a physical store using voice recognition and voice synthesis and receive appropriate answers in the form of voice and text.

[1823] "User registration information" refers to basic information such as name, email address, and password provided by the user to use the system.

[1824] "Database" means a digital storage system for storing and managing data such as user registration information and feedback.

[1825] A "generative AI system" is an artificial intelligence system that uses natural language processing to generate answers to questions entered by users.

[1826] "Speech recognition means" is a technology that converts a user's voice into text, thereby enabling tasks entered by voice to be sent to the generation AI system as text information.

[1827] "Speech synthesis means" refers to a technology that reproduces the answer obtained from the generative AI system as voice, and presents the answer in a way that is easy for the user to understand.

[1828] "Feedback" refers to opinions and impressions that users enter about the usefulness and areas for improvement of answers provided by the AI ​​generation.

[1829] An "API format" is a standardized data structure for sending data in an appropriate format to a generative AI system.

[1830] "Warnings and additional information" refers to notes and supplementary information that should be provided to users in addition to the answers obtained from the generative AI system.

[1831] A "user-friendly format" is a format in which information is processed and presented in a way that makes it easy for users to understand.

[1832] The present invention is a system for assisting users in smoothly searching for and using products in physical stores. This system is particularly useful for elderly people and users with low IT literacy, and is designed to enable them to easily obtain information. Specific embodiments for implementing the present invention are described below.

[1833] System configuration

[1834] 1. Hardware

[1835] Server: A central processing unit that manages the database and interacts with the generative AI system.

[1836] Smartphone (user's device): A device on which the user enters assignments and receives answers.

[1837] 2. Software

[1838] Firebase Authentication: Software for user authentication.

[1839] Firebase Firestore: A database for storing user registration information and feedback.

[1840] OpenAI's GPT-3 API: A generative AI system that generates answers in natural language to user challenges.

[1841] Google Cloud Speech-to-Text API: A speech recognition technology that converts user speech into text.

[1842] Google Cloud Text-to-Speech API: A speech synthesis technology that plays back answers obtained from generative AI systems.

[1843] Program processing

[1844] 1. User Registration

[1845] A user creates an account by entering basic information (name, email address, password) on a smartphone. The smartphone sends this information to the server, which authenticates it using Firebase Authentication and stores it in Firebase Firestore.

[1846] 2. Inputting the problem and querying the generation AI

[1847] The user uses the voice recognition function on their smartphone to input a specific question (e.g., "Where is the shelf for this product?"), which is then converted into text using the Google Cloud Speech-to-Text API and sent to the server.

[1848] The server converts this problem data into the appropriate API format and sends it to OpenAI's GPT-3 API to retrieve the answer.

[1849] 3. Obtaining and displaying answers

[1850] The server receives the answers from the AI ​​generation system, analyzes and processes them into a user-friendly format, and adds warnings and additional information if necessary.

[1851] The smartphone uses the Google Cloud Text-to-Speech API to present this information to the user in voice and text.

[1852] 4. Collect and use feedback

[1853] The user provides feedback on the generated AI's answers via voice or text, and the smartphone converts this feedback into text using the Google Cloud Speech-to-Text API and sends it to the server.

[1854] The server saves the feedback in Firebase Firestore and analyzes it. Based on the analysis results, new service proposals and feature improvements are made.

[1855] Specific examples

[1856] Prompt Sentence Examples

[1857] If the user is an elderly person using a smartphone for the first time and asks, "Where is this medicine cabinet?", the prompt to the generative AI would be:

[1858] User question: "Where is this medicine cabinet?"

[1859] Prompt for the generative AI: "The exact location of the medicine shelves will vary depending on the layout of each store, but generally, medicines are often located towards the back or middle of the store."

[1860] Example of a GPT-3 response

[1861] GPT-3's answer: "Medicine shelves are typically located towards the back of the store, but this varies depending on the layout of a particular store, so please ask a store associate or refer to the store's signage for guidance."

[1862] Through these processing steps, the system of the present invention allows users to easily utilize the generated AI and improve their shopping experience.

[1863] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1864] Step 1:

[1865] A user creates an account by entering basic information (name, email address, password) on their smartphone. The smartphone sends this information to the server. The server authenticates using Firebase Authentication and stores the authenticated information in Firebase Firestore. The input of this step is user information, and the output is the stored user information.

[1866] Step 2:

[1867] The user uses the voice recognition function on their smartphone to input a specific task by voice. The smartphone then converts this speech into text using the Google Cloud Speech-to-Text API and sends the generated text data to the server. The input for this step is voice data, and the output is text data.

[1868] Step 3:

[1869] The server receives the text data sent by the user and converts it into an API format suitable for the generative AI model. It then queries OpenAI's GPT-3 API. The input of this step is text data, and the output is the data converted into the API format.

[1870] Step 4:

[1871] The server receives the answer data obtained from the generative AI model (GPT-3). It analyzes the answer data and processes it into a user-friendly format with warnings and additional information. The input of this step is the answer data, and the output is the processed answer data.

[1872] Step 5:

[1873] The smartphone receives the processed answer data sent from the server. Using the Google Cloud Text-to-Speech API, this information is played back aloud and presented to the user in text form. The input of this step is the processed answer data, and the output is the answer presented in audio and text form.

[1874] Step 6:

[1875] The user provides feedback on the generated AI's answer via voice or text. The smartphone again uses the Google Cloud Speech-to-Text API to convert the user's voice feedback into text and send it to the server. The input for this step is voice feedback, and the output is text-formatted feedback.

[1876] Step 7:

[1877] The server receives user feedback data and stores it in Firebase Firestore. The received feedback is analyzed and new service suggestions and feature improvements are made based on the results. The input of this step is the feedback in text format, and the output is the analysis results and new service suggestions.

[1878] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1879] This invention is a system that uses generative AI to enable elderly people and users with low IT literacy to easily solve problems, and further provides a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. This system includes a part that receives user registration information and stores it in a database, a part that queries the generative AI about the problem entered by the user, a part that analyzes and processes the generated answer and displays it to the user, a part that collects and analyzes user feedback and proposes new services, and a part that uses an emotion engine to recognize the user's emotions.

[1880] System Programming and Processing

[1881] 1. User Registration

[1882] A user accesses the system and registers by entering information such as name, email address, and password, and then clicking the registration button.

[1883] The terminal sends this input information to the server.

[1884] The server stores the received user registration information in a database.

[1885] 2. Inputting the problem and querying the generation AI

[1886] The user enters a specific problem (e.g., "I want to know how to shop online") into an input form on the system.

[1887] The terminal sends this assignment data to the server.

[1888] The server converts the problem data into the appropriate API format and sends it to the generative AI system.

[1889] 3. Emotion Recognition Using an Emotion Engine

[1890] The server uses an emotion engine based on the user's input tasks and feedback to recognize the user's emotions.

[1891] The server reflects the recognized emotion data in the generative AI system.

[1892] 4. Retrieving and displaying answers

[1893] The generative AI system generates answers to the tasks while taking into account the user's emotions and sends them back to the server.

[1894] The server receives the answer from the generative AI system.

[1895] The server parses this response and formats it in a user-friendly way, adding warnings and additional information as needed.

[1896] The terminal displays the processed answer to the user.

[1897] 5. Collect and use feedback

[1898] The user enters feedback about the answer from the generating AI (e.g., usefulness of the answer and areas for improvement).

[1899] The device sends this feedback to the server.

[1900] The server stores the received feedback in a database and analyzes it.

[1901] The server generates new service proposals and improvement plans based on the analysis results and reflects them in the system.

[1902] Specific examples

[1903] User registration: To use the service, elderly person A creates an account by entering their name, email address, and password. The device sends this information to the server, which then stores it in a database.

[1904] Task input: Person A inputs, "I want to know how to shop online." The device sends this task information to the server, which then queries the generation AI system.

[1905] Use of emotion recognition: The server uses an emotion engine based on Mr. A's input to recognize his emotion as "anxiety" and sends that data to the generative AI system.

[1906] Obtaining and displaying answers: The generative AI system provides "specific online shopping procedures," which are received, analyzed, and processed by the server. The system then sends the results back to the device along with any necessary warnings (e.g., beware of phishing scams) and displays them to User A in a more reassuring manner.

[1907] Collecting and utilizing feedback: User A uses the provided information to shop online and enters feedback about the ease of use. The device sends this to the server, which analyzes the feedback and uses it to improve the service next time.

[1908] The system of the present invention allows users to easily use generative AI through these processing steps, and by utilizing an emotion engine, provides a more personalized experience. Collected feedback and emotion data are used to continuously improve the service, thereby increasing user satisfaction.

[1909] The processing flow will be explained below.

[1910] Step 1:

[1911] The user goes to the system access page and clicks the "New Registration" button.

[1912] Step 2:

[1913] The user enters their name, email address, and password and clicks the "Register" button.

[1914] Operation details: Information entered by the user is saved in an input form on the device.

[1915] Step 3:

[1916] The terminal sends the user's input information to the server.

[1917] What it does: An HTTP POST request containing the input data is generated and sent to the server.

[1918] Step 4:

[1919] The server stores the received user registration information in a database.

[1920] What it does: Validates user information, hashes passwords if necessary, and stores them securely in a database.

[1921] Step 5:

[1922] The server sends a message to the terminal indicating that the user registration was successful.

[1923] What it does: Generates and sends a response containing a success message and a link to the user's account page.

[1924] Step 6:

[1925] The terminal displays a registration completion message to the user.

[1926] Action details: A message will be displayed in the browser and a link will be provided to guide the user through the next steps.

[1927] Step 7:

[1928] The user goes to the "Enter Problem" page and enters the problem they want to solve (e.g., "I want to know how to shop online").

[1929] Step 8:

[1930] The user enters the assignment details and clicks the "Submit" button.

[1931] Operation details: The assignment content is saved in the input form.

[1932] Step 9:

[1933] The terminal sends the entered assignment data to the server.

[1934] What it does: Creates an HTTP POST request containing the issue data and sends it to the server.

[1935] Step 10:

[1936] The server analyzes the received task data and converts it into an API format suitable for the generative AI system.

[1937] Operational details: Through a data conversion routine, the task data is formatted into a format that can be understood by the generative AI system.

[1938] Step 11:

[1939] The server sends the converted task data to the generation AI system.

[1940] Operation details: Generates an API request and sends it to the generation AI system.

[1941] Step 12:

[1942] The generative AI system generates answers to the tasks and sends them back to the server.

[1943] Step 13:

[1944] The server receives the answer from the generative AI system.

[1945] Operation details: Receives the response as JSON format data.

[1946] Step 14:

[1947] The server parses the received response, adds warnings and additional information as needed, and processes it in a user-friendly format.

[1948] How it works: It uses an emotion engine to analyze the user's emotions and then formats the answer appropriately, taking into account the analysis results.

[1949] Step 15:

[1950] The server then sends the processed response data to the terminal.

[1951] Operation details: Generates HTML content to display to the user and returns it as a response.

[1952] Step 16:

[1953] The terminal displays the response received from the server to the user.

[1954] Operation details: Displays HTML content in the browser so that the user can visually confirm it.

[1955] Step 17:

[1956] The user acts based on the answers and enters the results and impressions as feedback.

[1957] Step 18:

[1958] The user enters their feedback and clicks the "Submit" button.

[1959] Operation details: Feedback content is saved in the input form.

[1960] Step 19:

[1961] The device sends the feedback data to the server.

[1962] Operational details: An HTTP POST request containing the feedback data is generated and sent to the server.

[1963] Step 20:

[1964] The server stores the received feedback in a database.

[1965] Operation details: Analyzes feedback data and stores it in a database.

[1966] Step 21:

[1967] The server analyzes the collected feedback and uses an emotion engine to generate new service proposals and improvement plans along with the user's emotional data.

[1968] How it works: Feedback and sentiment data is processed using analytics tools to extract insights for service improvement.

[1969] Step 22:

[1970] The server reflects new service proposals and improvement ideas into the system.

[1971] How it works: We use feedback and sentiment data to develop our next updates and new features.

[1972] Example 2

[1973] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1974] The purpose of this invention is to provide a system that enables elderly people and users with low IT literacy to easily solve problems. Another objective is to provide a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. This system reduces user stress and inconvenience and enables continuous improvement of services.

[1975] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving user registration information and storing it in a database, means for receiving a task entered by the user and querying the generation AI system, means for analyzing and processing the answer obtained from the generation AI system, means for using an emotion engine that recognizes emotions from the user's input content and feedback, and means for reflecting the recognized emotion data in the generation AI system. This makes it possible to provide an appropriate response that takes emotions into consideration for the task entered by the user.

[1976] "User registration information" refers to basic information such as name, email address, and password entered by a user accessing the system for the first time.

[1977] The "database" is an information management system for storing received user registration information, assignment data, feedback, and the like.

[1978] A "problem" is a specific problem or question that a user wants to solve through the system.

[1979] A "generative AI system" is an artificial intelligence system that generates appropriate responses and information based on a task posed by the user.

[1980] An "emotion engine" is software that analyzes and recognizes emotions from user input and feedback.

[1981] "Analysis" is the process of breaking down received information, generated answers, and feedback to extract meaning and patterns.

[1982] "Processing" is the process of reconstructing analyzed information into a format that is easy for users to understand.

[1983] "Feedback" refers to the user's evaluation and opinion of the answer provided by the generative AI system.

[1984] "API format" means an appropriate data format used for data exchange with a generative AI system.

[1985] "Recognition" is the process of understanding the user's emotional state using an emotion engine.

[1986] An "appropriate response" is the optimal answer or suggestion provided by a generative AI system that is tailored to the user's challenges and emotions.

[1987] A "personalized experience" is the provision of customized services that respond to the user's individual feelings and needs.

[1988] This invention is a system that enables elderly users and users with low IT literacy to easily solve problems. This system provides a more personalized experience by recognizing the user's emotions and reflecting them in responses and service improvements. The system includes a component that receives and stores user registration information in a database, a component that queries a generation AI for the problem entered by the user, a component that analyzes and processes the generated answer and displays it to the user, a component that collects and analyzes user feedback to propose new services, and a component that uses an emotion engine to recognize the user's emotions.

[1989] In concrete terms, a user first accesses the system and registers. This requires information such as name, email address, and password. This information is sent from the terminal to the server and received by the server. The server verifies the received data and, if there are no problems, stores it in the database.

[1990] The user then enters the problem they want to solve into an input form on the system. For example, they might enter a problem like "I want to know how to shop online." This problem data is sent from the device to the server. The server converts the problem data into an appropriate API format and sends it to the generative AI system. In this case, OpenAI's GPT-3 or another model can be used as the generative AI model.

[1991] Furthermore, the server uses an emotion engine based on the user's input tasks and feedback to recognize the user's emotions. The emotion engine can be IBM's Watson Tone Analyzer, for example. The recognized emotion data is fed into a generative AI system, which generates answers that take the user's emotions into consideration.

[1992] The answer from the AI ​​generation system is sent back to the server. The server receives this answer data and analyzes and processes it. Specifically, it reconstructs the answer into a user-friendly format, adding, for example, a warning about phishing scams or additional information. This processed answer is then sent to the device and displayed to the user.

[1993] The user enters feedback about the answer provided. For example, the user may say, "The answer was helpful" or "I would like more detailed information." The feedback data is sent from the device to the server, which then stores it in a database. This stored feedback data is periodically analyzed and used to propose new services and improvements.

[1994] As a specific example, elderly person A accesses the system and creates an account by entering their name, email address, and password. When A enters, "I want to know how to shop online," the problem data is sent to the generative AI system. The server analyzes A's problem using an emotion engine and recognizes that A is feeling "anxious." The generative AI system's response, which takes emotion into consideration, is returned to the server and displayed to A with a warning message and additional information added. Finally, A enters feedback, which the server analyzes and uses to improve the service next time.

[1995] Prompt Sentence Examples

[1996] 1. "Please enter your name, email address, and password to register."

[1997] 2. "Please enter the problem you want to solve in the form below. Example: 'I want to know how to shop online.'"

[1998] 3. "Please provide feedback about the answer provided. For example, how helpful the answer was and how it could be improved."

[1999] Through these components and processing steps, the present invention makes generative AI easily accessible to users, and by utilizing an emotion engine, provides a more personalized experience. Collected feedback and emotion data are used to continuously improve the service.

[2000] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2001] Step 1: Receiving and storing user registration information

[2002] Input: The user enters their name, email address, and password.

[2003] process:

[2004] The user accesses the system's new registration screen, enters the required information, and clicks the "Register" button.

[2005] The terminal converts this input data into JSON format and sends the data to the server by sending an HTTP request.

[2006] The server receives the request and validates the data (e.g., checking the format of the email address, checking the strength of the password).

[2007] If the server passes the validation, it stores the user information in its database.

[2008] Output: Returns a user registration completion message to the user.

[2009] Step 2: Enter and submit your assignment

[2010] Input: The user enters the problem they want to solve (e.g., "I want to know how to shop online").

[2011] process:

[2012] The user enters the problem they want to solve into an input form on the system.

[2013] The device sends the entered assignment data to the server in JSON format.

[2014] The server converts the problem data into the appropriate API format (e.g., for OpenAI GPT-3).

[2015] Output: Issue data converted to API format is sent to the generation AI system.

[2016] Step 3: Performing emotion recognition

[2017] Input: Assignment and past feedback data entered by the user.

[2018] process:

[2019] The server sends the task input data and feedback data to the emotion engine.

[2020] The server analyzes the emotion data obtained from the emotion engine and understands the user's current emotions (e.g., "anxiety," "confusion," etc.).

[2021] Output: The recognized emotion data is fed back to the generative AI system.

[2022] Step 4: Generative AI system generates answers

[2023] Input: Issue data and sentiment data.

[2024] process:

[2025] The generative AI system generates answers to tasks that take into account the user's emotions based on task data and emotion data.

[2026] Output: The generated answer data is sent back to the server.

[2027] Step 5: Analyze, process and display responses

[2028] Input: Answer data obtained from the generative AI system.

[2029] process:

[2030] The server analyzes the response data received from the generation AI system and reconstructs it into a user-friendly format (e.g., adding specific examples and warnings).

[2031] The server then sends the processed response data to the terminal.

[2032] The terminal displays the received response data on the user interface.

[2033] Output: The answer information displayed to the user.

[2034] Step 6: Collect and analyze feedback

[2035] Input: User feedback data (e.g., "The answer was helpful," "I'd like more information").

[2036] process:

[2037] The user enters feedback about the answers provided.

[2038] The device sends the input feedback data to the server in JSON format.

[2039] The server stores the received feedback data in a database.

[2040] The server periodically analyzes the stored feedback data.

[2041] Output: Based on the analysis results, new service proposals and improvement plans are generated and reflected in the system.

[2042] Specific examples

[2043] As a specific example, elderly person A accesses the system and creates an account by entering their name, email address, and password. When A enters, "I want to know how to shop online," the problem data is sent to the generative AI system. The server analyzes A's problem using an emotion engine and recognizes that A is feeling "anxious." The generative AI system's response, which takes emotion into consideration, is returned to the server and displayed to A with a warning message and additional information added. Finally, A enters feedback, which the server analyzes and uses to improve the service next time.

[2044] (Application example 2)

[2045] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2046] There is a problem that elderly people and users with low IT literacy have difficulty shopping smoothly in physical stores. In addition, conventional systems lack personalized responses that take into account the user's emotions, making it difficult to improve user satisfaction.

[2047] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2048] In this invention, the server includes a means for receiving user registration information and storing it in a database, a means for receiving tasks entered by the user and querying the AI ​​system, and a means for analyzing and processing the answers obtained from the AI ​​system. This allows even elderly people and users with low IT literacy to smoothly shop in physical stores. Furthermore, by including a means for recognizing user emotions using an emotion recognition engine and reflecting that data in the AI ​​system, and a means for providing product location information and detailed information in physical stores, a more personalized experience can be provided.

[2049] "User registration information" refers to basic personal information such as the system user's name, email address, and password.

[2050] A "database" is a system for systematically storing and managing collected user information, feedback, emotional data, etc.

[2051] A "generative AI system" refers to an artificial intelligence that generates a response to a task entered by a user.

[2052] An "emotion recognition engine" is software or hardware that analyzes emotions from user input and actions and reflects that data in the system.

[2053] "API format" refers to the standard data format used when exchanging data between systems.

[2054] "Means for analysis and processing" refers to the technology used to convert the answers obtained from the generative AI system into a user-friendly format and add warnings or additional information as necessary.

[2055] A "warning" is a statement that provides information about points that users should be aware of and risks.

[2056] "Additional information" refers to useful supplemental information added to the answer from the generative AI system.

[2057] "Product location information in physical stores" refers to information indicating where products are located in stores.

[2058] "Detailed information" refers to information that allows users to understand the product in detail, such as how to use it, its features, price, and related precautions.

[2059] The system of this invention is designed to support elderly people and users with low IT literacy, with the aim of making the shopping experience in brick-and-mortar stores smoother and more personalized. The specific system configuration and program processing are described below.

[2060] System configuration

[2061] Hardware used:

[2062] Smartphone (Android / iOS)

[2063] Software used:

[2064] Development environment: Android Studio, Xcode

[2065] Emotion recognition engine: Microsoft Azure's "Emotion API"

[2066] Database: Firebase Realtime Database

[2067] Generation AI: OpenAI API

[2068] What the program does

[2069] The system consists of the following main processing steps:

[2070] User Registration:

[2071] When a user launches the app for the first time, they are prompted to create an account by entering their name, email address, and password, which is automatically stored in the Firebase Realtime Database.

[2072] Inputting the issue and querying the AI:

[2073] When a user enters a specific task in the app (e.g., "Where is the rice?"), this task data is first sent to Firebase, which then converts the data into the appropriate API format and queries the OpenAI API.

[2074] Emotion recognition:

[2075] The task data and feedback entered by the user are analyzed for emotions using Microsoft Azure's Emotion API. This emotional data is reflected in the generative AI and used to generate answers.

[2076] Get and display answers:

[2077] The answers obtained from the generative AI system are analyzed and processed on the server, and then displayed on the smartphone in a user-friendly format. Warning messages and additional information may be added as needed.

[2078] Specific example explanation

[2079] Example 1: User registration

[2080] A user enters their name, email address, and password in the in-app registration form and clicks the "Register" button. The registration information is stored in the Firebase Realtime Database.

[2081] Example 2: Entering an assignment

[2082] The user types in "Where is the rice?" and submits it. The server converts this task data into the appropriate API format and queries the OpenAI API.

[2083] Example 3: Emotion Recognition

[2084] The Emotion API analyzes the emotion of the task entered by the user and recognizes it as "anxiety." This emotion data is used as feedback for the generative AI's answer generation process.

[2085] Example 4: Displaying answers

[2086] The AI ​​system generates a response such as, "The rice is in aisle 3 of the food aisle. If you can't find it, please ask a member of staff." This is received by the server, analyzed and processed, and then displayed in a user-friendly format on the smartphone screen.

[2087] Prompt Sentence Examples

[2088] User prompt: "Where is the rice?"

[2089] Emotion: "Anxiety"

[2090] In this way, by combining generative AI and emotion recognition technology throughout the system, we can provide users with a more personalized shopping experience.

[2091] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2092] Step 1: User Registration

[2093] When the user first starts up the device, they enter their name, email address, and password. The device sends this information to the Firebase Realtime Database. The server then stores the received user registration information in the database. This saves the registered user's information and makes it available for later processing.

[2094] Step 2: Enter your assignment

[2095] A user uses an input form within the app to enter a specific task, such as "Where is the rice?", and clicks the submit button. The device then sends this task data to the server, which converts it into the appropriate API format and queries the OpenAI API. This process passes the user's task to the generative AI system.

[2096] Step 3: Emotion Recognition

[2097] The server receives the task data entered by the user and performs emotion analysis using Microsoft Azure's Emotion API. For example, the task data may be recognized as containing the emotion "anxiety." The server then reflects this emotion data in the generative AI system. This allows the user's emotions to be taken into account when generating answers.

[2098] Step 4: Generative AI generates answers

[2099] The server sends a request containing task data and emotion data to the OpenAI API and receives a response from the generative AI model. The generative AI system generates an answer that reflects the user's emotion. For example, the generated answer might be, "The rice is in aisle 3 of the food aisle. If you can't find it, please ask a staff member."

[2100] Step 5: Analyze and process the responses

[2101] The server receives the answers sent by the AI ​​generation system and processes them into a user-friendly format by adding warnings about phishing scams and other scams, as well as additional information. This processing involves analyzing the content of the answers and adding appropriate additional information. For example, a warning about phishing scams may be added.

[2102] Step 6: View your answers

[2103] The server then sends the processed response to the user's smartphone, which then displays the response to the user, including any necessary warnings or additional information. This step provides the information in a format that is easy for the user to understand.

[2104] Step 7: Gather feedback

[2105] Users can enter and submit feedback about the answers provided within the app. The device then sends the feedback data to the server. The server stores the received feedback in a database and analyzes it for use in improving the service next time. For example, feedback entered by a user as "easy to use" regarding an answer is stored and analyzed.

[2106] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2107] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2108] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2109] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2110] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2111] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2112] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2113] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2114] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2115] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2116] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2117] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2118] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2119] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2120] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2121] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2122] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2123] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2124] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2125] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2126] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2127] The following is further disclosed regarding the above embodiment.

[2128] (Claim 1)

[2129] means for receiving and storing user registration information in a database;

[2130] A means for receiving the task input by the user and querying the generating AI system;

[2131] A means for analyzing and processing the answers obtained from the generative AI system; and

[2132] means for displaying the processed answer to the user;

[2133] means for receiving and storing user feedback in a database;

[2134] a means for analyzing the feedback and generating new service suggestions;

[2135] A system including:

[2136] (Claim 2)

[2137] 2. The system of claim 1, further comprising means for converting task data input by a user into an appropriate API format and transmitting the converted data to the generation AI system.

[2138] (Claim 3)

[2139] 10. The system of claim 1, further comprising means for processing the response from the generating AI system into a user-friendly format by adding warnings and additional information.

[2140] "Example 1"

[2141] (Claim 1)

[2142] means for receiving and storing user registration information in a database;

[2143] A means for receiving user-entered challenges and querying the generative AI model;

[2144] A means of analyzing and processing the answers obtained from the generative AI model; and

[2145] means for displaying the processed answer to the user;

[2146] means for receiving and storing user feedback in a database;

[2147] a means for analyzing the feedback and generating new service suggestions;

[2148] A system including:

[2149] (Claim 2)

[2150] 2. The system of claim 1, further comprising means for converting task data input by a user into an appropriate API format and transmitting the converted data to the generative AI model.

[2151] (Claim 3)

[2152] 10. The system of claim 1, further comprising means for processing the response from the generative AI model into a user-friendly format by adding warnings and additional information.

[2153] "Application Example 1"

[2154] (Claim 1)

[2155] means for receiving and storing user registration information in a database;

[2156] A means for receiving the task input by the user and querying the generating AI system;

[2157] A means for analyzing and processing the answers obtained from the generative AI system; and

[2158] means for displaying the processed answer to the user;

[2159] means for receiving and storing user feedback in a database;

[2160] a means for analyzing the feedback and generating new service suggestions;

[2161] means for converting the user's task into text using speech recognition means;

[2162] The system includes means for presenting a response processed by speech synthesis to a user.

[2163] (Claim 2)

[2164] 2. The system of claim 1, further comprising means for converting task data input by a user into an appropriate API format and transmitting the converted data to the generation AI system.

[2165] (Claim 3)

[2166] 10. The system of claim 1, further comprising means for processing the response from the generating AI system into a user-friendly format by adding warnings and additional information.

[2167] "Example 2: Combining Emotion Engines"

[2168] (Claim 1)

[2169] means for receiving and storing user registration information in a database;

[2170] A means for receiving the task input by the user and querying the generating AI system;

[2171] A means for analyzing and processing the answers obtained from the generative AI system; and

[2172] means for displaying the processed answer to the user;

[2173] means for receiving and storing user feedback in a database;

[2174] a means for analyzing the feedback and generating new service suggestions;

[2175] using an emotion engine to recognize emotions from user input and feedback;

[2176] A means for reflecting the recognized emotion data in the generative AI system;

[2177] A system including:

[2178] (Claim 2)

[2179] 2. The system of claim 1, further comprising means for converting task data input by a user into an appropriate API format and transmitting the converted data to the generation AI system.

[2180] (Claim 3)

[2181] 10. The system of claim 1, further comprising means for processing the response from the generating AI system into a user-friendly format by adding warnings and additional information.

[2182] "Application example 2 when combining emotion engines"

[2183] (Claim 1)

[2184] means for receiving and storing user registration information in a database;

[2185] A means for receiving the task input by the user and querying the generating AI system;

[2186] A means for analyzing and processing the answers obtained from the generative AI system; and

[2187] means for displaying the processed answer to the user;

[2188] means for receiving and storing user feedback in a database;

[2189] a means for analyzing the feedback and generating new service suggestions;

[2190] A means of recognizing user emotions using an emotion recognition engine and reflecting that data in the generative AI system;

[2191] A means to provide product location information and detailed information in physical stores,

[2192] A system including:

[2193] (Claim 2)

[2194] 2. The system of claim 1, further comprising means for converting task data input by a user into an appropriate API format and transmitting the converted data to the generation AI system.

[2195] (Claim 3)

[2196] 10. The system of claim 1, further comprising means for processing the response from the generating AI system into a user-friendly format by adding warnings and additional information. [Explanation of symbols]

[2197] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving and storing user registration information in a database; A means for receiving the task input by the user and querying the generating AI system; A means for analyzing and processing the answers obtained from the generative AI system; and means for displaying the processed answer to the user; means for receiving and storing user feedback in a database; a means for analyzing the feedback and generating new service suggestions; A system including:

2. 2. The system according to claim 1, further comprising means for converting task data input by a user into an appropriate API format and transmitting the converted data to the generation AI system.

3. 2. The system of claim 1, further comprising means for adding warnings and additional information to the response from the generating AI system to process it into a user-friendly format.

Images