system

Abstract

Provide a system. means for receiving a data retrieval request and retrieving data from an external data source; A means of converting the acquired data into a certain format and incorporating it into the artificial intelligence to be generated; A means for outputting the captured data in a data format according to an output request; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's business environment, many companies need to manage massive amounts of data and extract valuable information from it. However, traditional methods require knowledge of database schema design and SQL, which reduces business efficiency. Furthermore, data visualization requires specialized software, which is also a burden for users. To solve these problems, a system that can easily and effectively acquire, process, and output data is required. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for receiving a data acquisition request and acquiring data from an external data source, a means for converting the acquired data into a specific format and inputting it into a generated artificial intelligence, and a means for outputting the input data in a data format corresponding to an output request. Furthermore, by including a means for receiving an output request in natural language and issuing instructions to the generated artificial intelligence based on that natural language, a user can easily manipulate data without requiring special knowledge. Furthermore, by including a means for the generated artificial intelligence to display output data in a graph format, data visualization is also performed automatically, improving work efficiency.

[0006] A "data acquisition request" is an instruction received by a server or terminal to acquire data required by a user from an external data source.

[0007] An "external data source" is an external information source, such as a database or API, that the server connects to to obtain data.

[0008] A "means for obtaining data" is a function or process for obtaining data from an external data source.

[0009] "Converting to a certain format" refers to the process of converting the acquired data into a form that is easy to understand for the artificial intelligence that generates it.

[0010] "Generative AI" refers to AI that processes data based on input data in accordance with user requests.

[0011] An "output request" is an instruction received by a server or terminal to output the format and content of data desired by a user.

[0012] "Means for output" refers to a function or process for displaying or providing the data processed by the artificial intelligence in a format desired by the user.

[0013] "Natural language" refers to the normal language (e.g., Japanese, English, etc.) that a user uses to give instructions to the artificial intelligence they create.

[0014] "Display in graph format" means that artificial intelligence that generates a visual representation of data creates a graph and displays it to the user. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system that allows users to easily obtain data and view it in a desired format without requiring special knowledge. How this system is implemented will be described below in detail.

[0037] Overall system overview

[0038] First, the user accesses the system via a terminal. The user requests data acquisition and output in natural language. The terminal receives these requests and sends them to the server. The server accesses external data sources to acquire the required data and inputs it into the AI ​​to be generated. The AI ​​to be generated analyzes the data and outputs it in the format desired by the user. This system automates data acquisition, conversion, and output, significantly reducing the burden on the user.

[0039] Program processing flow

[0040] Data Import

[0041] 1. User Instructions

[0042] The user instructs the device to "import this month's sales data into the generation AI."

[0043] 2. Sending the request

[0044] The terminal analyzes the user's request and sends the request to the server.

[0045] 3. Data Acquisition

[0046] The server accesses an external data source to obtain the current month's sales data.

[0047] The server converts the acquired data into a certain format.

[0048] 4. Data Import

[0049] The server then inputs the converted data into the generation AI.

[0050] Data reference (output)

[0051] 1. User Instructions

[0052] The user instructs the terminal to "show this month's sales data in a graph."

[0053] 2. Sending the request

[0054] The terminal analyzes the user's request and sends the request to the server.

[0055] 3. Generate output data

[0056] The server instructs the generating AI to output the data in graph format.

[0057] The generation AI analyzes the data taken in and generates data in graph format.

[0058] 4. Data transmission and display

[0059] The server transmits the generated graph data to the terminal.

[0060] The terminal displays the received graph to the user.

[0061] Specific examples

[0062] For example, consider a sales representative who wants to view monthly sales data.

[0063] 1. Data Ingestion Request

[0064] The user (sales representative) instructs the device to "input this month's sales data into the generation AI."

[0065] The terminal sends this instruction to the server.

[0066] The server retrieves this month's data from the company's sales database and inputs it into the generation AI.

[0067] 2. Data output request

[0068] The user instructs, "Show me this month's sales data in a graph."

[0069] The terminal sends this instruction to the server.

[0070] The server instructs the generating AI to output the data in graph format.

[0071] The generative AI analyzes the data, generates a graph, and returns it to the server.

[0072] The server sends the graph data to the terminal, which displays it to the user.

[0073] Other examples

[0074] Additionally, if the user requests additional information, it is easy to accommodate.

[0075] 3. Additional requests

[0076] The user instructs, "Please also tell me the increase or decrease in sales this month compared to last month."

[0077] The terminal sends this instruction to the server.

[0078] The server instructs the generating AI to compare last month's data with this month's data and display any increases or decreases.

[0079] The generation AI analyzes the data, generates comparison results, and returns them to the server.

[0080] The server sends the results of the comparison to the terminal, which displays them to the user.

[0081] As described above, the system of the present invention allows users to acquire, analyze, and output data in natural language without requiring any special knowledge, thereby improving work efficiency.

[0082] The processing flow will be explained below.

[0083] Data Import

[0084] Step 1:

[0085] The user types into the terminal, "Enter this month's sales data into the generation AI."

[0086] Step 2:

[0087] The terminal analyzes the user's instructions and sends them to the server as a data acquisition request.

[0088] Step 3:

[0089] The server receives the data retrieval request and accesses an external data source, the company's sales database, to retrieve the current month's sales data.

[0090] Step 4:

[0091] The server converts the acquired data into a format that the generating AI can understand, such as JSON format.

[0092] Step 5:

[0093] The server calls an API to feed the converted data into the generation AI.

[0094] Step 6:

[0095] The generation AI stores the data received from the server internally and notifies the server that the data import is complete.

[0096] Step 7:

[0097] The server notifies the terminal of the completion of data import.

[0098] Step 8:

[0099] The terminal receives a notification from the server and notifies the user that the data import is complete.

[0100] Data reference (output)

[0101] Step 1:

[0102] The user inputs into the terminal, "Show me this month's sales data in a graph."

[0103] Step 2:

[0104] The terminal analyzes the user's instructions and sends them to the server as a data output request.

[0105] Step 3:

[0106] The server receives the data output request and instructs the generation AI to output this month's sales data in graph format.

[0107] Step 4:

[0108] The generation AI analyzes the data taken into it and generates graphs according to instructions.

[0109] Step 5:

[0110] The generation AI returns the generated graph data to the server.

[0111] Step 6:

[0112] The server sends the graph data received from the generation AI to the terminal.

[0113] Step 7:

[0114] The terminal receives the graph data and displays it to the user.

[0115] Specific examples

[0116] A salesperson wants to retrieve this month's sales data and display it as a graph.

[0117] Step 1:

[0118] The sales representative (user) instructs the terminal to "import this month's sales data into the generation AI."

[0119] Step 2:

[0120] The terminal analyzes the instruction and sends a data acquisition request to the server.

[0121] Step 3:

[0122] The server accesses the company's sales database and retrieves the current month's sales data.

[0123] Step 4:

[0124] The server converts the acquired data and feeds it into the generated AI.

[0125] Step 5:

[0126] The generation AI stores the data internally and notifies the server when the import is complete.

[0127] Step 6:

[0128] The server notifies the terminal that the import is complete.

[0129] Step 7:

[0130] The terminal notifies the sales representative that the import is complete.

[0131] Step 8:

[0132] The salesperson (user) instructs the terminal to "show this month's sales data in a graph."

[0133] Step 9:

[0134] The terminal analyzes the instruction and sends a data output request to the server.

[0135] Step 10:

[0136] The server instructs the generating AI to output the data in graph format.

[0137] Step 11:

[0138] The generative AI analyzes the data and generates graphs.

[0139] Step 12:

[0140] The generation AI sends the graph data back to the server.

[0141] Step 13:

[0142] The server transmits the graph data to the terminal.

[0143] Step 14:

[0144] The terminal displays the graph data and allows the sales representative to view it.

[0145] Through the above steps, users can acquire and visually refer to data without having any special technical knowledge.

[0146] Example 1

[0147] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0148] Conventional systems have the problem that it is difficult for users to acquire, analyze, and output data unless they have special knowledge. Furthermore, numerous manual operations are required from acquiring data to displaying it, resulting in reduced work efficiency. Furthermore, because data format conversion is performed manually, there is a high risk of errors and it is difficult to ensure the accuracy of the data.

[0149] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0150] In this invention, the server includes a means for receiving a data acquisition request and acquiring data from an external data source, a means for converting the acquired data into a specific format and incorporating it into a generated artificial intelligence model, and a means for outputting the acquired data in a data format corresponding to an output request. This allows users to instruct data acquisition and output in natural language, making it easy to handle data without special knowledge. Furthermore, since the process from data acquisition to display is automated, business efficiency is improved and data accuracy is also increased.

[0151] A "data acquisition request" is an instruction issued by a user to identify the data they require and acquire that data from an external data source.

[0152] An "external data source" is an information source that an external system, such as a server or database, has to provide specific data.

[0153] A "uniform format" is a standard data representation method for converting data into a unified structure or format.

[0154] A "generative artificial intelligence model" refers to an artificial intelligence algorithm or system that analyzes acquired data and outputs the data in a format that meets the user's requirements.

[0155] "Ingestion" refers to the act of inputting data obtained from external sources into an internal system or artificial intelligence model.

[0156] An "output request" is an instruction from the user to display or extract the analyzed data in a desired format.

[0157] "Natural language analysis means" refers to technology that analyzes the natural language (normal spoken or written language) entered by the user and converts it into appropriate system operations or instructions.

[0158] A "server" is a central computing device or system that acquires, analyzes, and ultimately outputs data in response to user requests.

[0159] A "terminal" is a device, such as a computer or smartphone, that a user uses to access the system and input instructions.

[0160] "Graph format" is a method of displaying data in the form of a bar graph, line graph, etc., to make it easier to understand visually.

[0161] MODE FOR CARRYING OUT THE INVENTION

[0162] The present invention relates to a system that allows users to easily obtain data and view it in a desired format without requiring special knowledge. How this system is implemented will be described below in detail.

[0163] Overall system overview

[0164] First, the user accesses the system via a terminal. Specific terminals include PCs and smartphones. The user issues data acquisition and output requests in natural language. The terminal receives these requests and sends them to the server.

[0165] The server accesses an external data source to obtain the necessary data. For example, a sales database is used as the external data source. MySQL (registered trademark) or PostgreSQL is used as the database management system. The server converts the obtained data into a specific format. The Pandas library is used for data conversion.

[0166] The converted data is then fed into a generative AI model. For example, OpenAI's GPT-4 (registered trademark) is used as the generative AI model. The generative AI model analyzes the fed data and outputs the data in the format desired by the user. This output data can be in the form of a graph or table.

[0167] Data import procedure

[0168] The user instructs the device to "input this month's sales data into the generation AI." The device uses Google's Natural Language API to parse the instruction and sends the request to the server.

[0169] Based on the received request, the server retrieves data from a database named "SELLS_DB" by executing an SQL query such as "SELECT FROM sales WHERE date >= '2023-10-01' AND date <= '2023-10-31'". The retrieved data is converted to JSON format using the Pandas library.

[0170] The server sends the converted JSON data to the generative AI model via an API for ingestion, and the generative AI model stores this data in its internal memory and makes it available for analysis.

[0171] Data reference (output) procedure

[0172] The user says, "Show me this month's sales data in a graph." The device again uses Google's Natural Language API to parse the request and sends the request to the server.

[0173] The server sends a prompt message to the generative AI model to output the data in graph format. The generative AI model analyzes the data stored internally and generates the data in the graph format desired by the user using a graph generation library such as Matplotlib or Plotly.

[0174] The generated graph data is sent from the server to the device, which then displays the data to the user. At this time, JavaScript (registered trademark) such as D3.js may be used for displaying the data.

[0175] As a concrete example, consider the case where a sales representative checks monthly sales data. The user (sales representative) instructs the terminal to "import this month's sales data into the generation AI," and then inputs "display this month's sales data as a graph." This causes this month's sales data to be retrieved from the sales database, analyzed by the generation AI model, and finally displayed as a graph on the terminal.

[0176] Prompt Sentence Examples

[0177] Prompt: "Feed this month's sales data into the Generative AI"

[0178] Prompt: "Show me this month's sales data in a graph."

[0179] As described above, the system of the present invention allows users to acquire, analyze, and output data in natural language without requiring any special knowledge, thereby improving work efficiency.

[0180] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0181] Step 1:

[0182] The user accesses the terminal and inputs instructions for data capture.

[0183] As a specific example of input, you can instruct the device to "input this month's sales data into the generation AI."

[0184] The terminal accepts this input and processes the input data.

[0185] The output is an HTTP request generated by the terminal, which undergoes natural language analysis and is sent to the server.

[0186] Step 2:

[0187] The terminal analyzes the user's instructions using natural language analysis means.

[0188] The input instructions are parsed using Google's Natural Language API.

[0189] From this analysis, the terminal constructs a specific data acquisition request.

[0190] The output is an HTTP request based on the parsed results, which is sent to the server.

[0191] Step 3:

[0192] The server receives the HTTP request and accesses the database to retrieve the required data.

[0193] The server executes SQL queries against the database management system.

[0194] As a specific example, issue the SQL query "SELECT FROM sales WHERE date >= '2023-10-01' AND date <= '2023-10-31'".

[0195] The input is an HTTP request based on the user's requirements.

[0196] The output is the retrieved sales data, which is converted to JSON format.

[0197] Step 4:

[0198] The server converts the data it receives into a specific format.

[0199] The Python Pandas library is used for data conversion.

[0200] The input is sales data obtained from a database.

[0201] The output is data converted into a certain format (e.g. JSON format).

[0202] Step 5:

[0203] The server then inputs the converted data into the generative AI model.

[0204] The server sends the data to a generative AI model (e.g., OpenAI's GPT-4) via an API for ingestion.

[0205] The input is data that has been converted into a certain format.

[0206] The output is the processing result of the generative AI model storing the data in its internal memory.

[0207] Step 6:

[0208] The user accesses the terminal again and inputs an instruction to output data.

[0209] As a specific example of input, the user might instruct the user to "display this month's sales data in a graph."

[0210] The terminal accepts this input and processes the input data.

[0211] The output is an HTTP request generated by the terminal, which is sent to the server after natural language analysis.

[0212] Step 7:

[0213] The terminal analyzes the user's output request using natural language analysis means.

[0214] The input instructions are parsed using Google's Natural Language API.

[0215] From this analysis, the terminal constructs a specific data output request.

[0216] The output is an HTTP request based on the parsed results, which is sent to the server.

[0217] Step 8:

[0218] The server receives the HTTP request and sends output instructions to the generative AI model.

[0219] The server sends prompt messages to the generative AI model to output the data in graph format.

[0220] The input is an HTTP request based on the user's output requirements.

[0221] The output is a prompt message sent to the generative AI model.

[0222] Step 9:

[0223] A generative AI model analyzes the data and generates the data in graph form.

[0224] The generative AI model analyzes the data stored in its internal memory.

[0225] To generate graphs, we use graph generation libraries such as Matplotlib and Plotly.

[0226] Input is data fed into a generative AI model.

[0227] The output is graphical data, generated as an image file or data structure.

[0228] Step 10:

[0229] The server transmits the generated graph data to the terminal.

[0230] The input is graph data received from a generative AI model.

[0231] The output is graph data sent to the terminal.

[0232] Step 11:

[0233] The terminal displays the received graph data to the user.

[0234] The terminal uses the JavaScript D3.js to display graph data.

[0235] The input is the graph data received from the server.

[0236] The output is a graph that is displayed on the user's terminal.

[0237] Through the above processing steps, the user can easily acquire, analyze, and display data.

[0238] (Application example 1)

[0239] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0240] In today's business environment, store managers and employees are required to acquire and analyze data in real time. However, current systems require specialized knowledge and complex operations, which reduces work efficiency. For this reason, a system is needed that can easily acquire data and display it in an appropriate output format.

[0241] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0242] In this invention, the server includes means for receiving a data acquisition request and acquiring information from an external information source, means for converting the acquired information into a certain format and inputting it into the generated artificial intelligence, means for outputting the acquired information in a data format corresponding to the output request, means for receiving and analyzing data acquisition and output requests in natural language, means for transmitting the analyzed natural language to the server, and means for displaying the analyzed and processed data on a user interface. This allows users to acquire data in natural language without having special knowledge and display the necessary information in real time, thereby significantly improving business efficiency.

[0243] A "data acquisition request" is a request made by a user to acquire specific information.

[0244] An "external information source" is an information provider such as a database or cloud service that exists outside the system.

[0245] "Information" refers to data or content obtained from external sources, such as sales data or inventory data.

[0246] A "uniform format" refers to a standard data structure or format suitable for data analysis and display.

[0247] "Generative AI" is an AI technology that analyzes acquired data and outputs it in an appropriate format according to the user's request.

[0248] The "means for receiving and analyzing data acquisition and output requests in natural language" refers to a means for analyzing requests in natural language spoken or input by a user and for the system to understand them.

[0249] The "means for transmitting the analyzed natural language to the server" refers to a means for transmitting the results of the natural language analysis to the server for further processing on the server side.

[0250] The "means for displaying the analyzed and processed data on a user interface" refers to a means for visually presenting the analyzed data to the user.

[0251] A "user interface" is a screen or device through which a user interacts with a system.

[0252] This invention relates to a system that allows users to easily acquire data and view it in the desired format without requiring special knowledge. This system includes data acquisition requests, data conversion, data analysis and output using generation AI, and visual display of data using a user interface.

[0253] Overall system overview

[0254] First, the user accesses the system via a device such as smart glasses or a smartphone. The user requests data acquisition or output using natural language. This request is analyzed by the device and sent to the server. The server then accesses external information sources (e.g., databases or cloud services) to obtain the required information, converts it into a specific format, and feeds it into the generation AI. The generation AI then analyzes the information it has received and outputs data in the format desired by the user. This output data is then sent to the device and displayed visually to the user.

[0255] Hardware and software used

[0256] Hardware:

[0257] Smart glasses (e.g., regular smart glasses)

[0258] Server (e.g. cloud server)

[0259] User device (smartphone, etc.)

[0260] software:

[0261] Natural language processing libraries (e.g., spaCy, BERT)

[0262] Data analysis tools (e.g., Pandas, NumPy)

[0263] Data visualization tools (e.g., Matplotlib, Plotly)

[0264] Generative AI libraries (e.g., OpenAI GPT-4)

[0265] Data acquisition and analysis flow

[0266] 1. User command: The user commands their smart glasses or smartphone to "show me today's sales data."

[0267] 2. Sending and parsing the request: The device parses this instruction using a natural language processing library (e.g., spaCy, BERT) and sends it to the server.

[0268] 3. Data Acquisition: The server accesses the store's sales database to acquire the latest sales data and converts it into a specific format (e.g., CSV, JSON).

[0269] 4. Data import and analysis: The server imports the converted data into a generative AI (e.g., OpenAI GPT-4) for analysis.

[0270] 5. Data transmission and display: Based on the analysis results, the generation AI converts the sales data into a graph format and sends it to the device, which then displays the received data on the screen of the smart glasses or smartphone.

[0271] Specific examples

[0272] For example, when a store manager wants to check inventory status in real time, he or she uses the system by following the steps below.

[0273] 1. Instruction: The store manager speaks to the smart glasses and says, "Tell me the current inventory status."

[0274] 2. Send: The smart glasses parse this request and send it to the server.

[0275] 3. Acquisition and conversion: The server acquires the latest inventory data from the inventory database and inputs it into the generation AI.

[0276] 4. Analysis and output: The generative AI analyzes the data and sends the current inventory status in a list format to the smart glasses.

[0277] 5. Display: The smart glasses visually display the results to the user.

[0278] Prompt Sentence Examples

[0279] "Please get yesterday's sales data and graph it."

[0280] "Show me this week's sales data compared by day of the week."

[0281] Please see a list of current inventory.

[0282] This system allows users to obtain data in natural language without requiring any special knowledge and check the information they need in real time, significantly improving work efficiency and supporting faster decision-making.

[0283] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0284] Step 1:

[0285] The user issues a command in natural language to their smart glasses or smartphone, such as "Show me today's sales data."

[0286] Input: User's voice or text instructions

[0287] Output: Natural language instructions data

[0288] Specific operation: The voice recognition function of smart glasses or smartphone captures the user's instructions as text data.

[0289] Step 2:

[0290] The device analyzes the user's instructions using a natural language processing library (e.g., spaCy, BERT) and sends the results to the server.

[0291] Input: Natural language instructions

[0292] Output: Parsed instruction data

[0293] Specific operation: The device uses a natural language processing library to analyze the intent of the instruction and sends the content of the instruction (e.g., "Get today's sales data") to the server as structured data.

[0294] Step 3:

[0295] The server accesses an external source (e.g., a sales database) and retrieves the required data.

[0296] Input: Parsed instruction data

[0297] Output: Sales data

[0298] Specific operation: The server accesses the sales database, queries today's sales data, and retrieves the data.

[0299] Step 4:

[0300] The server converts the data it receives into a specific format (e.g., CSV, JSON).

[0301] Input: Sales data

[0302] Output: Reformatted sales data

[0303] What happens: The server uses a data conversion library to convert raw sales data into a defined format (e.g. CSV, JSON).

[0304] Step 5:

[0305] The server then inputs the converted data into a generative AI model (e.g., OpenAI GPT-4) for analysis.

[0306] Input: Reformatted sales data

[0307] Output: Analysis result data (e.g., graph data)

[0308] Specific operation: The server inputs data into the generative AI model, analyzes the sales data, and generates the results in graph form.

[0309] Step 6:

[0310] The server transmits the generated graph data to the terminal.

[0311] Input: Analysis result data (graph data)

[0312] Output: Graph data sent to the terminal

[0313] Specific operation: The server uses a network protocol to send the generated graph data to the device.

[0314] Step 7:

[0315] The terminal visually displays the received graph data to the user.

[0316] Input: Graph data sent to the terminal

[0317] Output: Screen showing the graph

[0318] Specific operation: Graph data is drawn on the device's display screen so that the user can visually check it.

[0319] By performing the necessary data processing and calculations at each step in this way, users can easily obtain data in natural language and visually confirm the results.

[0320] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0321] The present invention relates to a system that uses artificial intelligence to recognize a user's natural language instructions and emotions, and acquire, convert, and generate data from external data sources based on those instructions and emotions, and output data in a desired format. In particular, by combining an emotion engine that recognizes the user's emotions, the system enables more adaptive data output according to the user's emotional state.

[0322] Overall system overview

[0323] The system mainly includes the following functional blocks: data acquisition means, data conversion means, artificial intelligence for generation, output means, and emotion engine.

[0324] Key components of the system

[0325] 1. Data Acquisition Request

[0326] A user inputs a data acquisition request in natural language via a terminal, which then transmits the request to a server.

[0327] 2. Data Acquisition and Transformation

[0328] The server retrieves the required data from an external data source (e.g., a company's sales database).

[0329] The acquired data is converted into a certain format (e.g. JSON) by the server.

[0330] 3. Inputting data into the generated AI

[0331] The server then feeds the converted data into the AI ​​that generates it, which stores the data internally and prepares it for processing.

[0332] 4. Functions of the Emotion Engine

[0333] The emotion engine analyzes emotions from the user's natural language instructions.

[0334] The analysis results are fed back to the generating AI, which then analyzes and outputs the data taking into account the emotional state.

[0335] 5. Data reference and output

[0336] The user instructs the server in natural language via the terminal which output format (e.g., graph format) he / she desires.

[0337] The server transmits the instructions to the AI ​​that generates the data, and the AI ​​outputs the data in the specified format based on the internal data. The output format is adjusted based on feedback from the emotion engine.

[0338] The output data is sent to the terminal via the server and can be viewed by the user.

[0339] Specific examples of processing

[0340] A sales rep reviews monthly sales data and analyzes sentiment:

[0341] 1. Data Ingestion Request

[0342] The sales representative (user) instructs the device to "input this month's sales data into the generation AI."

[0343] The terminal sends this instruction to the server as a data acquisition request.

[0344] 2. Data Acquisition and Transformation

[0345] The server accesses the company's sales database to retrieve the current month's sales data.

[0346] The server converts the acquired data into a certain format.

[0347] 3. Data Import

[0348] The server then inputs the converted data into the generation AI.

[0349] The generative AI stores the data internally.

[0350] 4. Emotion Engine Analysis

[0351] When a salesperson says, "Show me this month's sales data in a graph," the emotion engine analyzes the emotion from their natural language.

[0352] For example, if the user indicates that they are feeling stressed, the emotion engine will detect this.

[0353] 5. Data output request

[0354] The terminal sends an output request to the server.

[0355] The server instructs the generative AI to output data in a format that adapts to the user's emotions, taking into account feedback from the emotion engine.

[0356] 6. Data Output and Display

[0357] The generation AI generates graph data taking into account the analysis results of the emotion engine and sends it back to the server.

[0358] The server sends the generated data to the terminal, which displays the graph to the user.

[0359] The above process allows users to acquire, analyze, and output data without any special technical knowledge, and also enables adaptive data output according to the user's emotional state.

[0360] The processing flow will be explained below.

[0361] Data Import

[0362] Step 1:

[0363] The user types into the terminal, "Enter this month's sales data into the generation AI."

[0364] Step 2:

[0365] The terminal analyzes the user's instructions and sends them to the server as a data acquisition request.

[0366] Step 3:

[0367] The server receives the data acquisition request and accesses the company's sales database, which is an external data source, to acquire the current month's sales data.

[0368] Step 4:

[0369] The server converts the acquired data into a format that the generating AI can understand, such as JSON format.

[0370] Step 5:

[0371] The server calls an API to feed the converted data into the generation AI.

[0372] Step 6:

[0373] The generation AI stores the data received from the server internally and notifies the server that the data import is complete.

[0374] Step 7:

[0375] The server notifies the terminal of the completion of data import.

[0376] Step 8:

[0377] The terminal receives a notification from the server and notifies the user that the data import is complete.

[0378] Data lookup and sentiment analysis

[0379] Step 1:

[0380] The user inputs into the terminal, "Show me this month's sales data in a graph."

[0381] Step 2:

[0382] The terminal analyzes the user's instructions and transmits the user's natural language to the emotion engine.

[0383] Step 3:

[0384] The emotion engine analyzes emotions from the user's natural language and feeds the analysis results back to the generative AI.

[0385] Step 4:

[0386] The device sends the user's instructions and the results of emotion analysis to the server.

[0387] Step 5:

[0388] The server receives the data output request and the sentiment analysis results, and instructs the generation AI to output this month's sales data in graph format. The instruction also includes the sentiment analysis results.

[0389] Step 6:

[0390] The AI ​​analyzes the data and emotion analysis results and generates graphs according to instructions. For example, if the user is feeling stressed, it will use colors and layouts to reduce stress.

[0391] Step 7:

[0392] The generation AI returns the generated graph data to the server.

[0393] Step 8:

[0394] The server sends the graph data received from the generation AI to the terminal.

[0395] Step 9:

[0396] The terminal receives the graph data and displays it to the user.

[0397] Specific examples

[0398] A sales rep pulls this month's sales data and analyzes sentiment.

[0399] Step 1:

[0400] The sales representative (user) instructs the device to "input this month's sales data into the generation AI."

[0401] Step 2:

[0402] The terminal analyzes the instruction and sends a data acquisition request to the server.

[0403] Step 3:

[0404] The server accesses the company's sales database to retrieve the current month's sales data.

[0405] Step 4:

[0406] The server converts the acquired data and feeds it into the generated AI.

[0407] Step 5:

[0408] The generation AI stores the data internally and notifies the server when the import is complete.

[0409] Step 6:

[0410] The server notifies the terminal that the import is complete.

[0411] Step 7:

[0412] The terminal notifies the sales representative that the import is complete.

[0413] Step 8:

[0414] The sales representative (user) instructs, "Display this month's sales data in a graph."

[0415] Step 9:

[0416] The terminal sends the user's emotion along with the instruction to the emotion engine for analysis.

[0417] Step 10:

[0418] The emotion engine analyzes the instructions and detects when the user is feeling stressed.

[0419] Step 11:

[0420] The emotion engine feeds back the analysis results to the server.

[0421] Step 12:

[0422] The terminal sends a data output request and the emotion analysis results to the server.

[0423] Step 13:

[0424] The server instructs the generating AI to output data in graph format, taking into account the results of emotion analysis.

[0425] Step 14:

[0426] The generative AI generates appropriate graph data based on data and sentiment analysis results.

[0427] Step 15:

[0428] The generation AI sends the graph data back to the server.

[0429] Step 16:

[0430] The server transmits the graph data to the terminal.

[0431] Step 17:

[0432] The terminal displays the graph data to the user, who can then refer to it.

[0433] This specific processing flow allows users to obtain and output data through instructions in natural language, and by utilizing sentiment analysis in the process, it enables more user-friendly and effective data reference.

[0434] Example 2

[0435] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0436] In conventional data acquisition and output systems, it was difficult for users to not only give instructions in natural language, but also to recognize the emotional state of the user from the content of the instructions and adjust the data output accordingly. As a result, it was not possible to output data that was adapted to the user's emotional state, and this resulted in a problem of not being able to sufficiently improve the user experience.

[0437] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving a data acquisition request and acquiring data from an external data source, means for converting the acquired data into a certain format and inputting it into the AI ​​to be generated, means for outputting the input data in a data format corresponding to the output request based on the data, means for analyzing emotions from a user's natural language instruction and feeding back the emotional state to the AI ​​to be generated, and means for the AI ​​to analyze and output data taking the user's emotional state into consideration. This makes it possible to acquire, analyze, and output data adapted to the user's emotional state.

[0438] "Data retrieval request" refers to an instruction sent by a user to the system to retrieve data from an external data source.

[0439] "External data source" refers to an information source that exists outside the system and where the required data is stored.

[0440] "Means for obtaining data" refers to the functionality for accessing and obtaining the required data from an external data source.

[0441] "Means for converting data into a certain format and inputting it into the AI ​​to be generated" refers to the function for converting acquired data into a specific format (such as JSON format) and inputting the converted data into the internal memory of the AI ​​to be generated.

[0442] "Generative AI" refers to an AI model that is used to manipulate and analyze ingested data and generate output in response to user requests.

[0443] An "output request" refers to a user's request to the system to output data in a particular format.

[0444] "Output means" refers to a function for generating and outputting necessary data in a specific format in response to a user's output request.

[0445] "Natural language instructions" refers to instructions given by a user to a system in natural language (e.g., Japanese, English, etc.).

[0446] "Means for analyzing emotions" refers to a function for analyzing the user's emotional state from the user's natural language instructions, voice, etc.

[0447] An "emotion engine" refers to an analysis device that analyzes emotions from the user's natural language instructions and reflects the results in the system.

[0448] "Feedback" refers to transmitting analysis results and other information to the artificial intelligence that generates it, and then using that information for further processing and adjustments.

[0449] "Means for analyzing and outputting data" refers to a function for analyzing the captured data taking into account the emotional state of the user and outputting it in the most appropriate format.

[0450] This system recognizes the user's natural language instructions and emotions, and uses artificial intelligence to acquire, convert, and generate the necessary data from external data sources based on that information, outputting the data in the desired format. The main hardware and software components that make up the system are as follows:

[0451] Hardware and Software

[0452] 1. Device:

[0453] A device that allows users to input data acquisition and output requests. Examples include smartphones, tablets, and PCs.

[0454] 2. Server:

[0455] It is responsible for accessing external data sources, acquiring and transforming the data, feeding it into the generative AI model, receiving feedback from the emotion engine, and finally converting the data into an output format. It uses a regular web server or a cloud-based server.

[0456] 3. Generative AI Model:

[0457] It is an artificial intelligence model that analyzes and generates acquired data based on user requests. Typical generative AI models include GPT and BERT.

[0458] 4. Emotion Engine:

[0459] This is software or a module that analyzes emotions from users' natural language instructions and feeds the results back to a generative AI model. Natural language processing technology is used for emotion analysis.

[0460] Specific operations and procedures

[0461] 1. User data request:

[0462] The user issues instructions in natural language via the device, such as "Input this month's sales data into the generation AI." The device then sends this instruction to the server as text data.

[0463] 2. Data retrieval and transformation by the server:

[0464] Based on the received data retrieval request, the server accesses external data sources such as a company's sales database to retrieve the required data, and then converts the retrieved data into a specific format (such as JSON).

[0465] 3. Server imports data into the generated AI model:

[0466] The server sends the converted data to the generative AI model, which then stores the data in its internal memory.

[0467] 4. User sentiment analysis:

[0468] When a user requests data output, for example, "Show me this month's sales data in a graph," the emotion engine analyzes the user's emotional state (e.g., stress, joy, etc.) from this instruction and feeds the results back to the generative AI model.

[0469] 5. Data generation and output:

[0470] Based on feedback from the emotion engine, the generative AI model analyzes the data in a format that adapts to the user's emotional state, and finally generates the data in an output format (e.g., graph format).

[0471] The generated data is sent to the terminal via the server, and the terminal displays it to the user.

[0472] Specific examples

[0473] Prompt Sentence Examples

[0474] User: "Please input this month's sales data into the generation AI."

[0475] User: "Show me this month's sales data in a graph."

[0476] This allows users to easily acquire, analyze, and output data using natural language without any special technical knowledge.In addition, by introducing an emotion engine, it is possible to realize adaptive data output according to the user's emotional state.

[0477] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0478] Step 1:

[0479] Sending a data retrieval request

[0480] explanation

[0481] The user inputs a data acquisition request in natural language into the terminal, which then sends the request to the server.

[0482] input

[0483] Natural language instructions from the user (e.g., "Please feed this month's sales data into the Generative AI").

[0484] output

[0485] The request sent to the server to retrieve data.

[0486] Specific actions

[0487] The user types into the device, "Please input this month's sales data into the generation AI." The device converts the natural language instruction into text format and sends it to the server as an HTTP request.

[0488] Step 2:

[0489] Data Acquisition

[0490] explanation

[0491] When the server receives a data retrieval request, it accesses external data sources to retrieve the required data.

[0492] input

[0493] The data retrieval request sent to the server (e.g. received as a "fetch_data" command).

[0494] output

[0495] Data obtained from external sources.

[0496] Specific actions

[0497] The server runs an SQL query (e.g., "SELECT FROM sales_data WHERE month = 'current_month'") against the company's sales database to retrieve the current month's sales data.

[0498] Step 3:

[0499] Data transformation

[0500] explanation

[0501] The server converts the retrieved data into a certain format (e.g. JSON).

[0502] input

[0503] Raw data obtained from external sources.

[0504] output

[0505] The data converted to JSON format.

[0506] Specific actions

[0507] Convert the sales data acquired by the server into JSON format. Example: Convert as follows.

[0508] json

[0509] {

[0510] "month": "current_month",

[0511] "total_sales": 100000,

[0512] "details": [

[0513] {"product": "Product A", "sales": 50000},

[0514] {"product": "Product B", "sales": 50000}

[0515] ]

[0516] }

[0517] Step 4:

[0518] Ingesting data into a generative AI model

[0519] explanation

[0520] The server then feeds the converted data into a generative AI model, which stores the data internally and prepares it for processing.

[0521] input

[0522] The data converted to JSON format.

[0523] output

[0524] Data fed into generative AI models.

[0525] Specific actions

[0526] The server uses an internal API to send data to the generative AI model, which then stores the received data in its internal memory.

[0527] Step 5:

[0528] Sentiment analysis using natural language instructions

[0529] explanation

[0530] The user makes an output request in natural language, and the emotion engine analyzes the emotion from the natural language instruction.

[0531] input

[0532] Natural language commands from the user (e.g., "Show me this month's sales data in a graph").

[0533] output

[0534] Analyzed emotion data.

[0535] Specific actions

[0536] A user might say, "Show me this month's sales data in a graph." The emotion engine analyzes this natural language and identifies emotions such as "stress" or "joy."

[0537] Step 6:

[0538] Generating emotional feedback and sending it to the AI ​​model

[0539] explanation

[0540] The analysis results of the emotion engine are fed back to the generative AI model.

[0541] input

[0542] Analyzed emotion data.

[0543] output

[0544] Emotional feedback sent to a generative AI model.

[0545] Specific actions

[0546] The emotion engine sends the analysis results (e.g., "emotion": "stress", "confidence_score": 0.85") to the generative AI model.

[0547] Step 7:

[0548] Data generation and output

[0549] explanation

[0550] The generative AI model analyzes the data and generates an output format taking into account the user's emotional state. The server sends the generated data to the device and displays it on the device.

[0551] input

[0552] Data in a generative AI model that has received emotional feedback.

[0553] output

[0554] The output data generated and the data format sent to the terminal.

[0555] Specific actions

[0556] The generative AI model analyzes the data taking into account the emotional state and generates output data in the form of a graph. The server sends the generated graph data to the device, which then displays the graph to the user.

[0557] Through the above processing steps, the user can make a data acquisition request in natural language and the results can be displayed adaptively according to the user's emotional state.

[0558] (Application example 2)

[0559] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0560] Conventional food delivery systems process orders without considering the user's emotional state, making it difficult to provide food that suits the user's current mood or emotion. Furthermore, they are unable to respond to instructions in natural language, which can lead to a lack of user experience. This can lead to a decrease in user satisfaction and complicated ordering.

[0561] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a data acquisition request and acquiring data from an external data source, means for converting the acquired data into a certain format and inputting it into the generated artificial intelligence, means for outputting the acquired data in a data format corresponding to the output request, and means for recognizing the user's instructions and emotions in natural language and adjusting the data output according to the user's emotional state using an emotion engine. This allows users to easily order the dish that best suits their emotional state using natural language, significantly improving the convenience and satisfaction of food delivery.

[0562] A "data acquisition request" is a request from a user to acquire data from a specified external data source.

[0563] An "external data source" is an information source that exists outside the system and provides data, such as various databases and APIs.

[0564] A "consistent format" is a standardized format that makes data easier to process, such as JSON or XML.

[0565] "Generative AI" is a general term for AI that has the ability to generate new information or results based on acquired data.

[0566] "Natural language" refers to a language that humans use on a daily basis, and is not a programming language.

[0567] An "emotion engine" is a software component for analyzing and recognizing emotions from a user's natural language input.

[0568] "Output format" refers to the format or style in which data is presented to a user, visually or otherwise.

[0569] "Filtering" is the process of selecting or excluding data based on specific conditions.

[0570] "User experience" refers to the overall experience and satisfaction a user has when using a product or service.

[0571] To implement this invention, a system including the following functional blocks and procedures is required: This system comprises a data acquisition means, a data conversion means, an artificial intelligence for generation, an output means, and an emotion engine.

[0572] Hardware and software used

[0573] Hardware: Smartphone (iOS, ANDROID (registered trademark))

[0574] software:

[0575] Natural language processing libraries (e.g., Google Cloud Natural Language API)

[0576] Emotion recognition engine (e.g., Microsoft® Azure® Emotion API)

[0577] Database (e.g. Firebase Firestore)

[0578] RESTful API (for data retrieval)

[0579] Front-end frameworks (e.g., React Native)

[0580] Processing a data retrieval request

[0581] The server receives a data retrieval request from the user and accesses the food delivery service's database to retrieve the dish data for each restaurant. This data is then converted into a standard format such as JSON.

[0582] Data transformation and incorporation into generative artificial intelligence

[0583] The server converts the acquired data into a certain format (such as JSON) and inputs it into the AI ​​model to be generated. The AI ​​model to be generated stores the converted data internally and prepares it for subsequent processing.

[0584] Natural Language Processing and Emotion Recognition

[0585] The user's natural language instructions are received via the smartphone. These instructions are parsed using the Google Cloud Natural Language API to understand the user's request. The Microsoft Azure Emotion API is then used to recognize the user's emotional state. The emotion recognition engine determines emotions such as "relaxed," "stressed," or "happy."

[0586] Adjusting output requirements and data formats

[0587] The user inputs the desired dish format in natural language and sends it back to the server. The server instructs the generation AI to consider feedback from the emotion engine and output the dish that best suits the user's emotional state. The generation AI filters the dish data based on the analysis results of the emotion engine, generates a personalized dish list, and displays it on the smartphone.

[0588] Specific examples

[0589] 1. User input: "I'm a little tired today, so I want something relaxing to eat."

[0590] 2. Natural language processing: Analyze using the Google Cloud Natural Language API to understand the user's specifications.

[0591] 3. Emotion recognition: Determine whether the user is "tired" using the Microsoft Azure Emotion API.

[0592] 4. Example output: "Good work! Would you like some relaxing herbal tea and a light salad?"

[0593] This allows users to easily order the food that best suits their emotional state using natural language, significantly improving the convenience and satisfaction of food delivery.

[0594] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0595] Step 1:

[0596] The user inputs a data acquisition request in natural language via a smartphone, and this instruction (e.g., "Recommend me a dish that suits my current mood") is sent from the device to the server.

[0597] Input: User's natural language instructions.

[0598] Output: The data retrieval request sent to the server.

[0599] Step 2:

[0600] The server accesses the food delivery service's database to obtain the food data for each restaurant.

[0601] Input: Data retrieval request.

[0602] Output: Retrieved dish data (e.g., JSON format).

[0603] Step 3:

[0604] The server converts the acquired data into a certain format (e.g., JSON), and the converted data is fed into the AI ​​that generates it.

[0605] Input: The acquired food data.

[0606] Output: The converted data (e.g., in JSON format).

[0607] Step 4:

[0608] The user again inputs the desired cooking style in natural language via the smartphone, and the terminal transmits this output request to the server.

[0609] Input: The user's output request (e.g., "Show me relaxing recipes").

[0610] Output: The output request sent to the server.

[0611] Step 5:

[0612] The server uses the Google Cloud Natural Language API to parse the user's natural language output request and understand the user's request.

[0613] Input: A natural language output request.

[0614] Output: The parsed request content.

[0615] Step 6:

[0616] The server uses the Microsoft Azure Emotion API to recognize the user's emotional state. The emotion engine determines emotions such as "relaxed" or "stressed" from the user's natural language instructions.

[0617] Input: A natural language output request.

[0618] Output: Perceived emotional state.

[0619] Step 7:

[0620] The server provides feedback to the artificial intelligence it generates, instructing it to filter the food data based on the user's emotional state and generate a personalized list of dishes.

[0621] Input: Recognized emotional state and transformed food data.

[0622] Output: A filtered, personalized list of dishes.

[0623] Step 8:

[0624] The server sends the generated recipe list to the terminal, which displays it on the smartphone.

[0625] Input: A filtered list of dishes.

[0626] Output: The list of dishes displayed to the user.

[0627] Through the above processing steps, users can easily order the dish that best suits their emotional state using natural language.

[0628] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0629] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0630] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0631] [Second embodiment]

[0632] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0633] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0634] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0635] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0636] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0637] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0638] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0639] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0640] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0641] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0642] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0643] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0644] The present invention relates to a system that allows users to easily obtain data and view it in a desired format without requiring special knowledge. How this system is implemented will be described below in detail.

[0645] Overall system overview

[0646] First, the user accesses the system via a terminal. The user requests data acquisition and output in natural language. The terminal receives these requests and sends them to the server. The server accesses external data sources to acquire the required data and inputs it into the AI ​​to be generated. The AI ​​to be generated analyzes the data and outputs it in the format desired by the user. This system automates data acquisition, conversion, and output, significantly reducing the burden on the user.

[0647] Program processing flow

[0648] Data Import

[0649] 1. User Instructions

[0650] The user instructs the device to "import this month's sales data into the generation AI."

[0651] 2. Sending the request

[0652] The terminal analyzes the user's request and sends the request to the server.

[0653] 3. Data Acquisition

[0654] The server accesses an external data source to obtain the current month's sales data.

[0655] The server converts the acquired data into a certain format.

[0656] 4. Data Import

[0657] The server then inputs the converted data into the generation AI.

[0658] Data reference (output)

[0659] 1. User Instructions

[0660] The user instructs the terminal to "show this month's sales data in a graph."

[0661] 2. Sending the request

[0662] The terminal analyzes the user's request and sends the request to the server.

[0663] 3. Generate output data

[0664] The server instructs the generating AI to output the data in graph format.

[0665] The generation AI analyzes the data taken in and generates data in graph format.

[0666] 4. Data transmission and display

[0667] The server transmits the generated graph data to the terminal.

[0668] The terminal displays the received graph to the user.

[0669] Specific examples

[0670] For example, consider a sales representative who wants to view monthly sales data.

[0671] 1. Data Ingestion Request

[0672] The user (sales representative) instructs the device to "input this month's sales data into the generation AI."

[0673] The terminal sends this instruction to the server.

[0674] The server retrieves this month's data from the company's sales database and inputs it into the generation AI.

[0675] 2. Data output request

[0676] The user instructs, "Show me this month's sales data in a graph."

[0677] The terminal sends this instruction to the server.

[0678] The server instructs the generating AI to output the data in graph format.

[0679] The generative AI analyzes the data, generates a graph, and returns it to the server.

[0680] The server sends the graph data to the terminal, which displays it to the user.

[0681] Other examples

[0682] Additionally, if the user requests additional information, it is easy to accommodate.

[0683] 3. Additional requests

[0684] The user instructs, "Please also tell me the increase or decrease in sales this month compared to last month."

[0685] The terminal sends this instruction to the server.

[0686] The server instructs the generating AI to compare last month's data with this month's data and display any increases or decreases.

[0687] The generation AI analyzes the data, generates comparison results, and returns them to the server.

[0688] The server sends the results of the comparison to the terminal, which displays them to the user.

[0689] As described above, the system of the present invention allows users to acquire, analyze, and output data in natural language without requiring any special knowledge, thereby improving work efficiency.

[0690] The processing flow will be explained below.

[0691] Data Import

[0692] Step 1:

[0693] The user types into the terminal, "Enter this month's sales data into the generation AI."

[0694] Step 2:

[0695] The terminal analyzes the user's instructions and sends them to the server as a data acquisition request.

[0696] Step 3:

[0697] The server receives the data retrieval request and accesses an external data source, the company's sales database, to retrieve the current month's sales data.

[0698] Step 4:

[0699] The server converts the acquired data into a format that the generating AI can understand, such as JSON format.

[0700] Step 5:

[0701] The server calls an API to feed the converted data into the generation AI.

[0702] Step 6:

[0703] The generation AI stores the data received from the server internally and notifies the server that the data import is complete.

[0704] Step 7:

[0705] The server notifies the terminal of the completion of data import.

[0706] Step 8:

[0707] The terminal receives a notification from the server and notifies the user that the data import is complete.

[0708] Data reference (output)

[0709] Step 1:

[0710] The user inputs into the terminal, "Show me this month's sales data in a graph."

[0711] Step 2:

[0712] The terminal analyzes the user's instructions and sends them to the server as a data output request.

[0713] Step 3:

[0714] The server receives the data output request and instructs the generation AI to output this month's sales data in graph format.

[0715] Step 4:

[0716] The generation AI analyzes the data taken into it and generates graphs according to instructions.

[0717] Step 5:

[0718] The generation AI returns the generated graph data to the server.

[0719] Step 6:

[0720] The server sends the graph data received from the generation AI to the terminal.

[0721] Step 7:

[0722] The terminal receives the graph data and displays it to the user.

[0723] Specific examples

[0724] A salesperson wants to retrieve this month's sales data and display it as a graph.

[0725] Step 1:

[0726] The sales representative (user) instructs the terminal to "import this month's sales data into the generation AI."

[0727] Step 2:

[0728] The terminal analyzes the instruction and sends a data acquisition request to the server.

[0729] Step 3:

[0730] The server accesses the company's sales database and retrieves the current month's sales data.

[0731] Step 4:

[0732] The server converts the acquired data and feeds it into the generated AI.

[0733] Step 5:

[0734] The generation AI stores the data internally and notifies the server when the import is complete.

[0735] Step 6:

[0736] The server notifies the terminal that the import is complete.

[0737] Step 7:

[0738] The terminal notifies the sales representative that the import is complete.

[0739] Step 8:

[0740] The salesperson (user) instructs the terminal to "show this month's sales data in a graph."

[0741] Step 9:

[0742] The terminal analyzes the instruction and sends a data output request to the server.

[0743] Step 10:

[0744] The server instructs the generating AI to output the data in graph format.

[0745] Step 11:

[0746] The generative AI analyzes the data and generates graphs.

[0747] Step 12:

[0748] The generation AI sends the graph data back to the server.

[0749] Step 13:

[0750] The server transmits the graph data to the terminal.

[0751] Step 14:

[0752] The terminal displays the graph data and allows the sales representative to view it.

[0753] Through the above steps, users can acquire and visually refer to data without having any special technical knowledge.

[0754] Example 1

[0755] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0756] Conventional systems have the problem that it is difficult for users to acquire, analyze, and output data unless they have special knowledge. Furthermore, numerous manual operations are required from acquiring data to displaying it, resulting in reduced work efficiency. Furthermore, because data format conversion is performed manually, there is a high risk of errors and it is difficult to ensure the accuracy of the data.

[0757] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0758] In this invention, the server includes a means for receiving a data acquisition request and acquiring data from an external data source, a means for converting the acquired data into a specific format and incorporating it into a generated artificial intelligence model, and a means for outputting the acquired data in a data format corresponding to an output request. This allows users to instruct data acquisition and output in natural language, making it easy to handle data without special knowledge. Furthermore, since the process from data acquisition to display is automated, business efficiency is improved and data accuracy is also increased.

[0759] A "data acquisition request" is an instruction issued by a user to identify the data they require and acquire that data from an external data source.

[0760] An "external data source" is an information source that an external system, such as a server or database, has to provide specific data.

[0761] A "uniform format" is a standard data representation method for converting data into a unified structure or format.

[0762] A "generative artificial intelligence model" refers to an artificial intelligence algorithm or system that analyzes acquired data and outputs the data in a format that meets the user's requirements.

[0763] "Ingestion" refers to the act of inputting data obtained from external sources into an internal system or artificial intelligence model.

[0764] An "output request" is an instruction from the user to display or extract the analyzed data in a desired format.

[0765] "Natural language analysis means" refers to technology that analyzes the natural language (normal spoken or written language) entered by the user and converts it into appropriate system operations or instructions.

[0766] A "server" is a central computing device or system that acquires, analyzes, and ultimately outputs data in response to user requests.

[0767] A "terminal" is a device, such as a computer or smartphone, that a user uses to access the system and input instructions.

[0768] "Graph format" is a method of displaying data in the form of a bar graph, line graph, etc., to make it easier to understand visually.

[0769] MODE FOR CARRYING OUT THE INVENTION

[0770] The present invention relates to a system that allows users to easily obtain data and view it in a desired format without requiring special knowledge. How this system is implemented will be described below in detail.

[0771] Overall system overview

[0772] First, the user accesses the system via a terminal. Specific terminals include PCs and smartphones. The user issues data acquisition and output requests in natural language. The terminal receives these requests and sends them to the server.

[0773] The server accesses an external data source to obtain the necessary data. For example, a sales database is used as the external data source. A database management system such as MySQL or PostgreSQL is used. The server converts the obtained data into a specific format. The Pandas library is used for data conversion.

[0774] The converted data is then fed into a generative AI model. For example, OpenAI's GPT-4 is used as the generative AI model. The generative AI model analyzes the data and outputs it in the format desired by the user. This output data can be in the form of a graph or table.

[0775] Data import procedure

[0776] The user instructs the device to "feed this month's sales data into the generation AI," and the device uses Google's Natural Language API to parse the instruction and send the request to the server.

[0777] Based on the received request, the server retrieves data from a database named "SELLS_DB" by executing an SQL query such as "SELECT FROM sales WHERE date >= '2023-10-01' AND date <= '2023-10-31'". The retrieved data is converted to JSON format using the Pandas library.

[0778] The server sends the converted JSON data to the generative AI model via an API for ingestion, and the generative AI model stores this data in its internal memory and makes it available for analysis.

[0779] Data reference (output) procedure

[0780] The user says, "Show me this month's sales data in a graph." The device again uses Google's Natural Language API to parse the request and sends the request to the server.

[0781] The server sends a prompt message to the generative AI model to output the data in graph format. The generative AI model analyzes the data stored internally and generates the data in the graph format desired by the user using a graph generation library such as Matplotlib or Plotly.

[0782] The generated graph data is sent from the server to the device, which then displays the data to the user. In this case, JavaScript such as D3.js may be used for displaying the data.

[0783] As a concrete example, consider the case where a sales representative checks monthly sales data. The user (sales representative) instructs the terminal to "import this month's sales data into the generation AI," and then inputs "display this month's sales data as a graph." This causes this month's sales data to be retrieved from the sales database, analyzed by the generation AI model, and finally displayed as a graph on the terminal.

[0784] Prompt Sentence Examples

[0785] Prompt: "Feed this month's sales data into the Generative AI"

[0786] Prompt: "Show me this month's sales data in a graph."

[0787] As described above, the system of the present invention allows users to acquire, analyze, and output data in natural language without requiring any special knowledge, thereby improving work efficiency.

[0788] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0789] Step 1:

[0790] The user accesses the terminal and inputs instructions for data capture.

[0791] As a specific example of input, you can instruct the device to "input this month's sales data into the generation AI."

[0792] The terminal accepts this input and processes the input data.

[0793] The output is an HTTP request generated by the terminal, which undergoes natural language analysis and is sent to the server.

[0794] Step 2:

[0795] The terminal analyzes the user's instructions using natural language analysis means.

[0796] The input instructions are parsed using Google's Natural Language API.

[0797] From this analysis, the terminal constructs a specific data acquisition request.

[0798] The output is an HTTP request based on the parsed results, which is sent to the server.

[0799] Step 3:

[0800] The server receives the HTTP request and accesses the database to retrieve the required data.

[0801] The server executes SQL queries against the database management system.

[0802] As a specific example, issue the SQL query "SELECT FROM sales WHERE date >= '2023-10-01' AND date <= '2023-10-31'".

[0803] The input is an HTTP request based on the user's requirements.

[0804] The output is the retrieved sales data, which is converted to JSON format.

[0805] Step 4:

[0806] The server converts the data it receives into a specific format.

[0807] The Python Pandas library is used for data conversion.

[0808] The input is sales data obtained from a database.

[0809] The output is data converted into a certain format (e.g. JSON format).

[0810] Step 5:

[0811] The server then inputs the converted data into the generative AI model.

[0812] The server sends the data to a generative AI model (e.g., OpenAI's GPT-4) via an API for ingestion.

[0813] The input is data that has been converted into a certain format.

[0814] The output is the processing result of the generative AI model storing the data in its internal memory.

[0815] Step 6:

[0816] The user accesses the terminal again and inputs an instruction to output data.

[0817] As a specific example of input, the user might instruct the user to "display this month's sales data in a graph."

[0818] The terminal accepts this input and processes the input data.

[0819] The output is an HTTP request generated by the terminal, which is sent to the server after natural language analysis.

[0820] Step 7:

[0821] The terminal analyzes the user's output request using natural language analysis means.

[0822] The input instructions are parsed using Google's Natural Language API.

[0823] From this analysis, the terminal constructs a specific data output request.

[0824] The output is an HTTP request based on the parsed results, which is sent to the server.

[0825] Step 8:

[0826] The server receives the HTTP request and sends output instructions to the generative AI model.

[0827] The server sends prompt messages to the generative AI model to output the data in graph format.

[0828] The input is an HTTP request based on the user's output requirements.

[0829] The output is a prompt message sent to the generative AI model.

[0830] Step 9:

[0831] A generative AI model analyzes the data and generates the data in graph form.

[0832] The generative AI model analyzes the data stored in its internal memory.

[0833] To generate graphs, we use graph generation libraries such as Matplotlib and Plotly.

[0834] Input is data fed into a generative AI model.

[0835] The output is graphical data, generated as an image file or data structure.

[0836] Step 10:

[0837] The server transmits the generated graph data to the terminal.

[0838] The input is graph data received from a generative AI model.

[0839] The output is graph data sent to the terminal.

[0840] Step 11:

[0841] The terminal displays the received graph data to the user.

[0842] The terminal uses the JavaScript D3.js to display graph data.

[0843] The input is the graph data received from the server.

[0844] The output is a graph that is displayed on the user's terminal.

[0845] Through the above processing steps, the user can easily acquire, analyze, and display data.

[0846] (Application example 1)

[0847] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0848] In today's business environment, store managers and employees are required to acquire and analyze data in real time. However, current systems require specialized knowledge and complex operations, which reduces work efficiency. For this reason, a system is needed that can easily acquire data and display it in an appropriate output format.

[0849] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0850] In this invention, the server includes means for receiving a data acquisition request and acquiring information from an external information source, means for converting the acquired information into a certain format and inputting it into the generated artificial intelligence, means for outputting the acquired information in a data format corresponding to the output request, means for receiving and analyzing data acquisition and output requests in natural language, means for transmitting the analyzed natural language to the server, and means for displaying the analyzed and processed data on a user interface. This allows users to acquire data in natural language without having special knowledge and display the necessary information in real time, thereby significantly improving business efficiency.

[0851] A "data acquisition request" is a request made by a user to acquire specific information.

[0852] An "external information source" is an information provider such as a database or cloud service that exists outside the system.

[0853] "Information" refers to data or content obtained from external sources, such as sales data or inventory data.

[0854] A "uniform format" refers to a standard data structure or format suitable for data analysis and display.

[0855] "Generative AI" is an AI technology that analyzes acquired data and outputs it in an appropriate format according to the user's request.

[0856] The "means for receiving and analyzing data acquisition and output requests in natural language" refers to a means for analyzing requests in natural language spoken or input by a user and for the system to understand them.

[0857] The "means for transmitting the analyzed natural language to the server" refers to a means for transmitting the results of the natural language analysis to the server for further processing on the server side.

[0858] The "means for displaying the analyzed and processed data on a user interface" refers to a means for visually presenting the analyzed data to the user.

[0859] A "user interface" is a screen or device through which a user interacts with a system.

[0860] This invention relates to a system that allows users to easily acquire data and view it in the desired format without requiring special knowledge. This system includes data acquisition requests, data conversion, data analysis and output using generation AI, and visual display of data using a user interface.

[0861] Overall system overview

[0862] First, the user accesses the system via a device such as smart glasses or a smartphone. The user requests data acquisition or output using natural language. This request is analyzed by the device and sent to the server. The server then accesses external information sources (e.g., databases or cloud services) to obtain the required information, converts it into a specific format, and feeds it into the generation AI. The generation AI then analyzes the information it has received and outputs data in the format desired by the user. This output data is then sent to the device and displayed visually to the user.

[0863] Hardware and software used

[0864] Hardware:

[0865] Smart glasses (e.g., regular smart glasses)

[0866] Server (e.g. cloud server)

[0867] User device (smartphone, etc.)

[0868] software:

[0869] Natural language processing libraries (e.g., spaCy, BERT)

[0870] Data analysis tools (e.g., Pandas, NumPy)

[0871] Data visualization tools (e.g., Matplotlib, Plotly)

[0872] Generative AI libraries (e.g., OpenAI GPT-4)

[0873] Data acquisition and analysis flow

[0874] 1. User command: The user commands their smart glasses or smartphone to "show me today's sales data."

[0875] 2. Sending and parsing the request: The device parses this instruction using a natural language processing library (e.g., spaCy, BERT) and sends it to the server.

[0876] 3. Data Acquisition: The server accesses the store's sales database to acquire the latest sales data and converts it into a specific format (e.g., CSV, JSON).

[0877] 4. Data import and analysis: The server imports the converted data into a generative AI (e.g., OpenAI GPT-4) for analysis.

[0878] 5. Data transmission and display: Based on the analysis results, the generation AI converts the sales data into a graph format and sends it to the device, which then displays the received data on the screen of the smart glasses or smartphone.

[0879] Specific examples

[0880] For example, when a store manager wants to check inventory status in real time, he or she uses the system by following the steps below.

[0881] 1. Instruction: The store manager speaks to the smart glasses and says, "Tell me the current inventory status."

[0882] 2. Send: The smart glasses parse this request and send it to the server.

[0883] 3. Acquisition and conversion: The server acquires the latest inventory data from the inventory database and inputs it into the generation AI.

[0884] 4. Analysis and output: The generative AI analyzes the data and sends the current inventory status in a list format to the smart glasses.

[0885] 5. Display: The smart glasses visually display the results to the user.

[0886] Prompt Sentence Examples

[0887] "Please get yesterday's sales data and graph it."

[0888] "Show me this week's sales data compared by day of the week."

[0889] Please see a list of current inventory.

[0890] This system allows users to obtain data in natural language without requiring any special knowledge and check the information they need in real time, significantly improving work efficiency and supporting faster decision-making.

[0891] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0892] Step 1:

[0893] The user issues a command in natural language to their smart glasses or smartphone, such as "Show me today's sales data."

[0894] Input: User's voice or text instructions

[0895] Output: Natural language instructions data

[0896] Specific operation: The voice recognition function of smart glasses or smartphone captures the user's instructions as text data.

[0897] Step 2:

[0898] The device analyzes the user's instructions using a natural language processing library (e.g., spaCy, BERT) and sends the results to the server.

[0899] Input: Natural language instructions

[0900] Output: Parsed instruction data

[0901] Specific operation: The device uses a natural language processing library to analyze the intent of the instruction and sends the content of the instruction (e.g., "Get today's sales data") to the server as structured data.

[0902] Step 3:

[0903] The server accesses an external source (e.g., a sales database) and retrieves the required data.

[0904] Input: Parsed instruction data

[0905] Output: Sales data

[0906] Specific operation: The server accesses the sales database, queries today's sales data, and retrieves the data.

[0907] Step 4:

[0908] The server converts the data it receives into a specific format (e.g., CSV, JSON).

[0909] Input: Sales data

[0910] Output: Reformatted sales data

[0911] What happens: The server uses a data conversion library to convert raw sales data into a defined format (e.g. CSV, JSON).

[0912] Step 5:

[0913] The server then inputs the converted data into a generative AI model (e.g., OpenAI GPT-4) for analysis.

[0914] Input: Reformatted sales data

[0915] Output: Analysis result data (e.g., graph data)

[0916] Specific operation: The server inputs data into the generative AI model, analyzes the sales data, and generates the results in graph form.

[0917] Step 6:

[0918] The server transmits the generated graph data to the terminal.

[0919] Input: Analysis result data (graph data)

[0920] Output: Graph data sent to the terminal

[0921] Specific operation: The server uses a network protocol to send the generated graph data to the device.

[0922] Step 7:

[0923] The terminal visually displays the received graph data to the user.

[0924] Input: Graph data sent to the terminal

[0925] Output: Screen showing the graph

[0926] Specific operation: Graph data is drawn on the device's display screen so that the user can visually check it.

[0927] By performing the necessary data processing and calculations at each step in this way, users can easily obtain data in natural language and visually confirm the results.

[0928] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0929] The present invention relates to a system that uses artificial intelligence to recognize a user's natural language instructions and emotions, and acquire, convert, and generate data from external data sources based on those instructions and emotions, and output data in a desired format. In particular, by combining an emotion engine that recognizes the user's emotions, the system enables more adaptive data output according to the user's emotional state.

[0930] Overall system overview

[0931] The system mainly includes the following functional blocks: data acquisition means, data conversion means, artificial intelligence for generation, output means, and emotion engine.

[0932] Key components of the system

[0933] 1. Data Acquisition Request

[0934] A user inputs a data acquisition request in natural language via a terminal, which then transmits the request to a server.

[0935] 2. Data Acquisition and Transformation

[0936] The server retrieves the required data from an external data source (e.g., a company's sales database).

[0937] The acquired data is converted into a certain format (e.g. JSON) by the server.

[0938] 3. Inputting data into the generated AI

[0939] The server then feeds the converted data into the AI ​​that generates it, which stores the data internally and prepares it for processing.

[0940] 4. Functions of the Emotion Engine

[0941] The emotion engine analyzes emotions from the user's natural language instructions.

[0942] The analysis results are fed back to the generating AI, which then analyzes and outputs the data taking into account the emotional state.

[0943] 5. Data reference and output

[0944] The user instructs the server in natural language via the terminal which output format (e.g., graph format) he / she desires.

[0945] The server transmits the instructions to the AI ​​that generates the data, and the AI ​​outputs the data in the specified format based on the internal data. The output format is adjusted based on feedback from the emotion engine.

[0946] The output data is sent to the terminal via the server and can be viewed by the user.

[0947] Specific examples of processing

[0948] A sales rep reviews monthly sales data and analyzes sentiment:

[0949] 1. Data Ingestion Request

[0950] The sales representative (user) instructs the device to "input this month's sales data into the generation AI."

[0951] The terminal sends this instruction to the server as a data acquisition request.

[0952] 2. Data Acquisition and Transformation

[0953] The server accesses the company's sales database to retrieve the current month's sales data.

[0954] The server converts the acquired data into a certain format.

[0955] 3. Data Import

[0956] The server then inputs the converted data into the generation AI.

[0957] The generative AI stores the data internally.

[0958] 4. Emotion Engine Analysis

[0959] When a salesperson says, "Show me this month's sales data in a graph," the emotion engine analyzes the emotion from their natural language.

[0960] For example, if the user indicates that they are feeling stressed, the emotion engine will detect this.

[0961] 5. Data output request

[0962] The terminal sends an output request to the server.

[0963] The server instructs the generative AI to output data in a format that adapts to the user's emotions, taking into account feedback from the emotion engine.

[0964] 6. Data Output and Display

[0965] The generation AI generates graph data taking into account the analysis results of the emotion engine and sends it back to the server.

[0966] The server sends the generated data to the terminal, which displays the graph to the user.

[0967] The above process allows users to acquire, analyze, and output data without any special technical knowledge, and also enables adaptive data output according to the user's emotional state.

[0968] The processing flow will be explained below.

[0969] Data Import

[0970] Step 1:

[0971] The user types into the terminal, "Enter this month's sales data into the generation AI."

[0972] Step 2:

[0973] The terminal analyzes the user's instructions and sends them to the server as a data acquisition request.

[0974] Step 3:

[0975] The server receives the data acquisition request and accesses the company's sales database, which is an external data source, to acquire the current month's sales data.

[0976] Step 4:

[0977] The server converts the acquired data into a format that the generating AI can understand, such as JSON format.

[0978] Step 5:

[0979] The server calls an API to feed the converted data into the generation AI.

[0980] Step 6:

[0981] The generation AI stores the data received from the server internally and notifies the server that the data import is complete.

[0982] Step 7:

[0983] The server notifies the terminal of the completion of data import.

[0984] Step 8:

[0985] The terminal receives a notification from the server and notifies the user that the data import is complete.

[0986] Data lookup and sentiment analysis

[0987] Step 1:

[0988] The user inputs into the terminal, "Show me this month's sales data in a graph."

[0989] Step 2:

[0990] The terminal analyzes the user's instructions and transmits the user's natural language to the emotion engine.

[0991] Step 3:

[0992] The emotion engine analyzes emotions from the user's natural language and feeds the analysis results back to the generative AI.

[0993] Step 4:

[0994] The device sends the user's instructions and the results of emotion analysis to the server.

[0995] Step 5:

[0996] The server receives the data output request and the sentiment analysis results, and instructs the generation AI to output this month's sales data in graph format. The instruction also includes the sentiment analysis results.

[0997] Step 6:

[0998] The AI ​​analyzes the data and emotion analysis results and generates graphs according to instructions. For example, if the user is feeling stressed, it will use colors and layouts to reduce stress.

[0999] Step 7:

[1000] The generation AI returns the generated graph data to the server.

[1001] Step 8:

[1002] The server sends the graph data received from the generation AI to the terminal.

[1003] Step 9:

[1004] The terminal receives the graph data and displays it to the user.

[1005] Specific examples

[1006] A sales rep pulls this month's sales data and analyzes sentiment.

[1007] Step 1:

[1008] The sales representative (user) instructs the device to "input this month's sales data into the generation AI."

[1009] Step 2:

[1010] The terminal analyzes the instruction and sends a data acquisition request to the server.

[1011] Step 3:

[1012] The server accesses the company's sales database to retrieve the current month's sales data.

[1013] Step 4:

[1014] The server converts the acquired data and feeds it into the generated AI.

[1015] Step 5:

[1016] The generation AI stores the data internally and notifies the server when the import is complete.

[1017] Step 6:

[1018] The server notifies the terminal that the import is complete.

[1019] Step 7:

[1020] The terminal notifies the sales representative that the import is complete.

[1021] Step 8:

[1022] The sales representative (user) instructs, "Display this month's sales data in a graph."

[1023] Step 9:

[1024] The terminal sends the user's emotion along with the instruction to the emotion engine for analysis.

[1025] Step 10:

[1026] The emotion engine analyzes the instructions and detects when the user is feeling stressed.

[1027] Step 11:

[1028] The emotion engine feeds back the analysis results to the server.

[1029] Step 12:

[1030] The terminal sends a data output request and the emotion analysis results to the server.

[1031] Step 13:

[1032] The server instructs the generating AI to output data in graph format, taking into account the results of emotion analysis.

[1033] Step 14:

[1034] The generative AI generates appropriate graph data based on data and sentiment analysis results.

[1035] Step 15:

[1036] The generation AI sends the graph data back to the server.

[1037] Step 16:

[1038] The server transmits the graph data to the terminal.

[1039] Step 17:

[1040] The terminal displays the graph data to the user, who can then refer to it.

[1041] This specific processing flow allows users to obtain and output data through instructions in natural language, and by utilizing sentiment analysis in the process, it enables more user-friendly and effective data reference.

[1042] Example 2

[1043] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1044] In conventional data acquisition and output systems, it was difficult for users to not only give instructions in natural language, but also to recognize the emotional state of the user from the content of the instructions and adjust the data output accordingly. As a result, it was not possible to output data that was adapted to the user's emotional state, and this resulted in a problem of not being able to sufficiently improve the user experience.

[1045] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving a data acquisition request and acquiring data from an external data source, means for converting the acquired data into a certain format and inputting it into the AI ​​to be generated, means for outputting the input data in a data format corresponding to the output request based on the data, means for analyzing emotions from a user's natural language instruction and feeding back the emotional state to the AI ​​to be generated, and means for the AI ​​to analyze and output data taking the user's emotional state into consideration. This makes it possible to acquire, analyze, and output data adapted to the user's emotional state.

[1046] "Data retrieval request" refers to an instruction sent by a user to the system to retrieve data from an external data source.

[1047] "External data source" refers to an information source that exists outside the system and where the required data is stored.

[1048] "Means for obtaining data" refers to the functionality for accessing and obtaining the required data from an external data source.

[1049] "Means for converting data into a certain format and inputting it into the AI ​​to be generated" refers to the function for converting acquired data into a specific format (such as JSON format) and inputting the converted data into the internal memory of the AI ​​to be generated.

[1050] "Generative AI" refers to an AI model that is used to manipulate and analyze ingested data and generate output in response to user requests.

[1051] An "output request" refers to a user's request to the system to output data in a particular format.

[1052] "Output means" refers to a function for generating and outputting necessary data in a specific format in response to a user's output request.

[1053] "Natural language instructions" refers to instructions given by a user to a system in natural language (e.g., Japanese, English, etc.).

[1054] "Means for analyzing emotions" refers to a function for analyzing the user's emotional state from the user's natural language instructions, voice, etc.

[1055] An "emotion engine" refers to an analysis device that analyzes emotions from the user's natural language instructions and reflects the results in the system.

[1056] "Feedback" refers to transmitting analysis results and other information to the artificial intelligence that generates it, and then using that information for further processing and adjustments.

[1057] "Means for analyzing and outputting data" refers to a function for analyzing the captured data taking into account the emotional state of the user and outputting it in the most appropriate format.

[1058] This system recognizes the user's natural language instructions and emotions, and uses artificial intelligence to acquire, convert, and generate the necessary data from external data sources based on that information, outputting the data in the desired format. The main hardware and software components that make up the system are as follows:

[1059] Hardware and Software

[1060] 1. Device:

[1061] A device that allows users to input data acquisition and output requests. Examples include smartphones, tablets, and PCs.

[1062] 2. Server:

[1063] It is responsible for accessing external data sources, acquiring and transforming the data, feeding it into the generative AI model, receiving feedback from the emotion engine, and finally converting the data into an output format. It uses a regular web server or a cloud-based server.

[1064] 3. Generative AI Model:

[1065] It is an artificial intelligence model that analyzes and generates acquired data based on user requests. Typical generative AI models include GPT and BERT.

[1066] 4. Emotion Engine:

[1067] This is software or a module that analyzes emotions from users' natural language instructions and feeds the results back to a generative AI model. Natural language processing technology is used for emotion analysis.

[1068] Specific operations and procedures

[1069] 1. User data request:

[1070] The user issues instructions in natural language via the device, such as "Input this month's sales data into the generation AI." The device then sends this instruction to the server as text data.

[1071] 2. Data retrieval and transformation by the server:

[1072] Based on the received data retrieval request, the server accesses external data sources such as a company's sales database to retrieve the required data, and then converts the retrieved data into a specific format (such as JSON).

[1073] 3. Server imports data into the generated AI model:

[1074] The server sends the converted data to the generative AI model, which then stores the data in its internal memory.

[1075] 4. User sentiment analysis:

[1076] When a user requests data output, for example, "Show me this month's sales data in a graph," the emotion engine analyzes the user's emotional state (e.g., stress, joy, etc.) from this instruction and feeds the results back to the generative AI model.

[1077] 5. Data generation and output:

[1078] Based on feedback from the emotion engine, the generative AI model analyzes the data in a format that adapts to the user's emotional state, and finally generates the data in an output format (e.g., graph format).

[1079] The generated data is sent to the terminal via the server, and the terminal displays it to the user.

[1080] Specific examples

[1081] Prompt Sentence Examples

[1082] User: "Please input this month's sales data into the generation AI."

[1083] User: "Show me this month's sales data in a graph."

[1084] This allows users to easily acquire, analyze, and output data using natural language without any special technical knowledge.In addition, by introducing an emotion engine, it is possible to realize adaptive data output according to the user's emotional state.

[1085] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1086] Step 1:

[1087] Sending a data retrieval request

[1088] explanation

[1089] The user inputs a data acquisition request in natural language into the terminal, which then sends the request to the server.

[1090] input

[1091] Natural language instructions from the user (e.g., "Please feed this month's sales data into the Generative AI").

[1092] output

[1093] The request sent to the server to retrieve data.

[1094] Specific actions

[1095] The user types into the device, "Please input this month's sales data into the generation AI." The device converts the natural language instruction into text format and sends it to the server as an HTTP request.

[1096] Step 2:

[1097] Data Acquisition

[1098] explanation

[1099] When the server receives a data retrieval request, it accesses external data sources to retrieve the required data.

[1100] input

[1101] The data retrieval request sent to the server (e.g. received as a "fetch_data" command).

[1102] output

[1103] Data obtained from external sources.

[1104] Specific actions

[1105] The server runs an SQL query (e.g., "SELECT FROM sales_data WHERE month = 'current_month'") against the company's sales database to retrieve the current month's sales data.

[1106] Step 3:

[1107] Data transformation

[1108] explanation

[1109] The server converts the retrieved data into a certain format (e.g. JSON).

[1110] input

[1111] Raw data obtained from external sources.

[1112] output

[1113] The data converted to JSON format.

[1114] Specific actions

[1115] Convert the sales data acquired by the server into JSON format. Example: Convert as follows.

[1116] json

[1117] {

[1118] "month": "current_month",

[1119] "total_sales": 100000,

[1120] "details": [

[1121] {"product": "Product A", "sales": 50000},

[1122] {"product": "Product B", "sales": 50000}

[1123] ]

[1124] }

[1125] Step 4:

[1126] Ingesting data into a generative AI model

[1127] explanation

[1128] The server then feeds the converted data into a generative AI model, which stores the data internally and prepares it for processing.

[1129] input

[1130] The data converted to JSON format.

[1131] output

[1132] Data fed into generative AI models.

[1133] Specific actions

[1134] The server uses an internal API to send data to the generative AI model, which then stores the received data in its internal memory.

[1135] Step 5:

[1136] Sentiment analysis using natural language instructions

[1137] explanation

[1138] The user makes an output request in natural language, and the emotion engine analyzes the emotion from the natural language instruction.

[1139] input

[1140] Natural language commands from the user (e.g., "Show me this month's sales data in a graph").

[1141] output

[1142] Analyzed emotion data.

[1143] Specific actions

[1144] A user might say, "Show me this month's sales data in a graph." The emotion engine analyzes this natural language and identifies emotions such as "stress" or "joy."

[1145] Step 6:

[1146] Generating emotional feedback and sending it to the AI ​​model

[1147] explanation

[1148] The analysis results of the emotion engine are fed back to the generative AI model.

[1149] input

[1150] Analyzed emotion data.

[1151] output

[1152] Emotional feedback sent to a generative AI model.

[1153] Specific actions

[1154] The emotion engine sends the analysis results (e.g., "emotion": "stress", "confidence_score": 0.85") to the generative AI model.

[1155] Step 7:

[1156] Data generation and output

[1157] explanation

[1158] The generative AI model analyzes the data and generates an output format taking into account the user's emotional state. The server sends the generated data to the device and displays it on the device.

[1159] input

[1160] Data in a generative AI model that has received emotional feedback.

[1161] output

[1162] The output data generated and the data format sent to the terminal.

[1163] Specific actions

[1164] The generative AI model analyzes the data taking into account the emotional state and generates output data in the form of a graph. The server sends the generated graph data to the device, which then displays the graph to the user.

[1165] Through the above processing steps, the user can make a data acquisition request in natural language and the results can be displayed adaptively according to the user's emotional state.

[1166] (Application example 2)

[1167] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1168] Conventional food delivery systems process orders without considering the user's emotional state, making it difficult to provide food that suits the user's current mood or emotion. Furthermore, they are unable to respond to instructions in natural language, which can lead to a lack of user experience. This can lead to a decrease in user satisfaction and complicated ordering.

[1169] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a data acquisition request and acquiring data from an external data source, means for converting the acquired data into a certain format and inputting it into the generated artificial intelligence, means for outputting the acquired data in a data format corresponding to the output request, and means for recognizing the user's instructions and emotions in natural language and adjusting the data output according to the user's emotional state using an emotion engine. This allows users to easily order the dish that best suits their emotional state using natural language, significantly improving the convenience and satisfaction of food delivery.

[1170] A "data acquisition request" is a request from a user to acquire data from a specified external data source.

[1171] An "external data source" is an information source that exists outside the system and provides data, such as various databases and APIs.

[1172] A "consistent format" is a standardized format that makes data easier to process, such as JSON or XML.

[1173] "Generative AI" is a general term for AI that has the ability to generate new information or results based on acquired data.

[1174] "Natural language" refers to a language that humans use on a daily basis, and is not a programming language.

[1175] An "emotion engine" is a software component for analyzing and recognizing emotions from a user's natural language input.

[1176] "Output format" refers to the format or style in which data is presented to a user, visually or otherwise.

[1177] "Filtering" is the process of selecting or excluding data based on specific conditions.

[1178] "User experience" refers to the overall experience and satisfaction a user has when using a product or service.

[1179] To implement this invention, a system including the following functional blocks and procedures is required: This system comprises a data acquisition means, a data conversion means, an artificial intelligence for generation, an output means, and an emotion engine.

[1180] Hardware and software used

[1181] Hardware: Smartphone (iOS, Android)

[1182] software:

[1183] Natural language processing libraries (e.g., Google Cloud Natural Language API)

[1184] Emotion recognition engine (e.g. Microsoft Azure Emotion API)

[1185] Database (e.g. Firebase Firestore)

[1186] RESTful API (for data retrieval)

[1187] Front-end frameworks (e.g., React Native)

[1188] Processing a data retrieval request

[1189] The server receives a data retrieval request from the user and accesses the food delivery service's database to retrieve the dish data for each restaurant. This data is then converted into a standard format such as JSON.

[1190] Data transformation and incorporation into generative artificial intelligence

[1191] The server converts the acquired data into a certain format (such as JSON) and inputs it into the AI ​​model to be generated. The AI ​​model to be generated stores the converted data internally and prepares it for subsequent processing.

[1192] Natural Language Processing and Emotion Recognition

[1193] The user's natural language instructions are received via the smartphone. These instructions are parsed using the Google Cloud Natural Language API to understand the user's request. The Microsoft Azure Emotion API is then used to recognize the user's emotional state. The emotion recognition engine determines emotions such as "relaxed," "stressed," or "happy."

[1194] Adjusting output requirements and data formats

[1195] The user inputs the desired dish format in natural language and sends it back to the server. The server instructs the generation AI to consider feedback from the emotion engine and output the dish that best suits the user's emotional state. The generation AI filters the dish data based on the analysis results of the emotion engine, generates a personalized dish list, and displays it on the smartphone.

[1196] Specific examples

[1197] 1. User input: "I'm a little tired today, so I want something relaxing to eat."

[1198] 2. Natural language processing: Analyze using the Google Cloud Natural Language API to understand the user's specifications.

[1199] 3. Emotion recognition: Determine whether the user is "tired" using the Microsoft Azure Emotion API.

[1200] 4. Example output: "Good work! Would you like some relaxing herbal tea and a light salad?"

[1201] This allows users to easily order the food that best suits their emotional state using natural language, significantly improving the convenience and satisfaction of food delivery.

[1202] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1203] Step 1:

[1204] The user inputs a data acquisition request in natural language via a smartphone, and this instruction (e.g., "Recommend me a dish that suits my current mood") is sent from the device to the server.

[1205] Input: User's natural language instructions.

[1206] Output: The data retrieval request sent to the server.

[1207] Step 2:

[1208] The server accesses the food delivery service's database to obtain the food data for each restaurant.

[1209] Input: Data retrieval request.

[1210] Output: Retrieved dish data (e.g., JSON format).

[1211] Step 3:

[1212] The server converts the acquired data into a certain format (e.g., JSON), and the converted data is fed into the AI ​​that generates it.

[1213] Input: The acquired food data.

[1214] Output: The converted data (e.g., in JSON format).

[1215] Step 4:

[1216] The user again inputs the desired cooking style in natural language via the smartphone, and the terminal transmits this output request to the server.

[1217] Input: The user's output request (e.g., "Show me relaxing recipes").

[1218] Output: The output request sent to the server.

[1219] Step 5:

[1220] The server uses the Google Cloud Natural Language API to parse the user's natural language output request and understand the user's request.

[1221] Input: A natural language output request.

[1222] Output: The parsed request content.

[1223] Step 6:

[1224] The server uses the Microsoft Azure Emotion API to recognize the user's emotional state. The emotion engine determines emotions such as "relaxed" or "stressed" from the user's natural language instructions.

[1225] Input: A natural language output request.

[1226] Output: Perceived emotional state.

[1227] Step 7:

[1228] The server provides feedback to the artificial intelligence it generates, instructing it to filter the food data based on the user's emotional state and generate a personalized list of dishes.

[1229] Input: Recognized emotional state and transformed food data.

[1230] Output: A filtered, personalized list of dishes.

[1231] Step 8:

[1232] The server sends the generated recipe list to the terminal, which displays it on the smartphone.

[1233] Input: A filtered list of dishes.

[1234] Output: The list of dishes displayed to the user.

[1235] Through the above processing steps, users can easily order the dish that best suits their emotional state using natural language.

[1236] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1237] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1238] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1239] [Third embodiment]

[1240] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1241] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1242] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1243] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1244] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1245] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1246] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1247] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1248] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1249] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1250] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1251] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1252] The present invention relates to a system that allows users to easily obtain data and view it in a desired format without requiring special knowledge. How this system is implemented will be described below in detail.

[1253] Overall system overview

[1254] First, the user accesses the system via a terminal. The user requests data acquisition and output in natural language. The terminal receives these requests and sends them to the server. The server accesses external data sources to acquire the required data and inputs it into the AI ​​to be generated. The AI ​​to be generated analyzes the data and outputs it in the format desired by the user. This system automates data acquisition, conversion, and output, significantly reducing the burden on the user.

[1255] Program processing flow

[1256] Data Import

[1257] 1. User Instructions

[1258] The user instructs the device to "import this month's sales data into the generation AI."

[1259] 2. Sending the request

[1260] The terminal analyzes the user's request and sends the request to the server.

[1261] 3. Data Acquisition

[1262] The server accesses an external data source to obtain the current month's sales data.

[1263] The server converts the acquired data into a certain format.

[1264] 4. Data Import

[1265] The server then inputs the converted data into the generation AI.

[1266] Data reference (output)

[1267] 1. User Instructions

[1268] The user instructs the terminal to "show this month's sales data in a graph."

[1269] 2. Sending the request

[1270] The terminal analyzes the user's request and sends the request to the server.

[1271] 3. Generate output data

[1272] The server instructs the generating AI to output the data in graph format.

[1273] The generation AI analyzes the data taken in and generates data in graph format.

[1274] 4. Data transmission and display

[1275] The server transmits the generated graph data to the terminal.

[1276] The terminal displays the received graph to the user.

[1277] Specific examples

[1278] For example, consider a sales representative who wants to view monthly sales data.

[1279] 1. Data Ingestion Request

[1280] The user (sales representative) instructs the device to "input this month's sales data into the generation AI."

[1281] The terminal sends this instruction to the server.

[1282] The server retrieves this month's data from the company's sales database and inputs it into the generation AI.

[1283] 2. Data output request

[1284] The user instructs, "Show me this month's sales data in a graph."

[1285] The terminal sends this instruction to the server.

[1286] The server instructs the generating AI to output the data in graph format.

[1287] The generative AI analyzes the data, generates a graph, and returns it to the server.

[1288] The server sends the graph data to the terminal, which displays it to the user.

[1289] Other examples

[1290] Additionally, if the user requests additional information, it is easy to accommodate.

[1291] 3. Additional requests

[1292] The user instructs, "Please also tell me the increase or decrease in sales this month compared to last month."

[1293] The terminal sends this instruction to the server.

[1294] The server instructs the generating AI to compare last month's data with this month's data and display any increases or decreases.

[1295] The generation AI analyzes the data, generates comparison results, and returns them to the server.

[1296] The server sends the results of the comparison to the terminal, which displays them to the user.

[1297] As described above, the system of the present invention allows users to acquire, analyze, and output data in natural language without requiring any special knowledge, thereby improving work efficiency.

[1298] The processing flow will be explained below.

[1299] Data Import

[1300] Step 1:

[1301] The user types into the terminal, "Enter this month's sales data into the generation AI."

[1302] Step 2:

[1303] The terminal analyzes the user's instructions and sends them to the server as a data acquisition request.

[1304] Step 3:

[1305] The server receives the data retrieval request and accesses an external data source, the company's sales database, to retrieve the current month's sales data.

[1306] Step 4:

[1307] The server converts the acquired data into a format that the generating AI can understand, such as JSON format.

[1308] Step 5:

[1309] The server calls an API to feed the converted data into the generation AI.

[1310] Step 6:

[1311] The generation AI stores the data received from the server internally and notifies the server that the data import is complete.

[1312] Step 7:

[1313] The server notifies the terminal of the completion of data import.

[1314] Step 8:

[1315] The terminal receives a notification from the server and notifies the user that the data import is complete.

[1316] Data reference (output)

[1317] Step 1:

[1318] The user inputs into the terminal, "Show me this month's sales data in a graph."

[1319] Step 2:

[1320] The terminal analyzes the user's instructions and sends them to the server as a data output request.

[1321] Step 3:

[1322] The server receives the data output request and instructs the generation AI to output this month's sales data in graph format.

[1323] Step 4:

[1324] The generation AI analyzes the data taken into it and generates graphs according to instructions.

[1325] Step 5:

[1326] The generation AI returns the generated graph data to the server.

[1327] Step 6:

[1328] The server sends the graph data received from the generation AI to the terminal.

[1329] Step 7:

[1330] The terminal receives the graph data and displays it to the user.

[1331] Specific examples

[1332] A salesperson wants to retrieve this month's sales data and display it as a graph.

[1333] Step 1:

[1334] The sales representative (user) instructs the terminal to "import this month's sales data into the generation AI."

[1335] Step 2:

[1336] The terminal analyzes the instruction and sends a data acquisition request to the server.

[1337] Step 3:

[1338] The server accesses the company's sales database and retrieves the current month's sales data.

[1339] Step 4:

[1340] The server converts the acquired data and feeds it into the generated AI.

[1341] Step 5:

[1342] The generation AI stores the data internally and notifies the server when the import is complete.

[1343] Step 6:

[1344] The server notifies the terminal that the import is complete.

[1345] Step 7:

[1346] The terminal notifies the sales representative that the import is complete.

[1347] Step 8:

[1348] The salesperson (user) instructs the terminal to "show this month's sales data in a graph."

[1349] Step 9:

[1350] The terminal analyzes the instruction and sends a data output request to the server.

[1351] Step 10:

[1352] The server instructs the generating AI to output the data in graph format.

[1353] Step 11:

[1354] The generative AI analyzes the data and generates graphs.

[1355] Step 12:

[1356] The generation AI sends the graph data back to the server.

[1357] Step 13:

[1358] The server transmits the graph data to the terminal.

[1359] Step 14:

[1360] The terminal displays the graph data and allows the sales representative to view it.

[1361] Through the above steps, users can acquire and visually refer to data without having any special technical knowledge.

[1362] Example 1

[1363] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1364] Conventional systems have the problem that it is difficult for users to acquire, analyze, and output data unless they have special knowledge. Furthermore, numerous manual operations are required from acquiring data to displaying it, resulting in reduced work efficiency. Furthermore, because data format conversion is performed manually, there is a high risk of errors and it is difficult to ensure the accuracy of the data.

[1365] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1366] In this invention, the server includes a means for receiving a data acquisition request and acquiring data from an external data source, a means for converting the acquired data into a specific format and incorporating it into a generated artificial intelligence model, and a means for outputting the acquired data in a data format corresponding to an output request. This allows users to instruct data acquisition and output in natural language, making it easy to handle data without special knowledge. Furthermore, since the process from data acquisition to display is automated, business efficiency is improved and data accuracy is also increased.

[1367] A "data acquisition request" is an instruction issued by a user to identify the data they require and acquire that data from an external data source.

[1368] An "external data source" is an information source that an external system, such as a server or database, has to provide specific data.

[1369] A "uniform format" is a standard data representation method for converting data into a unified structure or format.

[1370] A "generative artificial intelligence model" refers to an artificial intelligence algorithm or system that analyzes acquired data and outputs the data in a format that meets the user's requirements.

[1371] "Ingestion" refers to the act of inputting data obtained from external sources into an internal system or artificial intelligence model.

[1372] An "output request" is an instruction from the user to display or extract the analyzed data in a desired format.

[1373] "Natural language analysis means" refers to technology that analyzes the natural language (normal spoken or written language) entered by the user and converts it into appropriate system operations or instructions.

[1374] A "server" is a central computing device or system that acquires, analyzes, and ultimately outputs data in response to user requests.

[1375] A "terminal" is a device, such as a computer or smartphone, that a user uses to access the system and input instructions.

[1376] "Graph format" is a method of displaying data in the form of a bar graph, line graph, etc., to make it easier to understand visually.

[1377] MODE FOR CARRYING OUT THE INVENTION

[1378] The present invention relates to a system that allows users to easily obtain data and view it in a desired format without requiring special knowledge. How this system is implemented will be described below in detail.

[1379] Overall system overview

[1380] First, the user accesses the system via a terminal. Specific terminals include PCs and smartphones. The user issues data acquisition and output requests in natural language. The terminal receives these requests and sends them to the server.

[1381] The server accesses an external data source to obtain the necessary data. For example, a sales database is used as the external data source. A database management system such as MySQL or PostgreSQL is used. The server converts the obtained data into a specific format. The Pandas library is used for data conversion.

[1382] The converted data is then fed into a generative AI model. For example, OpenAI's GPT-4 is used as the generative AI model. The generative AI model analyzes the data and outputs it in the format desired by the user. This output data can be in the form of a graph or table.

[1383] Data import procedure

[1384] The user instructs the device to "feed this month's sales data into the generation AI," and the device uses Google's Natural Language API to parse the instruction and send the request to the server.

[1385] Based on the received request, the server retrieves data from a database named "SELLS_DB" by executing an SQL query such as "SELECT FROM sales WHERE date >= '2023-10-01' AND date <= '2023-10-31'". The retrieved data is converted to JSON format using the Pandas library.

[1386] The server sends the converted JSON data to the generative AI model via an API for ingestion, and the generative AI model stores this data in its internal memory and makes it available for analysis.

[1387] Data reference (output) procedure

[1388] The user says, "Show me this month's sales data in a graph." The device again uses Google's Natural Language API to parse the request and sends the request to the server.

[1389] The server sends a prompt message to the generative AI model to output the data in graph format. The generative AI model analyzes the data stored internally and generates the data in the graph format desired by the user using a graph generation library such as Matplotlib or Plotly.

[1390] The generated graph data is sent from the server to the device, which then displays the data to the user. In this case, JavaScript such as D3.js may be used for displaying the data.

[1391] As a concrete example, consider the case where a sales representative checks monthly sales data. The user (sales representative) instructs the terminal to "import this month's sales data into the generation AI," and then inputs "display this month's sales data as a graph." This causes this month's sales data to be retrieved from the sales database, analyzed by the generation AI model, and finally displayed as a graph on the terminal.

[1392] Prompt Sentence Examples

[1393] Prompt: "Feed this month's sales data into the Generative AI"

[1394] Prompt: "Show me this month's sales data in a graph."

[1395] As described above, the system of the present invention allows users to acquire, analyze, and output data in natural language without requiring any special knowledge, thereby improving work efficiency.

[1396] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1397] Step 1:

[1398] The user accesses the terminal and inputs instructions for data capture.

[1399] As a specific example of input, you can instruct the device to "input this month's sales data into the generation AI."

[1400] The terminal accepts this input and processes the input data.

[1401] The output is an HTTP request generated by the terminal, which undergoes natural language analysis and is sent to the server.

[1402] Step 2:

[1403] The terminal analyzes the user's instructions using natural language analysis means.

[1404] The input instructions are parsed using Google's Natural Language API.

[1405] From this analysis, the terminal constructs a specific data acquisition request.

[1406] The output is an HTTP request based on the parsed results, which is sent to the server.

[1407] Step 3:

[1408] The server receives the HTTP request and accesses the database to retrieve the required data.

[1409] The server executes SQL queries against the database management system.

[1410] As a specific example, issue the SQL query "SELECT FROM sales WHERE date >= '2023-10-01' AND date <= '2023-10-31'".

[1411] The input is an HTTP request based on the user's requirements.

[1412] The output is the retrieved sales data, which is converted to JSON format.

[1413] Step 4:

[1414] The server converts the data it receives into a specific format.

[1415] The Python Pandas library is used for data conversion.

[1416] The input is sales data obtained from a database.

[1417] The output is data converted into a certain format (e.g. JSON format).

[1418] Step 5:

[1419] The server then inputs the converted data into the generative AI model.

[1420] The server sends the data to a generative AI model (e.g., OpenAI's GPT-4) via an API for ingestion.

[1421] The input is data that has been converted into a certain format.

[1422] The output is the processing result of the generative AI model storing the data in its internal memory.

[1423] Step 6:

[1424] The user accesses the terminal again and inputs an instruction to output data.

[1425] As a specific example of input, the user might instruct the user to "display this month's sales data in a graph."

[1426] The terminal accepts this input and processes the input data.

[1427] The output is an HTTP request generated by the terminal, which is sent to the server after natural language analysis.

[1428] Step 7:

[1429] The terminal analyzes the user's output request using natural language analysis means.

[1430] The input instructions are parsed using Google's Natural Language API.

[1431] From this analysis, the terminal constructs a specific data output request.

[1432] The output is an HTTP request based on the parsed results, which is sent to the server.

[1433] Step 8:

[1434] The server receives the HTTP request and sends output instructions to the generative AI model.

[1435] The server sends prompt messages to the generative AI model to output the data in graph format.

[1436] The input is an HTTP request based on the user's output requirements.

[1437] The output is a prompt message sent to the generative AI model.

[1438] Step 9:

[1439] A generative AI model analyzes the data and generates the data in graph form.

[1440] The generative AI model analyzes the data stored in its internal memory.

[1441] To generate graphs, we use graph generation libraries such as Matplotlib and Plotly.

[1442] Input is data fed into a generative AI model.

[1443] The output is graphical data, generated as an image file or data structure.

[1444] Step 10:

[1445] The server transmits the generated graph data to the terminal.

[1446] The input is graph data received from a generative AI model.

[1447] The output is graph data sent to the terminal.

[1448] Step 11:

[1449] The terminal displays the received graph data to the user.

[1450] The terminal uses the JavaScript D3.js to display graph data.

[1451] The input is the graph data received from the server.

[1452] The output is a graph that is displayed on the user's terminal.

[1453] Through the above processing steps, the user can easily acquire, analyze, and display data.

[1454] (Application example 1)

[1455] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1456] In today's business environment, store managers and employees are required to acquire and analyze data in real time. However, current systems require specialized knowledge and complex operations, which reduces work efficiency. For this reason, a system is needed that can easily acquire data and display it in an appropriate output format.

[1457] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1458] In this invention, the server includes means for receiving a data acquisition request and acquiring information from an external information source, means for converting the acquired information into a certain format and inputting it into the generated artificial intelligence, means for outputting the acquired information in a data format corresponding to the output request, means for receiving and analyzing data acquisition and output requests in natural language, means for transmitting the analyzed natural language to the server, and means for displaying the analyzed and processed data on a user interface. This allows users to acquire data in natural language without having special knowledge and display the necessary information in real time, thereby significantly improving business efficiency.

[1459] A "data acquisition request" is a request made by a user to acquire specific information.

[1460] An "external information source" is an information provider such as a database or cloud service that exists outside the system.

[1461] "Information" refers to data or content obtained from external sources, such as sales data or inventory data.

[1462] A "uniform format" refers to a standard data structure or format suitable for data analysis and display.

[1463] "Generative AI" is an AI technology that analyzes acquired data and outputs it in an appropriate format according to the user's request.

[1464] The "means for receiving and analyzing data acquisition and output requests in natural language" refers to a means for analyzing requests in natural language spoken or input by a user and for the system to understand them.

[1465] The "means for transmitting the analyzed natural language to the server" refers to a means for transmitting the results of the natural language analysis to the server for further processing on the server side.

[1466] The "means for displaying the analyzed and processed data on a user interface" refers to a means for visually presenting the analyzed data to the user.

[1467] A "user interface" is a screen or device through which a user interacts with a system.

[1468] This invention relates to a system that allows users to easily acquire data and view it in the desired format without requiring special knowledge. This system includes data acquisition requests, data conversion, data analysis and output using generation AI, and visual display of data using a user interface.

[1469] Overall system overview

[1470] First, the user accesses the system via a device such as smart glasses or a smartphone. The user requests data acquisition or output using natural language. This request is analyzed by the device and sent to the server. The server then accesses external information sources (e.g., databases or cloud services) to obtain the required information, converts it into a specific format, and feeds it into the generation AI. The generation AI then analyzes the information it has received and outputs data in the format desired by the user. This output data is then sent to the device and displayed visually to the user.

[1471] Hardware and software used

[1472] Hardware:

[1473] Smart glasses (e.g., regular smart glasses)

[1474] Server (e.g. cloud server)

[1475] User device (smartphone, etc.)

[1476] software:

[1477] Natural language processing libraries (e.g., spaCy, BERT)

[1478] Data analysis tools (e.g., Pandas, NumPy)

[1479] Data visualization tools (e.g., Matplotlib, Plotly)

[1480] Generative AI libraries (e.g., OpenAI GPT-4)

[1481] Data acquisition and analysis flow

[1482] 1. User command: The user commands their smart glasses or smartphone to "show me today's sales data."

[1483] 2. Sending and parsing the request: The device parses this instruction using a natural language processing library (e.g., spaCy, BERT) and sends it to the server.

[1484] 3. Data Acquisition: The server accesses the store's sales database to acquire the latest sales data and converts it into a specific format (e.g., CSV, JSON).

[1485] 4. Data import and analysis: The server imports the converted data into a generative AI (e.g., OpenAI GPT-4) for analysis.

[1486] 5. Data transmission and display: Based on the analysis results, the generation AI converts the sales data into a graph format and sends it to the device, which then displays the received data on the screen of the smart glasses or smartphone.

[1487] Specific examples

[1488] For example, when a store manager wants to check inventory status in real time, he or she uses the system by following the steps below.

[1489] 1. Instruction: The store manager speaks to the smart glasses and says, "Tell me the current inventory status."

[1490] 2. Send: The smart glasses parse this request and send it to the server.

[1491] 3. Acquisition and conversion: The server acquires the latest inventory data from the inventory database and inputs it into the generation AI.

[1492] 4. Analysis and output: The generative AI analyzes the data and sends the current inventory status in a list format to the smart glasses.

[1493] 5. Display: The smart glasses visually display the results to the user.

[1494] Prompt Sentence Examples

[1495] "Please get yesterday's sales data and graph it."

[1496] "Show me this week's sales data compared by day of the week."

[1497] Please see a list of current inventory.

[1498] This system allows users to obtain data in natural language without requiring any special knowledge and check the information they need in real time, significantly improving work efficiency and supporting faster decision-making.

[1499] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1500] Step 1:

[1501] The user issues a command in natural language to their smart glasses or smartphone, such as "Show me today's sales data."

[1502] Input: User's voice or text instructions

[1503] Output: Natural language instructions data

[1504] Specific operation: The voice recognition function of smart glasses or smartphone captures the user's instructions as text data.

[1505] Step 2:

[1506] The device analyzes the user's instructions using a natural language processing library (e.g., spaCy, BERT) and sends the results to the server.

[1507] Input: Natural language instructions

[1508] Output: Parsed instruction data

[1509] Specific operation: The device uses a natural language processing library to analyze the intent of the instruction and sends the content of the instruction (e.g., "Get today's sales data") to the server as structured data.

[1510] Step 3:

[1511] The server accesses an external source (e.g., a sales database) and retrieves the required data.

[1512] Input: Parsed instruction data

[1513] Output: Sales data

[1514] Specific operation: The server accesses the sales database, queries today's sales data, and retrieves the data.

[1515] Step 4:

[1516] The server converts the data it receives into a specific format (e.g., CSV, JSON).

[1517] Input: Sales data

[1518] Output: Reformatted sales data

[1519] What happens: The server uses a data conversion library to convert raw sales data into a defined format (e.g. CSV, JSON).

[1520] Step 5:

[1521] The server then inputs the converted data into a generative AI model (e.g., OpenAI GPT-4) for analysis.

[1522] Input: Reformatted sales data

[1523] Output: Analysis result data (e.g., graph data)

[1524] Specific operation: The server inputs data into the generative AI model, analyzes the sales data, and generates the results in graph form.

[1525] Step 6:

[1526] The server transmits the generated graph data to the terminal.

[1527] Input: Analysis result data (graph data)

[1528] Output: Graph data sent to the terminal

[1529] Specific operation: The server uses a network protocol to send the generated graph data to the device.

[1530] Step 7:

[1531] The terminal visually displays the received graph data to the user.

[1532] Input: Graph data sent to the terminal

[1533] Output: Screen showing the graph

[1534] Specific operation: Graph data is drawn on the device's display screen so that the user can visually check it.

[1535] By performing the necessary data processing and calculations at each step in this way, users can easily obtain data in natural language and visually confirm the results.

[1536] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1537] The present invention relates to a system that uses artificial intelligence to recognize a user's natural language instructions and emotions, and acquire, convert, and generate data from external data sources based on those instructions and emotions, and output data in a desired format. In particular, by combining an emotion engine that recognizes the user's emotions, the system enables more adaptive data output according to the user's emotional state.

[1538] Overall system overview

[1539] The system mainly includes the following functional blocks: data acquisition means, data conversion means, artificial intelligence for generation, output means, and emotion engine.

[1540] Key components of the system

[1541] 1. Data Acquisition Request

[1542] A user inputs a data acquisition request in natural language via a terminal, which then transmits the request to a server.

[1543] 2. Data Acquisition and Transformation

[1544] The server retrieves the required data from an external data source (e.g., a company's sales database).

[1545] The acquired data is converted into a certain format (e.g. JSON) by the server.

[1546] 3. Inputting data into the generated AI

[1547] The server then feeds the converted data into the AI ​​that generates it, which stores the data internally and prepares it for processing.

[1548] 4. Functions of the Emotion Engine

[1549] The emotion engine analyzes emotions from the user's natural language instructions.

[1550] The analysis results are fed back to the generating AI, which then analyzes and outputs the data taking into account the emotional state.

[1551] 5. Data reference and output

[1552] The user instructs the server in natural language via the terminal which output format (e.g., graph format) he / she desires.

[1553] The server transmits the instructions to the AI ​​that generates the data, and the AI ​​outputs the data in the specified format based on the internal data. The output format is adjusted based on feedback from the emotion engine.

[1554] The output data is sent to the terminal via the server and can be viewed by the user.

[1555] Specific examples of processing

[1556] A sales rep reviews monthly sales data and analyzes sentiment:

[1557] 1. Data Ingestion Request

[1558] The sales representative (user) instructs the device to "input this month's sales data into the generation AI."

[1559] The terminal sends this instruction to the server as a data acquisition request.

[1560] 2. Data Acquisition and Transformation

[1561] The server accesses the company's sales database to retrieve the current month's sales data.

[1562] The server converts the acquired data into a certain format.

[1563] 3. Data Import

[1564] The server then inputs the converted data into the generation AI.

[1565] The generative AI stores the data internally.

[1566] 4. Emotion Engine Analysis

[1567] When a salesperson says, "Show me this month's sales data in a graph," the emotion engine analyzes the emotion from their natural language.

[1568] For example, if the user indicates that they are feeling stressed, the emotion engine will detect this.

[1569] 5. Data output request

[1570] The terminal sends an output request to the server.

[1571] The server instructs the generative AI to output data in a format that adapts to the user's emotions, taking into account feedback from the emotion engine.

[1572] 6. Data Output and Display

[1573] The generation AI generates graph data taking into account the analysis results of the emotion engine and sends it back to the server.

[1574] The server sends the generated data to the terminal, which displays the graph to the user.

[1575] The above process allows users to acquire, analyze, and output data without any special technical knowledge, and also enables adaptive data output according to the user's emotional state.

[1576] The processing flow will be explained below.

[1577] Data Import

[1578] Step 1:

[1579] The user types into the terminal, "Enter this month's sales data into the generation AI."

[1580] Step 2:

[1581] The terminal analyzes the user's instructions and sends them to the server as a data acquisition request.

[1582] Step 3:

[1583] The server receives the data acquisition request and accesses the company's sales database, which is an external data source, to acquire the current month's sales data.

[1584] Step 4:

[1585] The server converts the acquired data into a format that the generating AI can understand, such as JSON format.

[1586] Step 5:

[1587] The server calls an API to feed the converted data into the generation AI.

[1588] Step 6:

[1589] The generation AI stores the data received from the server internally and notifies the server that the data import is complete.

[1590] Step 7:

[1591] The server notifies the terminal of the completion of data import.

[1592] Step 8:

[1593] The terminal receives a notification from the server and notifies the user that the data import is complete.

[1594] Data lookup and sentiment analysis

[1595] Step 1:

[1596] The user inputs into the terminal, "Show me this month's sales data in a graph."

[1597] Step 2:

[1598] The terminal analyzes the user's instructions and transmits the user's natural language to the emotion engine.

[1599] Step 3:

[1600] The emotion engine analyzes emotions from the user's natural language and feeds the analysis results back to the generative AI.

[1601] Step 4:

[1602] The device sends the user's instructions and the results of emotion analysis to the server.

[1603] Step 5:

[1604] The server receives the data output request and the sentiment analysis results, and instructs the generation AI to output this month's sales data in graph format. The instruction also includes the sentiment analysis results.

[1605] Step 6:

[1606] The AI ​​analyzes the data and emotion analysis results and generates graphs according to instructions. For example, if the user is feeling stressed, it will use colors and layouts to reduce stress.

[1607] Step 7:

[1608] The generation AI returns the generated graph data to the server.

[1609] Step 8:

[1610] The server sends the graph data received from the generation AI to the terminal.

[1611] Step 9:

[1612] The terminal receives the graph data and displays it to the user.

[1613] Specific examples

[1614] A sales rep pulls this month's sales data and analyzes sentiment.

[1615] Step 1:

[1616] The sales representative (user) instructs the device to "input this month's sales data into the generation AI."

[1617] Step 2:

[1618] The terminal analyzes the instruction and sends a data acquisition request to the server.

[1619] Step 3:

[1620] The server accesses the company's sales database to retrieve the current month's sales data.

[1621] Step 4:

[1622] The server converts the acquired data and feeds it into the generated AI.

[1623] Step 5:

[1624] The generation AI stores the data internally and notifies the server when the import is complete.

[1625] Step 6:

[1626] The server notifies the terminal that the import is complete.

[1627] Step 7:

[1628] The terminal notifies the sales representative that the import is complete.

[1629] Step 8:

[1630] The sales representative (user) instructs, "Display this month's sales data in a graph."

[1631] Step 9:

[1632] The terminal sends the user's emotion along with the instruction to the emotion engine for analysis.

[1633] Step 10:

[1634] The emotion engine analyzes the instructions and detects when the user is feeling stressed.

[1635] Step 11:

[1636] The emotion engine feeds back the analysis results to the server.

[1637] Step 12:

[1638] The terminal sends a data output request and the emotion analysis results to the server.

[1639] Step 13:

[1640] The server instructs the generating AI to output data in graph format, taking into account the results of emotion analysis.

[1641] Step 14:

[1642] The generative AI generates appropriate graph data based on data and sentiment analysis results.

[1643] Step 15:

[1644] The generation AI sends the graph data back to the server.

[1645] Step 16:

[1646] The server transmits the graph data to the terminal.

[1647] Step 17:

[1648] The terminal displays the graph data to the user, who can then refer to it.

[1649] This specific processing flow allows users to obtain and output data through instructions in natural language, and by utilizing sentiment analysis in the process, it enables more user-friendly and effective data reference.

[1650] Example 2

[1651] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1652] In conventional data acquisition and output systems, it was difficult for users to not only give instructions in natural language, but also to recognize the emotional state of the user from the content of the instructions and adjust the data output accordingly. As a result, it was not possible to output data that was adapted to the user's emotional state, and this resulted in a problem of not being able to sufficiently improve the user experience.

[1653] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving a data acquisition request and acquiring data from an external data source, means for converting the acquired data into a certain format and inputting it into the AI ​​to be generated, means for outputting the input data in a data format corresponding to the output request based on the data, means for analyzing emotions from a user's natural language instruction and feeding back the emotional state to the AI ​​to be generated, and means for the AI ​​to analyze and output data taking the user's emotional state into consideration. This makes it possible to acquire, analyze, and output data adapted to the user's emotional state.

[1654] "Data retrieval request" refers to an instruction sent by a user to the system to retrieve data from an external data source.

[1655] "External data source" refers to an information source that exists outside the system and where the required data is stored.

[1656] "Means for obtaining data" refers to the functionality for accessing and obtaining the required data from an external data source.

[1657] "Means for converting data into a certain format and inputting it into the AI ​​to be generated" refers to the function for converting acquired data into a specific format (such as JSON format) and inputting the converted data into the internal memory of the AI ​​to be generated.

[1658] "Generative AI" refers to an AI model that is used to manipulate and analyze ingested data and generate output in response to user requests.

[1659] An "output request" refers to a user's request to the system to output data in a particular format.

[1660] "Output means" refers to a function for generating and outputting necessary data in a specific format in response to a user's output request.

[1661] "Natural language instructions" refers to instructions given by a user to a system in natural language (e.g., Japanese, English, etc.).

[1662] "Means for analyzing emotions" refers to a function for analyzing the user's emotional state from the user's natural language instructions, voice, etc.

[1663] An "emotion engine" refers to an analysis device that analyzes emotions from the user's natural language instructions and reflects the results in the system.

[1664] "Feedback" refers to transmitting analysis results and other information to the artificial intelligence that generates it, and then using that information for further processing and adjustments.

[1665] "Means for analyzing and outputting data" refers to a function for analyzing the captured data taking into account the emotional state of the user and outputting it in the most appropriate format.

[1666] This system recognizes the user's natural language instructions and emotions, and uses artificial intelligence to acquire, convert, and generate the necessary data from external data sources based on that information, outputting the data in the desired format. The main hardware and software components that make up the system are as follows:

[1667] Hardware and Software

[1668] 1. Device:

[1669] A device that allows users to input data acquisition and output requests. Examples include smartphones, tablets, and PCs.

[1670] 2. Server:

[1671] It is responsible for accessing external data sources, acquiring and transforming the data, feeding it into the generative AI model, receiving feedback from the emotion engine, and finally converting the data into an output format. It uses a regular web server or a cloud-based server.

[1672] 3. Generative AI Model:

[1673] It is an artificial intelligence model that analyzes and generates acquired data based on user requests. Typical generative AI models include GPT and BERT.

[1674] 4. Emotion Engine:

[1675] This is software or a module that analyzes emotions from users' natural language instructions and feeds the results back to a generative AI model. Natural language processing technology is used for emotion analysis.

[1676] Specific operations and procedures

[1677] 1. User data request:

[1678] The user issues instructions in natural language via the device, such as "Input this month's sales data into the generation AI." The device then sends this instruction to the server as text data.

[1679] 2. Data retrieval and transformation by the server:

[1680] Based on the received data retrieval request, the server accesses external data sources such as a company's sales database to retrieve the required data, and then converts the retrieved data into a specific format (such as JSON).

[1681] 3. Server imports data into the generated AI model:

[1682] The server sends the converted data to the generative AI model, which then stores the data in its internal memory.

[1683] 4. User sentiment analysis:

[1684] When a user requests data output, for example, "Show me this month's sales data in a graph," the emotion engine analyzes the user's emotional state (e.g., stress, joy, etc.) from this instruction and feeds the results back to the generative AI model.

[1685] 5. Data generation and output:

[1686] Based on feedback from the emotion engine, the generative AI model analyzes the data in a format that adapts to the user's emotional state, and finally generates the data in an output format (e.g., graph format).

[1687] The generated data is sent to the terminal via the server, and the terminal displays it to the user.

[1688] Specific examples

[1689] Prompt Sentence Examples

[1690] User: "Please input this month's sales data into the generation AI."

[1691] User: "Show me this month's sales data in a graph."

[1692] This allows users to easily acquire, analyze, and output data using natural language without any special technical knowledge.In addition, by introducing an emotion engine, it is possible to realize adaptive data output according to the user's emotional state.

[1693] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1694] Step 1:

[1695] Sending a data retrieval request

[1696] explanation

[1697] The user inputs a data acquisition request in natural language into the terminal, which then sends the request to the server.

[1698] input

[1699] Natural language instructions from the user (e.g., "Please feed this month's sales data into the Generative AI").

[1700] output

[1701] The request sent to the server to retrieve data.

[1702] Specific actions

[1703] The user types into the device, "Please input this month's sales data into the generation AI." The device converts the natural language instruction into text format and sends it to the server as an HTTP request.

[1704] Step 2:

[1705] Data Acquisition

[1706] explanation

[1707] When the server receives a data retrieval request, it accesses external data sources to retrieve the required data.

[1708] input

[1709] The data retrieval request sent to the server (e.g. received as a "fetch_data" command).

[1710] output

[1711] Data obtained from external sources.

[1712] Specific actions

[1713] The server runs an SQL query (e.g., "SELECT FROM sales_data WHERE month = 'current_month'") against the company's sales database to retrieve the current month's sales data.

[1714] Step 3:

[1715] Data transformation

[1716] explanation

[1717] The server converts the retrieved data into a certain format (e.g. JSON).

[1718] input

[1719] Raw data obtained from external sources.

[1720] output

[1721] The data converted to JSON format.

[1722] Specific actions

[1723] Convert the sales data acquired by the server into JSON format. Example: Convert as follows.

[1724] json

[1725] {

[1726] "month": "current_month",

[1727] "total_sales": 100000,

[1728] "details": [

[1729] {"product": "Product A", "sales": 50000},

[1730] {"product": "Product B", "sales": 50000}

[1731] ]

[1732] }

[1733] Step 4:

[1734] Ingesting data into a generative AI model

[1735] explanation

[1736] The server then feeds the converted data into a generative AI model, which stores the data internally and prepares it for processing.

[1737] input

[1738] The data converted to JSON format.

[1739] output

[1740] Data fed into generative AI models.

[1741] Specific actions

[1742] The server uses an internal API to send data to the generative AI model, which then stores the received data in its internal memory.

[1743] Step 5:

[1744] Sentiment analysis using natural language instructions

[1745] explanation

[1746] The user makes an output request in natural language, and the emotion engine analyzes the emotion from the natural language instruction.

[1747] input

[1748] Natural language commands from the user (e.g., "Show me this month's sales data in a graph").

[1749] output

[1750] Analyzed emotion data.

[1751] Specific actions

[1752] A user might say, "Show me this month's sales data in a graph." The emotion engine analyzes this natural language and identifies emotions such as "stress" or "joy."

[1753] Step 6:

[1754] Generating emotional feedback and sending it to the AI ​​model

[1755] explanation

[1756] The analysis results of the emotion engine are fed back to the generative AI model.

[1757] input

[1758] Analyzed emotion data.

[1759] output

[1760] Emotional feedback sent to a generative AI model.

[1761] Specific actions

[1762] The emotion engine sends the analysis results (e.g., "emotion": "stress", "confidence_score": 0.85") to the generative AI model.

[1763] Step 7:

[1764] Data generation and output

[1765] explanation

[1766] The generative AI model analyzes the data and generates an output format taking into account the user's emotional state. The server sends the generated data to the device and displays it on the device.

[1767] input

[1768] Data in a generative AI model that has received emotional feedback.

[1769] output

[1770] The output data generated and the data format sent to the terminal.

[1771] Specific actions

[1772] The generative AI model analyzes the data taking into account the emotional state and generates output data in the form of a graph. The server sends the generated graph data to the device, which then displays the graph to the user.

[1773] Through the above processing steps, the user can make a data acquisition request in natural language and the results can be displayed adaptively according to the user's emotional state.

[1774] (Application example 2)

[1775] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1776] Conventional food delivery systems process orders without considering the user's emotional state, making it difficult to provide food that suits the user's current mood or emotion. Furthermore, they are unable to respond to instructions in natural language, which can lead to a lack of user experience. This can lead to a decrease in user satisfaction and complicated ordering.

[1777] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a data acquisition request and acquiring data from an external data source, means for converting the acquired data into a certain format and inputting it into the generated artificial intelligence, means for outputting the acquired data in a data format corresponding to the output request, and means for recognizing the user's instructions and emotions in natural language and adjusting the data output according to the user's emotional state using an emotion engine. This allows users to easily order the dish that best suits their emotional state using natural language, significantly improving the convenience and satisfaction of food delivery.

[1778] A "data acquisition request" is a request from a user to acquire data from a specified external data source.

[1779] An "external data source" is an information source that exists outside the system and provides data, such as various databases and APIs.

[1780] A "consistent format" is a standardized format that makes data easier to process, such as JSON or XML.

[1781] "Generative AI" is a general term for AI that has the ability to generate new information or results based on acquired data.

[1782] "Natural language" refers to a language that humans use on a daily basis, and is not a programming language.

[1783] An "emotion engine" is a software component for analyzing and recognizing emotions from a user's natural language input.

[1784] "Output format" refers to the format or style in which data is presented to a user, visually or otherwise.

[1785] "Filtering" is the process of selecting or excluding data based on specific conditions.

[1786] "User experience" refers to the overall experience and satisfaction a user has when using a product or service.

[1787] To implement this invention, a system including the following functional blocks and procedures is required: This system comprises a data acquisition means, a data conversion means, an artificial intelligence for generation, an output means, and an emotion engine.

[1788] Hardware and software used

[1789] Hardware: Smartphone (iOS, Android)

[1790] software:

[1791] Natural language processing libraries (e.g., Google Cloud Natural Language API)

[1792] Emotion recognition engine (e.g. Microsoft Azure Emotion API)

[1793] Database (e.g. Firebase Firestore)

[1794] RESTful API (for data retrieval)

[1795] Front-end frameworks (e.g., React Native)

[1796] Processing a data retrieval request

[1797] The server receives a data retrieval request from the user and accesses the food delivery service's database to retrieve the dish data for each restaurant. This data is then converted into a standard format such as JSON.

[1798] Data transformation and incorporation into generative artificial intelligence

[1799] The server converts the acquired data into a certain format (such as JSON) and inputs it into the AI ​​model to be generated. The AI ​​model to be generated stores the converted data internally and prepares it for subsequent processing.

[1800] Natural Language Processing and Emotion Recognition

[1801] The user's natural language instructions are received via the smartphone. These instructions are parsed using the Google Cloud Natural Language API to understand the user's request. The Microsoft Azure Emotion API is then used to recognize the user's emotional state. The emotion recognition engine determines emotions such as "relaxed," "stressed," or "happy."

[1802] Adjusting output requirements and data formats

[1803] The user inputs the desired dish format in natural language and sends it back to the server. The server instructs the generation AI to consider feedback from the emotion engine and output the dish that best suits the user's emotional state. The generation AI filters the dish data based on the analysis results of the emotion engine, generates a personalized dish list, and displays it on the smartphone.

[1804] Specific examples

[1805] 1. User input: "I'm a little tired today, so I want something relaxing to eat."

[1806] 2. Natural language processing: Analyze using the Google Cloud Natural Language API to understand the user's specifications.

[1807] 3. Emotion recognition: Determine whether the user is "tired" using the Microsoft Azure Emotion API.

[1808] 4. Example output: "Good work! Would you like some relaxing herbal tea and a light salad?"

[1809] This allows users to easily order the food that best suits their emotional state using natural language, significantly improving the convenience and satisfaction of food delivery.

[1810] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1811] Step 1:

[1812] The user inputs a data acquisition request in natural language via a smartphone, and this instruction (e.g., "Recommend me a dish that suits my current mood") is sent from the device to the server.

[1813] Input: User's natural language instructions.

[1814] Output: The data retrieval request sent to the server.

[1815] Step 2:

[1816] The server accesses the food delivery service's database to obtain the food data for each restaurant.

[1817] Input: Data retrieval request.

[1818] Output: Retrieved dish data (e.g., JSON format).

[1819] Step 3:

[1820] The server converts the acquired data into a certain format (e.g., JSON), and the converted data is fed into the AI ​​that generates it.

[1821] Input: The acquired food data.

[1822] Output: The converted data (e.g., in JSON format).

[1823] Step 4:

[1824] The user again inputs the desired cooking style in natural language via the smartphone, and the terminal transmits this output request to the server.

[1825] Input: The user's output request (e.g., "Show me relaxing recipes").

[1826] Output: The output request sent to the server.

[1827] Step 5:

[1828] The server uses the Google Cloud Natural Language API to parse the user's natural language output request and understand the user's request.

[1829] Input: A natural language output request.

[1830] Output: The parsed request content.

[1831] Step 6:

[1832] The server uses the Microsoft Azure Emotion API to recognize the user's emotional state. The emotion engine determines emotions such as "relaxed" or "stressed" from the user's natural language instructions.

[1833] Input: A natural language output request.

[1834] Output: Perceived emotional state.

[1835] Step 7:

[1836] The server provides feedback to the artificial intelligence it generates, instructing it to filter the food data based on the user's emotional state and generate a personalized list of dishes.

[1837] Input: Recognized emotional state and transformed food data.

[1838] Output: A filtered, personalized list of dishes.

[1839] Step 8:

[1840] The server sends the generated recipe list to the terminal, which displays it on the smartphone.

[1841] Input: A filtered list of dishes.

[1842] Output: The list of dishes displayed to the user.

[1843] Through the above processing steps, users can easily order the dish that best suits their emotional state using natural language.

[1844] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1845] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1846] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1847] [Fourth embodiment]

[1848] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1849] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1850] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1851] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1852] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1853] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1854] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1855] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1856] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1857] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1858] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1859] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1860] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1861] The present invention relates to a system that allows users to easily obtain data and view it in a desired format without requiring special knowledge. How this system is implemented will be described below in detail.

[1862] Overall system overview

[1863] First, the user accesses the system via a terminal. The user requests data acquisition and output in natural language. The terminal receives these requests and sends them to the server. The server accesses external data sources to acquire the required data and inputs it into the AI ​​to be generated. The AI ​​to be generated analyzes the data and outputs it in the format desired by the user. This system automates data acquisition, conversion, and output, significantly reducing the burden on the user.

[1864] Program processing flow

[1865] Data Import

[1866] 1. User Instructions

[1867] The user instructs the device to "import this month's sales data into the generation AI."

[1868] 2. Sending the request

[1869] The terminal analyzes the user's request and sends the request to the server.

[1870] 3. Data Acquisition

[1871] The server accesses an external data source to obtain the current month's sales data.

[1872] The server converts the acquired data into a certain format.

[1873] 4. Data Import

[1874] The server then inputs the converted data into the generation AI.

[1875] Data reference (output)

[1876] 1. User Instructions

[1877] The user instructs the terminal to "show this month's sales data in a graph."

[1878] 2. Sending the request

[1879] The terminal analyzes the user's request and sends the request to the server.

[1880] 3. Generate output data

[1881] The server instructs the generating AI to output the data in graph format.

[1882] The generation AI analyzes the data taken in and generates data in graph format.

[1883] 4. Data transmission and display

[1884] The server transmits the generated graph data to the terminal.

[1885] The terminal displays the received graph to the user.

[1886] Specific examples

[1887] For example, consider a sales representative who wants to view monthly sales data.

[1888] 1. Data Ingestion Request

[1889] The user (sales representative) instructs the device to "input this month's sales data into the generation AI."

[1890] The terminal sends this instruction to the server.

[1891] The server retrieves this month's data from the company's sales database and inputs it into the generation AI.

[1892] 2. Data output request

[1893] The user instructs, "Show me this month's sales data in a graph."

[1894] The terminal sends this instruction to the server.

[1895] The server instructs the generating AI to output the data in graph format.

[1896] The generative AI analyzes the data, generates a graph, and returns it to the server.

[1897] The server sends the graph data to the terminal, which displays it to the user.

[1898] Other examples

[1899] Additionally, if the user requests additional information, it is easy to accommodate.

[1900] 3. Additional requests

[1901] The user instructs, "Please also tell me the increase or decrease in sales this month compared to last month."

[1902] The terminal sends this instruction to the server.

[1903] The server instructs the generating AI to compare last month's data with this month's data and display any increases or decreases.

[1904] The generation AI analyzes the data, generates comparison results, and returns them to the server.

[1905] The server sends the results of the comparison to the terminal, which displays them to the user.

[1906] As described above, the system of the present invention allows users to acquire, analyze, and output data in natural language without requiring any special knowledge, thereby improving work efficiency.

[1907] The processing flow will be explained below.

[1908] Data Import

[1909] Step 1:

[1910] The user types into the terminal, "Enter this month's sales data into the generation AI."

[1911] Step 2:

[1912] The terminal analyzes the user's instructions and sends them to the server as a data acquisition request.

[1913] Step 3:

[1914] The server receives the data retrieval request and accesses an external data source, the company's sales database, to retrieve the current month's sales data.

[1915] Step 4:

[1916] The server converts the acquired data into a format that the generating AI can understand, such as JSON format.

[1917] Step 5:

[1918] The server calls an API to feed the converted data into the generation AI.

[1919] Step 6:

[1920] The generation AI stores the data received from the server internally and notifies the server that the data import is complete.

[1921] Step 7:

[1922] The server notifies the terminal of the completion of data import.

[1923] Step 8:

[1924] The terminal receives a notification from the server and notifies the user that the data import is complete.

[1925] Data reference (output)

[1926] Step 1:

[1927] The user inputs into the terminal, "Show me this month's sales data in a graph."

[1928] Step 2:

[1929] The terminal analyzes the user's instructions and sends them to the server as a data output request.

[1930] Step 3:

[1931] The server receives the data output request and instructs the generation AI to output this month's sales data in graph format.

[1932] Step 4:

[1933] The generation AI analyzes the data taken into it and generates graphs according to instructions.

[1934] Step 5:

[1935] The generation AI returns the generated graph data to the server.

[1936] Step 6:

[1937] The server sends the graph data received from the generation AI to the terminal.

[1938] Step 7:

[1939] The terminal receives the graph data and displays it to the user.

[1940] Specific examples

[1941] A salesperson wants to retrieve this month's sales data and display it as a graph.

[1942] Step 1:

[1943] The sales representative (user) instructs the terminal to "import this month's sales data into the generation AI."

[1944] Step 2:

[1945] The terminal analyzes the instruction and sends a data acquisition request to the server.

[1946] Step 3:

[1947] The server accesses the company's sales database and retrieves the current month's sales data.

[1948] Step 4:

[1949] The server converts the acquired data and feeds it into the generated AI.

[1950] Step 5:

[1951] The generation AI stores the data internally and notifies the server when the import is complete.

[1952] Step 6:

[1953] The server notifies the terminal that the import is complete.

[1954] Step 7:

[1955] The terminal notifies the sales representative that the import is complete.

[1956] Step 8:

[1957] The salesperson (user) instructs the terminal to "show this month's sales data in a graph."

[1958] Step 9:

[1959] The terminal analyzes the instruction and sends a data output request to the server.

[1960] Step 10:

[1961] The server instructs the generating AI to output the data in graph format.

[1962] Step 11:

[1963] The generative AI analyzes the data and generates graphs.

[1964] Step 12:

[1965] The generation AI sends the graph data back to the server.

[1966] Step 13:

[1967] The server transmits the graph data to the terminal.

[1968] Step 14:

[1969] The terminal displays the graph data and allows the sales representative to view it.

[1970] Through the above steps, users can acquire and visually refer to data without having any special technical knowledge.

[1971] Example 1

[1972] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1973] Conventional systems have the problem that it is difficult for users to acquire, analyze, and output data unless they have special knowledge. Furthermore, numerous manual operations are required from acquiring data to displaying it, resulting in reduced work efficiency. Furthermore, because data format conversion is performed manually, there is a high risk of errors and it is difficult to ensure the accuracy of the data.

[1974] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1975] In this invention, the server includes a means for receiving a data acquisition request and acquiring data from an external data source, a means for converting the acquired data into a specific format and incorporating it into a generated artificial intelligence model, and a means for outputting the acquired data in a data format corresponding to an output request. This allows users to instruct data acquisition and output in natural language, making it easy to handle data without special knowledge. Furthermore, since the process from data acquisition to display is automated, business efficiency is improved and data accuracy is also increased.

[1976] A "data acquisition request" is an instruction issued by a user to identify the data they require and acquire that data from an external data source.

[1977] An "external data source" is an information source that an external system, such as a server or database, has to provide specific data.

[1978] A "uniform format" is a standard data representation method for converting data into a unified structure or format.

[1979] A "generative artificial intelligence model" refers to an artificial intelligence algorithm or system that analyzes acquired data and outputs the data in a format that meets the user's requirements.

[1980] "Ingestion" refers to the act of inputting data obtained from external sources into an internal system or artificial intelligence model.

[1981] An "output request" is an instruction from the user to display or extract the analyzed data in a desired format.

[1982] "Natural language analysis means" refers to technology that analyzes the natural language (normal spoken or written language) entered by the user and converts it into appropriate system operations or instructions.

[1983] A "server" is a central computing device or system that acquires, analyzes, and ultimately outputs data in response to user requests.

[1984] A "terminal" is a device, such as a computer or smartphone, that a user uses to access the system and input instructions.

[1985] "Graph format" is a method of displaying data in the form of a bar graph, line graph, etc., to make it easier to understand visually.

[1986] MODE FOR CARRYING OUT THE INVENTION

[1987] The present invention relates to a system that allows users to easily obtain data and view it in a desired format without requiring special knowledge. How this system is implemented will be described below in detail.

[1988] Overall system overview

[1989] First, the user accesses the system via a terminal. Specific terminals include PCs and smartphones. The user issues data acquisition and output requests in natural language. The terminal receives these requests and sends them to the server.

[1990] The server accesses an external data source to obtain the necessary data. For example, a sales database is used as the external data source. A database management system such as MySQL or PostgreSQL is used. The server converts the obtained data into a specific format. The Pandas library is used for data conversion.

[1991] The converted data is then fed into a generative AI model. For example, OpenAI's GPT-4 is used as the generative AI model. The generative AI model analyzes the data and outputs it in the format desired by the user. This output data can be in the form of a graph or table.

[1992] Data import procedure

[1993] The user instructs the device to "feed this month's sales data into the generation AI," and the device uses Google's Natural Language API to parse the instruction and send the request to the server.

[1994] Based on the received request, the server retrieves data from a database named "SELLS_DB" by executing an SQL query such as "SELECT FROM sales WHERE date >= '2023-10-01' AND date <= '2023-10-31'". The retrieved data is converted to JSON format using the Pandas library.

[1995] The server sends the converted JSON data to the generative AI model via an API for ingestion, and the generative AI model stores this data in its internal memory and makes it available for analysis.

[1996] Data reference (output) procedure

[1997] The user says, "Show me this month's sales data in a graph." The device again uses Google's Natural Language API to parse the request and sends the request to the server.

[1998] The server sends a prompt message to the generative AI model to output the data in graph format. The generative AI model analyzes the data stored internally and generates the data in the graph format desired by the user using a graph generation library such as Matplotlib or Plotly.

[1999] The generated graph data is sent from the server to the device, which then displays the data to the user. In this case, JavaScript such as D3.js may be used for displaying the data.

[2000] As a concrete example, consider the case where a sales representative checks monthly sales data. The user (sales representative) instructs the terminal to "import this month's sales data into the generation AI," and then inputs "display this month's sales data as a graph." This causes this month's sales data to be retrieved from the sales database, analyzed by the generation AI model, and finally displayed as a graph on the terminal.

[2001] Prompt Sentence Examples

[2002] Prompt: "Feed this month's sales data into the Generative AI"

[2003] Prompt: "Show me this month's sales data in a graph."

[2004] As described above, the system of the present invention allows users to acquire, analyze, and output data in natural language without requiring any special knowledge, thereby improving work efficiency.

[2005] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2006] Step 1:

[2007] The user accesses the terminal and inputs instructions for data capture.

[2008] As a specific example of input, you can instruct the device to "input this month's sales data into the generation AI."

[2009] The terminal accepts this input and processes the input data.

[2010] The output is an HTTP request generated by the terminal, which undergoes natural language analysis and is sent to the server.

[2011] Step 2:

[2012] The terminal analyzes the user's instructions using natural language analysis means.

[2013] The input instructions are parsed using Google's Natural Language API.

[2014] From this analysis, the terminal constructs a specific data acquisition request.

[2015] The output is an HTTP request based on the parsed results, which is sent to the server.

[2016] Step 3:

[2017] The server receives the HTTP request and accesses the database to retrieve the required data.

[2018] The server executes SQL queries against the database management system.

[2019] As a specific example, issue the SQL query "SELECT FROM sales WHERE date >= '2023-10-01' AND date <= '2023-10-31'".

[2020] The input is an HTTP request based on the user's requirements.

[2021] The output is the retrieved sales data, which is converted to JSON format.

[2022] Step 4:

[2023] The server converts the data it receives into a specific format.

[2024] The Python Pandas library is used for data conversion.

[2025] The input is sales data obtained from a database.

[2026] The output is data converted into a certain format (e.g. JSON format).

[2027] Step 5:

[2028] The server then inputs the converted data into the generative AI model.

[2029] The server sends the data to a generative AI model (e.g., OpenAI's GPT-4) via an API for ingestion.

[2030] The input is data that has been converted into a certain format.

[2031] The output is the processing result of the generative AI model storing the data in its internal memory.

[2032] Step 6:

[2033] The user accesses the terminal again and inputs an instruction to output data.

[2034] As a specific example of input, the user might instruct the user to "display this month's sales data in a graph."

[2035] The terminal accepts this input and processes the input data.

[2036] The output is an HTTP request generated by the terminal, which is sent to the server after natural language analysis.

[2037] Step 7:

[2038] The terminal analyzes the user's output request using natural language analysis means.

[2039] The input instructions are parsed using Google's Natural Language API.

[2040] From this analysis, the terminal constructs a specific data output request.

[2041] The output is an HTTP request based on the parsed results, which is sent to the server.

[2042] Step 8:

[2043] The server receives the HTTP request and sends output instructions to the generative AI model.

[2044] The server sends prompt messages to the generative AI model to output the data in graph format.

[2045] The input is an HTTP request based on the user's output requirements.

[2046] The output is a prompt message sent to the generative AI model.

[2047] Step 9:

[2048] A generative AI model analyzes the data and generates the data in graph form.

[2049] The generative AI model analyzes the data stored in its internal memory.

[2050] To generate graphs, we use graph generation libraries such as Matplotlib and Plotly.

[2051] Input is data fed into a generative AI model.

[2052] The output is graphical data, generated as an image file or data structure.

[2053] Step 10:

[2054] The server transmits the generated graph data to the terminal.

[2055] The input is graph data received from a generative AI model.

[2056] The output is graph data sent to the terminal.

[2057] Step 11:

[2058] The terminal displays the received graph data to the user.

[2059] The terminal uses the JavaScript D3.js to display graph data.

[2060] The input is the graph data received from the server.

[2061] The output is a graph that is displayed on the user's terminal.

[2062] Through the above processing steps, the user can easily acquire, analyze, and display data.

[2063] (Application example 1)

[2064] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2065] In today's business environment, store managers and employees are required to acquire and analyze data in real time. However, current systems require specialized knowledge and complex operations, which reduces work efficiency. For this reason, a system is needed that can easily acquire data and display it in an appropriate output format.

[2066] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2067] In this invention, the server includes means for receiving a data acquisition request and acquiring information from an external information source, means for converting the acquired information into a certain format and inputting it into the generated artificial intelligence, means for outputting the acquired information in a data format corresponding to the output request, means for receiving and analyzing data acquisition and output requests in natural language, means for transmitting the analyzed natural language to the server, and means for displaying the analyzed and processed data on a user interface. This allows users to acquire data in natural language without having special knowledge and display the necessary information in real time, thereby significantly improving business efficiency.

[2068] A "data acquisition request" is a request made by a user to acquire specific information.

[2069] An "external information source" is an information provider such as a database or cloud service that exists outside the system.

[2070] "Information" refers to data or content obtained from external sources, such as sales data or inventory data.

[2071] A "uniform format" refers to a standard data structure or format suitable for data analysis and display.

[2072] "Generative AI" is an AI technology that analyzes acquired data and outputs it in an appropriate format according to the user's request.

[2073] The "means for receiving and analyzing data acquisition and output requests in natural language" refers to a means for analyzing requests in natural language spoken or input by a user and for the system to understand them.

[2074] The "means for transmitting the analyzed natural language to the server" refers to a means for transmitting the results of the natural language analysis to the server for further processing on the server side.

[2075] The "means for displaying the analyzed and processed data on a user interface" refers to a means for visually presenting the analyzed data to the user.

[2076] A "user interface" is a screen or device through which a user interacts with a system.

[2077] This invention relates to a system that allows users to easily acquire data and view it in the desired format without requiring special knowledge. This system includes data acquisition requests, data conversion, data analysis and output using generation AI, and visual display of data using a user interface.

[2078] Overall system overview

[2079] First, the user accesses the system via a device such as smart glasses or a smartphone. The user requests data acquisition or output using natural language. This request is analyzed by the device and sent to the server. The server then accesses external information sources (e.g., databases or cloud services) to obtain the required information, converts it into a specific format, and feeds it into the generation AI. The generation AI then analyzes the information it has received and outputs data in the format desired by the user. This output data is then sent to the device and displayed visually to the user.

[2080] Hardware and software used

[2081] Hardware:

[2082] Smart glasses (e.g., regular smart glasses)

[2083] Server (e.g. cloud server)

[2084] User device (smartphone, etc.)

[2085] software:

[2086] Natural language processing libraries (e.g., spaCy, BERT)

[2087] Data analysis tools (e.g., Pandas, NumPy)

[2088] Data visualization tools (e.g., Matplotlib, Plotly)

[2089] Generative AI libraries (e.g., OpenAI GPT-4)

[2090] Data acquisition and analysis flow

[2091] 1. User command: The user commands their smart glasses or smartphone to "show me today's sales data."

[2092] 2. Sending and parsing the request: The device parses this instruction using a natural language processing library (e.g., spaCy, BERT) and sends it to the server.

[2093] 3. Data Acquisition: The server accesses the store's sales database to acquire the latest sales data and converts it into a specific format (e.g., CSV, JSON).

[2094] 4. Data import and analysis: The server imports the converted data into a generative AI (e.g., OpenAI GPT-4) for analysis.

[2095] 5. Data transmission and display: Based on the analysis results, the generation AI converts the sales data into a graph format and sends it to the device, which then displays the received data on the screen of the smart glasses or smartphone.

[2096] Specific examples

[2097] For example, when a store manager wants to check inventory status in real time, he or she uses the system by following the steps below.

[2098] 1. Instruction: The store manager speaks to the smart glasses and says, "Tell me the current inventory status."

[2099] 2. Send: The smart glasses parse this request and send it to the server.

[2100] 3. Acquisition and conversion: The server acquires the latest inventory data from the inventory database and inputs it into the generation AI.

[2101] 4. Analysis and output: The generative AI analyzes the data and sends the current inventory status in a list format to the smart glasses.

[2102] 5. Display: The smart glasses visually display the results to the user.

[2103] Prompt Sentence Examples

[2104] "Please get yesterday's sales data and graph it."

[2105] "Show me this week's sales data compared by day of the week."

[2106] Please see a list of current inventory.

[2107] This system allows users to obtain data in natural language without requiring any special knowledge and check the information they need in real time, significantly improving work efficiency and supporting faster decision-making.

[2108] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2109] Step 1:

[2110] The user issues a command in natural language to their smart glasses or smartphone, such as "Show me today's sales data."

[2111] Input: User's voice or text instructions

[2112] Output: Natural language instructions data

[2113] Specific operation: The voice recognition function of smart glasses or smartphone captures the user's instructions as text data.

[2114] Step 2:

[2115] The device analyzes the user's instructions using a natural language processing library (e.g., spaCy, BERT) and sends the results to the server.

[2116] Input: Natural language instructions

[2117] Output: Parsed instruction data

[2118] Specific operation: The device uses a natural language processing library to analyze the intent of the instruction and sends the content of the instruction (e.g., "Get today's sales data") to the server as structured data.

[2119] Step 3:

[2120] The server accesses an external source (e.g., a sales database) and retrieves the required data.

[2121] Input: Parsed instruction data

[2122] Output: Sales data

[2123] Specific operation: The server accesses the sales database, queries today's sales data, and retrieves the data.

[2124] Step 4:

[2125] The server converts the data it receives into a specific format (e.g., CSV, JSON).

[2126] Input: Sales data

[2127] Output: Reformatted sales data

[2128] What happens: The server uses a data conversion library to convert raw sales data into a defined format (e.g. CSV, JSON).

[2129] Step 5:

[2130] The server then inputs the converted data into a generative AI model (e.g., OpenAI GPT-4) for analysis.

[2131] Input: Reformatted sales data

[2132] Output: Analysis result data (e.g., graph data)

[2133] Specific operation: The server inputs data into the generative AI model, analyzes the sales data, and generates the results in graph form.

[2134] Step 6:

[2135] The server transmits the generated graph data to the terminal.

[2136] Input: Analysis result data (graph data)

[2137] Output: Graph data sent to the terminal

[2138] Specific operation: The server uses a network protocol to send the generated graph data to the device.

[2139] Step 7:

[2140] The terminal visually displays the received graph data to the user.

[2141] Input: Graph data sent to the terminal

[2142] Output: Screen showing the graph

[2143] Specific operation: Graph data is drawn on the device's display screen so that the user can visually check it.

[2144] By performing the necessary data processing and calculations at each step in this way, users can easily obtain data in natural language and visually confirm the results.

[2145] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2146] The present invention relates to a system that uses artificial intelligence to recognize a user's natural language instructions and emotions, and acquire, convert, and generate data from external data sources based on those instructions and emotions, and output data in a desired format. In particular, by combining an emotion engine that recognizes the user's emotions, the system enables more adaptive data output according to the user's emotional state.

[2147] Overall system overview

[2148] The system mainly includes the following functional blocks: data acquisition means, data conversion means, artificial intelligence for generation, output means, and emotion engine.

[2149] Key components of the system

[2150] 1. Data Acquisition Request

[2151] A user inputs a data acquisition request in natural language via a terminal, which then transmits the request to a server.

[2152] 2. Data Acquisition and Transformation

[2153] The server retrieves the required data from an external data source (e.g., a company's sales database).

[2154] The acquired data is converted into a certain format (e.g. JSON) by the server.

[2155] 3. Inputting data into the generated AI

[2156] The server then feeds the converted data into the AI ​​that generates it, which stores the data internally and prepares it for processing.

[2157] 4. Functions of the Emotion Engine

[2158] The emotion engine analyzes emotions from the user's natural language instructions.

[2159] The analysis results are fed back to the generating AI, which then analyzes and outputs the data taking into account the emotional state.

[2160] 5. Data reference and output

[2161] The user instructs the server in natural language via the terminal which output format (e.g., graph format) he / she desires.

[2162] The server transmits the instructions to the AI ​​that generates the data, and the AI ​​outputs the data in the specified format based on the internal data. The output format is adjusted based on feedback from the emotion engine.

[2163] The output data is sent to the terminal via the server and can be viewed by the user.

[2164] Specific examples of processing

[2165] A sales rep reviews monthly sales data and analyzes sentiment:

[2166] 1. Data Ingestion Request

[2167] The sales representative (user) instructs the device to "input this month's sales data into the generation AI."

[2168] The terminal sends this instruction to the server as a data acquisition request.

[2169] 2. Data Acquisition and Transformation

[2170] The server accesses the company's sales database to retrieve the current month's sales data.

[2171] The server converts the acquired data into a certain format.

[2172] 3. Data Import

[2173] The server then inputs the converted data into the generation AI.

[2174] The generative AI stores the data internally.

[2175] 4. Emotion Engine Analysis

[2176] When a salesperson says, "Show me this month's sales data in a graph," the emotion engine analyzes the emotion from their natural language.

[2177] For example, if the user indicates that they are feeling stressed, the emotion engine will detect this.

[2178] 5. Data output request

[2179] The terminal sends an output request to the server.

[2180] The server instructs the generative AI to output data in a format that adapts to the user's emotions, taking into account feedback from the emotion engine.

[2181] 6. Data Output and Display

[2182] The generation AI generates graph data taking into account the analysis results of the emotion engine and sends it back to the server.

[2183] The server sends the generated data to the terminal, which displays the graph to the user.

[2184] The above process allows users to acquire, analyze, and output data without any special technical knowledge, and also enables adaptive data output according to the user's emotional state.

[2185] The processing flow will be explained below.

[2186] Data Import

[2187] Step 1:

[2188] The user types into the terminal, "Enter this month's sales data into the generation AI."

[2189] Step 2:

[2190] The terminal analyzes the user's instructions and sends them to the server as a data acquisition request.

[2191] Step 3:

[2192] The server receives the data acquisition request and accesses the company's sales database, which is an external data source, to acquire the current month's sales data.

[2193] Step 4:

[2194] The server converts the acquired data into a format that the generating AI can understand, such as JSON format.

[2195] Step 5:

[2196] The server calls an API to feed the converted data into the generation AI.

[2197] Step 6:

[2198] The generation AI stores the data received from the server internally and notifies the server that the data import is complete.

[2199] Step 7:

[2200] The server notifies the terminal of the completion of data import.

[2201] Step 8:

[2202] The terminal receives a notification from the server and notifies the user that the data import is complete.

[2203] Data lookup and sentiment analysis

[2204] Step 1:

[2205] The user inputs into the terminal, "Show me this month's sales data in a graph."

[2206] Step 2:

[2207] The terminal analyzes the user's instructions and transmits the user's natural language to the emotion engine.

[2208] Step 3:

[2209] The emotion engine analyzes emotions from the user's natural language and feeds the analysis results back to the generative AI.

[2210] Step 4:

[2211] The device sends the user's instructions and the results of emotion analysis to the server.

[2212] Step 5:

[2213] The server receives the data output request and the sentiment analysis results, and instructs the generation AI to output this month's sales data in graph format. The instruction also includes the sentiment analysis results.

[2214] Step 6:

[2215] The AI ​​analyzes the data and emotion analysis results and generates graphs according to instructions. For example, if the user is feeling stressed, it will use colors and layouts to reduce stress.

[2216] Step 7:

[2217] The generation AI returns the generated graph data to the server.

[2218] Step 8:

[2219] The server sends the graph data received from the generation AI to the terminal.

[2220] Step 9:

[2221] The terminal receives the graph data and displays it to the user.

[2222] Specific examples

[2223] A sales rep pulls this month's sales data and analyzes sentiment.

[2224] Step 1:

[2225] The sales representative (user) instructs the device to "input this month's sales data into the generation AI."

[2226] Step 2:

[2227] The terminal analyzes the instruction and sends a data acquisition request to the server.

[2228] Step 3:

[2229] The server accesses the company's sales database to retrieve the current month's sales data.

[2230] Step 4:

[2231] The server converts the acquired data and feeds it into the generated AI.

[2232] Step 5:

[2233] The generation AI stores the data internally and notifies the server when the import is complete.

[2234] Step 6:

[2235] The server notifies the terminal that the import is complete.

[2236] Step 7:

[2237] The terminal notifies the sales representative that the import is complete.

[2238] Step 8:

[2239] The sales representative (user) instructs, "Display this month's sales data in a graph."

[2240] Step 9:

[2241] The terminal sends the user's emotion along with the instruction to the emotion engine for analysis.

[2242] Step 10:

[2243] The emotion engine analyzes the instructions and detects when the user is feeling stressed.

[2244] Step 11:

[2245] The emotion engine feeds back the analysis results to the server.

[2246] Step 12:

[2247] The terminal sends a data output request and the emotion analysis results to the server.

[2248] Step 13:

[2249] The server instructs the generating AI to output data in graph format, taking into account the results of emotion analysis.

[2250] Step 14:

[2251] The generative AI generates appropriate graph data based on data and sentiment analysis results.

[2252] Step 15:

[2253] The generation AI sends the graph data back to the server.

[2254] Step 16:

[2255] The server transmits the graph data to the terminal.

[2256] Step 17:

[2257] The terminal displays the graph data to the user, who can then refer to it.

[2258] This specific processing flow allows users to obtain and output data through instructions in natural language, and by utilizing sentiment analysis in the process, it enables more user-friendly and effective data reference.

[2259] Example 2

[2260] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2261] In conventional data acquisition and output systems, it was difficult for users to not only give instructions in natural language, but also to recognize the emotional state of the user from the content of the instructions and adjust the data output accordingly. As a result, it was not possible to output data that was adapted to the user's emotional state, and this resulted in a problem of not being able to sufficiently improve the user experience.

[2262] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving a data acquisition request and acquiring data from an external data source, means for converting the acquired data into a certain format and inputting it into the AI ​​to be generated, means for outputting the input data in a data format corresponding to the output request based on the data, means for analyzing emotions from a user's natural language instruction and feeding back the emotional state to the AI ​​to be generated, and means for the AI ​​to analyze and output data taking the user's emotional state into consideration. This makes it possible to acquire, analyze, and output data adapted to the user's emotional state.

[2263] "Data retrieval request" refers to an instruction sent by a user to the system to retrieve data from an external data source.

[2264] "External data source" refers to an information source that exists outside the system and where the required data is stored.

[2265] "Means for obtaining data" refers to the functionality for accessing and obtaining the required data from an external data source.

[2266] "Means for converting data into a certain format and inputting it into the AI ​​to be generated" refers to the function for converting acquired data into a specific format (such as JSON format) and inputting the converted data into the internal memory of the AI ​​to be generated.

[2267] "Generative AI" refers to an AI model that is used to manipulate and analyze ingested data and generate output in response to user requests.

[2268] An "output request" refers to a user's request to the system to output data in a particular format.

[2269] "Output means" refers to a function for generating and outputting necessary data in a specific format in response to a user's output request.

[2270] "Natural language instructions" refers to instructions given by a user to a system in natural language (e.g., Japanese, English, etc.).

[2271] "Means for analyzing emotions" refers to a function for analyzing the user's emotional state from the user's natural language instructions, voice, etc.

[2272] An "emotion engine" refers to an analysis device that analyzes emotions from the user's natural language instructions and reflects the results in the system.

[2273] "Feedback" refers to transmitting analysis results and other information to the artificial intelligence that generates it, and then using that information for further processing and adjustments.

[2274] "Means for analyzing and outputting data" refers to a function for analyzing the captured data taking into account the emotional state of the user and outputting it in the most appropriate format.

[2275] This system recognizes the user's natural language instructions and emotions, and uses artificial intelligence to acquire, convert, and generate the necessary data from external data sources based on that information, outputting the data in the desired format. The main hardware and software components that make up the system are as follows:

[2276] Hardware and Software

[2277] 1. Device:

[2278] A device that allows users to input data acquisition and output requests. Examples include smartphones, tablets, and PCs.

[2279] 2. Server:

[2280] It is responsible for accessing external data sources, acquiring and transforming the data, feeding it into the generative AI model, receiving feedback from the emotion engine, and finally converting the data into an output format. It uses a regular web server or a cloud-based server.

[2281] 3. Generative AI Model:

[2282] It is an artificial intelligence model that analyzes and generates acquired data based on user requests. Typical generative AI models include GPT and BERT.

[2283] 4. Emotion Engine:

[2284] This is software or a module that analyzes emotions from users' natural language instructions and feeds the results back to a generative AI model. Natural language processing technology is used for emotion analysis.

[2285] Specific operations and procedures

[2286] 1. User data request:

[2287] The user issues instructions in natural language via the device, such as "Input this month's sales data into the generation AI." The device then sends this instruction to the server as text data.

[2288] 2. Data retrieval and transformation by the server:

[2289] Based on the received data retrieval request, the server accesses external data sources such as a company's sales database to retrieve the required data, and then converts the retrieved data into a specific format (such as JSON).

[2290] 3. Server imports data into the generated AI model:

[2291] The server sends the converted data to the generative AI model, which then stores the data in its internal memory.

[2292] 4. User sentiment analysis:

[2293] When a user requests data output, for example, "Show me this month's sales data in a graph," the emotion engine analyzes the user's emotional state (e.g., stress, joy, etc.) from this instruction and feeds the results back to the generative AI model.

[2294] 5. Data generation and output:

[2295] Based on feedback from the emotion engine, the generative AI model analyzes the data in a format that adapts to the user's emotional state, and finally generates the data in an output format (e.g., graph format).

[2296] The generated data is sent to the terminal via the server, and the terminal displays it to the user.

[2297] Specific examples

[2298] Prompt Sentence Examples

[2299] User: "Please input this month's sales data into the generation AI."

[2300] User: "Show me this month's sales data in a graph."

[2301] This allows users to easily acquire, analyze, and output data using natural language without any special technical knowledge.In addition, by introducing an emotion engine, it is possible to realize adaptive data output according to the user's emotional state.

[2302] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2303] Step 1:

[2304] Sending a data retrieval request

[2305] explanation

[2306] The user inputs a data acquisition request in natural language into the terminal, which then sends the request to the server.

[2307] input

[2308] Natural language instructions from the user (e.g., "Please feed this month's sales data into the Generative AI").

[2309] output

[2310] The request sent to the server to retrieve data.

[2311] Specific actions

[2312] The user types into the device, "Please input this month's sales data into the generation AI." The device converts the natural language instruction into text format and sends it to the server as an HTTP request.

[2313] Step 2:

[2314] Data Acquisition

[2315] explanation

[2316] When the server receives a data retrieval request, it accesses external data sources to retrieve the required data.

[2317] input

[2318] The data retrieval request sent to the server (e.g. received as a "fetch_data" command).

[2319] output

[2320] Data obtained from external sources.

[2321] Specific actions

[2322] The server runs an SQL query (e.g., "SELECT FROM sales_data WHERE month = 'current_month'") against the company's sales database to retrieve the current month's sales data.

[2323] Step 3:

[2324] Data transformation

[2325] explanation

[2326] The server converts the retrieved data into a certain format (e.g. JSON).

[2327] input

[2328] Raw data obtained from external sources.

[2329] output

[2330] The data converted to JSON format.

[2331] Specific actions

[2332] Convert the sales data acquired by the server into JSON format. Example: Convert as follows.

[2333] json

[2334] {

[2335] "month": "current_month",

[2336] "total_sales": 100000,

[2337] "details": [

[2338] {"product": "Product A", "sales": 50000},

[2339] {"product": "Product B", "sales": 50000}

[2340] ]

[2341] }

[2342] Step 4:

[2343] Ingesting data into a generative AI model

[2344] explanation

[2345] The server then feeds the converted data into a generative AI model, which stores the data internally and prepares it for processing.

[2346] input

[2347] The data converted to JSON format.

[2348] output

[2349] Data fed into generative AI models.

[2350] Specific actions

[2351] The server uses an internal API to send data to the generative AI model, which then stores the received data in its internal memory.

[2352] Step 5:

[2353] Sentiment analysis using natural language instructions

[2354] explanation

[2355] The user makes an output request in natural language, and the emotion engine analyzes the emotion from the natural language instruction.

[2356] input

[2357] Natural language commands from the user (e.g., "Show me this month's sales data in a graph").

[2358] output

[2359] Analyzed emotion data.

[2360] Specific actions

[2361] A user might say, "Show me this month's sales data in a graph." The emotion engine analyzes this natural language and identifies emotions such as "stress" or "joy."

[2362] Step 6:

[2363] Generating emotional feedback and sending it to the AI ​​model

[2364] explanation

[2365] The analysis results of the emotion engine are fed back to the generative AI model.

[2366] input

[2367] Analyzed emotion data.

[2368] output

[2369] Emotional feedback sent to a generative AI model.

[2370] Specific actions

[2371] The emotion engine sends the analysis results (e.g., "emotion": "stress", "confidence_score": 0.85") to the generative AI model.

[2372] Step 7:

[2373] Data generation and output

[2374] explanation

[2375] The generative AI model analyzes the data and generates an output format taking into account the user's emotional state. The server sends the generated data to the device and displays it on the device.

[2376] input

[2377] Data in a generative AI model that has received emotional feedback.

[2378] output

[2379] The output data generated and the data format sent to the terminal.

[2380] Specific actions

[2381] The generative AI model analyzes the data taking into account the emotional state and generates output data in the form of a graph. The server sends the generated graph data to the device, which then displays the graph to the user.

[2382] Through the above processing steps, the user can make a data acquisition request in natural language and the results can be displayed adaptively according to the user's emotional state.

[2383] (Application example 2)

[2384] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2385] Conventional food delivery systems process orders without considering the user's emotional state, making it difficult to provide food that suits the user's current mood or emotion. Furthermore, they are unable to respond to instructions in natural language, which can lead to a lack of user experience. This can lead to a decrease in user satisfaction and complicated ordering.

[2386] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a data acquisition request and acquiring data from an external data source, means for converting the acquired data into a certain format and inputting it into the generated artificial intelligence, means for outputting the acquired data in a data format corresponding to the output request, and means for recognizing the user's instructions and emotions in natural language and adjusting the data output according to the user's emotional state using an emotion engine. This allows users to easily order the dish that best suits their emotional state using natural language, significantly improving the convenience and satisfaction of food delivery.

[2387] A "data acquisition request" is a request from a user to acquire data from a specified external data source.

[2388] An "external data source" is an information source that exists outside the system and provides data, such as various databases and APIs.

[2389] A "consistent format" is a standardized format that makes data easier to process, such as JSON or XML.

[2390] "Generative AI" is a general term for AI that has the ability to generate new information or results based on acquired data.

[2391] "Natural language" refers to a language that humans use on a daily basis, and is not a programming language.

[2392] An "emotion engine" is a software component for analyzing and recognizing emotions from a user's natural language input.

[2393] "Output format" refers to the format or style in which data is presented to a user, visually or otherwise.

[2394] "Filtering" is the process of selecting or excluding data based on specific conditions.

[2395] "User experience" refers to the overall experience and satisfaction a user has when using a product or service.

[2396] To implement this invention, a system including the following functional blocks and procedures is required: This system comprises a data acquisition means, a data conversion means, an artificial intelligence for generation, an output means, and an emotion engine.

[2397] Hardware and software used

[2398] Hardware: Smartphone (iOS, Android)

[2399] software:

[2400] Natural language processing libraries (e.g., Google Cloud Natural Language API)

[2401] Emotion recognition engine (e.g. Microsoft Azure Emotion API)

[2402] Database (e.g. Firebase Firestore)

[2403] RESTful API (for data retrieval)

[2404] Front-end frameworks (e.g., React Native)

[2405] Processing a data retrieval request

[2406] The server receives a data retrieval request from the user and accesses the food delivery service's database to retrieve the dish data for each restaurant. This data is then converted into a standard format such as JSON.

[2407] Data transformation and incorporation into generative artificial intelligence

[2408] The server converts the acquired data into a certain format (such as JSON) and inputs it into the AI ​​model to be generated. The AI ​​model to be generated stores the converted data internally and prepares it for subsequent processing.

[2409] Natural Language Processing and Emotion Recognition

[2410] The user's natural language instructions are received via the smartphone. These instructions are parsed using the Google Cloud Natural Language API to understand the user's request. The Microsoft Azure Emotion API is then used to recognize the user's emotional state. The emotion recognition engine determines emotions such as "relaxed," "stressed," or "happy."

[2411] Adjusting output requirements and data formats

[2412] The user inputs the desired dish format in natural language and sends it back to the server. The server instructs the generation AI to consider feedback from the emotion engine and output the dish that best suits the user's emotional state. The generation AI filters the dish data based on the analysis results of the emotion engine, generates a personalized dish list, and displays it on the smartphone.

[2413] Specific examples

[2414] 1. User input: "I'm a little tired today, so I want something relaxing to eat."

[2415] 2. Natural language processing: Analyze using the Google Cloud Natural Language API to understand the user's specifications.

[2416] 3. Emotion recognition: Determine whether the user is "tired" using the Microsoft Azure Emotion API.

[2417] 4. Example output: "Good work! Would you like some relaxing herbal tea and a light salad?"

[2418] This allows users to easily order the food that best suits their emotional state using natural language, significantly improving the convenience and satisfaction of food delivery.

[2419] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2420] Step 1:

[2421] The user inputs a data acquisition request in natural language via a smartphone, and this instruction (e.g., "Recommend me a dish that suits my current mood") is sent from the device to the server.

[2422] Input: User's natural language instructions.

[2423] Output: The data retrieval request sent to the server.

[2424] Step 2:

[2425] The server accesses the food delivery service's database to obtain the food data for each restaurant.

[2426] Input: Data retrieval request.

[2427] Output: Retrieved dish data (e.g., JSON format).

[2428] Step 3:

[2429] The server converts the acquired data into a certain format (e.g., JSON), and the converted data is fed into the AI ​​that generates it.

[2430] Input: The acquired food data.

[2431] Output: The converted data (e.g., in JSON format).

[2432] Step 4:

[2433] The user again inputs the desired cooking style in natural language via the smartphone, and the terminal transmits this output request to the server.

[2434] Input: The user's output request (e.g., "Show me relaxing recipes").

[2435] Output: The output request sent to the server.

[2436] Step 5:

[2437] The server uses the Google Cloud Natural Language API to parse the user's natural language output request and understand the user's request.

[2438] Input: A natural language output request.

[2439] Output: The parsed request content.

[2440] Step 6:

[2441] The server uses the Microsoft Azure Emotion API to recognize the user's emotional state. The emotion engine determines emotions such as "relaxed" or "stressed" from the user's natural language instructions.

[2442] Input: A natural language output request.

[2443] Output: Perceived emotional state.

[2444] Step 7:

[2445] The server provides feedback to the artificial intelligence it generates, instructing it to filter the food data based on the user's emotional state and generate a personalized list of dishes.

[2446] Input: Recognized emotional state and transformed food data.

[2447] Output: A filtered, personalized list of dishes.

[2448] Step 8:

[2449] The server sends the generated recipe list to the terminal, which displays it on the smartphone.

[2450] Input: A filtered list of dishes.

[2451] Output: The list of dishes displayed to the user.

[2452] Through the above processing steps, users can easily order the dish that best suits their emotional state using natural language.

[2453] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2454] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2455] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2456] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2457] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2458] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2459] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2460] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2461] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2462] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2463] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2464] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2465] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2466] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2467] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2468] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2469] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2470] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2471] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2472] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2473] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2474] The following is further disclosed regarding the above embodiment.

[2475] (Claim 1)

[2476] means for receiving a data retrieval request and retrieving data from an external data source;

[2477] A means of converting the acquired data into a certain format and incorporating it into the artificial intelligence to be generated;

[2478] A means for outputting the captured data in a data format according to an output request;

[2479] A system including:

[2480] (Claim 2)

[2481] 2. The system according to claim 1, further comprising means for receiving an output request in natural language and issuing instructions to the generating artificial intelligence based on the natural language.

[2482] (Claim 3)

[2483] 10. The system of claim 1, wherein the generating artificial intelligence includes means for displaying the output data in a graphical format.

[2484] "Example 1"

[2485] (Claim 1)

[2486] means for receiving a data retrieval request and retrieving data from an external data source;

[2487] A means for converting the acquired data into a certain format and incorporating it into the artificial intelligence model to be generated;

[2488] A means for outputting the captured data in a data format according to an output request;

[2489] means for analyzing the user's input using natural language analysis means and transmitting the analysis result to a server;

[2490] means for the server to analyze the received data and obtain necessary data from a designated data source;

[2491] a means for causing an artificial intelligence model generated by the server to analyze the data and output the analyzed data in a graph format;

[2492] means for displaying the data in graph form received by the terminal from the server to the user;

[2493] A system including:

[2494] (Claim 2)

[2495] 10. The system of claim 1, further comprising means for receiving a natural language output request from a user and transmitting the parsed request to the server.

[2496] (Claim 3)

[2497] 10. The system of claim 1, wherein the generating artificial intelligence model includes means for displaying output data in graphical form.

[2498] "Application Example 1"

[2499] (Claim 1)

[2500] means for receiving data retrieval requests and retrieving information from external sources;

[2501] A means of converting the acquired information into a certain format and incorporating it into the artificial intelligence to be generated;

[2502] means for outputting the captured information in a data format according to an output request;

[2503] means for receiving and parsing data retrieval and output requests in natural language;

[2504] means for transmitting the parsed natural language to a server;

[2505] means for displaying the analyzed and processed data in a user interface;

[2506] A system including:

[2507] (Claim 2)

[2508] 2. The system according to claim 1, further comprising means for receiving an output request in natural language and issuing instructions to the generating artificial intelligence based on the natural language.

[2509] (Claim 3)

[2510] 10. The system of claim 1, wherein the generating artificial intelligence includes means for displaying the output data in a graphical format.

[2511] "Example 2: Combining Emotion Engines"

[2512] (Claim 1)

[2513] means for receiving a data retrieval request and retrieving data from an external data source;

[2514] A means of converting the acquired data into a certain format and incorporating it into the artificial intelligence to be generated;

[2515] A means for outputting the captured data in a data format according to an output request;

[2516] a means for analyzing emotions from a user's natural language instructions and feeding the analyzed emotions back to an artificial intelligence generating the emotional state;

[2517] A means for the generating artificial intelligence to analyze and output data taking into account the emotional state of the user;

[2518] A system including:

[2519] (Claim 2)

[2520] 2. The system according to claim 1, further comprising means for receiving an output request in natural language and issuing instructions to the generating artificial intelligence based on the natural language.

[2521] (Claim 3)

[2522] 10. The system of claim 1, wherein the generating artificial intelligence includes means for displaying the output data in a graphical format.

[2523] "Application example 2 when combining emotion engines"

[2524] (Claim 1)

[2525] means for receiving a data retrieval request and retrieving data from an external data source;

[2526] A means of converting the acquired data into a certain format and incorporating it into the artificial intelligence to be generated;

[2527] A means for outputting the captured data in a data format according to an output request;

[2528] means for recognizing a user's natural language instructions and emotions and for adjusting data output according to the user's emotional state using an emotion engine;

[2529] A system including:

[2530] (Claim 2)

[2531] 2. The system according to claim 1, further comprising means for receiving an output request in natural language and issuing instructions to the generating artificial intelligence based on the natural language.

[2532] (Claim 3)

[2533] 2. The system of claim 1, wherein the generating artificial intelligence includes means for displaying the output data in a graphical format or in a format that best suits the user's emotional state. [Explanation of symbols]

[2534] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving a data retrieval request and retrieving data from an external data source; A means of converting the acquired data into a certain format and incorporating it into the artificial intelligence to be generated; A means for outputting the captured data in a data format according to an output request; A system including:

2. 2. The system according to claim 1, further comprising means for receiving an output request in natural language and issuing instructions to the generating artificial intelligence based on the natural language.

3. 2. The system of claim 1, wherein the generating artificial intelligence includes means for displaying the output data in graphical form.

Images