system

Abstract

We provide the system. [Solution] A means for receiving requests from users in natural language, A means for analyzing a received request using natural language processing and generating program code or data queries based on the content of the request, Means for executing generated program code or data queries, A system that includes a means of returning the execution results to the user.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventionally, when creating programs and data queries, specialized knowledge and skills are required, and it has been difficult for ordinary users to perform this easily. As a result, there is a problem that the rapid generation and acquisition of programs and data are hindered, making it difficult to improve business efficiency and respond quickly. In addition, since complex requests cannot be directly input in natural language, it is difficult to construct a system that appropriately reflects requests from users.

Means for Solving the Problems

[0005] The present invention provides a system that receives a request in natural language from a user, analyzes it, and generates and executes program code or a data query based on the request content. This system includes the following means.

[0006] 1. Means for receiving requests from users in natural language.

[0007] 2. Means for analyzing received requests using natural language processing and generating program code or data queries based on the content of the requests.

[0008] 3. Means for executing the generated program code or data queries.

[0009] 4. Means of returning execution results to the user

[0010] This allows users to generate programs and data queries tailored to their needs with just a single instruction in natural language, without requiring any special knowledge or skills, and then quickly respond to and improve based on that. Specifically, the system converts received requests into JSON format and sends them to the server, where it compiles and deploys the generated program code or data queries, or executes queries against the database, and returns the results to the user. This results in increased system efficiency and simplified user operation.

[0011] A "user" refers to an ordinary user who uses the system to input requests in natural language.

[0012] "Natural language processing" refers to the technology that enables computers to understand and analyze the language that humans normally use.

[0013] A "request" refers to a request that a user enters in natural language, such as a request to retrieve data or generate a program.

[0014] "Program code" refers to a set of instructions written in a format that a computer can execute.

[0015] A "data query" refers to a command statement that expresses instructions to retrieve or manipulate specific data from a database.

[0016] "Execution result" refers to the outcome obtained by executing the generated program code or data query.

[0017] "Receiving means" refers to the method or technology for the system to capture requests in natural language from users.

[0018] "Analyzing means" refers to the method or technology for interpreting the received requests in natural language and converting them into specific program code or data queries.

[0019] "Generating means" refers to the method or technology for creating program code or data queries based on the analysis results.

[0020] "Executing means" refers to the method or technology for operating the generated program code or data query on a computer.

[0021] "Returning means" refers to the method or technology for sending and presenting the execution result to the user.

[0022] "JSON format" is a standard format for storing and transmitting data concisely, which is a text-based format that is also easy for humans to read.

[0023] "Server" refers to a central processing unit that receives requests from users, processes them, and returns the generated results to the users.

[0024] "Compile" refers to the process of converting program code into a form executable by a computer.

[0025] "Deploy" refers to the process of placing and operating the generated program in an execution environment.

Brief Description of Drawings

[0026] [Figure 1]It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0027] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0028] First, let's explain the terminology used in the following explanation.

[0029] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0030] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0031] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0032] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0033] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0034] [First Embodiment]

[0035] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0036] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0037] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0038] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0039] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0040] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0041] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0042] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0043] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0044] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0045] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0046] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0047] This invention relates to a system that automatically generates program code and data queries based on requests entered by the user in natural language, and provides the results to the user. This system can be implemented in the following way.

[0048] This system begins with the user entering a request in natural language through their device. For example, they might enter a request such as "I want the sales data for 2023" into a text field in their browser or a dedicated application. When the user clicks the submit button, the request is converted to JSON format and sent to the server.

[0049] The server receives requests from users and parses their content. The requests are understood using natural language processing techniques and translated into specific program code or data queries. This generates precise instructions tailored to the specific request.

[0050] The generated program code and data queries are executed on the server. If the generated item is a data query, the server executes the query against the database and retrieves the corresponding data. Conversely, if the generated item is program code, this code is compiled by the server and deployed to the specified location.

[0051] The server returns the execution results to the user's terminal. In the case of a data query, the data obtained as a result of the query is returned; in the case of program code, the URL of the deployment destination is notified to the user. The user can then use this to retrieve the necessary data or access the generated application.

[0052] To give a concrete example, if a user requests a "simple to-do app," the server analyzes this request and generates the necessary program code. The generated code is compiled on the server and deployed to the specified server. The user then receives a URL to the deployment location, such as "Your app is available at http: / / yourserver.com / todoapp," and can access and use the application from there.

[0053] This system allows users to easily acquire data and develop applications based on their own requirements without needing specific expertise. This improves operational efficiency and enables faster response times. Furthermore, because it can generate programs and data for various purposes with just a single instruction in natural language, it is usable by a wide range of users.

[0054] The following describes the processing flow.

[0055] Program processing flow

[0056] Step 1: Obtaining requests from the user

[0057] 1.1 The user enters the request

[0058] User: The user enters a request in natural language, such as "I want the sales data for 2023," into a text field in their device's browser or application.

[0059] 1.2 User submits request

[0060] User: Click the submit button to send the request to the server.

[0061] Terminal: Encodes the request into an appropriate format such as JSON and sends it to the server.

[0062] Step 2: Server receives and parses the request.

[0063] 2.1 The server receives the request

[0064] Server: Receives requests sent from user terminals as HTTP requests and decodes the data in JSON format.

[0065] 2.2 Analysis of requests using generative AI

[0066] Server: The server analyzes the received request using natural language processing technology to obtain the user's request.

[0067] Server: Based on the analysis results, the generative AI generates appropriate program code or data queries.

[0068] Step 3: Execute the generated code and queries.

[0069] 3.1 Executing the generated query

[0070] Server: If the generated query is a data query (SQL), it executes that query against the database.

[0071] 3.2 Execution of the generated program code

[0072] Server: If the generated code is a program (such as Java® code), it compiles the code and deploys it to the specified location.

[0073] Step 4: Return the execution results

[0074] 4.1 Sending query results to the user's terminal

[0075] Server: Sends the results of the SQL query execution to the user's terminal.

[0076] 4.2 Notification of Program Deployment URL

[0077] Server: Notifies the user of the URL where the program code is deployed.

[0078] Step 5: Displaying results to the user

[0079] 5.1 Receiving the Results

[0080] Terminal: Receives results from the server.

[0081] 5.2 Displaying Results

[0082] Terminal: Displays the received results to the user.

[0083] User: Review the results of the submitted data query and the URL of the generated application, and access or use them as needed.

[0084] Through the steps described above, the present invention can automatically analyze natural language requests from users, generate appropriate program code and data queries, and execute them. This allows users to quickly and efficiently obtain the necessary information and programs without requiring any special knowledge.

[0085] (Example 1)

[0086] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0087] In conventional systems, generating program code and data queries required specialized knowledge, and the lack of automation made efficient data acquisition and application development difficult. This resulted in users having to expend a significant amount of time and effort, which was a major problem.

[0088] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0089] In this invention, the server includes means for receiving requests from users in natural language, means for analyzing the received requests using natural language processing and generating program code or data queries based on the request content, means for executing the generated program code or data queries, means for returning the execution results to the user, means for converting the received requests into JSON format, and means for using a generative AI model for analysis. This enables users to efficiently acquire data and create applications through natural language input without requiring specialized knowledge.

[0090] "Natural language" generally refers to the language that humans use on a daily basis, and is not a specific program or machine language.

[0091] A "request" is when a user asks for specific information or an action using natural language.

[0092] "Means of receiving" refers to methods or devices for obtaining requests from users and inputting them into the system.

[0093] "Natural language processing" is the technology that enables computers to understand and analyze human natural language.

[0094] "Program code" is code that contains instructions for performing a specific task.

[0095] A "data query" is a command used to retrieve specific data from a database.

[0096] "Generating means" refers to methods or devices that automatically generate program code or data queries based on the results of analysis.

[0097] "Means of execution" refers to the methods or devices used to run the generated program code or data queries on a computer.

[0098] "Means of return" refers to methods or devices for communicating the execution results to the user.

[0099] "JSON format" is a simple data format used for data exchange, and is an abbreviation for JavaScript® Object Notation.

[0100] A "generative AI model" is an artificial intelligence model that understands human language and commands and produces appropriate responses and output.

[0101] "Compilation" is the process of converting program code into a format that a machine can understand.

[0102] "Deployment" refers to placing a created program on a specific server or environment and making it available for use.

[0103] "HTTP format" refers to the protocol format used for sending and receiving information over the internet.

[0104] This invention is a system that automatically generates program code and data queries based on requests entered by the user in natural language, and provides the results to the user. The specific form of implementing this system is described below.

[0105] Users enter their requests using a device (e.g., a PC or smartphone). These devices are equipped with browsers (e.g., Google Chrome® or Mozilla Firefox) or dedicated applications, allowing users to enter requests in natural language. For example, a user might enter "I want the sales data for 2023" and click the submit button.

[0106] The terminal converts the input request into JSON format and sends it to the server as an HTTP request. The server receives this HTTP request and parses the JSON request body. Specifically, it analyzes the received data using natural language processing (NLP) techniques to understand the request. For this analysis, it uses, for example, generative AI models such as OpenAI®'s GPT-4® or Google® Cloud Natural Language API.

[0107] The server generates appropriate program code and data queries based on the analysis results. For example, to retrieve sales data for a specific year, it generates an appropriate SQL query. The generated SQL query is executed against a database (e.g., MySQL® or PostgreSQL) to retrieve the corresponding data. If program code is generated, it is compiled on the server and deployed to the specified server environment.

[0108] The results of data queries and the execution results of generated program code are returned from the server to the user's terminal. In the case of data queries, the retrieved data is returned as an HTTP response. In the case of program code, the user is notified of the deployment URL (for example, "http: / / yourserver.com / todoapp"). The user can use the generated application by accessing this URL.

[0109] For example, if a user requests a "simple to-do app," the server parses this request and generates appropriate JavaScript code. This code is compiled on the server and deployed to the specified server. As a result, the user receives a deployment URL such as "Your app is available at http: / / yourserver.com / todoapp," and can access and use the application.

[0110] Examples of prompt statements include the following:

[0111] "Please generate an SQL query to retrieve sales data for 2023."

[0112] "Please create JavaScript code to generate a simple to-do app."

[0113] "Create a Python script to retrieve order history based on a specific customer ID."

[0114] This system allows users to efficiently acquire data and develop applications through natural language input without requiring specialized knowledge.

[0115] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0116] Step 1: User enters request

[0117] The user enters their request in natural language into a text field in a browser or dedicated application using their device. For example, they might type "I want the sales data for 2023" and click the submit button. This request becomes the user's input. As output, this request becomes the input data needed to proceed to the next step.

[0118] Step 2: Convert and send the request

[0119] After receiving a user request, the terminal converts it into JSON format. For example, it is converted into a JSON object like the following:

[0120] json

[0121] {

[0122] "request": "I would like sales data for 2023."

[0123] }

[0124] This JSON data is output and sent to the server as an HTTP request. The body of the HTTP request becomes the input data.

[0125] Step 3: Receiving and parsing the request

[0126] The server receives HTTP requests sent from terminals. It parses the received JSON data and understands the request using natural language processing techniques. Specifically, it uses a generative AI model (e.g., OpenAI GPT-4) to analyze the request and extract its content. This analysis result becomes the output for data processing and the input data for the next step.

[0127] Step 4: Generate program code or data queries

[0128] The server generates program code and data queries based on the analysis results. For example, from a request such as "I want sales data for 2023," it generates the following SQL query:

[0129] sql

[0130] SELECT FROM sales WHERE year = 2023;

[0131] The generated SQL query becomes the output and the input data for proceeding to the next step. The generated program code and data queries are output as concrete actions.

[0132] Step 5: Execute the code or query

[0133] The server executes the generated program code and data queries. In the case of SQL queries, the server executes the query against the database (e.g., MySQL) and retrieves the data. The generated SQL query is executed and the corresponding data is output from the database. In the case of program code, the server compiles the code and deploys it to the specified location. Compilation and deployment are the concrete actions.

[0134] Step 6: Return the execution results

[0135] The server returns the retrieved data or the execution results of the generated program to the user. In the case of a data query, the retrieved data is returned as an HTTP response in JSON format.

[0136] json

[0137] {

[0138] "data": [

[0139] {"id": 1, "amount": 1000, "date": "2023-01-01"},

[0140] {"id": 2, "amount": 1500, "date": "2023-02-01"},

[0141] ...

[0142] ]

[0143] }

[0144] In the case of program code, the deployment URL is notified to the user. For example, the URL is returned in JSON format, such as "Your app is available at http: / / yourserver.com / todoapp". This result becomes the output, and the user receives it and proceeds with their actions.

[0145] Through the above processing steps, users can efficiently acquire data and develop applications through natural language input, even without specialized knowledge.

[0146] (Application Example 1)

[0147] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0148] Traditional factory inventory management systems have made it difficult to accurately track inventory levels and automate necessary replenishment orders. Furthermore, the systems were complex to use, increasing the risk of errors for workers without specialized knowledge. This led to inventory shortages and excesses, hindering efficient production activities.

[0149] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0150] In this invention, the server includes means for receiving requests from users in natural language; means for analyzing the received requests using natural language processing and generating program code or data queries based on the request content; means for executing the generated data queries against an inventory database within the factory and obtaining the results; means for returning the execution results to the user; and means for compiling the program code and deploying it to a specified location, or executing the data queries against the database and obtaining the results, depending on the type of program code or data query generated. This makes it possible for even workers without specialized knowledge to easily check inventory status and issue efficient replenishment instructions.

[0151] A "user" is someone who uses a system to input requests in natural language and receives the results.

[0152] "Natural language" refers to the language that humans use on a daily basis, and is a form of linguistic expression composed in a natural way, rather than a specific programming language.

[0153] A "request" refers to the content of instructions or inquiries that a user makes to the system.

[0154] "Means of receiving" refers to the processes and functions that allow the system to receive requests from users.

[0155] "Natural language processing" is a technology for analyzing and understanding requests from users in natural language.

[0156] "Analysis" is the process by which a system interprets information in order to understand the content of a received request and perform appropriate processing.

[0157] "Program code" is a part of a computer program that a system automatically generates to meet user requirements.

[0158] A "data query" is a query command sent to a database to retrieve specific information.

[0159] "Generating means" refers to functions or processes that automatically generate program code or data queries based on requests.

[0160] "Means of execution" refers to the processes and functions that actually run the generated program code and data queries.

[0161] A "factory inventory database" is a database system used to manage inventory information within a factory.

[0162] "Execution result" refers to the output obtained after the generated program code or data query has been executed.

[0163] "Means of return" refers to the processes or functions that allow a system to return its execution results to the user.

[0164] JSON format is a lightweight data exchange format for structuring, storing, and transmitting data.

[0165] A "server" is a computer system that receives requests, analyzes them, and performs the necessary processing.

[0166] "Deployment" refers to placing the generated program code in a specified environment and making it operational.

[0167] The system for implementing this invention will streamline inventory management within a factory and automate replenishment instructions based on requests entered by the user in natural language. The system mainly consists of the following hardware and software:

[0168] 1. Hardware:

[0169] User devices: ANDROID® smartphones, iOS smartphones

[0170] Server: High-performance computer system

[0171] Factory database: A database system for managing inventory information.

[0172] 2. Software:

[0173] "Smart Inventory Manager" is a dedicated application installed on the user's terminal.

[0174] A framework that runs on a server (e.g., Flask)

[0175] Generative AI models for natural language processing (e.g., OpenAI's GPT-3®)

[0176] Database management systems (e.g., MySQL, PostgreSQL)

[0177] Users input requests in natural language through the "Smart Inventory Manager" application on their smartphones. For example, they might enter text such as "Show me the inventory levels for 2023" and press the send button to send the request from their device to the server.

[0178] The server receives user requests in JSON format and analyzes them using natural language processing technology. Generative AI models such as OpenAI's GPT-3 are used to convert the requests into appropriate program code or data queries. These converted data queries are then executed against the factory's inventory database to retrieve the requested information.

[0179] The acquired inventory information is returned to the user by the server. The returned information is displayed on the user's smartphone application, allowing the user to check the inventory status in the factory in real time.

[0180] As a concrete example, if a user enters "Show me the inventory for 2023," the system translates this into an SQL query, "SELECT FROM inventory WHERE year = 2023," and executes it against the database. The results are returned to the device in JSON format, and the user can view the results on their smartphone.

[0181] This process allows even workers without specialized knowledge to check and efficiently manage inventory levels within the factory using simple natural language operations. Furthermore, the system automatically handles situations requiring quick responses, thus improving operational efficiency.

[0182] For the generative AI model and prompt statements, use the following example prompt statements:

[0183] Example prompt: "Show me the inventory levels for 2023."

[0184] This allows the system to automate inventory management within the factory and provide the ability to acquire and display data according to user requirements without requiring specialized knowledge.

[0185] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0186] Step 1:

[0187] A user enters a request in natural language using the smartphone application "Smart Inventory Manager" and presses the submit button. The input is in text format, such as "Show inventory levels for 2023." This request is converted to JSON format and sent to the server. Here, the input is the user's request, and the output is the data obtained by converting the request to JSON format.

[0188] Step 2:

[0189] The server receives user requests in JSON format sent from the terminal. Since the received requests are parsed within the server, the input is request data in JSON format, and the output is natural language text data.

[0190] Step 3:

[0191] The server analyzes received natural language requests using natural language processing. This analysis uses a generative AI model (e.g., OpenAI's GPT-3). The input is natural language text data, and the output is appropriate program code or data query. Specifically, the request "Show inventory levels for 2023" is converted into the data query "SELECT FROM inventory WHERE year = 2023".

[0192] Step 4:

[0193] The server executes the generated data query. Here, the server connects to the factory's inventory database and sends the generated data query. The input is the data query, and the output is the inventory information retrieved from the database.

[0194] Step 5:

[0195] The server converts inventory information retrieved from the database into JSON format and sends it back to the user. The input is inventory information retrieved from the database, and the output is inventory information converted into JSON format.

[0196] Step 6:

[0197] The user terminal receives inventory information in JSON format from the server and displays it within the application. The input is inventory information in JSON format, and the output is inventory information displayed visually to the user. This allows the user to check the inventory status within the factory in real time.

[0198] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0199] This invention combines a system that automatically analyzes user requests in natural language, converts them into program code or data queries, and provides the results with an emotion engine that recognizes user emotions. This system improves the user experience and provides more appropriate feedback and results.

[0200] This system begins with the user entering a request in natural language using a terminal. For example, the user enters a request such as "I want the 2023 sales data" or "I want a new ToDo app" into a text field in a browser or application. Simultaneously with the input, an emotion engine activates, recognizing the user's emotional state by analyzing the user's input along with their facial expressions and tone of voice.

[0201] When a user clicks the submit button, the device encodes the request in JSON format and sends it to the server. The server analyzes the received request and integrates sentiment data from the sentiment engine into the analysis results. This allows the server to prioritize the request for faster results, for example, if the user is in a hurry.

[0202] Once the server has finished its analysis, the generative AI generates program code or data queries based on the request. The generated program code or data queries are executed while also taking into account feedback based on the user's emotions. For example, if the user is feeling anxious, additional confirmation messages or detailed logs can be generated to provide reassurance.

[0203] If a data query is generated, the server executes it against the database and retrieves the results. If program code is generated, the server compiles it and deploys it to the specified location. The server then returns the execution results to the user's terminal. In the case of a data query, the retrieved data is notified to the user; in the case of a program, the URL of the deployment destination is notified.

[0204] For example, if a user emotionally inputs "I have so many tasks today, I need a new to-do app," the emotion engine recognizes the user's stress level. The server determines that an immediate response is needed, quickly generates program code, and provides a deployment URL in the most user-friendly format possible. By notifying the user of the URL in the form of "Your app is available at http: / / yourserver.com / todoapp," the user can access the URL and start using the app immediately.

[0205] This system allows users to receive highly personalized responses quickly through natural language request input and emotion recognition. Furthermore, by utilizing an emotion engine, it enables feedback and adjustments tailored to the user's psychological state, providing an optimal user experience.

[0206] The following describes the processing flow.

[0207] Program processing flow

[0208] Step 1: Obtaining the user's needs and emotions

[0209] 1.1 The user enters the request

[0210] User: Enters requests in natural language into a text field in a browser or application, such as "I want the 2023 sales data" or "I want a new ToDo app."

[0211] 1.2 The emotion engine recognizes the user's emotions

[0212] Server: The emotion engine analyzes the user's facial expressions and voice tone in real time and acquires emotion data.

[0213] 1.3 Sending a request

[0214] User: Click the submit button to send the request to the server.

[0215] Terminal: Encodes the request in JSON format and sends it to the server along with sentiment data.

[0216] Step 2: Server receives and parses the request.

[0217] 2.1 The server receives the request and sentiment data.

[0218] Server: Receives requests and sentiment data sent from the user terminal as HTTP requests and decodes the data in JSON format.

[0219] 2.2 Analysis of requests using generative AI

[0220] Server: Analyzes received request content and sentiment data using natural language processing technology to obtain the user's request content and sentiment state.

[0221] 2.3 Generating Program Code or Data Queries

[0222] Server: The generative AI generates program code or data queries based on the request, and adjusts the generation results considering sentiment data.

[0223] Step 3: Execute the generated code and queries.

[0224] 3.1 Executing the generated query

[0225] Server: If the generated query is a data query (SQL), it executes that query against the database.

[0226] 3.2 Compiling and deploying the generated program code

[0227] Server: If the generated code is a program (such as Java code), it compiles that code and deploys it to the specified location.

[0228] Step 4: Return the execution results

[0229] 4.1 Sending query results to the user's terminal

[0230] Server: Sends the results of the SQL query execution to the user's terminal.

[0231] 4.2 Notification of Program Deployment URL

[0232] Server: Notifies the user of the URL where the program code is deployed.

[0233] Step 5: Displaying results to the user

[0234] 5.1 Receiving the Results

[0235] Terminal: Receives results from the server.

[0236] 5.2 Displaying and responding to results

[0237] Terminal: Displays the received results to the user.

[0238] User: Review the results of the submitted data query and the URL of the generated application, and access or use them as needed.

[0239] Server: Take user sentiment data into consideration and provide additional appropriate feedback.

[0240] As a concrete example, consider a scenario where a user emotionally inputs, "I'm under a lot of stress and finding task management difficult; I need a new to-do app." The emotion engine recognizes the user's stress level, and the server determines that a quick response is needed. The server quickly generates program code and provides a user-friendly URL for deployment, thereby supporting the user in reducing stress. A message like "Your app is available at http: / / yourserver.com / todoapp" is sent, allowing the user to access the URL and start using the app immediately.

[0241] In this way, this system automatically analyzes users' natural language requests, recognizes their emotions, and provides personalized responses, thereby significantly improving the user experience.

[0242] (Example 2)

[0243] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0244] While conventional systems could accurately analyze users' natural language requests and generate appropriate program code and data queries, they failed to adequately improve the user experience because they did not consider the user's emotional state. Furthermore, they struggled to respond flexibly to situations where users required quick responses or detailed explanations and confirmations.

[0245] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting a request from a user in natural language, means for analyzing the input request and recognizing the user's emotions, means for encoding the request content and recognized emotion data into JSON format, means for transmitting the encoded data to the server, means for analyzing the transmitted data using natural language processing technology, means for generating program code or data queries using a generation AI model based on the analysis results, means for compiling the generated program code and deploying it to a specified location, or means for executing the generated data query against a database and obtaining the results, and means for returning the execution results to the user. This enables highly personalized responses that take into account the user's emotional state, and improves the user experience by providing quick and appropriate feedback.

[0246] "Natural language" refers to the language that humans use on a daily basis, and is in the form of text or spoken language, rather than program code or data queries.

[0247] "Emotions" refers to the user's psychological state, such as stress, excitement, or calmness.

[0248] An "emotion engine" is a software or hardware technology that analyzes input data such as a user's facial expressions and tone of voice to recognize the user's emotional state.

[0249] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a text format for describing data structures.

[0250] "Natural language processing technology" refers to technologies that enable the analysis and understanding of natural language, and includes text analysis, morphological analysis, and contextual analysis.

[0251] A "generative AI model" is an artificial intelligence model that generates new text, code, or data queries based on requests. Generative models such as GPT fall into this category.

[0252] "Program code" is a collection of instructions and statements that make up a computer program, and is software in a form that a computer can execute.

[0253] A "data query" refers to a command used to search for or retrieve specific information from a database.

[0254] "Compilation" is the process of converting programming code from a high-level language into machine code.

[0255] "Deployment" refers to the process of placing the generated program code into the actual production environment and making it executable.

[0256] "User experience" refers to the overall satisfaction and convenience that users experience when using a system or application.

[0257] "Feedback" refers to information that a system provides in response to a user, including information provided based on the user's requests and emotional state.

[0258] A "natural language request" is a request or question made by a user using natural language, and includes tasks that the system should process or the retrieval of information.

[0259] This invention combines a system that automatically analyzes user requests in natural language, converts them into program code or data queries, and provides the results with an emotion engine that recognizes user emotions. This system improves the user experience and provides more appropriate feedback and results.

[0260] This system begins with the user entering a request in natural language using a terminal. For example, the user might enter a request in a text field in a browser or application, such as "I want the 2023 sales data" or "I want a new ToDo app." The hardware used is the user's PC or smartphone, and the software used is a browser or application.

[0261] The emotion engine starts operating simultaneously with the input. The emotion engine analyzes the user's input, facial expressions, and tone of voice to recognize the user's emotional state. For example, it can determine whether the user is anxious or calm. This analysis uses hardware such as a camera and microphone, and emotion analysis software is employed.

[0262] When the user clicks the send button, the device encodes the request into JSON format and sends it to the server. At this point, sentiment data is also included in the JSON. The server then analyzes the received request using natural language processing (NLP) techniques (e.g., using Python or Node.js).

[0263] The server integrates the received emotional data into the analysis results. For example, if the emotional state is recognized as "stress," the request is processed as a high-priority task. This ensures that if the user is in a hurry, the server takes action to quickly obtain the results.

[0264] Next, the server uses a generative AI model (e.g., GPT-4) to generate program code or data queries that respond to the request. The generated program code or data queries also take into account the user's emotional state. For example, if the user is anxious, detailed comments or confirmation messages will be included.

[0265] If the generated data query is generated, the server executes it against the database and retrieves the results. If the generated program code is generated, the server compiles it and deploys it to the specified location. The server then returns the execution results to the user's terminal. For example, the URL of the deployment destination is notified to the user. By accessing this URL, the user can immediately use the generated program.

[0266] For example, if a user emotionally inputs "I have so many tasks today, I need a new to-do app," the emotion engine recognizes that the user is stressed. The server determines that a quick response is needed and uses a generative AI model to generate code for a to-do app. The generated application is then deployed to "http: / / yourserver.com / todoapp," and the URL is notified to the user. The user can access this URL and immediately start using the new to-do app.

[0267] Examples of prompts for a generative AI model:

[0268] User input: "I have so many tasks today, I need a new to-do app."

[0269] User's emotion: Stress

[0270] The program code to be generated: Code for a to-do app that makes task management easy.

[0271] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0272] Step 1:

[0273] The user enters the request into the terminal.

[0274] The user opens a browser or an application on their PC or smartphone and enters a request such as "I want the sales data for 2023" or "I want a new ToDo app" in natural language in the text field. At this time, the input is saved as text data in the memory of the terminal.

[0275] Step 2:

[0276] The terminal analyzes the user's emotion

[0277] The emotion engine installed on the terminal operates and analyzes the expression and voice tone in real time using the camera and microphone in addition to the user's input content. The input data is captured by the terminal as text, audio, and video data, and the emotion engine processes these data to output the user's emotional state.

[0278] Step 3:

[0279] The terminal sends the request and emotion data to the server

[0280] When the user clicks the send button, the terminal encodes the input request content and the analyzed emotion data in JSON format. This encoded JSON data is sent to the server via the Internet.

[0281] Step 4:

[0282] The server analyzes the request and emotion data

[0283] The server deserializes and analyzes the received JSON data. First, the request content is analyzed using natural language processing (NLP) technology and converted into a specific task. At the same time, the emotion data from the emotion engine is analyzed to determine the priority of the request. The analysis results are saved as an internal data structure (e.g., object or dictionary).

[0284] Step 5:

[0285] The generative AI generates program code or data queries

[0286] Based on the analyzed request data, the server uses a generative AI model (e.g., GPT-4) to generate program code or data queries. The generative AI model receives the request content and sentiment data as inputs and outputs specific code or queries based on them.

[0287] Step 6:

[0288] The server obtains the execution result and notifies the user

[0289] In the case of the generated data query, the server executes it against the database and obtains the result. In the case of the generated program code, the server compiles it and deploys it to the specified location. Finally, the server returns the deployment destination URL or query result to the user. Specifically, it sends a message to the user's terminal using the notification function.

[0290] Through each of these steps, the user can quickly receive highly personalized responses through natural language request input and sentiment recognition.

[0291] (Application Example 2)

[0292] Next, Application Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart device 14 is referred to as the "terminal". [[ID=3D]]

[0293] In conventional natural language processing systems, since appropriate feedback according to the user's psychological state is not provided, it is difficult to relieve the user's stress and anxiety, and as a result, there is a problem that the user experience does not improve. In addition, by providing feedback or answers that ignore the user's feelings, the reliability and convenience may be impaired.

[0294] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving requests from users in natural language, means for analyzing the received requests using natural language processing and generating program code or data queries based on the content of the requests, means for analyzing the user's emotional state at the time of request input, including an emotion engine that recognizes the user's emotions, means for integrating the emotional data from the emotion engine into the request analysis and providing feedback according to the user's psychological state, means for executing the generated program code or data queries, and means for returning the execution results to the user. This makes it possible to provide optimal feedback and results according to the user's psychological state.

[0295] "Means for receiving requests from users in natural language" refers to a system for obtaining requests for voice or text input by users in natural language.

[0296] "Means of analysis using natural language processing" refers to a system that analyzes received natural language requests using machine learning and statistical methods to understand their content.

[0297] "Means for generating program code or data queries" refers to a mechanism for automatically generating the necessary program code or queries for database operations based on the analyzed requirements.

[0298] An "emotion engine that recognizes user emotions" is software or hardware that analyzes a user's tone of voice and facial expressions to recognize their emotional state in real time.

[0299] "Means for analyzing emotional states" refers to a system that uses data acquired by the emotion engine to determine what emotional state a user is in.

[0300] The "means for integrating emotional data into request analysis" is a mechanism for incorporating the recognized emotional data into the results of natural language analysis and providing feedback according to the user's psychological state.

[0301] The "means for providing feedback according to the user's psychological state" is a mechanism for realizing a better user experience by performing appropriate additional messages and confirmations based on the user's emotional state.

[0302] The "means for executing the generated program code or data query" is a mechanism for compiling the generated program code and placing it in a specified location, or executing the data query against a database to obtain results.

[0303] The "means for returning the execution result to the user" is a mechanism for notifying or displaying the results of the executed program code or data query to the user.

[0304] The system for implementing this invention consists of the following components:

[0305] 1. User terminal:

[0306] As a terminal for the user to input requests in natural language, a smartphone is mainly used. Voice input uses the Google Cloud Speech-to-Text API, and text input is performed by standard keyboard input.

[0307] 2. Emotion engine:

[0308] Using the camera and microphone installed in the smartphone, the user's voice tone and expression are collected. This data is analyzed using Google ML Kit or Amazon Rekognition to recognize the user's emotional state.

[0309] 3. Natural language processing system:

[0310] The natural language requests entered by the user are parsed using the Google Cloud Natural Language API. As a result, the content of the request is extracted and treated as structured data.

[0311] 4. Server:

[0312] a. Requirements analysis:

[0313] The received request data and sentiment data are encoded into JSON format and sent to the server. The server then receives this data, integrates the results of natural language processing and sentiment analysis, and processes them.

[0314] b. Program code and data query generation:

[0315] Based on the analysis results, a generative AI model is used to generate program code or data queries that respond to user requests. This provides optimal feedback and results that are tailored to the user's psychological state.

[0316] c. Execution of the results:

[0317] The generated program code is compiled and deployed to the specified location, or data queries are executed against the database. The results of this execution are then returned to the user's terminal.

[0318] 5. Content Management:

[0319] The server manages the execution results and records them in a database so that it can quickly respond when the user makes another inquiry.

[0320] Specific example

[0321] 1. Example of natural language input:

[0322] The user voice-inputs "Send 5,000 yen to Tanaka" into their smartphone.

[0323] When the emotion engine detects anxiety, the system displays an additional confirmation message. For example, "Are you sure you want to send 5,000 yen to Tanaka? Please confirm."

[0324] 2. Example of a prompt:

[0325] "The user wants to send money to a friend. Their emotion has been identified as anxiety. Please generate an appropriate system response."

[0326] This system provides optimal feedback and results tailored to the user's psychological state, thereby improving the user experience and increasing trust.

[0327] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0328] Step 1:

[0329] User request input

[0330] Users enter their requests using their smartphones via voice or text. For voice input, the Google Cloud Speech-to-Text API converts the speech to text. If the input data is voice, the voice data is used and output as text. If the input data is text, it is used as is.

[0331] Specific actions:

[0332] The user voice-inputs, "Send 5,000 yen to Tanaka."

[0333] The Google Cloud Speech-to-Text API receives the audio data and outputs it as text data: "Send 5,000 yen to Tanaka."

[0334] Step 2:

[0335] Recognition of user emotions

[0336] The device's camera and microphone are used to capture the user's facial expressions and voice tone. Google ML Kit or Amazon Rekognition analyzes this data to recognize the user's emotional state. Input data consists of video and audio data, while output data consists of tags representing the emotional state (e.g., "anxious," "relieved," etc.).

[0337] Specific actions:

[0338] The camera captures the user's facial expressions, and the microphone collects their voice tone.

[0339] The ML kit outputs the tag "anxiety".

[0340] Step 3:

[0341] Requirements analysis using natural language processing

[0342] The device sends text data obtained from voice input to the Google Cloud Natural Language API for parsing of the request. The input data is text data, and the output data is structured request content (e.g., "Recipient: Tanaka", "Amount: 5000 yen").

[0343] Specific actions:

[0344] The text data "Send 5000 yen to Tanaka" is sent to the Natural Language API.

[0345] The API outputs structured data such as "Recipient: Tanaka" and "Amount: 5000 yen".

[0346] Step 4:

[0347] Integration of emotion data and request data

[0348] The server receives request data and sentiment data sent from the user terminal in JSON format, automatically parses and integrates them. The input data consists of structured request content and sentiment status tags, and the output data is the integrated JSON data.

[0349] Specific actions:

[0350] Send the following information to the server in JSON format: "Recipient: Tanaka", "Amount: 5000 yen", "Emotion: Anxious".

[0351] The server analyzes and integrates the data.

[0352] Step 5:

[0353] Program code and data query generation

[0354] The server uses a generative AI model to generate program code or data queries based on integrated JSON data. The input data is integrated data in JSON format, and the output data is the generated program code or data query.

[0355] Specific actions:

[0356] The server generates integrated data with the following information: "Recipient: Tanaka", "Amount: 5000 yen", and "Emotion: Anxiety". This data is then input into the AI ​​model.

[0357] The generating AI model outputs a remittance processing code or data query.

[0358] Step 6:

[0359] Execution and return of results

[0360] The server compiles the generated program code and deploys it to the specified location, or executes data queries against the database and retrieves the results. The resulting data is retrieved in JSON format and returned to the user's terminal. The input data is the generated program code or data query, and the output data is the execution result.

[0361] Specific actions:

[0362] The server compiles and executes the money transfer processing code.

[0363] The execution result returns a "Transfer successful" message in JSON format to the user's terminal.

[0364] Step 7:

[0365] User Feedback

[0366] The server returns the execution results to the user's terminal and displays additional confirmation messages corresponding to the user's emotional state. The input data is the JSON data of the execution results, and the output data is the messages displayed on the user interface.

[0367] Specific actions:

[0368] The user receives a "transfer successful" message along with an additional confirmation message stating, "We understand you were feeling anxious, but the transfer has been completed successfully."

[0369] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0370] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0371] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0372] [Second Embodiment]

[0373] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0374] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0375] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0376] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0377] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0378] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0379] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0380] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0381] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0382] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0383] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0384] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0385] This invention relates to a system that automatically generates program code and data queries based on requests entered by the user in natural language, and provides the results to the user. This system can be implemented in the following way.

[0386] This system begins with the user entering a request in natural language through their device. For example, they might enter a request such as "I want the sales data for 2023" into a text field in their browser or a dedicated application. When the user clicks the submit button, the request is converted to JSON format and sent to the server.

[0387] The server receives requests from users and parses their content. The requests are understood using natural language processing techniques and translated into specific program code or data queries. This generates precise instructions tailored to the specific request.

[0388] The generated program code and data queries are executed on the server. If the generated item is a data query, the server executes the query against the database and retrieves the corresponding data. Conversely, if the generated item is program code, this code is compiled by the server and deployed to the specified location.

[0389] The server returns the execution results to the user's terminal. In the case of a data query, the data obtained as a result of the query is returned; in the case of program code, the URL of the deployment destination is notified to the user. The user can then use this to retrieve the necessary data or access the generated application.

[0390] To give a concrete example, if a user requests a "simple to-do app," the server analyzes this request and generates the necessary program code. The generated code is compiled on the server and deployed to the specified server. The user then receives a URL to the deployment location, such as "Your app is available at http: / / yourserver.com / todoapp," and can access and use the application from there.

[0391] This system allows users to easily acquire data and develop applications based on their own requirements without needing specific expertise. This improves operational efficiency and enables faster response times. Furthermore, because it can generate programs and data for various purposes with just a single instruction in natural language, it is usable by a wide range of users.

[0392] The following describes the processing flow.

[0393] Program processing flow

[0394] Step 1: Obtaining requests from users

[0395] 1.1 The user enters the request

[0396] User: The user enters a request in natural language, such as "I want the sales data for 2023," into a text field in their device's browser or application.

[0397] 1.2 User submits request

[0398] User: Click the submit button to send the request to the server.

[0399] Terminal: Encodes the request into an appropriate format such as JSON and sends it to the server.

[0400] Step 2: Server receives and parses the request.

[0401] 2.1 The server receives the request

[0402] Server: Receives requests sent from user terminals as HTTP requests and decodes the data in JSON format.

[0403] 2.2 Analysis of requests using generative AI

[0404] Server: The server analyzes the received request using natural language processing technology to obtain the user's request.

[0405] Server: Based on the analysis results, the generative AI generates appropriate program code or data queries.

[0406] Step 3: Execute the generated code and queries.

[0407] 3.1 Executing the generated query

[0408] Server: If the generated query is a data query (SQL), it executes that query against the database.

[0409] 3.2 Execution of the generated program code

[0410] Server: If the generated code is a program (such as Java code), it compiles that code and deploys it to the specified location.

[0411] Step 4: Return the execution results

[0412] 4.1 Sending query results to the user's terminal

[0413] Server: Sends the results of the SQL query execution to the user's terminal.

[0414] 4.2 Notification of Program Deployment URL

[0415] Server: Notifies the user of the URL where the program code is deployed.

[0416] Step 5: Displaying results to the user

[0417] 5.1 Receiving the Results

[0418] Terminal: Receives results from the server.

[0419] 5.2 Displaying Results

[0420] Terminal: Displays the received results to the user.

[0421] User: Review the results of the submitted data query and the URL of the generated application, and access or use them as needed.

[0422] Through the steps described above, the present invention can automatically analyze natural language requests from users, generate appropriate program code and data queries, and execute them. This allows users to quickly and efficiently obtain the necessary information and programs without requiring any special knowledge.

[0423] (Example 1)

[0424] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0425] In conventional systems, generating program code and data queries required specialized knowledge, and the lack of automation made efficient data acquisition and application development difficult. This resulted in users having to expend a significant amount of time and effort, which was a major problem.

[0426] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0427] In this invention, the server includes means for receiving requests from users in natural language, means for analyzing the received requests using natural language processing and generating program code or data queries based on the request content, means for executing the generated program code or data queries, means for returning the execution results to the user, means for converting the received requests into JSON format, and means for using a generative AI model for analysis. This enables users to efficiently acquire data and create applications through natural language input without requiring specialized knowledge.

[0428] "Natural language" generally refers to the language that humans use on a daily basis, and is not a specific program or machine language.

[0429] A "request" is when a user asks for specific information or an action using natural language.

[0430] "Means of receiving" refers to methods or devices for obtaining requests from users and inputting them into the system.

[0431] "Natural language processing" is the technology that enables computers to understand and analyze human natural language.

[0432] "Program code" is code that contains instructions for performing a specific task.

[0433] A "data query" is a command used to retrieve specific data from a database.

[0434] "Generating means" refers to methods or devices that automatically generate program code or data queries based on the results of analysis.

[0435] "Means of execution" refers to the methods or devices used to run the generated program code or data queries on a computer.

[0436] "Means of return" refers to methods or devices for communicating the execution results to the user.

[0437] "JSON format" is a simple data format used for data exchange, and it is an abbreviation for JavaScript Object Notation.

[0438] A "generative AI model" is an artificial intelligence model that understands human language and commands and produces appropriate responses or generated text.

[0439] "Compilation" is the process of converting program code into a format that a machine can understand.

[0440] "Deployment" refers to placing a created program on a specific server or environment and making it available for use.

[0441] "HTTP format" refers to the protocol format used for sending and receiving information over the internet.

[0442] This invention is a system that automatically generates program code and data queries based on requests entered by the user in natural language, and provides the results to the user. The specific form of implementing this system is described below.

[0443] Users enter their requests using a device (e.g., a PC or smartphone). These devices are equipped with a browser (e.g., Google Chrome or Mozilla Firefox) or a dedicated application, allowing users to enter requests in natural language. For example, a user might enter "I want the sales data for 2023" and click the submit button.

[0444] The terminal converts the input request into JSON format and sends it to the server as an HTTP request. The server receives this HTTP request and parses the JSON request body. Specifically, it analyzes the received data using natural language processing (NLP) techniques to understand the request. For this analysis, it uses, for example, generative AI models such as OpenAI's GPT-4 or the Google Cloud Natural Language API.

[0445] The server generates appropriate program code or data queries based on the analysis results. For example, to retrieve sales data for a specific year, it generates an appropriate SQL query. The generated SQL query is executed against a database (e.g., MySQL or PostgreSQL) to retrieve the corresponding data. If program code is generated, it is compiled on the server and deployed to the specified server environment.

[0446] The results of data queries and the execution results of generated program code are returned from the server to the user's terminal. In the case of data queries, the retrieved data is returned as an HTTP response. In the case of program code, the user is notified of the deployment URL (for example, "http: / / yourserver.com / todoapp"). The user can use the generated application by accessing this URL.

[0447] For example, if a user requests a "simple to-do app," the server parses this request and generates appropriate JavaScript code. This code is compiled on the server and deployed to the specified server. As a result, the user receives a deployment URL such as "Your app is available at http: / / yourserver.com / todoapp," and can access and use the application.

[0448] Examples of prompt statements include the following:

[0449] "Please generate an SQL query to retrieve sales data for 2023."

[0450] "Please create JavaScript code to generate a simple to-do app."

[0451] "Create a Python script to retrieve order history based on a specific customer ID."

[0452] This system allows users to efficiently acquire data and develop applications through natural language input without requiring specialized knowledge.

[0453] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0454] Step 1: User enters request

[0455] The user enters their request in natural language into a text field in a browser or dedicated application using their device. For example, they might type "I want the sales data for 2023" and click the submit button. This request becomes the user's input. As output, this request becomes the input data needed to proceed to the next step.

[0456] Step 2: Convert and send the request

[0457] After receiving a user request, the terminal converts it into JSON format. For example, it is converted into a JSON object like the following:

[0458] json

[0459] {

[0460] "request": "I would like sales data for 2023."

[0461] }

[0462] This JSON data is output and sent to the server as an HTTP request. The body of the HTTP request becomes the input data.

[0463] Step 3: Receiving and parsing the request

[0464] The server receives HTTP requests sent from terminals. It parses the received JSON data and understands the request using natural language processing techniques. Specifically, it uses a generative AI model (e.g., OpenAI GPT-4) to analyze the request and extract its content. This analysis result becomes the output for data processing and the input data for the next step.

[0465] Step 4: Generate program code or data queries

[0466] The server generates program code and data queries based on the analysis results. For example, from a request such as "I want sales data for 2023," it generates the following SQL query:

[0467] sql

[0468] SELECT FROM sales WHERE year = 2023;

[0469] The generated SQL query becomes the output and the input data for proceeding to the next step. The generated program code and data queries are output as concrete actions.

[0470] Step 5: Execute the code or query

[0471] The server executes the generated program code and data queries. In the case of SQL queries, the server executes the query against the database (e.g., MySQL) and retrieves the data. The generated SQL query is executed and the corresponding data is output from the database. In the case of program code, the server compiles the code and deploys it to the specified location. Compilation and deployment are the concrete actions.

[0472] Step 6: Return the execution results

[0473] The server returns the retrieved data or the execution results of the generated program to the user. In the case of a data query, the retrieved data is returned as an HTTP response in JSON format.

[0474] json

[0475] {

[0476] "data": [

[0477] {"id": 1, "amount": 1000, "date": "2023-01-01"},

[0478] {"id": 2, "amount": 1500, "date": "2023-02-01"},

[0479] ...

[0480] ]

[0481] }

[0482] In the case of program code, the deployment URL is notified to the user. For example, the URL is returned in JSON format, such as "Your app is available at http: / / yourserver.com / todoapp". This result becomes the output, and the user receives it and proceeds with their actions.

[0483] Through the above processing steps, users can efficiently acquire data and develop applications through natural language input, even without specialized knowledge.

[0484] (Application Example 1)

[0485] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0486] Traditional factory inventory management systems have made it difficult to accurately track inventory levels and automate necessary replenishment orders. Furthermore, the systems were complex to use, increasing the risk of errors for workers without specialized knowledge. This led to inventory shortages and excesses, hindering efficient production activities.

[0487] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0488] In this invention, the server includes means for receiving requests from users in natural language; means for analyzing the received requests using natural language processing and generating program code or data queries based on the request content; means for executing the generated data queries against an inventory database within the factory and obtaining the results; means for returning the execution results to the user; and means for compiling the program code and deploying it to a specified location, or executing the data queries against the database and obtaining the results, depending on the type of program code or data query generated. This makes it possible for even workers without specialized knowledge to easily check inventory status and issue efficient replenishment instructions.

[0489] A "user" is someone who uses a system to input requests in natural language and receives the results.

[0490] "Natural language" refers to the language that humans use on a daily basis, and is a form of linguistic expression composed in a natural way, rather than a specific programming language.

[0491] A "request" refers to the content of instructions or inquiries that a user makes to the system.

[0492] "Means of receiving" refers to the processes and functions that allow the system to receive requests from users.

[0493] "Natural language processing" is a technology for analyzing and understanding requests from users in natural language.

[0494] "Analysis" is the process by which a system interprets information in order to understand the content of a received request and perform appropriate processing.

[0495] "Program code" is a part of a computer program that a system automatically generates to meet user requirements.

[0496] A "data query" is a query command sent to a database to retrieve specific information.

[0497] "Generating means" refers to functions or processes that automatically generate program code or data queries based on requests.

[0498] "Means of execution" refers to the processes and functions that actually run the generated program code and data queries.

[0499] A "factory inventory database" is a database system used to manage inventory information within a factory.

[0500] "Execution result" refers to the output obtained after the generated program code or data query has been executed.

[0501] "Means of return" refers to the processes or functions that allow a system to return its execution results to the user.

[0502] JSON format is a lightweight data exchange format for structuring, storing, and transmitting data.

[0503] A "server" is a computer system that receives requests, analyzes them, and performs the necessary processing.

[0504] "Deployment" refers to placing the generated program code in a specified environment and making it operational.

[0505] The system for implementing this invention will streamline inventory management within a factory and automate replenishment instructions based on requests entered by the user in natural language. The system mainly consists of the following hardware and software:

[0506] 1. Hardware:

[0507] User devices: Android smartphones, iOS smartphones

[0508] Server: High-performance computer system

[0509] Factory database: A database system for managing inventory information.

[0510] 2. Software:

[0511] "Smart Inventory Manager" is a dedicated application installed on the user's terminal.

[0512] A framework that runs on a server (e.g., Flask)

[0513] Generative AI models for natural language processing (e.g., OpenAI's GPT-3)

[0514] Database management systems (e.g., MySQL, PostgreSQL)

[0515] Users input requests in natural language through the "Smart Inventory Manager" application on their smartphones. For example, they might enter text such as "Show me the inventory levels for 2023" and press the send button to send the request from their device to the server.

[0516] The server receives user requests in JSON format and analyzes them using natural language processing technology. Generative AI models such as OpenAI's GPT-3 are used to convert the requests into appropriate program code or data queries. These converted data queries are then executed against the factory's inventory database to retrieve the requested information.

[0517] The acquired inventory information is returned to the user by the server. The returned information is displayed on the user's smartphone application, allowing the user to check the inventory status in the factory in real time.

[0518] As a concrete example, if a user enters "Show me the inventory for 2023," the system translates this into an SQL query, "SELECT FROM inventory WHERE year = 2023," and executes it against the database. The results are returned to the device in JSON format, and the user can view the results on their smartphone.

[0519] This process allows even workers without specialized knowledge to check and efficiently manage inventory levels within the factory using simple natural language operations. Furthermore, the system automatically handles situations requiring quick responses, thus improving operational efficiency.

[0520] For the generative AI model and prompt statements, use the following example prompt statements:

[0521] Example prompt: "Show me the inventory levels for 2023."

[0522] This allows the system to automate inventory management within the factory and provide the ability to acquire and display data according to user requirements without requiring specialized knowledge.

[0523] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0524] Step 1:

[0525] A user enters a request in natural language using the smartphone application "Smart Inventory Manager" and presses the submit button. The input is in text format, such as "Show inventory levels for 2023." This request is converted to JSON format and sent to the server. Here, the input is the user's request, and the output is the data obtained by converting the request to JSON format.

[0526] Step 2:

[0527] The server receives user requests in JSON format sent from the terminal. Since the received requests are parsed within the server, the input is request data in JSON format, and the output is natural language text data.

[0528] Step 3:

[0529] The server analyzes received natural language requests using natural language processing. This analysis uses a generative AI model (e.g., OpenAI's GPT-3). The input is natural language text data, and the output is appropriate program code or data query. Specifically, the request "Show inventory levels for 2023" is converted into the data query "SELECT FROM inventory WHERE year = 2023".

[0530] Step 4:

[0531] The server executes the generated data query. Here, the server connects to the factory's inventory database and sends the generated data query. The input is the data query, and the output is the inventory information retrieved from the database.

[0532] Step 5:

[0533] The server converts inventory information retrieved from the database into JSON format and sends it back to the user. The input is inventory information retrieved from the database, and the output is inventory information converted into JSON format.

[0534] Step 6:

[0535] The user terminal receives inventory information in JSON format from the server and displays it within the application. The input is inventory information in JSON format, and the output is inventory information displayed visually to the user. This allows the user to check the inventory status within the factory in real time.

[0536] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0537] This invention combines a system that automatically analyzes user requests in natural language, converts them into program code or data queries, and provides the results with an emotion engine that recognizes user emotions. This system improves the user experience and provides more appropriate feedback and results.

[0538] This system begins with the user entering a request in natural language using a terminal. For example, the user enters a request such as "I want the 2023 sales data" or "I want a new ToDo app" into a text field in a browser or application. Simultaneously with the input, an emotion engine activates, recognizing the user's emotional state by analyzing the user's input along with their facial expressions and tone of voice.

[0539] When a user clicks the submit button, the device encodes the request in JSON format and sends it to the server. The server analyzes the received request and integrates sentiment data from the sentiment engine into the analysis results. This allows the server to prioritize the request for faster results, for example, if the user is in a hurry.

[0540] Once the server has finished its analysis, the generative AI generates program code or data queries based on the request. The generated program code or data queries are executed while also taking into account feedback based on the user's emotions. For example, if the user is feeling anxious, additional confirmation messages or detailed logs can be generated to provide reassurance.

[0541] If a data query is generated, the server executes it against the database and retrieves the results. If program code is generated, the server compiles it and deploys it to the specified location. The server then returns the execution results to the user's terminal. In the case of a data query, the retrieved data is notified to the user; in the case of a program, the URL of the deployment destination is notified.

[0542] For example, if a user emotionally inputs "I have so many tasks today, I need a new to-do app," the emotion engine recognizes the user's stress level. The server determines that an immediate response is needed, quickly generates program code, and provides a deployment URL in the most user-friendly format possible. By notifying the user of the URL in the form of "Your app is available at http: / / yourserver.com / todoapp," the user can access the URL and start using the app immediately.

[0543] This system allows users to receive highly personalized responses quickly through natural language request input and emotion recognition. Furthermore, by utilizing an emotion engine, it enables feedback and adjustments tailored to the user's psychological state, providing an optimal user experience.

[0544] The following describes the processing flow.

[0545] Program processing flow

[0546] Step 1: Obtaining the user's needs and emotions

[0547] 1.1 The user enters the request

[0548] User: Enters requests in natural language into a text field in a browser or application, such as "I want the 2023 sales data" or "I want a new ToDo app."

[0549] 1.2 The emotion engine recognizes the user's emotions

[0550] Server: The emotion engine analyzes the user's facial expressions and voice tone in real time and acquires emotion data.

[0551] 1.3 Sending a request

[0552] User: Click the submit button to send the request to the server.

[0553] Terminal: Encodes the request in JSON format and sends it to the server along with sentiment data.

[0554] Step 2: Server receives and parses the request.

[0555] 2.1 The server receives the request and sentiment data.

[0556] Server: Receives requests and sentiment data sent from the user terminal as HTTP requests and decodes the data in JSON format.

[0557] 2.2 Analysis of requests using generative AI

[0558] Server: Analyzes received request content and sentiment data using natural language processing technology to obtain the user's request content and sentiment state.

[0559] 2.3 Generating Program Code or Data Queries

[0560] Server: The generative AI generates program code or data queries based on the request, and adjusts the generation results considering sentiment data.

[0561] Step 3: Execute the generated code and queries.

[0562] 3.1 Executing the generated query

[0563] Server: If the generated query is a data query (SQL), it executes that query against the database.

[0564] 3.2 Compiling and deploying the generated program code

[0565] Server: If the generated code is a program (such as Java code), it compiles that code and deploys it to the specified location.

[0566] Step 4: Return the execution results

[0567] 4.1 Sending query results to the user's terminal

[0568] Server: Sends the results of the SQL query execution to the user's terminal.

[0569] 4.2 Notification of Program Deployment URL

[0570] Server: Notifies the user of the URL where the program code is deployed.

[0571] Step 5: Displaying results to the user

[0572] 5.1 Receiving the Results

[0573] Terminal: Receives results from the server.

[0574] 5.2 Displaying and responding to results

[0575] Terminal: Displays the received results to the user.

[0576] User: Review the results of the submitted data query and the URL of the generated application, and access or use them as needed.

[0577] Server: Take user sentiment data into consideration and provide additional appropriate feedback.

[0578] As a concrete example, consider a scenario where a user emotionally inputs, "I'm under a lot of stress and finding task management difficult; I need a new to-do app." The emotion engine recognizes the user's stress level, and the server determines that a quick response is needed. The server quickly generates program code and provides a user-friendly URL for deployment, thereby supporting the user in reducing stress. A message like "Your app is available at http: / / yourserver.com / todoapp" is sent, allowing the user to access the URL and start using the app immediately.

[0579] In this way, this system automatically analyzes users' natural language requests, recognizes their emotions, and provides personalized responses, thereby significantly improving the user experience.

[0580] (Example 2)

[0581] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0582] While conventional systems could accurately analyze users' natural language requests and generate appropriate program code and data queries, they failed to adequately improve the user experience because they did not consider the user's emotional state. Furthermore, they struggled to respond flexibly to situations where users required quick responses or detailed explanations and confirmations.

[0583] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting a request from a user in natural language, means for analyzing the input request and recognizing the user's emotions, means for encoding the request content and recognized emotion data into JSON format, means for transmitting the encoded data to the server, means for analyzing the transmitted data using natural language processing technology, means for generating program code or data queries using a generation AI model based on the analysis results, means for compiling the generated program code and deploying it to a specified location, or means for executing the generated data query against a database and obtaining the results, and means for returning the execution results to the user. This enables highly personalized responses that take into account the user's emotional state, and improves the user experience by providing quick and appropriate feedback.

[0584] "Natural language" refers to the language that humans use on a daily basis, and is in the form of text or spoken language, rather than program code or data queries.

[0585] "Emotions" refers to the user's psychological state, such as stress, excitement, or calmness.

[0586] An "emotion engine" is a software or hardware technology that analyzes input data such as a user's facial expressions and tone of voice to recognize the user's emotional state.

[0587] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a text format for describing data structures.

[0588] "Natural language processing technology" refers to technologies that enable the analysis and understanding of natural language, and includes text analysis, morphological analysis, and contextual analysis.

[0589] A "generative AI model" is an artificial intelligence model that generates new text, code, or data queries based on requests. Generative models such as GPT fall into this category.

[0590] "Program code" is a collection of instructions and statements that make up a computer program, and is software in a form that a computer can execute.

[0591] A "data query" refers to a command used to search for or retrieve specific information from a database.

[0592] "Compilation" is the process of converting programming code from a high-level language into machine code.

[0593] "Deployment" refers to the process of placing the generated program code into the actual production environment and making it executable.

[0594] "User experience" refers to the overall satisfaction and convenience that users experience when using a system or application.

[0595] "Feedback" refers to information that a system provides in response to a user, including information provided based on the user's requests and emotional state.

[0596] A "natural language request" is a request or question made by a user using natural language, and includes tasks that the system should process or the retrieval of information.

[0597] This invention combines a system that automatically analyzes user requests in natural language, converts them into program code or data queries, and provides the results with an emotion engine that recognizes user emotions. This system improves the user experience and provides more appropriate feedback and results.

[0598] This system begins with the user entering a request in natural language using a terminal. For example, the user might enter a request in a text field in a browser or application, such as "I want the 2023 sales data" or "I want a new ToDo app." The hardware used is the user's PC or smartphone, and the software used is a browser or application.

[0599] The emotion engine starts operating simultaneously with the input. The emotion engine analyzes the user's input, facial expressions, and tone of voice to recognize the user's emotional state. For example, it can determine whether the user is anxious or calm. This analysis uses hardware such as a camera and microphone, and emotion analysis software is employed.

[0600] When the user clicks the send button, the device encodes the request into JSON format and sends it to the server. At this point, sentiment data is also included in the JSON. The server then analyzes the received request using natural language processing (NLP) techniques (e.g., using Python or Node.js).

[0601] The server integrates the received emotional data into the analysis results. For example, if the emotional state is recognized as "stress," the request is processed as a high-priority task. This ensures that if the user is in a hurry, the server takes action to quickly obtain the results.

[0602] Next, the server uses a generative AI model (e.g., GPT-4) to generate program code or data queries that respond to the request. The generated program code or data queries also take into account the user's emotional state. For example, if the user is anxious, detailed comments or confirmation messages will be included.

[0603] If the generated data query is generated, the server executes it against the database and retrieves the results. If the generated program code is generated, the server compiles it and deploys it to the specified location. The server then returns the execution results to the user's terminal. For example, the URL of the deployment destination is notified to the user. By accessing this URL, the user can immediately use the generated program.

[0604] For example, if a user emotionally inputs "I have so many tasks today, I need a new to-do app," the emotion engine recognizes that the user is stressed. The server determines that a quick response is needed and uses a generative AI model to generate code for a to-do app. The generated application is then deployed to "http: / / yourserver.com / todoapp," and the URL is notified to the user. The user can access this URL and immediately start using the new to-do app.

[0605] Examples of prompts for a generative AI model:

[0606] User input: "I have so many tasks today, I need a new to-do app."

[0607] User's emotion: Stress

[0608] The program code to be generated: Code for a to-do app that makes task management easy.

[0609] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0610] Step 1:

[0611] The user enters the request into the terminal.

[0612] Users open a browser or application on their PC or smartphone and enter requests in natural language into a text field, such as "I want the 2023 sales data" or "I want a new ToDo app." This input is then saved as text data in the device's memory.

[0613] Step 2:

[0614] The device analyzes the user's emotions.

[0615] The device's built-in emotion engine operates, analyzing the user's input, as well as their facial expressions and voice tone in real time using the camera and microphone. The input data is captured by the device as text, audio, and video data, and the emotion engine processes this data to output the user's emotional state.

[0616] Step 3:

[0617] The device sends request and sentiment data to the server.

[0618] When the user clicks the submit button, the device encodes the entered request and analyzed sentiment data into JSON format. This encoded JSON data is then sent to the server via the internet.

[0619] Step 4:

[0620] The server analyzes requests and sentiment data.

[0621] The server deserializes and parses the received JSON data. First, it analyzes the request content using natural language processing (NLP) techniques and converts it into a specific task. Simultaneously, it analyzes sentiment data from the sentiment engine to determine the priority of the request. The analysis results are stored as internal data structures (e.g., objects or dictionaries).

[0622] Step 5:

[0623] The generative AI generates program code or data queries.

[0624] The server generates program code or data queries using a generative AI model (e.g., GPT-4) based on the analyzed request data. The generative AI model receives the request content and sentiment data as input and outputs specific code or queries based on them.

[0625] Step 6:

[0626] The server retrieves the execution results and notifies the user.

[0627] For generated data queries, the server executes them against the database and retrieves the results. For generated program code, the server compiles it and deploys it to the specified location. Finally, the server returns the deployment URL and query results to the user. Specifically, it sends a message to the user's device using a notification function.

[0628] Each of these steps allows users to receive highly personalized responses quickly through natural language request input and emotion recognition.

[0629] (Application Example 2)

[0630] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0631] Conventional natural language processing systems fail to provide appropriate feedback tailored to the user's psychological state, making it difficult to alleviate user stress and anxiety, and consequently hindering the improvement of the user experience. Furthermore, providing feedback and answers that disregard the user's emotions can compromise reliability and usability.

[0632] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving requests from users in natural language, means for analyzing the received requests using natural language processing and generating program code or data queries based on the content of the requests, means for analyzing the user's emotional state at the time of request input, including an emotion engine that recognizes the user's emotions, means for integrating the emotional data from the emotion engine into the request analysis and providing feedback according to the user's psychological state, means for executing the generated program code or data queries, and means for returning the execution results to the user. This makes it possible to provide optimal feedback and results according to the user's psychological state.

[0633] "Means for receiving requests from users in natural language" refers to a system for obtaining requests for voice or text input by users in natural language.

[0634] "Means of analysis using natural language processing" refers to a system that analyzes received natural language requests using machine learning and statistical methods to understand their content.

[0635] "Means for generating program code or data queries" refers to a mechanism for automatically generating the necessary program code or queries for database operations based on the analyzed requirements.

[0636] An "emotion engine that recognizes user emotions" is software or hardware that analyzes a user's tone of voice and facial expressions to recognize their emotional state in real time.

[0637] "Means for analyzing emotional states" refers to a system that uses data acquired by the emotion engine to determine what emotional state a user is in.

[0638] "Means for integrating emotional data into request analysis" refers to a system that incorporates recognized emotional data into the results of natural language processing to provide feedback tailored to the user's psychological state.

[0639] "Means of providing feedback tailored to the user's psychological state" refers to a system that aims to create a better user experience by providing appropriate additional messages or confirmations based on the user's emotional state.

[0640] "Means for executing generated program code or data queries" refers to a mechanism for compiling generated program code and placing it in a specified location, or for executing data queries against a database and retrieving results.

[0641] "Means of returning execution results to the user" refers to a mechanism for notifying or displaying the results of executed program code or data queries to the user.

[0642] The system for carrying out this invention consists of the following components:

[0643] 1. User terminal:

[0644] Smartphones are primarily used as the terminal for users to input requests in natural language. Voice input is handled using the Google Cloud Speech-to-Text API, while text input is done via standard keyboard input.

[0645] 2. Emotional Engine:

[0646] The system uses the camera and microphone on the smartphone to collect the user's voice tone and facial expressions. This data is then analyzed using Google ML Kit or Amazon Rekognition to recognize the user's emotional state.

[0647] 3. Natural Language Processing Systems:

[0648] The natural language requests entered by the user are parsed using the Google Cloud Natural Language API. As a result, the content of the request is extracted and treated as structured data.

[0649] 4. Server:

[0650] a. Requirements analysis:

[0651] The received request data and sentiment data are encoded into JSON format and sent to the server. The server then receives this data, integrates the results of natural language processing and sentiment analysis, and processes them.

[0652] b. Program code and data query generation:

[0653] Based on the analysis results, a generative AI model is used to generate program code or data queries that respond to user requests. This provides optimal feedback and results that are tailored to the user's psychological state.

[0654] c. Execution of the results:

[0655] The generated program code is compiled and deployed to the specified location, or data queries are executed against the database. The results of this execution are then returned to the user's terminal.

[0656] 5. Content Management:

[0657] The server manages the execution results and records them in a database so that it can quickly respond when the user makes another inquiry.

[0658] Specific example

[0659] 1. Example of natural language input:

[0660] The user voice-inputs "Send 5,000 yen to Tanaka" into their smartphone.

[0661] When the emotion engine detects anxiety, the system displays an additional confirmation message. For example, "Are you sure you want to send 5,000 yen to Tanaka? Please confirm."

[0662] 2. Example of a prompt:

[0663] "The user wants to send money to a friend. Their emotion has been identified as anxiety. Please generate an appropriate system response."

[0664] This system provides optimal feedback and results tailored to the user's psychological state, thereby improving the user experience and increasing trust.

[0665] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0666] Step 1:

[0667] User request input

[0668] Users enter their requests using their smartphones via voice or text. For voice input, the Google Cloud Speech-to-Text API converts the speech to text. If the input data is voice, the voice data is used and output as text. If the input data is text, it is used as is.

[0669] Specific actions:

[0670] The user voice-inputs, "Send 5,000 yen to Tanaka."

[0671] The Google Cloud Speech-to-Text API receives the audio data and outputs it as text data: "Send 5,000 yen to Tanaka."

[0672] Step 2:

[0673] Recognition of user emotions

[0674] The device's camera and microphone are used to capture the user's facial expressions and voice tone. Google ML Kit or Amazon Rekognition analyzes this data to recognize the user's emotional state. Input data consists of video and audio data, while output data consists of tags representing the emotional state (e.g., "anxious," "relieved").

[0675] Specific actions:

[0676] The camera captures the user's facial expressions, and the microphone collects their voice tone.

[0677] The ML kit outputs the tag "anxiety".

[0678] Step 3:

[0679] Requirements analysis using natural language processing

[0680] The device sends text data obtained from voice input to the Google Cloud Natural Language API for parsing of the request. The input data is text data, and the output data is structured request content (e.g., "Recipient: Tanaka", "Amount: 5000 yen").

[0681] Specific actions:

[0682] The text data "Send 5000 yen to Tanaka" is sent to the Natural Language API.

[0683] The API outputs structured data such as "Recipient: Tanaka" and "Amount: 5000 yen".

[0684] Step 4:

[0685] Integration of emotion data and request data

[0686] The server receives request data and sentiment data sent from the user terminal in JSON format, automatically parses and integrates them. The input data consists of structured request content and sentiment status tags, and the output data is the integrated JSON data.

[0687] Specific actions:

[0688] Send the following information to the server in JSON format: "Recipient: Tanaka", "Amount: 5000 yen", "Emotion: Anxious".

[0689] The server analyzes and integrates the data.

[0690] Step 5:

[0691] Program code and data query generation

[0692] The server uses a generative AI model to generate program code or data queries based on integrated JSON data. The input data is integrated data in JSON format, and the output data is the generated program code or data query.

[0693] Specific actions:

[0694] The server generates integrated data with the following information: "Recipient: Tanaka", "Amount: 5000 yen", and "Emotion: Anxiety". This data is then input into the AI ​​model.

[0695] The generating AI model outputs a remittance processing code or data query.

[0696] Step 6:

[0697] Execution and return of results

[0698] The server compiles the generated program code and deploys it to the specified location, or executes data queries against the database and retrieves the results. The resulting data is retrieved in JSON format and returned to the user's terminal. The input data is the generated program code or data query, and the output data is the execution result.

[0699] Specific actions:

[0700] The server compiles and executes the money transfer processing code.

[0701] The execution result returns a "Transfer successful" message in JSON format to the user's terminal.

[0702] Step 7:

[0703] User Feedback

[0704] The server returns the execution results to the user's terminal and displays additional confirmation messages corresponding to the user's emotional state. The input data is the JSON data of the execution results, and the output data is the messages displayed on the user interface.

[0705] Specific actions:

[0706] The user receives a "transfer successful" message along with an additional confirmation message stating, "We understand you were feeling anxious, but the transfer has been completed successfully."

[0707] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0708] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0709] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0710] [Third Embodiment]

[0711] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0712] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0713] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0714] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0715] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0716] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0717] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0718] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0719] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0720] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0721] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0722] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0723] This invention relates to a system that automatically generates program code and data queries based on requests entered by the user in natural language, and provides the results to the user. This system can be implemented in the following way.

[0724] This system begins with the user entering a request in natural language through their device. For example, they might enter a request such as "I want the sales data for 2023" into a text field in their browser or a dedicated application. When the user clicks the submit button, the request is converted to JSON format and sent to the server.

[0725] The server receives requests from users and parses their content. The requests are understood using natural language processing techniques and translated into specific program code or data queries. This generates precise instructions tailored to the specific request.

[0726] The generated program code and data queries are executed on the server. If the generated item is a data query, the server executes the query against the database and retrieves the corresponding data. Conversely, if the generated item is program code, this code is compiled by the server and deployed to the specified location.

[0727] The server returns the execution results to the user's terminal. In the case of a data query, the data obtained as a result of the query is returned; in the case of program code, the URL of the deployment destination is notified to the user. The user can then use this to retrieve the necessary data or access the generated application.

[0728] To give a concrete example, if a user requests a "simple to-do app," the server analyzes this request and generates the necessary program code. The generated code is compiled on the server and deployed to the specified server. The user then receives a URL to the deployment location, such as "Your app is available at http: / / yourserver.com / todoapp," and can access and use the application from there.

[0729] This system allows users to easily acquire data and develop applications based on their own requirements without needing specific expertise. This improves operational efficiency and enables faster response times. Furthermore, because it can generate programs and data for various purposes with just a single instruction in natural language, it is usable by a wide range of users.

[0730] The following describes the processing flow.

[0731] Program processing flow

[0732] Step 1: Obtaining requests from users

[0733] 1.1 The user enters the request

[0734] User: The user enters a request in natural language, such as "I want the sales data for 2023," into a text field in their device's browser or application.

[0735] 1.2 User submits request

[0736] User: Click the submit button to send the request to the server.

[0737] Terminal: Encodes the request into an appropriate format such as JSON and sends it to the server.

[0738] Step 2: Server receives and parses the request.

[0739] 2.1 The server receives the request

[0740] Server: Receives requests sent from user terminals as HTTP requests and decodes the data in JSON format.

[0741] 2.2 Analysis of requests using generative AI

[0742] Server: The server analyzes the received request using natural language processing technology to obtain the user's request.

[0743] Server: Based on the analysis results, the generative AI generates appropriate program code or data queries.

[0744] Step 3: Execute the generated code and queries.

[0745] 3.1 Executing the generated query

[0746] Server: If the generated query is a data query (SQL), it executes that query against the database.

[0747] 3.2 Execution of the generated program code

[0748] Server: If the generated code is a program (such as Java code), it compiles that code and deploys it to the specified location.

[0749] Step 4: Return the execution results

[0750] 4.1 Sending query results to the user's terminal

[0751] Server: Sends the results of the SQL query execution to the user's terminal.

[0752] 4.2 Notification of Program Deployment URL

[0753] Server: Notifies the user of the URL where the program code is deployed.

[0754] Step 5: Displaying results to the user

[0755] 5.1 Receiving the Results

[0756] Terminal: Receives results from the server.

[0757] 5.2 Displaying Results

[0758] Terminal: Displays the received results to the user.

[0759] User: Review the results of the submitted data query and the URL of the generated application, and access or use them as needed.

[0760] Through the steps described above, the present invention can automatically analyze natural language requests from users, generate appropriate program code and data queries, and execute them. This allows users to quickly and efficiently obtain the necessary information and programs without requiring any special knowledge.

[0761] (Example 1)

[0762] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0763] In conventional systems, generating program code and data queries required specialized knowledge, and the lack of automation made efficient data acquisition and application development difficult. This resulted in users having to expend a significant amount of time and effort, which was a major problem.

[0764] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0765] In this invention, the server includes means for receiving requests from users in natural language, means for analyzing the received requests using natural language processing and generating program code or data queries based on the request content, means for executing the generated program code or data queries, means for returning the execution results to the user, means for converting the received requests into JSON format, and means for using a generative AI model for analysis. This enables users to efficiently acquire data and create applications through natural language input without requiring specialized knowledge.

[0766] "Natural language" generally refers to the language that humans use on a daily basis, and is not a specific program or machine language.

[0767] A "request" is when a user asks for specific information or an action using natural language.

[0768] "Means of receiving" refers to methods or devices for obtaining requests from users and inputting them into the system.

[0769] "Natural language processing" is the technology that enables computers to understand and analyze human natural language.

[0770] "Program code" is code that contains instructions for performing a specific task.

[0771] A "data query" is a command used to retrieve specific data from a database.

[0772] "Generating means" refers to methods or devices that automatically generate program code or data queries based on the results of analysis.

[0773] "Means of execution" refers to the methods or devices used to run the generated program code or data queries on a computer.

[0774] "Means of return" refers to methods or devices for communicating the execution results to the user.

[0775] "JSON format" is a simple data format used for data exchange, and it is an abbreviation for JavaScript Object Notation.

[0776] A "generative AI model" is an artificial intelligence model that understands human language and commands and produces appropriate responses or generated text.

[0777] "Compilation" is the process of converting program code into a format that a machine can understand.

[0778] "Deployment" refers to placing a created program on a specific server or environment and making it available for use.

[0779] "HTTP format" refers to the protocol format used for sending and receiving information over the internet.

[0780] This invention is a system that automatically generates program code and data queries based on requests entered by the user in natural language, and provides the results to the user. The specific form of implementing this system is described below.

[0781] Users enter their requests using a device (e.g., a PC or smartphone). These devices are equipped with a browser (e.g., Google Chrome or Mozilla Firefox) or a dedicated application, allowing users to enter requests in natural language. For example, a user might enter "I want the sales data for 2023" and click the submit button.

[0782] The terminal converts the input request into JSON format and sends it to the server as an HTTP request. The server receives this HTTP request and parses the JSON request body. Specifically, it analyzes the received data using natural language processing (NLP) techniques to understand the request. For this analysis, it uses, for example, generative AI models such as OpenAI's GPT-4 or the Google Cloud Natural Language API.

[0783] The server generates appropriate program code or data queries based on the analysis results. For example, to retrieve sales data for a specific year, it generates an appropriate SQL query. The generated SQL query is executed against a database (e.g., MySQL or PostgreSQL) to retrieve the corresponding data. If program code is generated, it is compiled on the server and deployed to the specified server environment.

[0784] The results of data queries and the execution results of generated program code are returned from the server to the user's terminal. In the case of data queries, the retrieved data is returned as an HTTP response. In the case of program code, the user is notified of the deployment URL (for example, "http: / / yourserver.com / todoapp"). The user can use the generated application by accessing this URL.

[0785] For example, if a user requests a "simple to-do app," the server parses this request and generates appropriate JavaScript code. This code is compiled on the server and deployed to the specified server. As a result, the user receives a deployment URL such as "Your app is available at http: / / yourserver.com / todoapp," and can access and use the application.

[0786] Examples of prompt statements include the following:

[0787] "Please generate an SQL query to retrieve sales data for 2023."

[0788] "Please create JavaScript code to generate a simple to-do app."

[0789] "Create a Python script to retrieve order history based on a specific customer ID."

[0790] This system allows users to efficiently acquire data and develop applications through natural language input without requiring specialized knowledge.

[0791] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0792] Step 1: User enters request

[0793] The user enters their request in natural language into a text field in a browser or dedicated application using their device. For example, they might type "I want the sales data for 2023" and click the submit button. This request becomes the user's input. As output, this request becomes the input data needed to proceed to the next step.

[0794] Step 2: Convert and send the request

[0795] After receiving a user request, the terminal converts it into JSON format. For example, it is converted into a JSON object like the following:

[0796] json

[0797] {

[0798] "request": "I would like sales data for 2023."

[0799] }

[0800] This JSON data is output and sent to the server as an HTTP request. The body of the HTTP request becomes the input data.

[0801] Step 3: Receiving and parsing the request

[0802] The server receives HTTP requests sent from terminals. It parses the received JSON data and understands the request using natural language processing techniques. Specifically, it uses a generative AI model (e.g., OpenAI GPT-4) to analyze the request and extract its content. This analysis result becomes the output for data processing and the input data for the next step.

[0803] Step 4: Generate program code or data queries

[0804] The server generates program code and data queries based on the analysis results. For example, from a request such as "I want sales data for 2023," it generates the following SQL query:

[0805] sql

[0806] SELECT FROM sales WHERE year = 2023;

[0807] The generated SQL query becomes the output and the input data for proceeding to the next step. The generated program code and data queries are output as concrete actions.

[0808] Step 5: Execute the code or query

[0809] The server executes the generated program code and data queries. In the case of SQL queries, the server executes the query against the database (e.g., MySQL) and retrieves the data. The generated SQL query is executed and the corresponding data is output from the database. In the case of program code, the server compiles the code and deploys it to the specified location. Compilation and deployment are the concrete actions.

[0810] Step 6: Return the execution results

[0811] The server returns the retrieved data or the execution results of the generated program to the user. In the case of a data query, the retrieved data is returned as an HTTP response in JSON format.

[0812] json

[0813] {

[0814] "data": [

[0815] {"id": 1, "amount": 1000, "date": "2023-01-01"},

[0816] {"id": 2, "amount": 1500, "date": "2023-02-01"},

[0817] ...

[0818] ]

[0819] }

[0820] In the case of program code, the deployment URL is notified to the user. For example, the URL is returned in JSON format, such as "Your app is available at http: / / yourserver.com / todoapp". This result becomes the output, and the user receives it and proceeds with their actions.

[0821] Through the above processing steps, users can efficiently acquire data and develop applications through natural language input, even without specialized knowledge.

[0822] (Application Example 1)

[0823] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0824] Traditional factory inventory management systems have made it difficult to accurately track inventory levels and automate necessary replenishment orders. Furthermore, the systems were complex to use, increasing the risk of errors for workers without specialized knowledge. This led to inventory shortages and excesses, hindering efficient production activities.

[0825] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0826] In this invention, the server includes means for receiving requests from users in natural language; means for analyzing the received requests using natural language processing and generating program code or data queries based on the request content; means for executing the generated data queries against an inventory database within the factory and obtaining the results; means for returning the execution results to the user; and means for compiling the program code and deploying it to a specified location, or executing the data queries against the database and obtaining the results, depending on the type of program code or data query generated. This makes it possible for even workers without specialized knowledge to easily check inventory status and issue efficient replenishment instructions.

[0827] A "user" is someone who uses a system to input requests in natural language and receives the results.

[0828] "Natural language" refers to the language that humans use on a daily basis, and is a form of linguistic expression composed in a natural way, rather than a specific programming language.

[0829] A "request" refers to the content of instructions or inquiries that a user makes to the system.

[0830] "Means of receiving" refers to the processes and functions that allow the system to receive requests from users.

[0831] "Natural language processing" is a technology for analyzing and understanding requests from users in natural language.

[0832] "Analysis" is the process by which a system interprets information in order to understand the content of a received request and perform appropriate processing.

[0833] "Program code" is a part of a computer program that a system automatically generates to meet user requirements.

[0834] A "data query" is a query command sent to a database to retrieve specific information.

[0835] "Generating means" refers to functions or processes that automatically generate program code or data queries based on requests.

[0836] "Means of execution" refers to the processes and functions that actually run the generated program code and data queries.

[0837] A "factory inventory database" is a database system used to manage inventory information within a factory.

[0838] "Execution result" refers to the output obtained after the generated program code or data query has been executed.

[0839] "Means of return" refers to the processes or functions that allow a system to return its execution results to the user.

[0840] JSON format is a lightweight data exchange format for structuring, storing, and transmitting data.

[0841] A "server" is a computer system that receives requests, analyzes them, and performs the necessary processing.

[0842] "Deployment" refers to placing the generated program code in a specified environment and making it operational.

[0843] The system for implementing this invention will streamline inventory management within a factory and automate replenishment instructions based on requests entered by the user in natural language. The system mainly consists of the following hardware and software:

[0844] 1. Hardware:

[0845] User devices: Android smartphones, iOS smartphones

[0846] Server: High-performance computer system

[0847] Factory database: A database system for managing inventory information.

[0848] 2. Software:

[0849] "Smart Inventory Manager" is a dedicated application installed on the user's terminal.

[0850] A framework that runs on a server (e.g., Flask)

[0851] Generative AI models for natural language processing (e.g., OpenAI's GPT-3)

[0852] Database management systems (e.g., MySQL, PostgreSQL)

[0853] Users input requests in natural language through the "Smart Inventory Manager" application on their smartphones. For example, they might enter text such as "Show me the inventory levels for 2023" and press the send button to send the request from their device to the server.

[0854] The server receives user requests in JSON format and analyzes them using natural language processing technology. Generative AI models such as OpenAI's GPT-3 are used to convert the requests into appropriate program code or data queries. These converted data queries are then executed against the factory's inventory database to retrieve the requested information.

[0855] The acquired inventory information is returned to the user by the server. The returned information is displayed on the user's smartphone application, allowing the user to check the inventory status in the factory in real time.

[0856] As a concrete example, if a user enters "Show me the inventory for 2023," the system translates this into an SQL query, "SELECT FROM inventory WHERE year = 2023," and executes it against the database. The results are returned to the device in JSON format, and the user can view the results on their smartphone.

[0857] This process allows even workers without specialized knowledge to check and efficiently manage inventory levels within the factory using simple natural language operations. Furthermore, the system automatically handles situations requiring quick responses, thus improving operational efficiency.

[0858] For the generative AI model and prompt statements, use the following example prompt statements:

[0859] Example prompt: "Show me the inventory levels for 2023."

[0860] This allows the system to automate inventory management within the factory and provide the ability to acquire and display data according to user requirements without requiring specialized knowledge.

[0861] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0862] Step 1:

[0863] A user enters a request in natural language using the smartphone application "Smart Inventory Manager" and presses the submit button. The input is in text format, such as "Show inventory levels for 2023." This request is converted to JSON format and sent to the server. Here, the input is the user's request, and the output is the data obtained by converting the request to JSON format.

[0864] Step 2:

[0865] The server receives user requests in JSON format sent from the terminal. Since the received requests are parsed within the server, the input is request data in JSON format, and the output is natural language text data.

[0866] Step 3:

[0867] The server analyzes received natural language requests using natural language processing. This analysis uses a generative AI model (e.g., OpenAI's GPT-3). The input is natural language text data, and the output is appropriate program code or data query. Specifically, the request "Show inventory levels for 2023" is converted into the data query "SELECT FROM inventory WHERE year = 2023".

[0868] Step 4:

[0869] The server executes the generated data query. Here, the server connects to the factory's inventory database and sends the generated data query. The input is the data query, and the output is the inventory information retrieved from the database.

[0870] Step 5:

[0871] The server converts inventory information retrieved from the database into JSON format and sends it back to the user. The input is inventory information retrieved from the database, and the output is inventory information converted into JSON format.

[0872] Step 6:

[0873] The user terminal receives inventory information in JSON format from the server and displays it within the application. The input is inventory information in JSON format, and the output is inventory information displayed visually to the user. This allows the user to check the inventory status within the factory in real time.

[0874] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0875] This invention combines a system that automatically analyzes user requests in natural language, converts them into program code or data queries, and provides the results with an emotion engine that recognizes user emotions. This system improves the user experience and provides more appropriate feedback and results.

[0876] This system begins with the user entering a request in natural language using a terminal. For example, the user enters a request such as "I want the 2023 sales data" or "I want a new ToDo app" into a text field in a browser or application. Simultaneously with the input, an emotion engine activates, recognizing the user's emotional state by analyzing the user's input along with their facial expressions and tone of voice.

[0877] When a user clicks the submit button, the device encodes the request in JSON format and sends it to the server. The server analyzes the received request and integrates sentiment data from the sentiment engine into the analysis results. This allows the server to prioritize the request for faster results, for example, if the user is in a hurry.

[0878] Once the server has finished its analysis, the generative AI generates program code or data queries based on the request. The generated program code or data queries are executed while also taking into account feedback based on the user's emotions. For example, if the user is feeling anxious, additional confirmation messages or detailed logs can be generated to provide reassurance.

[0879] If a data query is generated, the server executes it against the database and retrieves the results. If program code is generated, the server compiles it and deploys it to the specified location. The server then returns the execution results to the user's terminal. In the case of a data query, the retrieved data is notified to the user; in the case of a program, the URL of the deployment destination is notified.

[0880] For example, if a user emotionally inputs "I have so many tasks today, I need a new to-do app," the emotion engine recognizes the user's stress level. The server determines that an immediate response is needed, quickly generates program code, and provides a deployment URL in the most user-friendly format possible. By notifying the user of the URL in the form of "Your app is available at http: / / yourserver.com / todoapp," the user can access the URL and start using the app immediately.

[0881] This system allows users to receive highly personalized responses quickly through natural language request input and emotion recognition. Furthermore, by utilizing an emotion engine, it enables feedback and adjustments tailored to the user's psychological state, providing an optimal user experience.

[0882] The following describes the processing flow.

[0883] Program processing flow

[0884] Step 1: Obtaining the user's needs and emotions

[0885] 1.1 The user enters the request

[0886] User: Enters requests in natural language into a text field in a browser or application, such as "I want the 2023 sales data" or "I want a new ToDo app."

[0887] 1.2 The emotion engine recognizes the user's emotions

[0888] Server: The emotion engine analyzes the user's facial expressions and voice tone in real time and acquires emotion data.

[0889] 1.3 Sending a request

[0890] User: Click the submit button to send the request to the server.

[0891] Terminal: Encodes the request in JSON format and sends it to the server along with sentiment data.

[0892] Step 2: Server receives and parses the request.

[0893] 2.1 The server receives the request and sentiment data.

[0894] Server: Receives requests and sentiment data sent from the user terminal as HTTP requests and decodes the data in JSON format.

[0895] 2.2 Analysis of requests using generative AI

[0896] Server: Analyzes received request content and sentiment data using natural language processing technology to obtain the user's request content and sentiment state.

[0897] 2.3 Generating Program Code or Data Queries

[0898] Server: The generative AI generates program code or data queries based on the request, and adjusts the generation results considering sentiment data.

[0899] Step 3: Execute the generated code and queries.

[0900] 3.1 Executing the generated query

[0901] Server: If the generated query is a data query (SQL), it executes that query against the database.

[0902] 3.2 Compiling and deploying the generated program code

[0903] Server: If the generated code is a program (such as Java code), it compiles that code and deploys it to the specified location.

[0904] Step 4: Return the execution results

[0905] 4.1 Sending query results to the user's terminal

[0906] Server: Sends the results of the SQL query execution to the user's terminal.

[0907] 4.2 Notification of Program Deployment URL

[0908] Server: Notifies the user of the URL where the program code is deployed.

[0909] Step 5: Displaying results to the user

[0910] 5.1 Receiving the Results

[0911] Terminal: Receives results from the server.

[0912] 5.2 Displaying and responding to results

[0913] Terminal: Displays the received results to the user.

[0914] User: Review the results of the submitted data query and the URL of the generated application, and access or use them as needed.

[0915] Server: Take user sentiment data into consideration and provide additional appropriate feedback.

[0916] As a concrete example, consider a scenario where a user emotionally inputs, "I'm under a lot of stress and finding task management difficult; I need a new to-do app." The emotion engine recognizes the user's stress level, and the server determines that a quick response is needed. The server quickly generates program code and provides a user-friendly URL for deployment, thereby supporting the user in reducing stress. A message like "Your app is available at http: / / yourserver.com / todoapp" is sent, allowing the user to access the URL and start using the app immediately.

[0917] In this way, this system automatically analyzes users' natural language requests, recognizes their emotions, and provides personalized responses, thereby significantly improving the user experience.

[0918] (Example 2)

[0919] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0920] While conventional systems could accurately analyze users' natural language requests and generate appropriate program code and data queries, they failed to adequately improve the user experience because they did not consider the user's emotional state. Furthermore, they struggled to respond flexibly to situations where users required quick responses or detailed explanations and confirmations.

[0921] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting a request from a user in natural language, means for analyzing the input request and recognizing the user's emotions, means for encoding the request content and recognized emotion data into JSON format, means for transmitting the encoded data to the server, means for analyzing the transmitted data using natural language processing technology, means for generating program code or data queries using a generation AI model based on the analysis results, means for compiling the generated program code and deploying it to a specified location, or means for executing the generated data query against a database and obtaining the results, and means for returning the execution results to the user. This enables highly personalized responses that take into account the user's emotional state, and improves the user experience by providing quick and appropriate feedback.

[0922] "Natural language" refers to the language that humans use on a daily basis, and is in the form of text or spoken language, rather than program code or data queries.

[0923] "Emotions" refers to the user's psychological state, such as stress, excitement, or calmness.

[0924] An "emotion engine" is a software or hardware technology that analyzes input data such as a user's facial expressions and tone of voice to recognize the user's emotional state.

[0925] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a text format for describing data structures.

[0926] "Natural language processing technology" refers to technologies that enable the analysis and understanding of natural language, and includes text analysis, morphological analysis, and contextual analysis.

[0927] A "generative AI model" is an artificial intelligence model that generates new text, code, or data queries based on requests. Generative models such as GPT fall into this category.

[0928] "Program code" is a collection of instructions and statements that make up a computer program, and is software in a form that a computer can execute.

[0929] A "data query" refers to a command used to search for or retrieve specific information from a database.

[0930] "Compilation" is the process of converting programming code from a high-level language into machine code.

[0931] "Deployment" refers to the process of placing the generated program code into the actual production environment and making it executable.

[0932] "User experience" refers to the overall satisfaction and convenience that users experience when using a system or application.

[0933] "Feedback" refers to information that a system provides in response to a user, including information provided based on the user's requests and emotional state.

[0934] A "natural language request" is a request or question made by a user using natural language, and includes tasks that the system should process or the retrieval of information.

[0935] This invention combines a system that automatically analyzes user requests in natural language, converts them into program code or data queries, and provides the results with an emotion engine that recognizes user emotions. This system improves the user experience and provides more appropriate feedback and results.

[0936] This system begins with the user entering a request in natural language using a terminal. For example, the user might enter a request in a text field in a browser or application, such as "I want the 2023 sales data" or "I want a new ToDo app." The hardware used is the user's PC or smartphone, and the software used is a browser or application.

[0937] The emotion engine starts operating simultaneously with the input. The emotion engine analyzes the user's input, facial expressions, and tone of voice to recognize the user's emotional state. For example, it can determine whether the user is anxious or calm. This analysis uses hardware such as a camera and microphone, and emotion analysis software is employed.

[0938] When the user clicks the send button, the device encodes the request into JSON format and sends it to the server. At this point, sentiment data is also included in the JSON. The server then analyzes the received request using natural language processing (NLP) techniques (e.g., using Python or Node.js).

[0939] The server integrates the received emotional data into the analysis results. For example, if the emotional state is recognized as "stress," the request is processed as a high-priority task. This ensures that if the user is in a hurry, the server takes action to quickly obtain the results.

[0940] Next, the server uses a generative AI model (e.g., GPT-4) to generate program code or data queries that respond to the request. The generated program code or data queries also take into account the user's emotional state. For example, if the user is anxious, detailed comments or confirmation messages will be included.

[0941] If the generated data query is generated, the server executes it against the database and retrieves the results. If the generated program code is generated, the server compiles it and deploys it to the specified location. The server then returns the execution results to the user's terminal. For example, the URL of the deployment destination is notified to the user. By accessing this URL, the user can immediately use the generated program.

[0942] For example, if a user emotionally inputs "I have so many tasks today, I need a new to-do app," the emotion engine recognizes that the user is stressed. The server determines that a quick response is needed and uses a generative AI model to generate code for a to-do app. The generated application is then deployed to "http: / / yourserver.com / todoapp," and the URL is notified to the user. The user can access this URL and immediately start using the new to-do app.

[0943] Examples of prompts for a generative AI model:

[0944] User input: "I have so many tasks today, I need a new to-do app."

[0945] User's emotion: Stress

[0946] The program code to be generated: Code for a to-do app that makes task management easy.

[0947] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0948] Step 1:

[0949] The user enters the request into the terminal.

[0950] Users open a browser or application on their PC or smartphone and enter requests in natural language into a text field, such as "I want the 2023 sales data" or "I want a new ToDo app." This input is then saved as text data in the device's memory.

[0951] Step 2:

[0952] The device analyzes the user's emotions.

[0953] The device's built-in emotion engine operates, analyzing the user's input, as well as their facial expressions and voice tone in real time using the camera and microphone. The input data is captured by the device as text, audio, and video data, and the emotion engine processes this data to output the user's emotional state.

[0954] Step 3:

[0955] The device sends request and sentiment data to the server.

[0956] When the user clicks the submit button, the device encodes the entered request and analyzed sentiment data into JSON format. This encoded JSON data is then sent to the server via the internet.

[0957] Step 4:

[0958] The server analyzes requests and sentiment data.

[0959] The server deserializes and parses the received JSON data. First, it analyzes the request content using natural language processing (NLP) techniques and converts it into a specific task. Simultaneously, it analyzes sentiment data from the sentiment engine to determine the priority of the request. The analysis results are stored as internal data structures (e.g., objects or dictionaries).

[0960] Step 5:

[0961] The generative AI generates program code or data queries.

[0962] The server generates program code or data queries using a generative AI model (e.g., GPT-4) based on the analyzed request data. The generative AI model receives the request content and sentiment data as input and outputs specific code or queries based on them.

[0963] Step 6:

[0964] The server retrieves the execution results and notifies the user.

[0965] For generated data queries, the server executes them against the database and retrieves the results. For generated program code, the server compiles it and deploys it to the specified location. Finally, the server returns the deployment URL and query results to the user. Specifically, it sends a message to the user's device using a notification function.

[0966] Each of these steps allows users to receive highly personalized responses quickly through natural language request input and emotion recognition.

[0967] (Application Example 2)

[0968] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0969] Conventional natural language processing systems fail to provide appropriate feedback tailored to the user's psychological state, making it difficult to alleviate user stress and anxiety, and consequently hindering the improvement of the user experience. Furthermore, providing feedback and answers that disregard the user's emotions can compromise reliability and usability.

[0970] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving requests from users in natural language, means for analyzing the received requests using natural language processing and generating program code or data queries based on the content of the requests, means for analyzing the user's emotional state at the time of request input, including an emotion engine that recognizes the user's emotions, means for integrating the emotional data from the emotion engine into the request analysis and providing feedback according to the user's psychological state, means for executing the generated program code or data queries, and means for returning the execution results to the user. This makes it possible to provide optimal feedback and results according to the user's psychological state.

[0971] "Means for receiving requests from users in natural language" refers to a system for obtaining requests for voice or text input by users in natural language.

[0972] "Means of analysis using natural language processing" refers to a system that analyzes received natural language requests using machine learning and statistical methods to understand their content.

[0973] "Means for generating program code or data queries" refers to a mechanism for automatically generating the necessary program code or queries for database operations based on the analyzed requirements.

[0974] An "emotion engine that recognizes user emotions" is software or hardware that analyzes a user's tone of voice and facial expressions to recognize their emotional state in real time.

[0975] "Means for analyzing emotional states" refers to a system that uses data acquired by the emotion engine to determine what emotional state a user is in.

[0976] "Means for integrating emotional data into request analysis" refers to a system that incorporates recognized emotional data into the results of natural language processing to provide feedback tailored to the user's psychological state.

[0977] "Means of providing feedback tailored to the user's psychological state" refers to a system that aims to create a better user experience by providing appropriate additional messages or confirmations based on the user's emotional state.

[0978] "Means for executing generated program code or data queries" refers to a mechanism for compiling generated program code and placing it in a specified location, or for executing data queries against a database and retrieving results.

[0979] "Means of returning execution results to the user" refers to a mechanism for notifying or displaying the results of executed program code or data queries to the user.

[0980] The system for carrying out this invention consists of the following components:

[0981] 1. User terminal:

[0982] Smartphones are primarily used as the terminal for users to input requests in natural language. Voice input is handled using the Google Cloud Speech-to-Text API, while text input is done via standard keyboard input.

[0983] 2. Emotional Engine:

[0984] The system uses the camera and microphone on the smartphone to collect the user's voice tone and facial expressions. This data is then analyzed using Google ML Kit or Amazon Rekognition to recognize the user's emotional state.

[0985] 3. Natural Language Processing Systems:

[0986] The natural language requests entered by the user are parsed using the Google Cloud Natural Language API. As a result, the content of the request is extracted and treated as structured data.

[0987] 4. Server:

[0988] a. Requirements analysis:

[0989] The received request data and sentiment data are encoded into JSON format and sent to the server. The server then receives this data, integrates the results of natural language processing and sentiment analysis, and processes them.

[0990] b. Program code and data query generation:

[0991] Based on the analysis results, a generative AI model is used to generate program code or data queries that respond to user requests. This provides optimal feedback and results that are tailored to the user's psychological state.

[0992] c. Execution of the results:

[0993] The generated program code is compiled and deployed to the specified location, or data queries are executed against the database. The results of this execution are then returned to the user's terminal.

[0994] 5. Content Management:

[0995] The server manages the execution results and records them in a database so that it can quickly respond when the user makes another inquiry.

[0996] Specific example

[0997] 1. Example of natural language input:

[0998] The user voice-inputs "Send 5,000 yen to Tanaka" into their smartphone.

[0999] When the emotion engine detects anxiety, the system displays an additional confirmation message. For example, "Are you sure you want to send 5,000 yen to Tanaka? Please confirm."

[1000] 2. Example of a prompt:

[1001] "The user wants to send money to a friend. Their emotion has been identified as anxiety. Please generate an appropriate system response."

[1002] This system provides optimal feedback and results tailored to the user's psychological state, thereby improving the user experience and increasing trust.

[1003] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1004] Step 1:

[1005] User request input

[1006] Users enter their requests using their smartphones via voice or text. For voice input, the Google Cloud Speech-to-Text API converts the speech to text. If the input data is voice, the voice data is used and output as text. If the input data is text, it is used as is.

[1007] Specific actions:

[1008] The user voice-inputs, "Send 5,000 yen to Tanaka."

[1009] The Google Cloud Speech-to-Text API receives the audio data and outputs it as text data: "Send 5,000 yen to Tanaka."

[1010] Step 2:

[1011] Recognition of user emotions

[1012] The device's camera and microphone are used to capture the user's facial expressions and voice tone. Google ML Kit or Amazon Rekognition analyzes this data to recognize the user's emotional state. Input data consists of video and audio data, while output data consists of tags representing the emotional state (e.g., "anxious," "relieved").

[1013] Specific actions:

[1014] The camera captures the user's facial expressions, and the microphone collects their voice tone.

[1015] The ML kit outputs the tag "anxiety".

[1016] Step 3:

[1017] Requirements analysis using natural language processing

[1018] The device sends text data obtained from voice input to the Google Cloud Natural Language API for parsing of the request. The input data is text data, and the output data is structured request content (e.g., "Recipient: Tanaka", "Amount: 5000 yen").

[1019] Specific actions:

[1020] The text data "Send 5000 yen to Tanaka" is sent to the Natural Language API.

[1021] The API outputs structured data such as "Recipient: Tanaka" and "Amount: 5000 yen".

[1022] Step 4:

[1023] Integration of emotion data and request data

[1024] The server receives request data and sentiment data sent from the user terminal in JSON format, automatically parses and integrates them. The input data consists of structured request content and sentiment status tags, and the output data is the integrated JSON data.

[1025] Specific actions:

[1026] Send the following information to the server in JSON format: "Recipient: Tanaka", "Amount: 5000 yen", "Emotion: Anxious".

[1027] The server analyzes and integrates the data.

[1028] Step 5:

[1029] Program code and data query generation

[1030] The server uses a generative AI model to generate program code or data queries based on integrated JSON data. The input data is integrated data in JSON format, and the output data is the generated program code or data query.

[1031] Specific actions:

[1032] The server generates integrated data with the following information: "Recipient: Tanaka", "Amount: 5000 yen", and "Emotion: Anxiety". This data is then input into the AI ​​model.

[1033] The generating AI model outputs a remittance processing code or data query.

[1034] Step 6:

[1035] Execution and return of results

[1036] The server compiles the generated program code and deploys it to the specified location, or executes data queries against the database and retrieves the results. The resulting data is retrieved in JSON format and returned to the user's terminal. The input data is the generated program code or data query, and the output data is the execution result.

[1037] Specific actions:

[1038] The server compiles and executes the money transfer processing code.

[1039] The execution result returns a "Transfer successful" message in JSON format to the user's terminal.

[1040] Step 7:

[1041] User Feedback

[1042] The server returns the execution results to the user's terminal and displays additional confirmation messages corresponding to the user's emotional state. The input data is the JSON data of the execution results, and the output data is the messages displayed on the user interface.

[1043] Specific actions:

[1044] The user receives a "transfer successful" message along with an additional confirmation message stating, "We understand you were feeling anxious, but the transfer has been completed successfully."

[1045] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1046] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1047] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1048] [Fourth Embodiment]

[1049] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1050] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1051] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1052] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1053] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1054] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1055] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1056] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1057] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1058] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1059] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1060] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1061] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1062] This invention relates to a system that automatically generates program code and data queries based on requests entered by the user in natural language, and provides the results to the user. This system can be implemented in the following way.

[1063] This system begins with the user entering a request in natural language through their device. For example, they might enter a request such as "I want the sales data for 2023" into a text field in their browser or a dedicated application. When the user clicks the submit button, the request is converted to JSON format and sent to the server.

[1064] The server receives requests from users and parses their content. The requests are understood using natural language processing techniques and translated into specific program code or data queries. This generates precise instructions tailored to the specific request.

[1065] The generated program code and data queries are executed on the server. If the generated item is a data query, the server executes the query against the database and retrieves the corresponding data. Conversely, if the generated item is program code, this code is compiled by the server and deployed to the specified location.

[1066] The server returns the execution results to the user's terminal. In the case of a data query, the data obtained as a result of the query is returned; in the case of program code, the URL of the deployment destination is notified to the user. The user can then use this to retrieve the necessary data or access the generated application.

[1067] To give a concrete example, if a user requests a "simple to-do app," the server analyzes this request and generates the necessary program code. The generated code is compiled on the server and deployed to the specified server. The user then receives a URL to the deployment location, such as "Your app is available at http: / / yourserver.com / todoapp," and can access and use the application from there.

[1068] This system allows users to easily acquire data and develop applications based on their own requirements without needing specific expertise. This improves operational efficiency and enables faster response times. Furthermore, because it can generate programs and data for various purposes with just a single instruction in natural language, it is usable by a wide range of users.

[1069] The following describes the processing flow.

[1070] Program processing flow

[1071] Step 1: Obtaining requests from users

[1072] 1.1 The user enters the request

[1073] User: The user enters a request in natural language, such as "I want the sales data for 2023," into a text field in their device's browser or application.

[1074] 1.2 User submits request

[1075] User: Click the submit button to send the request to the server.

[1076] Terminal: Encodes the request into an appropriate format such as JSON and sends it to the server.

[1077] Step 2: Server receives and parses the request.

[1078] 2.1 The server receives the request

[1079] Server: Receives requests sent from user terminals as HTTP requests and decodes the data in JSON format.

[1080] 2.2 Analysis of requests using generative AI

[1081] Server: The server analyzes the received request using natural language processing technology to obtain the user's request.

[1082] Server: Based on the analysis results, the generative AI generates appropriate program code or data queries.

[1083] Step 3: Execute the generated code and queries.

[1084] 3.1 Executing the generated query

[1085] Server: If the generated query is a data query (SQL), it executes that query against the database.

[1086] 3.2 Execution of the generated program code

[1087] Server: If the generated code is a program (such as Java code), it compiles that code and deploys it to the specified location.

[1088] Step 4: Return the execution results

[1089] 4.1 Sending query results to the user's terminal

[1090] Server: Sends the results of the SQL query execution to the user's terminal.

[1091] 4.2 Notification of Program Deployment URL

[1092] Server: Notifies the user of the URL where the program code is deployed.

[1093] Step 5: Displaying results to the user

[1094] 5.1 Receiving the Results

[1095] Terminal: Receives results from the server.

[1096] 5.2 Displaying Results

[1097] Terminal: Displays the received results to the user.

[1098] User: Review the results of the submitted data query and the URL of the generated application, and access or use them as needed.

[1099] Through the steps described above, the present invention can automatically analyze natural language requests from users, generate appropriate program code and data queries, and execute them. This allows users to quickly and efficiently obtain the necessary information and programs without requiring any special knowledge.

[1100] (Example 1)

[1101] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1102] In conventional systems, generating program code and data queries required specialized knowledge, and the lack of automation made efficient data acquisition and application development difficult. This resulted in users having to expend a significant amount of time and effort, which was a major problem.

[1103] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1104] In this invention, the server includes means for receiving requests from users in natural language, means for analyzing the received requests using natural language processing and generating program code or data queries based on the request content, means for executing the generated program code or data queries, means for returning the execution results to the user, means for converting the received requests into JSON format, and means for using a generative AI model for analysis. This enables users to efficiently acquire data and create applications through natural language input without requiring specialized knowledge.

[1105] "Natural language" generally refers to the language that humans use on a daily basis, and is not a specific program or machine language.

[1106] A "request" is when a user asks for specific information or an action using natural language.

[1107] "Means of receiving" refers to methods or devices for obtaining requests from users and inputting them into the system.

[1108] "Natural language processing" is the technology that enables computers to understand and analyze human natural language.

[1109] "Program code" is code that contains instructions for performing a specific task.

[1110] A "data query" is a command used to retrieve specific data from a database.

[1111] "Generating means" refers to methods or devices that automatically generate program code or data queries based on the results of analysis.

[1112] "Means of execution" refers to the methods or devices used to run the generated program code or data queries on a computer.

[1113] "Means of return" refers to methods or devices for communicating the execution results to the user.

[1114] "JSON format" is a simple data format used for data exchange, and it is an abbreviation for JavaScript Object Notation.

[1115] A "generative AI model" is an artificial intelligence model that understands human language and commands and produces appropriate responses or generated text.

[1116] "Compilation" is the process of converting program code into a format that a machine can understand.

[1117] "Deployment" refers to placing a created program on a specific server or environment and making it available for use.

[1118] "HTTP format" refers to the protocol format used for sending and receiving information over the internet.

[1119] This invention is a system that automatically generates program code and data queries based on requests entered by the user in natural language, and provides the results to the user. The specific form of implementing this system is described below.

[1120] Users enter their requests using a device (e.g., a PC or smartphone). These devices are equipped with a browser (e.g., Google Chrome or Mozilla Firefox) or a dedicated application, allowing users to enter requests in natural language. For example, a user might enter "I want the sales data for 2023" and click the submit button.

[1121] The terminal converts the input request into JSON format and sends it to the server as an HTTP request. The server receives this HTTP request and parses the JSON request body. Specifically, it analyzes the received data using natural language processing (NLP) techniques to understand the request. For this analysis, it uses, for example, generative AI models such as OpenAI's GPT-4 or the Google Cloud Natural Language API.

[1122] The server generates appropriate program code or data queries based on the analysis results. For example, to retrieve sales data for a specific year, it generates an appropriate SQL query. The generated SQL query is executed against a database (e.g., MySQL or PostgreSQL) to retrieve the corresponding data. If program code is generated, it is compiled on the server and deployed to the specified server environment.

[1123] The results of data queries and the execution results of generated program code are returned from the server to the user's terminal. In the case of data queries, the retrieved data is returned as an HTTP response. In the case of program code, the user is notified of the deployment URL (for example, "http: / / yourserver.com / todoapp"). The user can use the generated application by accessing this URL.

[1124] For example, if a user requests a "simple to-do app," the server parses this request and generates appropriate JavaScript code. This code is compiled on the server and deployed to the specified server. As a result, the user receives a deployment URL such as "Your app is available at http: / / yourserver.com / todoapp," and can access and use the application.

[1125] Examples of prompt statements include the following:

[1126] "Please generate an SQL query to retrieve sales data for 2023."

[1127] "Please create JavaScript code to generate a simple to-do app."

[1128] "Create a Python script to retrieve order history based on a specific customer ID."

[1129] This system allows users to efficiently acquire data and develop applications through natural language input without requiring specialized knowledge.

[1130] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1131] Step 1: User enters request

[1132] The user enters their request in natural language into a text field in a browser or dedicated application using their device. For example, they might type "I want the sales data for 2023" and click the submit button. This request becomes the user's input. As output, this request becomes the input data needed to proceed to the next step.

[1133] Step 2: Convert and send the request

[1134] After receiving a user request, the terminal converts it into JSON format. For example, it is converted into a JSON object like the following:

[1135] json

[1136] {

[1137] "request": "I would like sales data for 2023."

[1138] }

[1139] This JSON data is output and sent to the server as an HTTP request. The body of the HTTP request becomes the input data.

[1140] Step 3: Receiving and parsing the request

[1141] The server receives HTTP requests sent from terminals. It parses the received JSON data and understands the request using natural language processing techniques. Specifically, it uses a generative AI model (e.g., OpenAI GPT-4) to analyze the request and extract its content. This analysis result becomes the output for data processing and the input data for the next step.

[1142] Step 4: Generate program code or data queries

[1143] The server generates program code and data queries based on the analysis results. For example, from a request such as "I want sales data for 2023," it generates the following SQL query:

[1144] sql

[1145] SELECT FROM sales WHERE year = 2023;

[1146] The generated SQL query becomes the output and the input data for proceeding to the next step. The generated program code and data queries are output as concrete actions.

[1147] Step 5: Execute the code or query

[1148] The server executes the generated program code and data queries. In the case of SQL queries, the server executes the query against the database (e.g., MySQL) and retrieves the data. The generated SQL query is executed and the corresponding data is output from the database. In the case of program code, the server compiles the code and deploys it to the specified location. Compilation and deployment are the concrete actions.

[1149] Step 6: Return the execution results

[1150] The server returns the retrieved data or the execution results of the generated program to the user. In the case of a data query, the retrieved data is returned as an HTTP response in JSON format.

[1151] json

[1152] {

[1153] "data": [

[1154] {"id": 1, "amount": 1000, "date": "2023-01-01"},

[1155] {"id": 2, "amount": 1500, "date": "2023-02-01"},

[1156] ...

[1157] ]

[1158] }

[1159] In the case of program code, the deployment URL is notified to the user. For example, the URL is returned in JSON format, such as "Your app is available at http: / / yourserver.com / todoapp". This result becomes the output, and the user receives it and proceeds with their actions.

[1160] Through the above processing steps, users can efficiently acquire data and develop applications through natural language input, even without specialized knowledge.

[1161] (Application Example 1)

[1162] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1163] Traditional factory inventory management systems have made it difficult to accurately track inventory levels and automate necessary replenishment orders. Furthermore, the systems were complex to use, increasing the risk of errors for workers without specialized knowledge. This led to inventory shortages and excesses, hindering efficient production activities.

[1164] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1165] In this invention, the server includes means for receiving requests from users in natural language; means for analyzing the received requests using natural language processing and generating program code or data queries based on the request content; means for executing the generated data queries against an inventory database within the factory and obtaining the results; means for returning the execution results to the user; and means for compiling the program code and deploying it to a specified location, or executing the data queries against the database and obtaining the results, depending on the type of program code or data query generated. This makes it possible for even workers without specialized knowledge to easily check inventory status and issue efficient replenishment instructions.

[1166] A "user" is someone who uses a system to input requests in natural language and receives the results.

[1167] "Natural language" refers to the language that humans use on a daily basis, and is a form of linguistic expression composed in a natural way, rather than a specific programming language.

[1168] A "request" refers to the content of instructions or inquiries that a user makes to the system.

[1169] "Means of receiving" refers to the processes and functions that allow the system to receive requests from users.

[1170] "Natural language processing" is a technology for analyzing and understanding requests from users in natural language.

[1171] "Analysis" is the process by which a system interprets information in order to understand the content of a received request and perform appropriate processing.

[1172] "Program code" is a part of a computer program that a system automatically generates to meet user requirements.

[1173] A "data query" is a query command sent to a database to retrieve specific information.

[1174] "Generating means" refers to functions or processes that automatically generate program code or data queries based on requests.

[1175] "Means of execution" refers to the processes and functions that actually run the generated program code and data queries.

[1176] A "factory inventory database" is a database system used to manage inventory information within a factory.

[1177] "Execution result" refers to the output obtained after the generated program code or data query has been executed.

[1178] "Means of return" refers to the processes or functions that allow a system to return its execution results to the user.

[1179] JSON format is a lightweight data exchange format for structuring, storing, and transmitting data.

[1180] A "server" is a computer system that receives requests, analyzes them, and performs the necessary processing.

[1181] "Deployment" refers to placing the generated program code in a specified environment and making it operational.

[1182] The system for implementing this invention will streamline inventory management within a factory and automate replenishment instructions based on requests entered by the user in natural language. The system mainly consists of the following hardware and software:

[1183] 1. Hardware:

[1184] User devices: Android smartphones, iOS smartphones

[1185] Server: High-performance computer system

[1186] Factory database: A database system for managing inventory information.

[1187] 2. Software:

[1188] "Smart Inventory Manager" is a dedicated application installed on the user's terminal.

[1189] A framework that runs on a server (e.g., Flask)

[1190] Generative AI models for natural language processing (e.g., OpenAI's GPT-3)

[1191] Database management systems (e.g., MySQL, PostgreSQL)

[1192] Users input requests in natural language through the "Smart Inventory Manager" application on their smartphones. For example, they might enter text such as "Show me the inventory levels for 2023" and press the send button to send the request from their device to the server.

[1193] The server receives user requests in JSON format and analyzes them using natural language processing technology. Generative AI models such as OpenAI's GPT-3 are used to convert the requests into appropriate program code or data queries. These converted data queries are then executed against the factory's inventory database to retrieve the requested information.

[1194] The acquired inventory information is returned to the user by the server. The returned information is displayed on the user's smartphone application, allowing the user to check the inventory status in the factory in real time.

[1195] As a concrete example, if a user enters "Show me the inventory for 2023," the system translates this into an SQL query, "SELECT FROM inventory WHERE year = 2023," and executes it against the database. The results are returned to the device in JSON format, and the user can view the results on their smartphone.

[1196] This process allows even workers without specialized knowledge to check and efficiently manage inventory levels within the factory using simple natural language operations. Furthermore, the system automatically handles situations requiring quick responses, thus improving operational efficiency.

[1197] For the generative AI model and prompt statements, use the following example prompt statements:

[1198] Example prompt: "Show me the inventory levels for 2023."

[1199] This allows the system to automate inventory management within the factory and provide the ability to acquire and display data according to user requirements without requiring specialized knowledge.

[1200] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1201] Step 1:

[1202] A user enters a request in natural language using the smartphone application "Smart Inventory Manager" and presses the submit button. The input is in text format, such as "Show inventory levels for 2023." This request is converted to JSON format and sent to the server. Here, the input is the user's request, and the output is the data obtained by converting the request to JSON format.

[1203] Step 2:

[1204] The server receives user requests in JSON format sent from the terminal. Since the received requests are parsed within the server, the input is request data in JSON format, and the output is natural language text data.

[1205] Step 3:

[1206] The server analyzes received natural language requests using natural language processing. This analysis uses a generative AI model (e.g., OpenAI's GPT-3). The input is natural language text data, and the output is appropriate program code or data query. Specifically, the request "Show inventory levels for 2023" is converted into the data query "SELECT FROM inventory WHERE year = 2023".

[1207] Step 4:

[1208] The server executes the generated data query. Here, the server connects to the factory's inventory database and sends the generated data query. The input is the data query, and the output is the inventory information retrieved from the database.

[1209] Step 5:

[1210] The server converts inventory information retrieved from the database into JSON format and sends it back to the user. The input is inventory information retrieved from the database, and the output is inventory information converted into JSON format.

[1211] Step 6:

[1212] The user terminal receives inventory information in JSON format from the server and displays it within the application. The input is inventory information in JSON format, and the output is inventory information displayed visually to the user. This allows the user to check the inventory status within the factory in real time.

[1213] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1214] This invention combines a system that automatically analyzes user requests in natural language, converts them into program code or data queries, and provides the results with an emotion engine that recognizes user emotions. This system improves the user experience and provides more appropriate feedback and results.

[1215] This system begins with the user entering a request in natural language using a terminal. For example, the user enters a request such as "I want the 2023 sales data" or "I want a new ToDo app" into a text field in a browser or application. Simultaneously with the input, an emotion engine activates, recognizing the user's emotional state by analyzing the user's input along with their facial expressions and tone of voice.

[1216] When a user clicks the submit button, the device encodes the request in JSON format and sends it to the server. The server analyzes the received request and integrates sentiment data from the sentiment engine into the analysis results. This allows the server to prioritize the request for faster results, for example, if the user is in a hurry.

[1217] Once the server has finished its analysis, the generative AI generates program code or data queries based on the request. The generated program code or data queries are executed while also taking into account feedback based on the user's emotions. For example, if the user is feeling anxious, additional confirmation messages or detailed logs can be generated to provide reassurance.

[1218] If a data query is generated, the server executes it against the database and retrieves the results. If program code is generated, the server compiles it and deploys it to the specified location. The server then returns the execution results to the user's terminal. In the case of a data query, the retrieved data is notified to the user; in the case of a program, the URL of the deployment destination is notified.

[1219] For example, if a user emotionally inputs "I have so many tasks today, I need a new to-do app," the emotion engine recognizes the user's stress level. The server determines that an immediate response is needed, quickly generates program code, and provides a deployment URL in the most user-friendly format possible. By notifying the user of the URL in the form of "Your app is available at http: / / yourserver.com / todoapp," the user can access the URL and start using the app immediately.

[1220] This system allows users to receive highly personalized responses quickly through natural language request input and emotion recognition. Furthermore, by utilizing an emotion engine, it enables feedback and adjustments tailored to the user's psychological state, providing an optimal user experience.

[1221] The following describes the processing flow.

[1222] Program processing flow

[1223] Step 1: Obtaining the user's needs and emotions

[1224] 1.1 The user enters the request

[1225] User: Enters requests in natural language into a text field in a browser or application, such as "I want the 2023 sales data" or "I want a new ToDo app."

[1226] 1.2 The emotion engine recognizes the user's emotions

[1227] Server: The emotion engine analyzes the user's facial expressions and voice tone in real time and acquires emotion data.

[1228] 1.3 Sending a request

[1229] User: Click the submit button to send the request to the server.

[1230] Terminal: Encodes the request in JSON format and sends it to the server along with sentiment data.

[1231] Step 2: Server receives and parses the request.

[1232] 2.1 The server receives the request and sentiment data.

[1233] Server: Receives requests and sentiment data sent from the user terminal as HTTP requests and decodes the data in JSON format.

[1234] 2.2 Analysis of requests using generative AI

[1235] Server: Analyzes received request content and sentiment data using natural language processing technology to obtain the user's request content and sentiment state.

[1236] 2.3 Generating Program Code or Data Queries

[1237] Server: The generative AI generates program code or data queries based on the request, and adjusts the generation results considering sentiment data.

[1238] Step 3: Execute the generated code and queries.

[1239] 3.1 Executing the generated query

[1240] Server: If the generated query is a data query (SQL), it executes that query against the database.

[1241] 3.2 Compiling and deploying the generated program code

[1242] Server: If the generated code is a program (such as Java code), it compiles that code and deploys it to the specified location.

[1243] Step 4: Return the execution results

[1244] 4.1 Sending query results to the user's terminal

[1245] Server: Sends the results of the SQL query execution to the user's terminal.

[1246] 4.2 Notification of Program Deployment URL

[1247] Server: Notifies the user of the URL where the program code is deployed.

[1248] Step 5: Displaying results to the user

[1249] 5.1 Receiving the Results

[1250] Terminal: Receives results from the server.

[1251] 5.2 Displaying and responding to results

[1252] Terminal: Displays the received results to the user.

[1253] User: Review the results of the submitted data query and the URL of the generated application, and access or use them as needed.

[1254] Server: Take user sentiment data into consideration and provide additional appropriate feedback.

[1255] As a concrete example, consider a scenario where a user emotionally inputs, "I'm under a lot of stress and finding task management difficult; I need a new to-do app." The emotion engine recognizes the user's stress level, and the server determines that a quick response is needed. The server quickly generates program code and provides a user-friendly URL for deployment, thereby supporting the user in reducing stress. A message like "Your app is available at http: / / yourserver.com / todoapp" is sent, allowing the user to access the URL and start using the app immediately.

[1256] In this way, this system automatically analyzes users' natural language requests, recognizes their emotions, and provides personalized responses, thereby significantly improving the user experience.

[1257] (Example 2)

[1258] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1259] While conventional systems could accurately analyze users' natural language requests and generate appropriate program code and data queries, they failed to adequately improve the user experience because they did not consider the user's emotional state. Furthermore, they struggled to respond flexibly to situations where users required quick responses or detailed explanations and confirmations.

[1260] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting a request from a user in natural language, means for analyzing the input request and recognizing the user's emotions, means for encoding the request content and recognized emotion data into JSON format, means for transmitting the encoded data to the server, means for analyzing the transmitted data using natural language processing technology, means for generating program code or data queries using a generation AI model based on the analysis results, means for compiling the generated program code and deploying it to a specified location, or means for executing the generated data query against a database and obtaining the results, and means for returning the execution results to the user. This enables highly personalized responses that take into account the user's emotional state, and improves the user experience by providing quick and appropriate feedback.

[1261] "Natural language" refers to the language that humans use on a daily basis, and is in the form of text or spoken language, rather than program code or data queries.

[1262] "Emotions" refers to the user's psychological state, such as stress, excitement, or calmness.

[1263] An "emotion engine" is a software or hardware technology that analyzes input data such as a user's facial expressions and tone of voice to recognize the user's emotional state.

[1264] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a text format for describing data structures.

[1265] "Natural language processing technology" refers to technologies that enable the analysis and understanding of natural language, and includes text analysis, morphological analysis, and contextual analysis.

[1266] A "generative AI model" is an artificial intelligence model that generates new text, code, or data queries based on requests. Generative models such as GPT fall into this category.

[1267] "Program code" is a collection of instructions and statements that make up a computer program, and is software in a form that a computer can execute.

[1268] A "data query" refers to a command used to search for or retrieve specific information from a database.

[1269] "Compilation" is the process of converting programming code from a high-level language into machine code.

[1270] "Deployment" refers to the process of placing the generated program code into the actual production environment and making it executable.

[1271] "User experience" refers to the overall satisfaction and convenience that users experience when using a system or application.

[1272] "Feedback" refers to information that a system provides in response to a user, including information provided based on the user's requests and emotional state.

[1273] A "natural language request" is a request or question made by a user using natural language, and includes tasks that the system should process or the retrieval of information.

[1274] This invention combines a system that automatically analyzes user requests in natural language, converts them into program code or data queries, and provides the results with an emotion engine that recognizes user emotions. This system improves the user experience and provides more appropriate feedback and results.

[1275] This system begins with the user entering a request in natural language using a terminal. For example, the user might enter a request in a text field in a browser or application, such as "I want the 2023 sales data" or "I want a new ToDo app." The hardware used is the user's PC or smartphone, and the software used is a browser or application.

[1276] The emotion engine starts operating simultaneously with the input. The emotion engine analyzes the user's input, facial expressions, and tone of voice to recognize the user's emotional state. For example, it can determine whether the user is anxious or calm. This analysis uses hardware such as a camera and microphone, and emotion analysis software is employed.

[1277] When the user clicks the send button, the device encodes the request into JSON format and sends it to the server. At this point, sentiment data is also included in the JSON. The server then analyzes the received request using natural language processing (NLP) techniques (e.g., using Python or Node.js).

[1278] The server integrates the received emotional data into the analysis results. For example, if the emotional state is recognized as "stress," the request is processed as a high-priority task. This ensures that if the user is in a hurry, the server takes action to quickly obtain the results.

[1279] Next, the server uses a generative AI model (e.g., GPT-4) to generate program code or data queries that respond to the request. The generated program code or data queries also take into account the user's emotional state. For example, if the user is anxious, detailed comments or confirmation messages will be included.

[1280] If the generated data query is generated, the server executes it against the database and retrieves the results. If the generated program code is generated, the server compiles it and deploys it to the specified location. The server then returns the execution results to the user's terminal. For example, the URL of the deployment destination is notified to the user. By accessing this URL, the user can immediately use the generated program.

[1281] For example, if a user emotionally inputs "I have so many tasks today, I need a new to-do app," the emotion engine recognizes that the user is stressed. The server determines that a quick response is needed and uses a generative AI model to generate code for a to-do app. The generated application is then deployed to "http: / / yourserver.com / todoapp," and the URL is notified to the user. The user can access this URL and immediately start using the new to-do app.

[1282] Examples of prompts for a generative AI model:

[1283] User input: "I have so many tasks today, I need a new to-do app."

[1284] User's emotion: Stress

[1285] The program code to be generated: Code for a to-do app that makes task management easy.

[1286] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1287] Step 1:

[1288] The user enters the request into the terminal.

[1289] Users open a browser or application on their PC or smartphone and enter requests in natural language into a text field, such as "I want the 2023 sales data" or "I want a new ToDo app." This input is then saved as text data in the device's memory.

[1290] Step 2:

[1291] The device analyzes the user's emotions.

[1292] The device's built-in emotion engine operates, analyzing the user's input, as well as their facial expressions and voice tone in real time using the camera and microphone. The input data is captured by the device as text, audio, and video data, and the emotion engine processes this data to output the user's emotional state.

[1293] Step 3:

[1294] The device sends request and sentiment data to the server.

[1295] When the user clicks the submit button, the device encodes the entered request and analyzed sentiment data into JSON format. This encoded JSON data is then sent to the server via the internet.

[1296] Step 4:

[1297] The server analyzes requests and sentiment data.

[1298] The server deserializes and parses the received JSON data. First, it analyzes the request content using natural language processing (NLP) techniques and converts it into a specific task. Simultaneously, it analyzes sentiment data from the sentiment engine to determine the priority of the request. The analysis results are stored as internal data structures (e.g., objects or dictionaries).

[1299] Step 5:

[1300] The generative AI generates program code or data queries.

[1301] The server generates program code or data queries using a generative AI model (e.g., GPT-4) based on the analyzed request data. The generative AI model receives the request content and sentiment data as input and outputs specific code or queries based on them.

[1302] Step 6:

[1303] The server retrieves the execution results and notifies the user.

[1304] For generated data queries, the server executes them against the database and retrieves the results. For generated program code, the server compiles it and deploys it to the specified location. Finally, the server returns the deployment URL and query results to the user. Specifically, it sends a message to the user's device using a notification function.

[1305] Each of these steps allows users to receive highly personalized responses quickly through natural language request input and emotion recognition.

[1306] (Application Example 2)

[1307] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1308] Conventional natural language processing systems fail to provide appropriate feedback tailored to the user's psychological state, making it difficult to alleviate user stress and anxiety, and consequently hindering the improvement of the user experience. Furthermore, providing feedback and answers that disregard the user's emotions can compromise reliability and usability.

[1309] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving requests from users in natural language, means for analyzing the received requests using natural language processing and generating program code or data queries based on the content of the requests, means for analyzing the user's emotional state at the time of request input, including an emotion engine that recognizes the user's emotions, means for integrating the emotional data from the emotion engine into the request analysis and providing feedback according to the user's psychological state, means for executing the generated program code or data queries, and means for returning the execution results to the user. This makes it possible to provide optimal feedback and results according to the user's psychological state.

[1310] "Means for receiving requests from users in natural language" refers to a system for obtaining requests for voice or text input by users in natural language.

[1311] "Means of analysis using natural language processing" refers to a system that analyzes received natural language requests using machine learning and statistical methods to understand their content.

[1312] "Means for generating program code or data queries" refers to a mechanism for automatically generating the necessary program code or queries for database operations based on the analyzed requirements.

[1313] An "emotion engine that recognizes user emotions" is software or hardware that analyzes a user's tone of voice and facial expressions to recognize their emotional state in real time.

[1314] "Means for analyzing emotional states" refers to a system that uses data acquired by the emotion engine to determine what emotional state a user is in.

[1315] "Means for integrating emotional data into request analysis" refers to a system that incorporates recognized emotional data into the results of natural language processing to provide feedback tailored to the user's psychological state.

[1316] "Means of providing feedback tailored to the user's psychological state" refers to a system that aims to create a better user experience by providing appropriate additional messages or confirmations based on the user's emotional state.

[1317] "Means for executing generated program code or data queries" refers to a mechanism for compiling generated program code and placing it in a specified location, or for executing data queries against a database and retrieving results.

[1318] "Means of returning execution results to the user" refers to a mechanism for notifying or displaying the results of executed program code or data queries to the user.

[1319] The system for carrying out this invention consists of the following components:

[1320] 1. User terminal:

[1321] Smartphones are primarily used as the terminal for users to input requests in natural language. Voice input is handled using the Google Cloud Speech-to-Text API, while text input is done via standard keyboard input.

[1322] 2. Emotional Engine:

[1323] The system uses the camera and microphone on the smartphone to collect the user's voice tone and facial expressions. This data is then analyzed using Google ML Kit or Amazon Rekognition to recognize the user's emotional state.

[1324] 3. Natural Language Processing Systems:

[1325] The natural language requests entered by the user are parsed using the Google Cloud Natural Language API. As a result, the content of the request is extracted and treated as structured data.

[1326] 4. Server:

[1327] a. Requirements analysis:

[1328] The received request data and sentiment data are encoded into JSON format and sent to the server. The server then receives this data, integrates the results of natural language processing and sentiment analysis, and processes them.

[1329] b. Program code and data query generation:

[1330] Based on the analysis results, a generative AI model is used to generate program code or data queries that respond to user requests. This provides optimal feedback and results that are tailored to the user's psychological state.

[1331] c. Execution of the results:

[1332] The generated program code is compiled and deployed to the specified location, or data queries are executed against the database. The results of this execution are then returned to the user's terminal.

[1333] 5. Content Management:

[1334] The server manages the execution results and records them in a database so that it can quickly respond when the user makes another inquiry.

[1335] Specific example

[1336] 1. Example of natural language input:

[1337] The user voice-inputs "Send 5,000 yen to Tanaka" into their smartphone.

[1338] When the emotion engine detects anxiety, the system displays an additional confirmation message. For example, "Are you sure you want to send 5,000 yen to Tanaka? Please confirm."

[1339] 2. Example of a prompt:

[1340] "The user wants to send money to a friend. Their emotion has been identified as anxiety. Please generate an appropriate system response."

[1341] This system provides optimal feedback and results tailored to the user's psychological state, thereby improving the user experience and increasing trust.

[1342] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1343] Step 1:

[1344] User request input

[1345] Users enter their requests using their smartphones via voice or text. For voice input, the Google Cloud Speech-to-Text API converts the speech to text. If the input data is voice, the voice data is used and output as text. If the input data is text, it is used as is.

[1346] Specific actions:

[1347] The user voice-inputs, "Send 5,000 yen to Tanaka."

[1348] The Google Cloud Speech-to-Text API receives the audio data and outputs it as text data: "Send 5,000 yen to Tanaka."

[1349] Step 2:

[1350] Recognition of user emotions

[1351] The device's camera and microphone are used to capture the user's facial expressions and voice tone. Google ML Kit or Amazon Rekognition analyzes this data to recognize the user's emotional state. Input data consists of video and audio data, while output data consists of tags representing the emotional state (e.g., "anxious," "relieved").

[1352] Specific actions:

[1353] The camera captures the user's facial expressions, and the microphone collects their voice tone.

[1354] The ML kit outputs the tag "anxiety".

[1355] Step 3:

[1356] Requirements analysis using natural language processing

[1357] The device sends text data obtained from voice input to the Google Cloud Natural Language API for parsing of the request. The input data is text data, and the output data is structured request content (e.g., "Recipient: Tanaka", "Amount: 5000 yen").

[1358] Specific actions:

[1359] The text data "Send 5000 yen to Tanaka" is sent to the Natural Language API.

[1360] The API outputs structured data such as "Recipient: Tanaka" and "Amount: 5000 yen".

[1361] Step 4:

[1362] Integration of emotion data and request data

[1363] The server receives request data and sentiment data sent from the user terminal in JSON format, automatically parses and integrates them. The input data consists of structured request content and sentiment status tags, and the output data is the integrated JSON data.

[1364] Specific actions:

[1365] Send the following information to the server in JSON format: "Recipient: Tanaka", "Amount: 5000 yen", "Emotion: Anxious".

[1366] The server analyzes and integrates the data.

[1367] Step 5:

[1368] Program code and data query generation

[1369] The server uses a generative AI model to generate program code or data queries based on integrated JSON data. The input data is integrated data in JSON format, and the output data is the generated program code or data query.

[1370] Specific actions:

[1371] The server generates integrated data with the following information: "Recipient: Tanaka", "Amount: 5000 yen", and "Emotion: Anxiety". This data is then input into the AI ​​model.

[1372] The generating AI model outputs a remittance processing code or data query.

[1373] Step 6:

[1374] Execution and return of results

[1375] The server compiles the generated program code and deploys it to the specified location, or executes data queries against the database and retrieves the results. The resulting data is retrieved in JSON format and returned to the user's terminal. The input data is the generated program code or data query, and the output data is the execution result.

[1376] Specific actions:

[1377] The server compiles and executes the money transfer processing code.

[1378] The execution result returns a "Transfer successful" message in JSON format to the user's terminal.

[1379] Step 7:

[1380] User Feedback

[1381] The server returns the execution results to the user's terminal and displays additional confirmation messages corresponding to the user's emotional state. The input data is the JSON data of the execution results, and the output data is the messages displayed on the user interface.

[1382] Specific actions:

[1383] The user receives a "transfer successful" message along with an additional confirmation message stating, "We understand you were feeling anxious, but the transfer has been completed successfully."

[1384] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1385] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1386] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1387] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1388] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1389] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1390] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1391] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1392] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1393] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1394] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1395] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1396] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1397] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1398] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1399] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1400] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1401] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1402] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1403] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1404] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1405] The following is further disclosed regarding the embodiments described above.

[1406] (Claim 1)

[1407] A means for receiving requests from users in natural language,

[1408] A means for analyzing a received request using natural language processing and generating program code or data queries based on the content of the request,

[1409] Means for executing generated program code or data queries,

[1410] A system that includes a means of returning the execution results to the user.

[1411] (Claim 2)

[1412] The system according to claim 1, which converts a received request into JSON format and sends it to a server.

[1413] (Claim 3)

[1414] The system according to claim 1, which, depending on the type of program code or data query generated, compiles the program code and deploys it to a specified location, or executes the data query against a database and retrieves the results.

[1415] (Claim 4)

[1416] The system according to claim 1, which receives requests from users via a browser or an input field in an application.

[1417] (Claim 5)

[1418] The system according to claim 1, which notifies the user of the results of executing a data query or the URL of the deployment destination.

[1419] "Example 1"

[1420] (Claim 1)

[1421] A means for receiving requests from users in natural language,

[1422] A means for analyzing a received request using natural language processing and generating program code or data queries based on the content of the request,

[1423] Means for executing generated program code or data queries,

[1424] A means of returning the execution result to the user,

[1425] A means of converting the received request into JSON format,

[1426] A system that includes means for using generative AI models for analysis.

[1427] (Claim 2)

[1428] The system according to claim 1, which, depending on the type of program code or data query, compiles program code and deploys it to a specified location, or executes a data query against a database and retrieves the results.

[1429] (Claim 3)

[1430] The system according to claim 1, which returns the execution result to the user terminal in HTTP format.

[1431] "Application Example 1"

[1432] (Claim 1)

[1433] A means for receiving requests from users in natural language,

[1434] A means for analyzing a received request using natural language processing and generating program code or data queries based on the content of the request,

[1435] Means for executing generated program code or data queries,

[1436] A means of executing the generated data query against the factory's inventory database and obtaining the results,

[1437] A system that includes a means of returning the execution results to the user.

[1438] (Claim 2)

[1439] The system according to claim 1, which converts a received request into JSON format and sends it to a server.

[1440] (Claim 3)

[1441] The system according to claim 1, which, depending on the type of program code or data query generated, compiles the program code and deploys it to a specified location, or executes the data query against a factory database and retrieves the results.

[1442] "Example 2 of combining an emotion engine"

[1443] (Claim 1)

[1444] A means of inputting requests from the user in natural language,

[1445] A means of analyzing input requests and recognizing user emotions,

[1446] A means for encoding the request content and recognized emotion data into JSON format,

[1447] A means of sending encoded data to a server,

[1448] A means of analyzing the transmitted data using natural language processing techniques,

[1449] A means for generating program code or data queries using a generated AI model based on the analysis results,

[1450] Means for compiling the generated program code and deploying it to a specified location, or means for executing the generated data queries against a database and obtaining the results,

[1451] A means of returning the execution result to the user,

[1452] A system that includes this.

[1453] (Claim 2)

[1454] The system according to claim 1, wherein recognized emotion data is reflected in the analysis results and response priority.

[1455] (Claim 3)

[1456] The system according to claim 1, which generates confirmation messages and detailed logs in accordance with the user's emotional state.

[1457] "Application example 2 of combining emotional engines"

[1458] (Claim 1)

[1459] A means for receiving requests from users in natural language,

[1460] A means for analyzing a received request using natural language processing and generating program code or data queries based on the content of the request,

[1461] Includes an emotion engine that recognizes the user's emotions, and means for analyzing the user's emotional state at the time of request input,

[1462] A means of integrating emotional data from an emotion engine into request analysis and providing feedback tailored to the user's psychological state,

[1463] Means for executing generated program code or data queries,

[1464] A system that includes a means of returning the execution results to the user.

[1465] (Claim 2)

[1466] The system according to claim 1, which converts a received request into JSON format and sends it to a server.

[1467] (Claim 3)

[1468] The system according to claim 1, which, depending on the type of program code or data query generated, compiles the program code and deploys it to a specified location, or executes the data query against a database and retrieves the results. [Explanation of Symbols]

[1469] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving requests from users in natural language, A means for analyzing a received request using natural language processing and generating program code or data queries based on the content of the request, Means for executing generated program code or data queries, A system that includes a means of returning the execution results to the user.

2. The system according to claim 1, which converts a received request into JSON format and sends it to the server.

3. The system according to claim 1, which, depending on the type of program code or data query generated, compiles the program code and deploys it to a specified location, or executes the data query against a database and retrieves the results.

4. The system according to claim 1, which receives requests from users via a browser or application input field.

5. The system according to claim 1, which notifies the user of the results of executing a data query or the URL of the deployment destination.

Images