system

Abstract

We provide the system. [Solution] A user interface means for the user to input information, Analysis means for analyzing information input through the user interface means, A data transmission and reception means for transmitting the information analyzed by the aforementioned analysis means to a generative AI and receiving a response, A display means for displaying the response received by the data transmission and reception means to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Based on old customs of official offices and the like, paper culture and analog culture remain deeply entrenched, which is a problem that the efficiency of business operations has declined. For this reason, the increasing overtime hours and inefficient work have become problems. An object of this invention is to provide a system for solving these problems of business efficiency improvement and achieving productivity improvement.

Means for Solving the Problems

[0005] To solve this problem, a system is provided that includes a user interface means for the user to input information, an analysis means for analyzing the information input through the user interface means, a data transmission and reception means for transmitting the information analyzed by the analysis means to a generative AI and receiving a response, and a display means for displaying the response received by the data transmission and reception means to the user. With this system, the user can easily input information and receive specific advice and suggestions to efficiently carry out their work, thereby improving work efficiency and productivity.

[0006] "User interface means" refers to interactive screens or input devices for users to input information.

[0007] "Analysis means" refers to a device or program that has the function of analyzing information input through a user interface and converting it into a format that can be processed by the system.

[0008] "Data transmission and reception means" refers to a communication device or program for transmitting analyzed information to a generative AI and receiving a response from the generative AI.

[0009] "Generative AI" refers to artificial intelligence that generates responses, suggestions, and other information based on input data.

[0010] "Display means" refers to a device or program for visually displaying a response received by a data transmission means to the user.

[0011] "System" refers to a collection of devices and programs that include user interface means, analysis means, data transmission and reception means, and display means.

[0012] "Information" refers to data entered by users and data processed by generative AI. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, when an emotion engine is combined. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention relates to a system aimed at improving the productivity of civil servants, in which a user inputs information, which is then analyzed and transmitted to a generative AI, and a response is received. The specific program processing of this system is described below in natural language, along with concrete examples.

[0035] System Overview

[0036] 1. Providing an input form

[0037] The system provides an input form that users can access through a web browser. This form has multiple input fields where users enter the necessary information.

[0038] 2. Data Analysis

[0039] When a user enters data into an input form and presses the submit button, the device uses JavaScript (registered trademark) to collect the form data and convert it into JSON format. This ensures that the data is sent to the server efficiently.

[0040] 3. Sending data and receiving responses

[0041] The server receives data in JSON format sent from the terminal. The server analyzes the received data and sends a request to the generative AI's API. The generative AI generates a response based on the input data. The server receives this response, converts it back to JSON format, and sends it back to the terminal.

[0042] 4. Display of response

[0043] The terminal receives response data sent back from the server. The received data is analyzed and displayed in the browser in a format that is easy for the user to understand. This allows the user to obtain specific advice and suggestions for efficiently carrying out their work.

[0044] Specific example

[0045] 1. Using the input form

[0046] A user (for example, an employee of the General Affairs Department of the city hall) enters "General Affairs Department of City Hall" in the "Role" field, enters "I would like to know the progress of paperless operations" in the "Request" field, and presses the submit button.

[0047] 2. Data transmission and analysis

[0048] The terminal converts the input data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI asking "How should the General Affairs Department of the city hall proceed with paperless operations?"

[0049] 3. Response from a generative AI

[0050] The generative AI generates advice on specific methods for implementing a paperless system based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal.

[0051] 4. Display of response

[0052] The terminal analyzes the response data it receives and displays it to the user as specific advice, such as "To proceed with paperless operations, please take the following steps..." The user can then take concrete action based on this information.

[0053] This system can streamline inefficient processes caused by outdated practices in government offices, thereby improving productivity. Users can easily input information and receive specific suggestions from generative AI, leading to improved work processes, reduced overtime, and a move towards a paperless environment.

[0054] The following describes the processing flow.

[0055] Step 1:

[0056] The user opens a web browser and accesses the specified URL. This displays an input form.

[0057] Step 2:

[0058] The user enters the necessary information into each field of the input form (e.g., "Role", "Request").

[0059] Step 3:

[0060] The user clicks the "Submit" button. This collects the input data.

[0061] Step 4:

[0062] The JavaScript on the device converts the collected data into JSON format.

[0063] Step 5:

[0064] The terminal sends the converted JSON data to the server as a POST request.

[0065] Step 6:

[0066] The server receives the POST request and extracts JSON data from the request body.

[0067] Step 7:

[0068] The server parses the extracted JSON data and splits it into individual fields (e.g., role, request).

[0069] Step 8:

[0070] The server sends the analyzed data as a request to the generative AI's API.

[0071] Step 9:

[0072] The generative AI receives the request and generates a response based on the input data.

[0073] Step 10:

[0074] The generative AI generates a response which is then sent back to the server.

[0075] Step 11:

[0076] The server receives the response from the generative AI and converts it into the required format.

[0077] Step 12:

[0078] The server returns the converted response data to the terminal in JSON format.

[0079] Step 13:

[0080] The terminal receives and parses the JSON data sent back from the server.

[0081] Step 14:

[0082] The data analyzed by the device is embedded in an HTML element to be displayed visually to the user.

[0083] Step 15:

[0084] Users review the displayed response data and take specific actions to improve operations and increase productivity.

[0085] (Example 1)

[0086] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0087] In modern public institutions, productivity is low because many tasks are performed manually or on paper. Furthermore, this leads to delays in work progress, slow feedback and decision-making, and ultimately, a decrease in overall operational efficiency. A system is needed to address this challenge and efficiently improve productivity.

[0088] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0089] In this invention, the server includes a user interface means for the user to input information, a conversion means for collecting the information input through the user interface means and converting it into JSON format, a data transmission means for analyzing the JSON format information converted by the conversion means and sending a request to a generative AI model, and a display means for converting the response received from the generative AI model back into JSON format and displaying it to the user. This makes it possible to efficiently analyze the input data and provide appropriate advice to the user.

[0090] A "user interface means" is a tool that provides an interface (UI) for users to input information.

[0091] A "conversion tool" is a tool that collects information entered through a user interface and performs the process of converting it into a specific format (e.g., JSON format).

[0092] A "data transmission means" is a tool that performs the process of analyzing the information converted by the conversion means, sending a request to a generative AI model, and receiving a response.

[0093] A "display means" is a tool that transforms the response received from a generative AI model and displays it in a format that is easy for the user to understand.

[0094] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight data exchange format for structuring and exchanging data.

[0095] A "generative AI model" is an artificial intelligence model that analyzes data provided by users and generates appropriate responses based on that data.

[0096] An "input field" refers to a specific area in a user interface where a user enters information.

[0097] This invention aims to improve the productivity of civil servants by providing a system in which a user inputs information, which is then analyzed and sent to a generative AI model, and a response is received. Specific embodiments of this system are described below.

[0098] Providing an input form

[0099] This system utilizes an input form accessible through a web browser. Specifically, browsers such as Google Chrome® and Mozilla Firefox can be used. This input form has multiple fields where users enter the necessary information. For example, information can be entered in fields such as "Role" and "Requests." The entire input form is constructed using HTML and CSS, and a dynamic user interface is implemented using JavaScript.

[0100] Data analysis

[0101] When a user enters information into an input form and presses the submit button, the device uses JavaScript to collect the form data. Next, this collected data is converted into JSON format. JSON is a lightweight data exchange format for efficient data exchange, and the data is sent to the server in this format.

[0102] Sending data and receiving responses

[0103] The JSON data sent from the device is received by the server. The server receives this data via an HTTP request and first verifies its contents. Next, it sends the verified data to a generative AI model. The generative AI model generates an appropriate response based on the user's input data. The response generated by the generative AI model is sent back to the server. The server receives this response, converts it back to JSON format if necessary, and sends it back to the device.

[0104] Display of response

[0105] The response data sent from the server to the terminal is received again by the terminal. The terminal parses this response data and uses JavaScript to display it in a user-friendly format on the browser. This allows the user to obtain specific advice and suggestions to efficiently carry out their work.

[0106] Specific example

[0107] For example, an employee of the General Affairs Division of a city hall enters "General Affairs Division of City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field, then presses the submit button. The terminal converts the input data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI model asking "How will the General Affairs Division of City Hall proceed with paperless initiatives?" The generative AI model generates advice on specific methods for proceeding with paperless initiatives based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal. The terminal analyzes the received response data and displays it to the user as specific advice such as "To proceed with paperless initiatives, follow these steps..."

[0108] Example of a prompt

[0109] Examples of prompt statements include the following:

[0110] "I'm a staff member in the General Affairs Department. I'd like to promote a paperless system, but could you tell me what steps I should take?"

[0111] By inputting such prompt messages into the AI ​​model, advice can be obtained on specific methods for implementing a paperless system.

[0112] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0113] Step 1:

[0114] The user accesses the system using a web browser. The user enters information into the input form. For example, the user might enter "General Affairs Department, City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field. The user proceeds to the next step by pressing the "Submit" button. The input is the information entered by the user into the input form, and the output is the data entered into the input form converted into JSON format.

[0115] Step 2:

[0116] The device uses JavaScript to collect form data and convert it into JSON format. Specifically, JavaScript retrieves data from input fields and converts it into a JSON object. For example, it might be converted to a format like {"Role": "General Affairs Department, City Hall", "Request": "I want to know the progress of paperless initiatives"}. The input is information entered by the user, and the output is data generated in JSON format.

[0117] Step 3:

[0118] The device uses an HTTP POST request to send the generated JSON data to the server. Specifically, it uses JavaScript's fetch API or similar to send data to a specific endpoint on the server. For example, it sends JSON data to POST / api / request. The input is data in JSON format, and the output is the request that was sent to the server.

[0119] Step 4:

[0120] The server receives an HTTP request and parses the JSON data in the request body. Specifically, the server-side program (e.g., Node.js or Python) parses the request data. For example, it parses the JSON data and extracts the values ​​of each field. The input is the JSON data sent from the terminal, and the output is the parsed data.

[0121] Step 5:

[0122] The server sends the analyzed data to the generative AI model. Specifically, it issues an HTTP request to the generative AI model's API. For example, it sends data containing a prompt to the generative AI model's endpoint. The input is the analyzed data, and the output is the request that was sent to the generative AI model.

[0123] Step 6:

[0124] The generative AI model generates a response and sends it back to the server. Specifically, the generative AI model analyzes the received data and generates an appropriate response based on that analysis. For example, it might generate a response such as, "To proceed with paperless operations, please follow these steps..." The input is prompt data sent from the server, and the output is the generated response data.

[0125] Step 7:

[0126] The server converts the response data received from the generative AI model back into JSON format and sends it to the terminal. Specifically, it structures the received response data and converts it into JSON format. The input is the response data received from the generative AI model, and the output is the data converted into JSON format.

[0127] Step 8:

[0128] The terminal receives and parses response data from the server. Specifically, it uses JavaScript to parse JSON data and retrieve data for each item. The input is the JSON data sent from the server, and the output is the parsed response data.

[0129] Step 9:

[0130] The terminal analyzes response data and displays it to the user. Specifically, it inserts the response data into an HTML element and displays it in the browser. For example, it might display specific advice such as, "To proceed with paperless operations, follow these steps..." The input is the analyzed response data, and the output is the content displayed to the user.

[0131] (Application Example 1)

[0132] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0133] In virtual stores, there is a need to respond to customer inquiries quickly and appropriately. However, conventional systems lack sufficient automation to provide optimal answers tailored to the content of inquiries, making it difficult to improve customer satisfaction. Furthermore, even simple inquiries required human intervention, resulting in increased effort and cost. To solve this problem, there is a need to automate inquiry handling using generative AI.

[0134] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0135] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, a data transmission and reception means for sending the analyzed information to a generative AI and receiving a response, and an API integration means for calling the generative AI's API and automatically generating appropriate responses to customer inquiries in the virtual store. This makes it possible to automate customer inquiries in the virtual store quickly and accurately.

[0136] "User interface means" refers to an interface for users to input information, and specifically includes input forms, buttons, and the like.

[0137] "Analysis means" refers to means for analyzing information input through a user interface and handling it as data.

[0138] "Data transmission and reception means" refers to means for transmitting information analyzed by the analysis means to the generative AI and receiving a response from the generative AI.

[0139] "Display means" refers to means for displaying the response received by the data transmission means to the user.

[0140] "Generative AI" refers to artificial intelligence that automatically generates appropriate responses based on input data.

[0141] "API integration means" refers to a method for calling the API of a generative AI and exchanging data in conjunction with an external system.

[0142] A "virtual store" is an online store that provides goods and services via the internet.

[0143] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a method for representing data in a lightweight and structured format.

[0144] As a concrete example of implementing this invention, a customer service system for a virtual store is provided. The system includes a user interface means for users to input information. This user interface means includes an input form with multiple different input fields, allowing customers to easily input their inquiries.

[0145] The input information is analyzed by an analysis tool. This analysis tool uses a programming language such as JavaScript to convert the input data into JSON format. The converted data is then sent to the server via the analysis tool.

[0146] The server processes the received JSON data using a data transmission / reception mechanism and calls the generative AI API. This generative AI API automatically generates an appropriate response based on the input data, and can use, for example, the OpenAI® API. The server converts the response from the generative AI back into JSON format and sends it back to the terminal.

[0147] The terminal displays the received response data to the customer via a display device. This series of processes enables quick and efficient customer service in the virtual store.

[0148] Hardware and software to use

[0149] 1. Hardware

[0150] Server: Cloud server (e.g., Amazon Web Services, Google Cloud Platform)

[0151] Device: The iOS or Android® device used by the user.

[0152] 2. Software

[0153] Framework: Flask (a web framework written in Python)

[0154] HTTP Library: Requests (Python's HTTP library)

[0155] Generative AI: OpenAI API, etc.

[0156] The server receives the user's inquiry data, converts it to JSON format, and works with a generative AI API to generate an appropriate response. The generated response is converted back to JSON format and sent to the user's device. The device then displays this response to the user.

[0157] Specific example

[0158] The following example prompt will be used to generate a response from the generative AI.

[0159] Example of a prompt:

[0160] User inquiry: "What is the best way to use the specified product?"

[0161] This system enables the rapid and accurate automation of customer inquiries at virtual stores, leading to improved customer satisfaction. Furthermore, because responses are generated automatically without the need for human intervention, it also reduces labor and costs.

[0162] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0163] Step 1:

[0164] The user enters information.

[0165] Users enter their inquiries using the user interface of a smartphone application. Specifically, they enter information into multiple input fields in an inquiry form and press the submit button. The input data is in text format and may include questions such as, "What is the best way to use the specified product?"

[0166] Step 2:

[0167] The device converts the data to JSON format.

[0168] The terminal converts the data entered by the user into JSON format using methods such as JavaScript. This conversion structures the data, making it easier to send to the server. For example, the input data is converted into JSON format as follows:

[0169] json

[0170] {

[0171] "query": "What is the best way to use the specified product?"

[0172] }

[0173] Step 3:

[0174] Send data to the server

[0175] The terminal sends the converted JSON data to the server using an HTTP request. The transmitted data is then passed directly to the analysis tool.

[0176] Step 4:

[0177] The server analyzes the data.

[0178] The server analyzes the received JSON data using parsing tools. This analysis helps understand what information the query requires. Specifically, it extracts the contents of the "query" field of the data and prepares it for the next processing step.

[0179] Step 5:

[0180] Call the API of a generative AI.

[0181] The server sends the analyzed query content as a request to the generative AI's API. Using the API integration method, the query content is sent as a prompt message to the OpenAI API. An example of a prompt message sent is as follows:

[0182] User inquiry: "What is the best way to use the specified product?"

[0183] Step 6:

[0184] Generative AI generates responses.

[0185] The generative AI generates a response based on the received prompt. This response includes specific advice, such as "This product is best used in the following procedure." The generated response is sent to the server in JSON format.

[0186] Step 7:

[0187] The server converts the response back to JSON format.

[0188] The server converts the response received from the generative AI back into JSON format. This response data is also in a structured format, for example, as follows:

[0189] json

[0190] {

[0191] "Response": "This product is best used in the following way:"

[0192] }

[0193] Step 8:

[0194] The device receives a response.

[0195] The terminal receives response data in JSON format sent from the server. It then parses this data and prepares to display the necessary information on the screen.

[0196] Step 9:

[0197] The terminal displays a response to the user.

[0198] The terminal displays the received response data to the user through a display mechanism. The user can see specific advice on the screen, such as, "This product is best used in the following way."

[0199] This series of processes allows users to receive quick and appropriate responses, streamlining customer inquiries in virtual stores.

[0200] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0201] This invention relates to a system in which a user inputs information, and a response is provided by combining a generative AI and an emotion recognition means. The specific program processing of this system is described below in natural language, along with concrete examples.

[0202] System Overview

[0203] 1. Providing an input form

[0204] The system provides an input form that users can access through a web browser. This form has multiple input fields where users enter the necessary information.

[0205] 2. Data Analysis

[0206] When a user enters data into an input form and presses the submit button, the device uses JavaScript to collect the form data and convert it into JSON format. This ensures that the data is sent to the server efficiently.

[0207] 3. Emotion recognition

[0208] The device is equipped with emotion recognition capabilities to analyze the user's facial expressions, voice, or biosignals. The device collects and analyzes the user's emotion data. This emotion data is also converted to JSON format and sent to the server.

[0209] 4. Sending data and receiving responses

[0210] The server receives data in JSON format sent from the terminal. The server analyzes the received data and sends a request to the generative AI's API. The generative AI generates a response based on the input data. The server receives this response, converts it back to JSON format, and sends it back to the terminal.

[0211] 5. Display of response

[0212] The device receives response data sent back from the server. The received data is analyzed and displayed in the browser in a user-friendly format. Furthermore, customized feedback based on the user's emotions is also displayed.

[0213] Specific example

[0214] 1. Using the input form

[0215] A user (for example, an employee of the General Affairs Department of the city hall) enters "General Affairs Department of City Hall" in the "Role" field, enters "I would like to know the progress of paperless operations" in the "Request" field, and presses the submit button.

[0216] 2. Collection of emotional data

[0217] The emotion recognition system identifies emotions such as "interest" or "confusion" from the user's facial expressions and voice. This emotion data is also sent to the server.

[0218] 3. Data transmission and analysis

[0219] The terminal converts input data and sentiment data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI asking "How should the general affairs department of the city hall proceed with paperless operations?"

[0220] 4. Response from a generative AI

[0221] The generative AI generates advice on specific methods for implementing a paperless system based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal.

[0222] 5. Display of response

[0223] The device analyzes the response data it receives and displays it to the user as specific advice, such as "To proceed with paperless operations, please take the following steps..." It also displays customized feedback, including additional explanations and links to resources, based on the user's emotions, such as "interest" or "confusion." The user can then take specific actions based on this feedback.

[0224] This system can streamline inefficient processes caused by outdated practices in government offices, thereby improving productivity. Users can easily input information and receive specific suggestions and emotion-based feedback from generative AI, leading to improved work processes, reduced overtime, and a shift towards a paperless environment.

[0225] The following describes the processing flow.

[0226] Step 1:

[0227] The user opens a web browser and accesses the specified URL. This displays an input form.

[0228] Step 2:

[0229] The user enters the necessary information into each field of the input form (e.g., "Role", "Request").

[0230] Step 3:

[0231] The user clicks the "Submit" button. This collects the input data.

[0232] Step 4:

[0233] The JavaScript on the device converts the collected data into JSON format.

[0234] Step 5:

[0235] The device sends JSON data to the server as a POST request.

[0236] Step 6:

[0237] The server receives the POST request and extracts JSON data from the request body.

[0238] Step 7:

[0239] The server parses the extracted JSON data and splits it into individual fields (e.g., role, request).

[0240] Step 8:

[0241] The user's webcam, microphone, and other sensors collect the user's facial expressions, voice, and biosignals.

[0242] Step 9:

[0243] The emotion recognition system analyzes the collected data to identify the user's emotions.

[0244] Step 10:

[0245] The device converts the emotional data into JSON format and sends it to the server.

[0246] Step 11:

[0247] The server receives and analyzes emotional data.

[0248] Step 12:

[0249] The server sends the analysis data and emotion data as requests to the generative AI's API.

[0250] Step 13:

[0251] A generative AI receives a request and generates a response based on the input data and sentiment data.

[0252] Step 14:

[0253] The generative AI generates a response which is then sent back to the server.

[0254] Step 15:

[0255] The server receives the response from the generative AI and converts it into the required format.

[0256] Step 16:

[0257] The server returns the converted response data to the terminal in JSON format.

[0258] Step 17:

[0259] The terminal receives and parses the JSON data sent back from the server.

[0260] Step 18:

[0261] The data analyzed by the device is embedded in an HTML element to be displayed visually to the user.

[0262] Step 19:

[0263] The device also displays customized feedback based on the user's emotions.

[0264] Step 20:

[0265] Users review the displayed response data and customized feedback to take concrete actions for business improvement and productivity enhancement.

[0266] (Example 2)

[0267] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0268] In modern business processes, there is a need for systems that generate appropriate responses to information entered by users. However, conventional systems cannot take user emotions into account, making it difficult to provide specific feedback tailored to individual needs and feelings. Furthermore, efficient and highly accurate processing is required in data analysis and exchange with generative artificial intelligence. To solve these problems, a response generation system that incorporates user emotion data is necessary.

[0269] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0270] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, and a data transmission and reception means for sending data to a generative artificial intelligence and receiving a response. This enables the provision of customized feedback based on the user's emotions by including an emotion recognition means for collecting and analyzing the user's emotion data, a data transmission and reception means for sending the emotion data to the generative artificial intelligence, and a display means for displaying the received response to the user.

[0271] "User interface means" refers to an interface for users to input information, and includes devices such as input forms and buttons.

[0272] "Analysis means" refers to software or hardware devices for analyzing information input through a user interface means.

[0273] "Generative artificial intelligence" refers to artificial intelligence technologies that generate responses based on user input, and specifically includes natural language processing models.

[0274] A "data transmission and reception means" is a device that has network communication capabilities for transmitting information analyzed by an analysis means to a generative artificial intelligence and receiving a response.

[0275] "Display means" refers to devices such as displays and monitors that visually display responses received by data transmission means to the user.

[0276] "Emotion recognition means" refers to software or hardware devices that analyze a user's facial expressions, voice, or biosignals to collect emotional data.

[0277] "Structured data format" is a general term for data formats that represent data in a standardized format such as JSON and allow for its exchange with other data.

[0278] An "input form" is a form with multiple different input fields, and is an interface that users use to enter specific information.

[0279] This invention is a system in which a user inputs information and a response is provided by combining generative artificial intelligence and emotion recognition means. In this system, the user can input information via a web browser and receive a response from generative artificial intelligence. Furthermore, by collecting the user's emotion data and reflecting it in the response, the system provides more personalized feedback.

[0280] Specifically, the following hardware and software will be used.

[0281] Hardware:

[0282] 1. Server - A computer used for high-speed and stable data processing and communication.

[0283] 2. Device - A device used by the user, such as a PC, smartphone, or tablet.

[0284] 3. Camera and microphone - An input device for capturing the user's expressions and voice.

[0285] Software:

[0286] 1. Web browser - Software used as user interface means.

[0287] 2. HTML, CSS, JavaScript - Languages for constructing input forms and user interfaces.

[0288] 3. OpenCV, TENSORFLOW (registered trademark) - Libraries used as emotion recognition means.

[0289] 4. Generative AI API - An interface of artificial intelligence used to generate responses (e.g., GPT-3 (registered trademark)).

[0290] Specific examples are shown below:

[0291] 1. User information input:

[0292] A user (e.g., a clerk in the General Affairs Section of a city hall) enters "General Affairs Section of the city hall" as the "role" field and "Want to know the progress of paperless operation" as the "request" field in the input form on the web browser. Then, the user clicks the "Send" button.

[0293] 2. Data analysis and transmission:

[0294] The terminal uses JavaScript to collect the input information and converts it into JSON format using the JSON.stringify() function. The converted data is sent to the server as an HTTP POST request.

[0295] 3. Collection and analysis of emotion data:

[0296] The device uses its camera and microphone to capture the user's facial expressions and voice, and performs emotion analysis using OpenCV and TensorFlow. The analysis results (e.g., "interested" or "confused") are converted to JSON format and sent to the server.

[0297] 4. Data transmission to generative artificial intelligence:

[0298] The server analyzes the user's input data and sentiment data, and sends requests to a generative artificial intelligence API (e.g., GPT-3). The following is an example of a prompt:

[0299] Text format: "How should the general affairs department of the city hall proceed with paperless operations?"

[0300] Text format: "How can we provide user feedback based on their emotions?"

[0301] 5. Receiving and displaying responses:

[0302] The server receives the response from the generative artificial intelligence, converts it to JSON format, and sends it to the terminal. The terminal parses this and displays it in a user-friendly format in the browser. For example, it might display advice such as, "To proceed with paperless operations, follow these steps..." It also displays additional customized feedback based on the user's interests and concerns.

[0303] In this way, users can easily input information and receive specific suggestions and emotion-based feedback from generative artificial intelligence. This can lead to improvements and increased efficiency in their work.

[0304] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0305] Step 1:

[0306] Providing an input form

[0307] The server provides an input form for the user to input information. The server generates the input form using HTML, CSS, and JavaScript and sends it to the user's web browser.

[0308] Input: HTML, CSS, and JavaScript code by the server.

[0309] Output: An input form displayed on the user's web browser.

[0310] Specific operations:

[0311] The user inputs information such as "role" and "request" into this form. For example, input "General Affairs Section of the City Hall" for "role" and "Want to know the progress of paperless transformation" for "request".

[0312] Step 2:

[0313] Data collection and transmission

[0314] When the user clicks the "Submit" button, the terminal uses JavaScript to collect the input information. The collected data is converted into JSON format and sent to the server as an HTTP POST request.

[0315] Input: Information input by the user.

[0316] Output: Data converted into JSON format is sent to the server.

[0317] Specific operations:

[0318] The terminal uses an event listener to detect the click of the "Submit" button, obtains the form data with JavaScript, and converts it into JSON format using the JSON.stringify() function. This is sent to the server as an HTTP POST request.

[0319] Step 3:

[0320] Collection and analysis of emotional data

[0321] The device uses its camera and microphone to capture the user's facial expressions and voice, and analyzes them using emotion recognition technology. The analysis results are also converted into JSON format and sent to the server.

[0322] Input: Real-time data from camera and microphone.

[0323] Output: The analyzed emotion data is converted to JSON format and sent to the server.

[0324] Specific actions:

[0325] The device uses OpenCV and TensorFlow to analyze the user's facial expressions and voice, collecting emotional data such as "interest" and "confusion." This data is then converted to JSON format and sent to the server as an additional HTTP POST request.

[0326] Step 4:

[0327] Sending data to generative artificial intelligence

[0328] The server analyzes user input data and sentiment data and sends a request to a generative artificial intelligence. It generates a request that includes a prompt and sentiment data.

[0329] Input: User input data and sentiment data.

[0330] Output: Request sent to the generative artificial intelligence.

[0331] Specific actions:

[0332] The server analyzes the received data using Python, Node.js, etc., and generates appropriate prompts for a generative artificial intelligence (e.g., GPT-3). For example, it might generate a prompt such as "How should the general affairs department of the city hall proceed with paperless operations?" and send it to the generative AI.

[0333] Step 5:

[0334] Receiving responses and sending data from generative artificial intelligence.

[0335] The server receives the response from the generative artificial intelligence and converts it into JSON format. It then sends the converted data to the terminal.

[0336] Input: Response data from a generative artificial intelligence.

[0337] Output: JSON-formatted response data sent to the terminal.

[0338] Specific actions:

[0339] The server receives the response from the generative artificial intelligence and converts it into JSON format. This converted data is then sent to the terminal as an HTTP response.

[0340] Step 6:

[0341] Display of response and feedback

[0342] The device parses the received JSON response data and displays it in a user-friendly format on the browser. It also provides customized feedback based on the user's emotions.

[0343] Input: JSON-formatted response data received from the server.

[0344] Output: Analyzed response data and sentiment-based feedback displayed in the user's browser.

[0345] Specific actions:

[0346] The JSON data received by the device is displayed in the browser using HTML / CSS. For example, it might display specific advice such as, "To proceed with paperless operations, follow these steps..." Furthermore, it displays links to additional information and resources based on the user's "interests" and "concerns."

[0347] This system allows users to easily input information and receive specific suggestions and emotion-based feedback from generative artificial intelligence. This can lead to improvements and increased efficiency in work processes.

[0348] (Application Example 2)

[0349] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0350] Traditional in-store customer service often suffers from insufficient personalization, making it difficult to enhance customer satisfaction. Furthermore, store staff struggle to accurately understand customers' emotions and interests, hindering their ability to provide optimal product recommendations. In addition, inconsistent staff service can often degrade the quality of the customer experience.

[0351] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0352] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, a data transmission and reception means for transmitting the analyzed information to a generative AI and receiving a response, a display means for displaying the received response to the user, an emotion recognition means for analyzing the user's emotions using an facial recognition camera, and a data transmission and reception means for transmitting emotion data to the generative AI. This enables personalized product suggestions based on customer emotions and interests in physical stores. Furthermore, it allows store staff to provide consistently high-quality customer service, thereby improving customer satisfaction.

[0353] "User interface means" refers to an interface that a user uses to input information.

[0354] "Analysis means" refers to means for analyzing information input through a user interface.

[0355] "Data transmission and reception means" refers to means for transmitting information analyzed by the analysis means to the generative AI and receiving a response from the generative AI.

[0356] "Display means" refers to means for displaying the response received by the data transmission means to the user.

[0357] An "input form" refers to a web form that has multiple different input fields for users to enter information.

[0358] A "facial recognition camera" refers to a camera device that captures a user's facial expressions and analyzes emotional data.

[0359] "Emotion recognition means" refers to a method for analyzing a user's emotions using a facial recognition camera.

[0360] "Generative AI" refers to an artificial intelligence system that generates responses based on input data.

[0361] A "custom support app for physical stores" refers to a mobile application used by store staff in physical stores to provide optimal product recommendations through dialogue with customers.

[0362] One embodiment of this invention is a system for personalizing customer service in physical stores and improving customer satisfaction. This system takes user input and provides a response by combining generative AI and emotion recognition means.

[0363] First, the system provides a user interface. This interface is for use by in-store staff via smartphones or tablets and provides an input form with multiple input fields for entering customer information, purchase history, interests, and more.

[0364] Next, the information entered on the terminal is analyzed using an analysis tool. This analysis tool primarily analyzes text data to understand customer needs and interests. This data is converted to JSON format and sent to the server. The server is equipped with a data transmission and reception mechanism, which sends the analyzed information to a generative AI and receives the response.

[0365] This system includes a facial recognition camera that captures the user's facial expressions. The facial expression data is analyzed using emotion recognition technology to understand the user's emotions. This emotion data is also converted to JSON format and sent to the server. All information, including the emotion data, is processed by generative AI to generate appropriate product suggestions and customized feedback.

[0366] As a concrete example, consider a scenario where a user shows interest in "casual fashion" in a physical store. Based on the user's purchase history, interests, and the emotion of "interest" inferred from their facial expressions, the generative AI makes optimal product suggestions. The prompt text in this case would be as follows:

[0367] The customer's name is Taro Yamada. His purchase history includes shirts and shoes, and his interest lies in casual fashion. He appears interested based on his expression. Please provide him with the most suitable product suggestions and additional information.

[0368] The server converts the response received from the generative AI into JSON format and sends it back to the terminal. The terminal analyzes the received data and displays it as information for in-store staff to make suggestions to customers. In addition, more personalized feedback is provided based on sentiment data. This system not only improves customer satisfaction but also ensures consistent service quality from store staff.

[0369] The main hardware and software used to implement this system include smartphones, tablets, facial recognition cameras, and generative AI models (e.g., GPT-3). Additionally, server-side APIs and associated analysis software are required for data transmission, reception, and analysis.

[0370] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0371] Step 1:

[0372] Users use their smartphones in physical stores to input customer information through a user interface. Specifically, they enter their purchase history and interests into input fields. An example of input data is "Name: Taro Yamada," "Purchase History: Shirts, Shoes," and "Interests: Casual Fashion." The entered data is temporarily stored internally.

[0373] Step 2:

[0374] The device uses a facial recognition camera to capture customer facial data. The facial data captured by the camera is transmitted in real time to an emotion recognition system for analysis. For example, the emotion of "interested" is analyzed. This emotion data is also temporarily stored internally.

[0375] Step 3:

[0376] The terminal converts the analyzed input data and sentiment data into JSON format. The converted data is in the following format as an example:

[0377] json

[0378] {

[0379] "name": "Yamada Taro",

[0380] "purchase_history": ["shirt", "shoes"],

[0381] "interests": "casual fashion",

[0382] "emotion": "Interested"

[0383] }

[0384] Step 4:

[0385] The terminal sends the data, converted to JSON format, to the server. A data transmission method is used for this process. The data is sent as an HTTP POST request.

[0386] Step 5:

[0387] The server analyzes the received data and sends a request to a generative AI model (e.g., GPT-3). This request includes data and a corresponding prompt. Specifically, the prompt used might be: "The customer's name is Yamada Taro. His purchase history includes shirts and shoes, and his interest is in casual fashion. His facial expression suggests he is interested. Please provide him with the most suitable product suggestions and additional information."

[0388] Step 6:

[0389] The generative AI model generates a response based on the prompt and data. An example of the generated response might be, "We recommend these new jackets and sneakers, perfect for casual fashion. For more information, please refer to this link." This response data is then sent back to the server.

[0390] Step 7:

[0391] The server receives the response from the generative AI model and converts it into JSON format. The converted response data is in the following format:

[0392] json

[0393] {

[0394] "recommendations": ["New jacket", "Sneakers"],

[0395] "additional_info": "For more detailed information, please refer to this link."

[0396] }

[0397] Step 8:

[0398] The server sends response data in JSON format back to the terminal. The returned data is then sent again as an HTTP POST request.

[0399] Step 9:

[0400] The device analyzes the received response data and displays it to the user. The displayed content may include customized feedback based on the customer's sentiment. For example, it might display: "For those interested in casual fashion, we recommend our new jackets and sneakers. For more information, please see this link."

[0401] The above steps enable personalized customer service in physical stores.

[0402] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0403] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0404] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0405] [Second Embodiment]

[0406] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0407] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0408] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0409] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0410] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0411] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0412] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0413] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0414] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0415] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0416] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0417] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0418] This invention relates to a system aimed at improving the productivity of civil servants, in which a user inputs information, which is then analyzed and transmitted to a generative AI, and a response is received. The specific program processing of this system is described below in natural language, along with concrete examples.

[0419] System Overview

[0420] 1. Providing an input form

[0421] The system provides an input form that users can access through a web browser. This form has multiple input fields where users enter the necessary information.

[0422] 2. Data Analysis

[0423] When a user enters data into an input form and presses the submit button, the device uses JavaScript to collect the form data and convert it into JSON format. This ensures that the data is sent to the server efficiently.

[0424] 3. Sending data and receiving responses

[0425] The server receives data in JSON format sent from the terminal. The server analyzes the received data and sends a request to the generative AI's API. The generative AI generates a response based on the input data. The server receives this response, converts it back to JSON format, and sends it back to the terminal.

[0426] 4. Display of response

[0427] The terminal receives response data sent back from the server. The received data is analyzed and displayed in the browser in a format that is easy for the user to understand. This allows the user to obtain specific advice and suggestions for efficiently carrying out their work.

[0428] Specific example

[0429] 1. Using the input form

[0430] A user (for example, an employee of the General Affairs Department of the city hall) enters "General Affairs Department of City Hall" in the "Role" field, enters "I would like to know the progress of paperless operations" in the "Request" field, and presses the submit button.

[0431] 2. Data transmission and analysis

[0432] The terminal converts the input data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI asking "How should the General Affairs Department of the city hall proceed with paperless operations?"

[0433] 3. Response from a generative AI

[0434] The generative AI generates advice on specific methods for implementing a paperless system based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal.

[0435] 4. Display of response

[0436] The terminal analyzes the response data it receives and displays it to the user as specific advice, such as "To proceed with paperless operations, please take the following steps..." The user can then take concrete action based on this information.

[0437] This system can streamline inefficient processes caused by outdated practices in government offices, thereby improving productivity. Users can easily input information and receive specific suggestions from generative AI, leading to improved work processes, reduced overtime, and a move towards a paperless environment.

[0438] The following describes the processing flow.

[0439] Step 1:

[0440] The user opens a web browser and accesses the specified URL. This displays an input form.

[0441] Step 2:

[0442] The user enters the necessary information into each field of the input form (e.g., "Role", "Request").

[0443] Step 3:

[0444] The user clicks the "Submit" button. This collects the input data.

[0445] Step 4:

[0446] The JavaScript on the device converts the collected data into JSON format.

[0447] Step 5:

[0448] The terminal sends the converted JSON data to the server as a POST request.

[0449] Step 6:

[0450] The server receives the POST request and extracts JSON data from the request body.

[0451] Step 7:

[0452] The server parses the extracted JSON data and splits it into individual fields (e.g., role, request).

[0453] Step 8:

[0454] The server sends the analyzed data as a request to the generative AI's API.

[0455] Step 9:

[0456] The generative AI receives the request and generates a response based on the input data.

[0457] Step 10:

[0458] The generative AI generates a response which is then sent back to the server.

[0459] Step 11:

[0460] The server receives the response from the generative AI and converts it into the required format.

[0461] Step 12:

[0462] The server returns the converted response data to the terminal in JSON format.

[0463] Step 13:

[0464] The terminal receives and parses the JSON data sent back from the server.

[0465] Step 14:

[0466] The data analyzed by the device is embedded in an HTML element to be displayed visually to the user.

[0467] Step 15:

[0468] Users review the displayed response data and take specific actions to improve operations and increase productivity.

[0469] (Example 1)

[0470] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0471] In modern public institutions, productivity is low because many tasks are performed manually or on paper. Furthermore, this leads to delays in work progress, slow feedback and decision-making, and ultimately, a decrease in overall operational efficiency. A system is needed to address this challenge and efficiently improve productivity.

[0472] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0473] In this invention, the server includes a user interface means for the user to input information, a conversion means for collecting the information input through the user interface means and converting it into JSON format, a data transmission means for analyzing the JSON format information converted by the conversion means and sending a request to a generative AI model, and a display means for converting the response received from the generative AI model back into JSON format and displaying it to the user. This makes it possible to efficiently analyze the input data and provide appropriate advice to the user.

[0474] A "user interface means" is a tool that provides an interface (UI) for users to input information.

[0475] A "conversion tool" is a tool that collects information entered through a user interface and performs the process of converting it into a specific format (e.g., JSON format).

[0476] A "data transmission means" is a tool that performs the process of analyzing the information converted by the conversion means, sending a request to a generative AI model, and receiving a response.

[0477] A "display means" is a tool that transforms the response received from a generative AI model and displays it in a format that is easy for the user to understand.

[0478] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight data exchange format for structuring and exchanging data.

[0479] A "generative AI model" is an artificial intelligence model that analyzes data provided by users and generates appropriate responses based on that data.

[0480] An "input field" refers to a specific area in a user interface where a user enters information.

[0481] This invention aims to improve the productivity of civil servants by providing a system in which a user inputs information, which is then analyzed and sent to a generative AI model, and a response is received. Specific embodiments of this system are described below.

[0482] Providing an input form

[0483] This system utilizes an input form accessible through a web browser. Specifically, browsers such as Google Chrome and Mozilla Firefox can be used. This input form contains multiple fields where users enter the necessary information. For example, information can be entered in fields such as "Role" and "Requests." The entire input form is constructed using HTML and CSS, and a dynamic user interface is implemented using JavaScript.

[0484] Data analysis

[0485] When a user enters information into an input form and presses the submit button, the device uses JavaScript to collect the form data. Next, this collected data is converted into JSON format. JSON is a lightweight data exchange format for efficient data exchange, and the data is sent to the server in this format.

[0486] Sending data and receiving responses

[0487] The JSON data sent from the device is received by the server. The server receives this data via an HTTP request and first verifies its contents. Next, it sends the verified data to a generative AI model. The generative AI model generates an appropriate response based on the user's input data. The response generated by the generative AI model is sent back to the server. The server receives this response, converts it back to JSON format if necessary, and sends it back to the device.

[0488] Display of response

[0489] The response data sent from the server to the terminal is received again by the terminal. The terminal parses this response data and uses JavaScript to display it in a user-friendly format on the browser. This allows the user to obtain specific advice and suggestions to efficiently carry out their work.

[0490] Specific example

[0491] For example, an employee of the General Affairs Division of a city hall enters "General Affairs Division of City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field, then presses the submit button. The terminal converts the input data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI model asking "How will the General Affairs Division of City Hall proceed with paperless initiatives?" The generative AI model generates advice on specific methods for proceeding with paperless initiatives based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal. The terminal analyzes the received response data and displays it to the user as specific advice such as "To proceed with paperless initiatives, follow these steps..."

[0492] Example of a prompt

[0493] Examples of prompt statements include the following:

[0494] "I'm a staff member in the General Affairs Department. I'd like to promote a paperless system, but could you tell me what steps I should take?"

[0495] By inputting such prompt messages into the AI ​​model, advice can be obtained on specific methods for implementing a paperless system.

[0496] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0497] Step 1:

[0498] The user accesses the system using a web browser. The user enters information into the input form. For example, the user might enter "General Affairs Department, City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field. The user proceeds to the next step by pressing the "Submit" button. The input is the information entered by the user into the input form, and the output is the data entered into the input form converted into JSON format.

[0499] Step 2:

[0500] The device uses JavaScript to collect form data and convert it into JSON format. Specifically, JavaScript retrieves data from input fields and converts it into a JSON object. For example, it might be converted to a format like {"Role": "General Affairs Department, City Hall", "Request": "I want to know the progress of paperless initiatives"}. The input is information entered by the user, and the output is data generated in JSON format.

[0501] Step 3:

[0502] The device uses an HTTP POST request to send the generated JSON data to the server. Specifically, it uses JavaScript's fetch API or similar to send data to a specific endpoint on the server. For example, it sends JSON data to POST / api / request. The input is data in JSON format, and the output is the request that was sent to the server.

[0503] Step 4:

[0504] The server receives an HTTP request and parses the JSON data in the request body. Specifically, the server-side program (e.g., Node.js or Python) parses the request data. For example, it parses the JSON data and extracts the values ​​of each field. The input is the JSON data sent from the terminal, and the output is the parsed data.

[0505] Step 5:

[0506] The server sends the analyzed data to the generative AI model. Specifically, it issues an HTTP request to the generative AI model's API. For example, it sends data containing a prompt to the generative AI model's endpoint. The input is the analyzed data, and the output is the request that was sent to the generative AI model.

[0507] Step 6:

[0508] The generative AI model generates a response and sends it back to the server. Specifically, the generative AI model analyzes the received data and generates an appropriate response based on that analysis. For example, it might generate a response such as, "To proceed with paperless operations, please follow these steps..." The input is prompt data sent from the server, and the output is the generated response data.

[0509] Step 7:

[0510] The server converts the response data received from the generative AI model back into JSON format and sends it to the terminal. Specifically, it structures the received response data and converts it into JSON format. The input is the response data received from the generative AI model, and the output is the data converted into JSON format.

[0511] Step 8:

[0512] The terminal receives and parses response data from the server. Specifically, it uses JavaScript to parse JSON data and retrieve data for each item. The input is the JSON data sent from the server, and the output is the parsed response data.

[0513] Step 9:

[0514] The terminal analyzes response data and displays it to the user. Specifically, it inserts the response data into an HTML element and displays it in the browser. For example, it might display specific advice such as, "To proceed with paperless operations, follow these steps..." The input is the analyzed response data, and the output is the content displayed to the user.

[0515] (Application Example 1)

[0516] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0517] In virtual stores, there is a need to respond to customer inquiries quickly and appropriately. However, conventional systems lack sufficient automation to provide optimal answers tailored to the content of inquiries, making it difficult to improve customer satisfaction. Furthermore, even simple inquiries required human intervention, resulting in increased effort and cost. To solve this problem, there is a need to automate inquiry handling using generative AI.

[0518] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0519] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, a data transmission and reception means for sending the analyzed information to a generative AI and receiving a response, and an API integration means for calling the generative AI's API and automatically generating appropriate responses to customer inquiries in the virtual store. This makes it possible to automate customer inquiries in the virtual store quickly and accurately.

[0520] "User interface means" refers to an interface for users to input information, and specifically includes input forms, buttons, and the like.

[0521] "Analysis means" refers to means for analyzing information input through a user interface and handling it as data.

[0522] "Data transmission and reception means" refers to means for transmitting information analyzed by the analysis means to the generative AI and receiving a response from the generative AI.

[0523] "Display means" refers to means for displaying the response received by the data transmission means to the user.

[0524] "Generative AI" refers to artificial intelligence that automatically generates appropriate responses based on input data.

[0525] "API integration means" refers to a method for calling the API of a generative AI and exchanging data in conjunction with an external system.

[0526] A "virtual store" is an online store that provides goods and services via the internet.

[0527] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a method for representing data in a lightweight and structured format.

[0528] As a concrete example of implementing this invention, a customer service system for a virtual store is provided. The system includes a user interface means for users to input information. This user interface means includes an input form with multiple different input fields, allowing customers to easily input their inquiries.

[0529] The input information is analyzed by an analysis tool. This analysis tool uses a programming language such as JavaScript to convert the input data into JSON format. The converted data is then sent to the server via the analysis tool.

[0530] The server processes the received JSON data using a data transmission and reception mechanism and calls the generative AI API. This generative AI API automatically generates an appropriate response based on the input data, and can use, for example, the OpenAI API. The server converts the response from the generative AI back into JSON format and sends it back to the terminal.

[0531] The terminal displays the received response data to the customer via a display device. This series of processes enables quick and efficient customer service in the virtual store.

[0532] Hardware and software to use

[0533] 1. Hardware

[0534] Server: Cloud server (e.g., Amazon Web Services, Google Cloud Platform)

[0535] Device: The iOS or Android device used by the user.

[0536] 2. Software

[0537] Framework: Flask (a web framework written in Python)

[0538] HTTP Library: Requests (Python's HTTP library)

[0539] Generative AI: OpenAI API, etc.

[0540] The server receives the user's inquiry data, converts it to JSON format, and works with a generative AI API to generate an appropriate response. The generated response is converted back to JSON format and sent to the user's device. The device then displays this response to the user.

[0541] Specific example

[0542] The following example prompt will be used to generate a response from the generative AI.

[0543] Example of a prompt:

[0544] User inquiry: "What is the best way to use the specified product?"

[0545] This system enables the rapid and accurate automation of customer inquiries at virtual stores, leading to improved customer satisfaction. Furthermore, because responses are generated automatically without the need for human intervention, it also reduces labor and costs.

[0546] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0547] Step 1:

[0548] The user enters information.

[0549] Users enter their inquiries using the user interface of a smartphone application. Specifically, they enter information into multiple input fields in an inquiry form and press the submit button. The input data is in text format and may include questions such as, "What is the best way to use the specified product?"

[0550] Step 2:

[0551] The device converts the data to JSON format.

[0552] The terminal converts the data entered by the user into JSON format using methods such as JavaScript. This conversion structures the data, making it easier to send to the server. For example, the input data is converted into JSON format as follows:

[0553] json

[0554] {

[0555] "query": "What is the best way to use the specified product?"

[0556] }

[0557] Step 3:

[0558] Send data to the server

[0559] The terminal sends the converted JSON data to the server using an HTTP request. The transmitted data is then passed directly to the analysis tool.

[0560] Step 4:

[0561] The server analyzes the data.

[0562] The server analyzes the received JSON data using parsing tools. This analysis helps understand what information the query requires. Specifically, it extracts the contents of the "query" field of the data and prepares it for the next processing step.

[0563] Step 5:

[0564] Call the API of a generative AI.

[0565] The server sends the analyzed query content as a request to the generative AI's API. Using the API integration method, the query content is sent as a prompt message to the OpenAI API. An example of a prompt message sent is as follows:

[0566] User inquiry: "What is the best way to use the specified product?"

[0567] Step 6:

[0568] Generative AI generates responses.

[0569] The generative AI generates a response based on the received prompt. This response includes specific advice, such as "This product is best used in the following procedure." The generated response is sent to the server in JSON format.

[0570] Step 7:

[0571] The server converts the response back to JSON format.

[0572] The server converts the response received from the generative AI back into JSON format. This response data is also in a structured format, for example, as follows:

[0573] json

[0574] {

[0575] "Response": "This product is best used in the following way:"

[0576] }

[0577] Step 8:

[0578] The device receives a response.

[0579] The terminal receives response data in JSON format sent from the server. It then parses this data and prepares to display the necessary information on the screen.

[0580] Step 9:

[0581] The terminal displays a response to the user.

[0582] The terminal displays the received response data to the user through a display mechanism. The user can see specific advice on the screen, such as, "This product is best used in the following way."

[0583] This series of processes allows users to receive quick and appropriate responses, streamlining customer inquiries in virtual stores.

[0584] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0585] This invention relates to a system in which a user inputs information, and a response is provided by combining a generative AI and an emotion recognition means. The specific program processing of this system is described below in natural language, along with concrete examples.

[0586] System Overview

[0587] 1. Providing an input form

[0588] The system provides an input form that users can access through a web browser. This form has multiple input fields where users enter the necessary information.

[0589] 2. Data Analysis

[0590] When a user enters data into an input form and presses the submit button, the device uses JavaScript to collect the form data and convert it into JSON format. This ensures that the data is sent to the server efficiently.

[0591] 3. Emotion recognition

[0592] The device is equipped with emotion recognition capabilities to analyze the user's facial expressions, voice, or biosignals. The device collects and analyzes the user's emotion data. This emotion data is also converted to JSON format and sent to the server.

[0593] 4. Sending data and receiving responses

[0594] The server receives data in JSON format sent from the terminal. The server analyzes the received data and sends a request to the generative AI's API. The generative AI generates a response based on the input data. The server receives this response, converts it back to JSON format, and sends it back to the terminal.

[0595] 5. Display of response

[0596] The device receives response data sent back from the server. The received data is analyzed and displayed in the browser in a user-friendly format. Furthermore, customized feedback based on the user's emotions is also displayed.

[0597] Specific example

[0598] 1. Using the input form

[0599] A user (for example, an employee of the General Affairs Department of the city hall) enters "General Affairs Department of City Hall" in the "Role" field, enters "I would like to know the progress of paperless operations" in the "Request" field, and presses the submit button.

[0600] 2. Collection of emotional data

[0601] The emotion recognition system identifies emotions such as "interest" or "confusion" from the user's facial expressions and voice. This emotion data is also sent to the server.

[0602] 3. Data transmission and analysis

[0603] The terminal converts input data and sentiment data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI asking "How should the general affairs department of the city hall proceed with paperless operations?"

[0604] 4. Response from a generative AI

[0605] The generative AI generates advice on specific methods for implementing a paperless system based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal.

[0606] 5. Display of response

[0607] The device analyzes the response data it receives and displays it to the user as specific advice, such as "To proceed with paperless operations, please take the following steps..." It also displays customized feedback, including additional explanations and links to resources, based on the user's emotions, such as "interest" or "confusion." The user can then take specific actions based on this feedback.

[0608] This system can streamline inefficient processes caused by outdated practices in government offices, thereby improving productivity. Users can easily input information and receive specific suggestions and emotion-based feedback from generative AI, leading to improved work processes, reduced overtime, and a shift towards a paperless environment.

[0609] The following describes the processing flow.

[0610] Step 1:

[0611] The user opens a web browser and accesses the specified URL. This displays an input form.

[0612] Step 2:

[0613] The user enters the necessary information into each field of the input form (e.g., "Role", "Request").

[0614] Step 3:

[0615] The user clicks the "Submit" button. This collects the input data.

[0616] Step 4:

[0617] The JavaScript on the device converts the collected data into JSON format.

[0618] Step 5:

[0619] The device sends JSON data to the server as a POST request.

[0620] Step 6:

[0621] The server receives the POST request and extracts JSON data from the request body.

[0622] Step 7:

[0623] The server parses the extracted JSON data and splits it into individual fields (e.g., role, request).

[0624] Step 8:

[0625] The user's webcam, microphone, and other sensors collect the user's facial expressions, voice, and biosignals.

[0626] Step 9:

[0627] The emotion recognition system analyzes the collected data to identify the user's emotions.

[0628] Step 10:

[0629] The device converts the emotional data into JSON format and sends it to the server.

[0630] Step 11:

[0631] The server receives and analyzes emotional data.

[0632] Step 12:

[0633] The server sends the analysis data and emotion data as requests to the generative AI's API.

[0634] Step 13:

[0635] A generative AI receives a request and generates a response based on the input data and sentiment data.

[0636] Step 14:

[0637] The generative AI generates a response which is then sent back to the server.

[0638] Step 15:

[0639] The server receives the response from the generative AI and converts it into the required format.

[0640] Step 16:

[0641] The server returns the converted response data to the terminal in JSON format.

[0642] Step 17:

[0643] The terminal receives and parses the JSON data sent back from the server.

[0644] Step 18:

[0645] The data analyzed by the device is embedded in an HTML element to be displayed visually to the user.

[0646] Step 19:

[0647] The device also displays customized feedback based on the user's emotions.

[0648] Step 20:

[0649] Users review the displayed response data and customized feedback to take concrete actions for business improvement and productivity enhancement.

[0650] (Example 2)

[0651] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0652] In modern business processes, there is a need for systems that generate appropriate responses to information entered by users. However, conventional systems cannot take user emotions into account, making it difficult to provide specific feedback tailored to individual needs and feelings. Furthermore, efficient and highly accurate processing is required in data analysis and exchange with generative artificial intelligence. To solve these problems, a response generation system that incorporates user emotion data is necessary.

[0653] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0654] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, and a data transmission and reception means for sending data to a generative artificial intelligence and receiving a response. This enables the provision of customized feedback based on the user's emotions by including an emotion recognition means for collecting and analyzing the user's emotion data, a data transmission and reception means for sending the emotion data to the generative artificial intelligence, and a display means for displaying the received response to the user.

[0655] "User interface means" refers to an interface for users to input information, and includes devices such as input forms and buttons.

[0656] "Analysis means" refers to software or hardware devices for analyzing information input through a user interface means.

[0657] "Generative artificial intelligence" refers to artificial intelligence technologies that generate responses based on user input, and specifically includes natural language processing models.

[0658] A "data transmission and reception means" is a device that has network communication capabilities for transmitting information analyzed by an analysis means to a generative artificial intelligence and receiving a response.

[0659] "Display means" refers to devices such as displays and monitors that visually display responses received by data transmission means to the user.

[0660] "Emotion recognition means" refers to software or hardware devices that analyze a user's facial expressions, voice, or biosignals to collect emotional data.

[0661] "Structured data format" is a general term for data formats that represent data in a standardized format such as JSON and allow for its exchange with other data.

[0662] An "input form" is a form with multiple different input fields, and is an interface that users use to enter specific information.

[0663] This invention is a system in which a user inputs information and a response is provided by combining generative artificial intelligence and emotion recognition means. In this system, the user can input information via a web browser and receive a response from generative artificial intelligence. Furthermore, by collecting the user's emotion data and reflecting it in the response, the system provides more personalized feedback.

[0664] Specifically, the following hardware and software will be used.

[0665] Hardware:

[0666] 1. Server - A computer used for high-speed and stable data processing and communication.

[0667] 2. Device - A device used by the user, such as a PC, smartphone, or tablet.

[0668] 3. Camera and microphone - Input devices for capturing the user's facial expressions and voice.

[0669] software:

[0670] 1. Web browser - Software used as a user interface.

[0671] 2. HTML, CSS, JavaScript - Languages ​​used to build input forms and user interfaces.

[0672] 3. OpenCV, TensorFlow - Libraries used for emotion recognition.

[0673] 4. Generative AI APIs - AI interfaces used to generate responses (e.g., GPT-3).

[0674] Here are some specific examples:

[0675] 1. User information input:

[0676] A user (for example, an employee of the general affairs department of a city hall) enters "General Affairs Department of City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field of an input form on a web browser. Then they press the "Submit" button.

[0677] 2. Data analysis and transmission:

[0678] The device uses JavaScript to collect the entered information and converts it to JSON format using the JSON.stringify() function. The converted data is then sent to the server as an HTTP POST request.

[0679] 3. Collection and analysis of emotional data:

[0680] The device uses its camera and microphone to capture the user's facial expressions and voice, and performs emotion analysis using OpenCV and TensorFlow. The analysis results (e.g., "interested" or "confused") are converted to JSON format and sent to the server.

[0681] 4. Data transmission to generative artificial intelligence:

[0682] The server analyzes the user's input data and sentiment data, and sends requests to a generative artificial intelligence API (e.g., GPT-3). The following is an example of a prompt:

[0683] Text format: "How should the general affairs department of the city hall proceed with paperless operations?"

[0684] Text format: "How can we provide user feedback based on their emotions?"

[0685] 5. Receiving and displaying responses:

[0686] The server receives the response from the generative artificial intelligence, converts it to JSON format, and sends it to the terminal. The terminal parses this and displays it in a user-friendly format in the browser. For example, it might display advice such as, "To proceed with paperless operations, follow these steps..." It also displays additional customized feedback based on the user's interests and concerns.

[0687] In this way, users can easily input information and receive specific suggestions and emotion-based feedback from generative artificial intelligence. This can lead to improvements and increased efficiency in their work.

[0688] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0689] Step 1:

[0690] Providing an input form

[0691] The server provides an input form for the user to enter information. The server generates the input form using HTML, CSS, and JavaScript, and sends it to the user's web browser.

[0692] Input: HTML, CSS, and JavaScript code from the server.

[0693] Output: An input form displayed in the user's web browser.

[0694] Specific actions:

[0695] Users enter information such as their "role" and "requests" into this form. For example, they might enter "General Affairs Department, City Hall" for "role" and "I would like to know the progress of the paperless initiative" for "request."

[0696] Step 2:

[0697] Data collection and transmission

[0698] When the user presses the "Submit" button, the device uses JavaScript to collect the entered information. The collected data is converted to JSON format and sent to the server as an HTTP POST request.

[0699] Input: Information entered by the user.

[0700] Output: The data, converted to JSON format, is sent to the server.

[0701] Specific actions:

[0702] The device uses an event listener to detect a click of the "Submit" button, retrieves the form data using JavaScript, and converts it to JSON format using the JSON.stringify() function. This is then sent to the server as an HTTP POST request.

[0703] Step 3:

[0704] Collection and analysis of emotional data

[0705] The device uses its camera and microphone to capture the user's facial expressions and voice, and analyzes them using emotion recognition technology. The analysis results are also converted into JSON format and sent to the server.

[0706] Input: Real-time data from camera and microphone.

[0707] Output: The analyzed emotion data is converted to JSON format and sent to the server.

[0708] Specific actions:

[0709] The device uses OpenCV and TensorFlow to analyze the user's facial expressions and voice, collecting emotional data such as "interest" and "confusion." This data is then converted to JSON format and sent to the server as an additional HTTP POST request.

[0710] Step 4:

[0711] Sending data to generative artificial intelligence

[0712] The server analyzes user input data and sentiment data and sends a request to a generative artificial intelligence. It generates a request that includes a prompt and sentiment data.

[0713] Input: User input data and sentiment data.

[0714] Output: Request sent to the generative artificial intelligence.

[0715] Specific actions:

[0716] The server analyzes the received data using Python, Node.js, etc., and generates appropriate prompts for a generative artificial intelligence (e.g., GPT-3). For example, it might generate a prompt such as "How should the general affairs department of the city hall proceed with paperless operations?" and send it to the generative AI.

[0717] Step 5:

[0718] Receiving responses and sending data from generative artificial intelligence.

[0719] The server receives the response from the generative artificial intelligence and converts it into JSON format. It then sends the converted data to the terminal.

[0720] Input: Response data from a generative artificial intelligence.

[0721] Output: JSON-formatted response data sent to the terminal.

[0722] Specific actions:

[0723] The server receives the response from the generative artificial intelligence and converts it into JSON format. This converted data is then sent to the terminal as an HTTP response.

[0724] Step 6:

[0725] Display of response and feedback

[0726] The device parses the received JSON response data and displays it in a user-friendly format on the browser. It also provides customized feedback based on the user's emotions.

[0727] Input: JSON-formatted response data received from the server.

[0728] Output: Analyzed response data and sentiment-based feedback displayed in the user's browser.

[0729] Specific actions:

[0730] The JSON data received by the device is displayed in the browser using HTML / CSS. For example, it might display specific advice such as, "To proceed with paperless operations, follow these steps..." Furthermore, it displays links to additional information and resources based on the user's "interests" and "concerns."

[0731] This system allows users to easily input information and receive specific suggestions and emotion-based feedback from generative artificial intelligence. This can lead to improvements and increased efficiency in work processes.

[0732] (Application Example 2)

[0733] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0734] Traditional in-store customer service often suffers from insufficient personalization, making it difficult to enhance customer satisfaction. Furthermore, store staff struggle to accurately understand customers' emotions and interests, hindering their ability to provide optimal product recommendations. In addition, inconsistent staff service can often degrade the quality of the customer experience.

[0735] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0736] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, a data transmission and reception means for transmitting the analyzed information to a generative AI and receiving a response, a display means for displaying the received response to the user, an emotion recognition means for analyzing the user's emotions using an facial recognition camera, and a data transmission and reception means for transmitting emotion data to the generative AI. This enables personalized product suggestions based on customer emotions and interests in physical stores. Furthermore, it allows store staff to provide consistently high-quality customer service, thereby improving customer satisfaction.

[0737] "User interface means" refers to an interface that a user uses to input information.

[0738] "Analysis means" refers to means for analyzing information input through a user interface.

[0739] "Data transmission and reception means" refers to means for transmitting information analyzed by the analysis means to the generative AI and receiving a response from the generative AI.

[0740] "Display means" refers to means for displaying the response received by the data transmission means to the user.

[0741] An "input form" refers to a web form that has multiple different input fields for users to enter information.

[0742] A "facial recognition camera" refers to a camera device that captures a user's facial expressions and analyzes emotional data.

[0743] "Emotion recognition means" refers to a method for analyzing a user's emotions using a facial recognition camera.

[0744] "Generative AI" refers to an artificial intelligence system that generates responses based on input data.

[0745] A "custom support app for physical stores" refers to a mobile application used by store staff in physical stores to provide optimal product recommendations through dialogue with customers.

[0746] One embodiment of this invention is a system for personalizing customer service in physical stores and improving customer satisfaction. This system takes user input and provides a response by combining generative AI and emotion recognition means.

[0747] First, the system provides a user interface. This interface is for use by in-store staff via smartphones or tablets and provides an input form with multiple input fields for entering customer information, purchase history, interests, and more.

[0748] Next, the information entered on the terminal is analyzed using an analysis tool. This analysis tool primarily analyzes text data to understand customer needs and interests. This data is converted to JSON format and sent to the server. The server is equipped with a data transmission and reception mechanism, which sends the analyzed information to a generative AI and receives the response.

[0749] This system includes a facial recognition camera that captures the user's facial expressions. The facial expression data is analyzed using emotion recognition technology to understand the user's emotions. This emotion data is also converted to JSON format and sent to the server. All information, including the emotion data, is processed by generative AI to generate appropriate product suggestions and customized feedback.

[0750] As a concrete example, consider a scenario where a user shows interest in "casual fashion" in a physical store. Based on the user's purchase history, interests, and the emotion of "interest" inferred from their facial expressions, the generative AI makes optimal product suggestions. The prompt text in this case would be as follows:

[0751] The customer's name is Taro Yamada. His purchase history includes shirts and shoes, and his interest lies in casual fashion. He appears interested based on his expression. Please provide him with the most suitable product suggestions and additional information.

[0752] The server converts the response received from the generative AI into JSON format and sends it back to the terminal. The terminal analyzes the received data and displays it as information for in-store staff to make suggestions to customers. In addition, more personalized feedback is provided based on sentiment data. This system not only improves customer satisfaction but also ensures consistent service quality from store staff.

[0753] The main hardware and software used to implement this system include smartphones, tablets, facial recognition cameras, and generative AI models (e.g., GPT-3). Additionally, server-side APIs and associated analysis software are required for data transmission, reception, and analysis.

[0754] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0755] Step 1:

[0756] Users use their smartphones in physical stores to input customer information through a user interface. Specifically, they enter their purchase history and interests into input fields. An example of input data is "Name: Taro Yamada," "Purchase History: Shirts, Shoes," and "Interests: Casual Fashion." The entered data is temporarily stored internally.

[0757] Step 2:

[0758] The device uses a facial recognition camera to capture customer facial data. The facial data captured by the camera is transmitted in real time to an emotion recognition system for analysis. For example, the emotion of "interested" is analyzed. This emotion data is also temporarily stored internally.

[0759] Step 3:

[0760] The terminal converts the analyzed input data and sentiment data into JSON format. The converted data is in the following format as an example:

[0761] json

[0762] {

[0763] "name": "Yamada Taro",

[0764] "purchase_history": ["shirt", "shoes"],

[0765] "interests": "casual fashion",

[0766] "emotion": "Interested"

[0767] }

[0768] Step 4:

[0769] The terminal sends the data, converted to JSON format, to the server. A data transmission method is used for this process. The data is sent as an HTTP POST request.

[0770] Step 5:

[0771] The server analyzes the received data and sends a request to a generative AI model (e.g., GPT-3). This request includes data and a corresponding prompt. Specifically, the prompt used might be: "The customer's name is Yamada Taro. His purchase history includes shirts and shoes, and his interest is in casual fashion. His facial expression suggests he is interested. Please provide him with the most suitable product suggestions and additional information."

[0772] Step 6:

[0773] The generative AI model generates a response based on the prompt and data. An example of the generated response might be, "We recommend these new jackets and sneakers, perfect for casual fashion. For more information, please refer to this link." This response data is then sent back to the server.

[0774] Step 7:

[0775] The server receives the response from the generative AI model and converts it into JSON format. The converted response data is in the following format:

[0776] json

[0777] {

[0778] "recommendations": ["New jacket", "Sneakers"],

[0779] "additional_info": "For more detailed information, please refer to this link."

[0780] }

[0781] Step 8:

[0782] The server sends response data in JSON format back to the terminal. The returned data is then sent again as an HTTP POST request.

[0783] Step 9:

[0784] The device analyzes the received response data and displays it to the user. The displayed content may include customized feedback based on the customer's sentiment. For example, it might display: "For those interested in casual fashion, we recommend our new jackets and sneakers. For more information, please see this link."

[0785] The above steps enable personalized customer service in physical stores.

[0786] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0787] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0788] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0789] [Third Embodiment]

[0790] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0791] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0792] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0793] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0794] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0795] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0796] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0797] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0798] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0799] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0800] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0801] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0802] This invention relates to a system aimed at improving the productivity of civil servants, in which a user inputs information, which is then analyzed and transmitted to a generative AI, and a response is received. The specific program processing of this system is described below in natural language, along with concrete examples.

[0803] System Overview

[0804] 1. Providing an input form

[0805] The system provides an input form that users can access through a web browser. This form has multiple input fields where users enter the necessary information.

[0806] 2. Data Analysis

[0807] When a user enters data into an input form and presses the submit button, the device uses JavaScript to collect the form data and convert it into JSON format. This ensures that the data is sent to the server efficiently.

[0808] 3. Sending data and receiving responses

[0809] The server receives data in JSON format sent from the terminal. The server analyzes the received data and sends a request to the generative AI's API. The generative AI generates a response based on the input data. The server receives this response, converts it back to JSON format, and sends it back to the terminal.

[0810] 4. Display of response

[0811] The terminal receives response data sent back from the server. The received data is analyzed and displayed in the browser in a format that is easy for the user to understand. This allows the user to obtain specific advice and suggestions for efficiently carrying out their work.

[0812] Specific example

[0813] 1. Using the input form

[0814] A user (for example, an employee of the General Affairs Department of the city hall) enters "General Affairs Department of City Hall" in the "Role" field, enters "I would like to know the progress of paperless operations" in the "Request" field, and presses the submit button.

[0815] 2. Data transmission and analysis

[0816] The terminal converts the input data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI asking "How should the General Affairs Department of the city hall proceed with paperless operations?"

[0817] 3. Response from a generative AI

[0818] The generative AI generates advice on specific methods for implementing a paperless system based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal.

[0819] 4. Display of response

[0820] The terminal analyzes the response data it receives and displays it to the user as specific advice, such as "To proceed with paperless operations, please take the following steps..." The user can then take concrete action based on this information.

[0821] This system can streamline inefficient processes caused by outdated practices in government offices, thereby improving productivity. Users can easily input information and receive specific suggestions from generative AI, leading to improved work processes, reduced overtime, and a move towards a paperless environment.

[0822] The following describes the processing flow.

[0823] Step 1:

[0824] The user opens a web browser and accesses the specified URL. This displays an input form.

[0825] Step 2:

[0826] The user enters the necessary information into each field of the input form (e.g., "Role", "Request").

[0827] Step 3:

[0828] The user clicks the "Submit" button. This collects the input data.

[0829] Step 4:

[0830] The JavaScript on the device converts the collected data into JSON format.

[0831] Step 5:

[0832] The terminal sends the converted JSON data to the server as a POST request.

[0833] Step 6:

[0834] The server receives the POST request and extracts JSON data from the request body.

[0835] Step 7:

[0836] The server parses the extracted JSON data and splits it into individual fields (e.g., role, request).

[0837] Step 8:

[0838] The server sends the analyzed data as a request to the generative AI's API.

[0839] Step 9:

[0840] The generative AI receives the request and generates a response based on the input data.

[0841] Step 10:

[0842] The generative AI generates a response which is then sent back to the server.

[0843] Step 11:

[0844] The server receives the response from the generative AI and converts it into the required format.

[0845] Step 12:

[0846] The server returns the converted response data to the terminal in JSON format.

[0847] Step 13:

[0848] The terminal receives and parses the JSON data sent back from the server.

[0849] Step 14:

[0850] The data analyzed by the device is embedded in an HTML element to be displayed visually to the user.

[0851] Step 15:

[0852] Users review the displayed response data and take specific actions to improve operations and increase productivity.

[0853] (Example 1)

[0854] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0855] In modern public institutions, productivity is low because many tasks are performed manually or on paper. Furthermore, this leads to delays in work progress, slow feedback and decision-making, and ultimately, a decrease in overall operational efficiency. A system is needed to address this challenge and efficiently improve productivity.

[0856] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0857] In this invention, the server includes a user interface means for the user to input information, a conversion means for collecting the information input through the user interface means and converting it into JSON format, a data transmission means for analyzing the JSON format information converted by the conversion means and sending a request to a generative AI model, and a display means for converting the response received from the generative AI model back into JSON format and displaying it to the user. This makes it possible to efficiently analyze the input data and provide appropriate advice to the user.

[0858] A "user interface means" is a tool that provides an interface (UI) for users to input information.

[0859] A "conversion tool" is a tool that collects information entered through a user interface and performs the process of converting it into a specific format (e.g., JSON format).

[0860] A "data transmission means" is a tool that performs the process of analyzing the information converted by the conversion means, sending a request to a generative AI model, and receiving a response.

[0861] A "display means" is a tool that transforms the response received from a generative AI model and displays it in a format that is easy for the user to understand.

[0862] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight data exchange format for structuring and exchanging data.

[0863] A "generative AI model" is an artificial intelligence model that analyzes data provided by users and generates appropriate responses based on that data.

[0864] An "input field" refers to a specific area in a user interface where a user enters information.

[0865] This invention aims to improve the productivity of civil servants by providing a system in which a user inputs information, which is then analyzed and sent to a generative AI model, and a response is received. Specific embodiments of this system are described below.

[0866] Providing an input form

[0867] This system utilizes an input form accessible through a web browser. Specifically, browsers such as Google Chrome and Mozilla Firefox can be used. This input form contains multiple fields where users enter the necessary information. For example, information can be entered in fields such as "Role" and "Requests." The entire input form is constructed using HTML and CSS, and a dynamic user interface is implemented using JavaScript.

[0868] Data analysis

[0869] When a user enters information into an input form and presses the submit button, the device uses JavaScript to collect the form data. Next, this collected data is converted into JSON format. JSON is a lightweight data exchange format for efficient data exchange, and the data is sent to the server in this format.

[0870] Sending data and receiving responses

[0871] The JSON data sent from the device is received by the server. The server receives this data via an HTTP request and first verifies its contents. Next, it sends the verified data to a generative AI model. The generative AI model generates an appropriate response based on the user's input data. The response generated by the generative AI model is sent back to the server. The server receives this response, converts it back to JSON format if necessary, and sends it back to the device.

[0872] Display of response

[0873] The response data sent from the server to the terminal is received again by the terminal. The terminal parses this response data and uses JavaScript to display it in a user-friendly format on the browser. This allows the user to obtain specific advice and suggestions to efficiently carry out their work.

[0874] Specific example

[0875] For example, an employee of the General Affairs Division of a city hall enters "General Affairs Division of City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field, then presses the submit button. The terminal converts the input data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI model asking "How will the General Affairs Division of City Hall proceed with paperless initiatives?" The generative AI model generates advice on specific methods for proceeding with paperless initiatives based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal. The terminal analyzes the received response data and displays it to the user as specific advice such as "To proceed with paperless initiatives, follow these steps..."

[0876] Example of a prompt

[0877] Examples of prompt statements include the following:

[0878] "I'm a staff member in the General Affairs Department. I'd like to promote a paperless system, but could you tell me what steps I should take?"

[0879] By inputting such prompt messages into the AI ​​model, advice can be obtained on specific methods for implementing a paperless system.

[0880] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0881] Step 1:

[0882] The user accesses the system using a web browser. The user enters information into the input form. For example, the user might enter "General Affairs Department, City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field. The user proceeds to the next step by pressing the "Submit" button. The input is the information entered by the user into the input form, and the output is the data entered into the input form converted into JSON format.

[0883] Step 2:

[0884] The device uses JavaScript to collect form data and convert it into JSON format. Specifically, JavaScript retrieves data from input fields and converts it into a JSON object. For example, it might be converted to a format like {"Role": "General Affairs Department, City Hall", "Request": "I want to know the progress of paperless initiatives"}. The input is information entered by the user, and the output is data generated in JSON format.

[0885] Step 3:

[0886] The device uses an HTTP POST request to send the generated JSON data to the server. Specifically, it uses JavaScript's fetch API or similar to send data to a specific endpoint on the server. For example, it sends JSON data to POST / api / request. The input is data in JSON format, and the output is the request that was sent to the server.

[0887] Step 4:

[0888] The server receives an HTTP request and parses the JSON data in the request body. Specifically, the server-side program (e.g., Node.js or Python) parses the request data. For example, it parses the JSON data and extracts the values ​​of each field. The input is the JSON data sent from the terminal, and the output is the parsed data.

[0889] Step 5:

[0890] The server sends the analyzed data to the generative AI model. Specifically, it issues an HTTP request to the generative AI model's API. For example, it sends data containing a prompt to the generative AI model's endpoint. The input is the analyzed data, and the output is the request that was sent to the generative AI model.

[0891] Step 6:

[0892] The generative AI model generates a response and sends it back to the server. Specifically, the generative AI model analyzes the received data and generates an appropriate response based on that analysis. For example, it might generate a response such as, "To proceed with paperless operations, please follow these steps..." The input is prompt data sent from the server, and the output is the generated response data.

[0893] Step 7:

[0894] The server converts the response data received from the generative AI model back into JSON format and sends it to the terminal. Specifically, it structures the received response data and converts it into JSON format. The input is the response data received from the generative AI model, and the output is the data converted into JSON format.

[0895] Step 8:

[0896] The terminal receives and parses response data from the server. Specifically, it uses JavaScript to parse JSON data and retrieve data for each item. The input is the JSON data sent from the server, and the output is the parsed response data.

[0897] Step 9:

[0898] The terminal analyzes response data and displays it to the user. Specifically, it inserts the response data into an HTML element and displays it in the browser. For example, it might display specific advice such as, "To proceed with paperless operations, follow these steps..." The input is the analyzed response data, and the output is the content displayed to the user.

[0899] (Application Example 1)

[0900] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0901] In virtual stores, there is a need to respond to customer inquiries quickly and appropriately. However, conventional systems lack sufficient automation to provide optimal answers tailored to the content of inquiries, making it difficult to improve customer satisfaction. Furthermore, even simple inquiries required human intervention, resulting in increased effort and cost. To solve this problem, there is a need to automate inquiry handling using generative AI.

[0902] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0903] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, a data transmission and reception means for sending the analyzed information to a generative AI and receiving a response, and an API integration means for calling the generative AI's API and automatically generating appropriate responses to customer inquiries in the virtual store. This makes it possible to automate customer inquiries in the virtual store quickly and accurately.

[0904] "User interface means" refers to an interface for users to input information, and specifically includes input forms, buttons, and the like.

[0905] "Analysis means" refers to means for analyzing information input through a user interface and handling it as data.

[0906] "Data transmission and reception means" refers to means for transmitting information analyzed by the analysis means to the generative AI and receiving a response from the generative AI.

[0907] "Display means" refers to means for displaying the response received by the data transmission means to the user.

[0908] "Generative AI" refers to artificial intelligence that automatically generates appropriate responses based on input data.

[0909] "API integration means" refers to a method for calling the API of a generative AI and exchanging data in conjunction with an external system.

[0910] A "virtual store" is an online store that provides goods and services via the internet.

[0911] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a method for representing data in a lightweight and structured format.

[0912] As a concrete example of implementing this invention, a customer service system for a virtual store is provided. The system includes a user interface means for users to input information. This user interface means includes an input form with multiple different input fields, allowing customers to easily input their inquiries.

[0913] The input information is analyzed by an analysis tool. This analysis tool uses a programming language such as JavaScript to convert the input data into JSON format. The converted data is then sent to the server via the analysis tool.

[0914] The server processes the received JSON data using a data transmission and reception mechanism and calls the generative AI API. This generative AI API automatically generates an appropriate response based on the input data, and can use, for example, the OpenAI API. The server converts the response from the generative AI back into JSON format and sends it back to the terminal.

[0915] The terminal displays the received response data to the customer via a display device. This series of processes enables quick and efficient customer service in the virtual store.

[0916] Hardware and software to use

[0917] 1. Hardware

[0918] Server: Cloud server (e.g., Amazon Web Services, Google Cloud Platform)

[0919] Device: The iOS or Android device used by the user.

[0920] 2. Software

[0921] Framework: Flask (a web framework written in Python)

[0922] HTTP Library: Requests (Python's HTTP library)

[0923] Generative AI: OpenAI API, etc.

[0924] The server receives the user's inquiry data, converts it to JSON format, and works with a generative AI API to generate an appropriate response. The generated response is converted back to JSON format and sent to the user's device. The device then displays this response to the user.

[0925] Specific example

[0926] The following example prompt will be used to generate a response from the generative AI.

[0927] Example of a prompt:

[0928] User inquiry: "What is the best way to use the specified product?"

[0929] This system enables the rapid and accurate automation of customer inquiries at virtual stores, leading to improved customer satisfaction. Furthermore, because responses are generated automatically without the need for human intervention, it also reduces labor and costs.

[0930] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0931] Step 1:

[0932] The user enters information.

[0933] Users enter their inquiries using the user interface of a smartphone application. Specifically, they enter information into multiple input fields in an inquiry form and press the submit button. The input data is in text format and may include questions such as, "What is the best way to use the specified product?"

[0934] Step 2:

[0935] The device converts the data to JSON format.

[0936] The terminal converts the data entered by the user into JSON format using methods such as JavaScript. This conversion structures the data, making it easier to send to the server. For example, the input data is converted into JSON format as follows:

[0937] json

[0938] {

[0939] "query": "What is the best way to use the specified product?"

[0940] }

[0941] Step 3:

[0942] Send data to the server

[0943] The terminal sends the converted JSON data to the server using an HTTP request. The transmitted data is then passed directly to the analysis tool.

[0944] Step 4:

[0945] The server analyzes the data.

[0946] The server analyzes the received JSON data using parsing tools. This analysis helps understand what information the query requires. Specifically, it extracts the contents of the "query" field of the data and prepares it for the next processing step.

[0947] Step 5:

[0948] Call the API of a generative AI.

[0949] The server sends the analyzed query content as a request to the generative AI's API. Using the API integration method, the query content is sent as a prompt message to the OpenAI API. An example of a prompt message sent is as follows:

[0950] User inquiry: "What is the best way to use the specified product?"

[0951] Step 6:

[0952] Generative AI generates responses.

[0953] The generative AI generates a response based on the received prompt. This response includes specific advice, such as "This product is best used in the following procedure." The generated response is sent to the server in JSON format.

[0954] Step 7:

[0955] The server converts the response back to JSON format.

[0956] The server converts the response received from the generative AI back into JSON format. This response data is also in a structured format, for example, as follows:

[0957] json

[0958] {

[0959] "Response": "This product is best used in the following way:"

[0960] }

[0961] Step 8:

[0962] The device receives a response.

[0963] The terminal receives response data in JSON format sent from the server. It then parses this data and prepares to display the necessary information on the screen.

[0964] Step 9:

[0965] The terminal displays a response to the user.

[0966] The terminal displays the received response data to the user through a display mechanism. The user can see specific advice on the screen, such as, "This product is best used in the following way."

[0967] This series of processes allows users to receive quick and appropriate responses, streamlining customer inquiries in virtual stores.

[0968] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0969] This invention relates to a system in which a user inputs information, and a response is provided by combining a generative AI and an emotion recognition means. The specific program processing of this system is described below in natural language, along with concrete examples.

[0970] System Overview

[0971] 1. Providing an input form

[0972] The system provides an input form that users can access through a web browser. This form has multiple input fields where users enter the necessary information.

[0973] 2. Data Analysis

[0974] When a user enters data into an input form and presses the submit button, the device uses JavaScript to collect the form data and convert it into JSON format. This ensures that the data is sent to the server efficiently.

[0975] 3. Emotion recognition

[0976] The device is equipped with emotion recognition capabilities to analyze the user's facial expressions, voice, or biosignals. The device collects and analyzes the user's emotion data. This emotion data is also converted to JSON format and sent to the server.

[0977] 4. Sending data and receiving responses

[0978] The server receives data in JSON format sent from the terminal. The server analyzes the received data and sends a request to the generative AI's API. The generative AI generates a response based on the input data. The server receives this response, converts it back to JSON format, and sends it back to the terminal.

[0979] 5. Display of response

[0980] The device receives response data sent back from the server. The received data is analyzed and displayed in the browser in a user-friendly format. Furthermore, customized feedback based on the user's emotions is also displayed.

[0981] Specific example

[0982] 1. Using the input form

[0983] A user (for example, an employee of the General Affairs Department of the city hall) enters "General Affairs Department of City Hall" in the "Role" field, enters "I would like to know the progress of paperless operations" in the "Request" field, and presses the submit button.

[0984] 2. Collection of emotional data

[0985] The emotion recognition system identifies emotions such as "interest" or "confusion" from the user's facial expressions and voice. This emotion data is also sent to the server.

[0986] 3. Data transmission and analysis

[0987] The terminal converts input data and sentiment data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI asking "How should the general affairs department of the city hall proceed with paperless operations?"

[0988] 4. Response from a generative AI

[0989] The generative AI generates advice on specific methods for implementing a paperless system based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal.

[0990] 5. Display of response

[0991] The device analyzes the response data it receives and displays it to the user as specific advice, such as "To proceed with paperless operations, please take the following steps..." It also displays customized feedback, including additional explanations and links to resources, based on the user's emotions, such as "interest" or "confusion." The user can then take specific actions based on this feedback.

[0992] This system can streamline inefficient processes caused by outdated practices in government offices, thereby improving productivity. Users can easily input information and receive specific suggestions and emotion-based feedback from generative AI, leading to improved work processes, reduced overtime, and a shift towards a paperless environment.

[0993] The following describes the processing flow.

[0994] Step 1:

[0995] The user opens a web browser and accesses the specified URL. This displays an input form.

[0996] Step 2:

[0997] The user enters the necessary information into each field of the input form (e.g., "Role", "Request").

[0998] Step 3:

[0999] The user clicks the "Submit" button. This collects the input data.

[1000] Step 4:

[1001] The JavaScript on the device converts the collected data into JSON format.

[1002] Step 5:

[1003] The device sends JSON data to the server as a POST request.

[1004] Step 6:

[1005] The server receives the POST request and extracts JSON data from the request body.

[1006] Step 7:

[1007] The server parses the extracted JSON data and splits it into individual fields (e.g., role, request).

[1008] Step 8:

[1009] The user's webcam, microphone, and other sensors collect the user's facial expressions, voice, and biosignals.

[1010] Step 9:

[1011] The emotion recognition system analyzes the collected data to identify the user's emotions.

[1012] Step 10:

[1013] The device converts the emotional data into JSON format and sends it to the server.

[1014] Step 11:

[1015] The server receives and analyzes emotional data.

[1016] Step 12:

[1017] The server sends the analysis data and emotion data as requests to the generative AI's API.

[1018] Step 13:

[1019] A generative AI receives a request and generates a response based on the input data and sentiment data.

[1020] Step 14:

[1021] The generative AI generates a response which is then sent back to the server.

[1022] Step 15:

[1023] The server receives the response from the generative AI and converts it into the required format.

[1024] Step 16:

[1025] The server returns the converted response data to the terminal in JSON format.

[1026] Step 17:

[1027] The terminal receives and parses the JSON data sent back from the server.

[1028] Step 18:

[1029] The data analyzed by the device is embedded in an HTML element to be displayed visually to the user.

[1030] Step 19:

[1031] The device also displays customized feedback based on the user's emotions.

[1032] Step 20:

[1033] Users review the displayed response data and customized feedback to take concrete actions for business improvement and productivity enhancement.

[1034] (Example 2)

[1035] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1036] In modern business processes, there is a need for systems that generate appropriate responses to information entered by users. However, conventional systems cannot take user emotions into account, making it difficult to provide specific feedback tailored to individual needs and feelings. Furthermore, efficient and highly accurate processing is required in data analysis and exchange with generative artificial intelligence. To solve these problems, a response generation system that incorporates user emotion data is necessary.

[1037] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1038] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, and a data transmission and reception means for sending data to a generative artificial intelligence and receiving a response. This enables the provision of customized feedback based on the user's emotions by including an emotion recognition means for collecting and analyzing the user's emotion data, a data transmission and reception means for sending the emotion data to the generative artificial intelligence, and a display means for displaying the received response to the user.

[1039] "User interface means" refers to an interface for users to input information, and includes devices such as input forms and buttons.

[1040] "Analysis means" refers to software or hardware devices for analyzing information input through a user interface means.

[1041] "Generative artificial intelligence" refers to artificial intelligence technologies that generate responses based on user input, and specifically includes natural language processing models.

[1042] A "data transmission and reception means" is a device that has network communication capabilities for transmitting information analyzed by an analysis means to a generative artificial intelligence and receiving a response.

[1043] "Display means" refers to devices such as displays and monitors that visually display responses received by data transmission means to the user.

[1044] "Emotion recognition means" refers to software or hardware devices that analyze a user's facial expressions, voice, or biosignals to collect emotional data.

[1045] "Structured data format" is a general term for data formats that represent data in a standardized format such as JSON and allow for its exchange with other data.

[1046] An "input form" is a form with multiple different input fields, and is an interface that users use to enter specific information.

[1047] This invention is a system in which a user inputs information and a response is provided by combining generative artificial intelligence and emotion recognition means. In this system, the user can input information via a web browser and receive a response from generative artificial intelligence. Furthermore, by collecting the user's emotion data and reflecting it in the response, the system provides more personalized feedback.

[1048] Specifically, the following hardware and software will be used.

[1049] Hardware:

[1050] 1. Server - A computer used for high-speed and stable data processing and communication.

[1051] 2. Device - A device used by the user, such as a PC, smartphone, or tablet.

[1052] 3. Camera and microphone - Input devices for capturing the user's facial expressions and voice.

[1053] software:

[1054] 1. Web browser - Software used as a user interface.

[1055] 2. HTML, CSS, JavaScript - Languages ​​used to build input forms and user interfaces.

[1056] 3. OpenCV, TensorFlow - Libraries used for emotion recognition.

[1057] 4. Generative AI APIs - AI interfaces used to generate responses (e.g., GPT-3).

[1058] Here are some specific examples:

[1059] 1. User information input:

[1060] A user (for example, an employee of the general affairs department of a city hall) enters "General Affairs Department of City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field of an input form on a web browser. Then they press the "Submit" button.

[1061] 2. Data analysis and transmission:

[1062] The device uses JavaScript to collect the entered information and converts it to JSON format using the JSON.stringify() function. The converted data is then sent to the server as an HTTP POST request.

[1063] 3. Collection and analysis of emotional data:

[1064] The device uses its camera and microphone to capture the user's facial expressions and voice, and performs emotion analysis using OpenCV and TensorFlow. The analysis results (e.g., "interested" or "confused") are converted to JSON format and sent to the server.

[1065] 4. Data transmission to generative artificial intelligence:

[1066] The server analyzes the user's input data and sentiment data, and sends requests to a generative artificial intelligence API (e.g., GPT-3). The following is an example of a prompt:

[1067] Text format: "How should the general affairs department of the city hall proceed with paperless operations?"

[1068] Text format: "How can we provide user feedback based on their emotions?"

[1069] 5. Receiving and displaying responses:

[1070] The server receives the response from the generative artificial intelligence, converts it to JSON format, and sends it to the terminal. The terminal parses this and displays it in a user-friendly format in the browser. For example, it might display advice such as, "To proceed with paperless operations, follow these steps..." It also displays additional customized feedback based on the user's interests and concerns.

[1071] In this way, users can easily input information and receive specific suggestions and emotion-based feedback from generative artificial intelligence. This can lead to improvements and increased efficiency in their work.

[1072] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1073] Step 1:

[1074] Providing an input form

[1075] The server provides an input form for the user to enter information. The server generates the input form using HTML, CSS, and JavaScript, and sends it to the user's web browser.

[1076] Input: HTML, CSS, and JavaScript code from the server.

[1077] Output: An input form displayed in the user's web browser.

[1078] Specific actions:

[1079] Users enter information such as their "role" and "requests" into this form. For example, they might enter "General Affairs Department, City Hall" for "role" and "I would like to know the progress of the paperless initiative" for "request."

[1080] Step 2:

[1081] Data collection and transmission

[1082] When the user presses the "Submit" button, the device uses JavaScript to collect the entered information. The collected data is converted to JSON format and sent to the server as an HTTP POST request.

[1083] Input: Information entered by the user.

[1084] Output: The data, converted to JSON format, is sent to the server.

[1085] Specific actions:

[1086] The device uses an event listener to detect a click of the "Submit" button, retrieves the form data using JavaScript, and converts it to JSON format using the JSON.stringify() function. This is then sent to the server as an HTTP POST request.

[1087] Step 3:

[1088] Collection and analysis of emotional data

[1089] The device uses its camera and microphone to capture the user's facial expressions and voice, and analyzes them using emotion recognition technology. The analysis results are also converted into JSON format and sent to the server.

[1090] Input: Real-time data from camera and microphone.

[1091] Output: The analyzed emotion data is converted to JSON format and sent to the server.

[1092] Specific actions:

[1093] The device uses OpenCV and TensorFlow to analyze the user's facial expressions and voice, collecting emotional data such as "interest" and "confusion." This data is then converted to JSON format and sent to the server as an additional HTTP POST request.

[1094] Step 4:

[1095] Sending data to generative artificial intelligence

[1096] The server analyzes user input data and sentiment data and sends a request to a generative artificial intelligence. It generates a request that includes a prompt and sentiment data.

[1097] Input: User input data and sentiment data.

[1098] Output: Request sent to the generative artificial intelligence.

[1099] Specific actions:

[1100] The server analyzes the received data using Python, Node.js, etc., and generates appropriate prompts for a generative artificial intelligence (e.g., GPT-3). For example, it might generate a prompt such as "How should the general affairs department of the city hall proceed with paperless operations?" and send it to the generative AI.

[1101] Step 5:

[1102] Receiving responses and sending data from generative artificial intelligence.

[1103] The server receives the response from the generative artificial intelligence and converts it into JSON format. It then sends the converted data to the terminal.

[1104] Input: Response data from a generative artificial intelligence.

[1105] Output: JSON-formatted response data sent to the terminal.

[1106] Specific actions:

[1107] The server receives the response from the generative artificial intelligence and converts it into JSON format. This converted data is then sent to the terminal as an HTTP response.

[1108] Step 6:

[1109] Display of response and feedback

[1110] The device parses the received JSON response data and displays it in a user-friendly format on the browser. It also provides customized feedback based on the user's emotions.

[1111] Input: JSON-formatted response data received from the server.

[1112] Output: Analyzed response data and sentiment-based feedback displayed in the user's browser.

[1113] Specific actions:

[1114] The JSON data received by the device is displayed in the browser using HTML / CSS. For example, it might display specific advice such as, "To proceed with paperless operations, follow these steps..." Furthermore, it displays links to additional information and resources based on the user's "interests" and "concerns."

[1115] This system allows users to easily input information and receive specific suggestions and emotion-based feedback from generative artificial intelligence. This can lead to improvements and increased efficiency in work processes.

[1116] (Application Example 2)

[1117] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1118] Traditional in-store customer service often suffers from insufficient personalization, making it difficult to enhance customer satisfaction. Furthermore, store staff struggle to accurately understand customers' emotions and interests, hindering their ability to provide optimal product recommendations. In addition, inconsistent staff service can often degrade the quality of the customer experience.

[1119] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1120] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, a data transmission and reception means for transmitting the analyzed information to a generative AI and receiving a response, a display means for displaying the received response to the user, an emotion recognition means for analyzing the user's emotions using an facial recognition camera, and a data transmission and reception means for transmitting emotion data to the generative AI. This enables personalized product suggestions based on customer emotions and interests in physical stores. Furthermore, it allows store staff to provide consistently high-quality customer service, thereby improving customer satisfaction.

[1121] "User interface means" refers to an interface that a user uses to input information.

[1122] "Analysis means" refers to means for analyzing information input through a user interface.

[1123] "Data transmission and reception means" refers to means for transmitting information analyzed by the analysis means to the generative AI and receiving a response from the generative AI.

[1124] "Display means" refers to means for displaying the response received by the data transmission means to the user.

[1125] An "input form" refers to a web form that has multiple different input fields for users to enter information.

[1126] A "facial recognition camera" refers to a camera device that captures a user's facial expressions and analyzes emotional data.

[1127] "Emotion recognition means" refers to a method for analyzing a user's emotions using a facial recognition camera.

[1128] "Generative AI" refers to an artificial intelligence system that generates responses based on input data.

[1129] A "custom support app for physical stores" refers to a mobile application used by store staff in physical stores to provide optimal product recommendations through dialogue with customers.

[1130] One embodiment of this invention is a system for personalizing customer service in physical stores and improving customer satisfaction. This system takes user input and provides a response by combining generative AI and emotion recognition means.

[1131] First, the system provides a user interface. This interface is for use by in-store staff via smartphones or tablets and provides an input form with multiple input fields for entering customer information, purchase history, interests, and more.

[1132] Next, the information entered on the terminal is analyzed using an analysis tool. This analysis tool primarily analyzes text data to understand customer needs and interests. This data is converted to JSON format and sent to the server. The server is equipped with a data transmission and reception mechanism, which sends the analyzed information to a generative AI and receives the response.

[1133] This system includes a facial recognition camera that captures the user's facial expressions. The facial expression data is analyzed using emotion recognition technology to understand the user's emotions. This emotion data is also converted to JSON format and sent to the server. All information, including the emotion data, is processed by generative AI to generate appropriate product suggestions and customized feedback.

[1134] As a concrete example, consider a scenario where a user shows interest in "casual fashion" in a physical store. Based on the user's purchase history, interests, and the emotion of "interest" inferred from their facial expressions, the generative AI makes optimal product suggestions. The prompt text in this case would be as follows:

[1135] The customer's name is Taro Yamada. His purchase history includes shirts and shoes, and his interest lies in casual fashion. He appears interested based on his expression. Please provide him with the most suitable product suggestions and additional information.

[1136] The server converts the response received from the generative AI into JSON format and sends it back to the terminal. The terminal analyzes the received data and displays it as information for in-store staff to make suggestions to customers. In addition, more personalized feedback is provided based on sentiment data. This system not only improves customer satisfaction but also ensures consistent service quality from store staff.

[1137] The main hardware and software used to implement this system include smartphones, tablets, facial recognition cameras, and generative AI models (e.g., GPT-3). Additionally, server-side APIs and associated analysis software are required for data transmission, reception, and analysis.

[1138] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1139] Step 1:

[1140] Users use their smartphones in physical stores to input customer information through a user interface. Specifically, they enter their purchase history and interests into input fields. An example of input data is "Name: Taro Yamada," "Purchase History: Shirts, Shoes," and "Interests: Casual Fashion." The entered data is temporarily stored internally.

[1141] Step 2:

[1142] The device uses a facial recognition camera to capture customer facial data. The facial data captured by the camera is transmitted in real time to an emotion recognition system for analysis. For example, the emotion of "interested" is analyzed. This emotion data is also temporarily stored internally.

[1143] Step 3:

[1144] The terminal converts the analyzed input data and sentiment data into JSON format. The converted data is in the following format as an example:

[1145] json

[1146] {

[1147] "name": "Yamada Taro",

[1148] "purchase_history": ["shirt", "shoes"],

[1149] "interests": "casual fashion",

[1150] "emotion": "Interested"

[1151] }

[1152] Step 4:

[1153] The terminal sends the data, converted to JSON format, to the server. A data transmission method is used for this process. The data is sent as an HTTP POST request.

[1154] Step 5:

[1155] The server analyzes the received data and sends a request to a generative AI model (e.g., GPT-3). This request includes data and a corresponding prompt. Specifically, the prompt used might be: "The customer's name is Yamada Taro. His purchase history includes shirts and shoes, and his interest is in casual fashion. His facial expression suggests he is interested. Please provide him with the most suitable product suggestions and additional information."

[1156] Step 6:

[1157] The generative AI model generates a response based on the prompt and data. An example of the generated response might be, "We recommend these new jackets and sneakers, perfect for casual fashion. For more information, please refer to this link." This response data is then sent back to the server.

[1158] Step 7:

[1159] The server receives the response from the generative AI model and converts it into JSON format. The converted response data is in the following format:

[1160] json

[1161] {

[1162] "recommendations": ["New jacket", "Sneakers"],

[1163] "additional_info": "For more detailed information, please refer to this link."

[1164] }

[1165] Step 8:

[1166] The server sends response data in JSON format back to the terminal. The returned data is then sent again as an HTTP POST request.

[1167] Step 9:

[1168] The device analyzes the received response data and displays it to the user. The displayed content may include customized feedback based on the customer's sentiment. For example, it might display: "For those interested in casual fashion, we recommend our new jackets and sneakers. For more information, please see this link."

[1169] The above steps enable personalized customer service in physical stores.

[1170] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1171] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1172] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1173] [Fourth Embodiment]

[1174] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1175] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1176] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1177] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1178] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1179] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1180] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1181] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1182] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1183] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1184] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1185] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1186] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1187] This invention relates to a system aimed at improving the productivity of civil servants, in which a user inputs information, which is then analyzed and transmitted to a generative AI, and a response is received. The specific program processing of this system is described below in natural language, along with concrete examples.

[1188] System Overview

[1189] 1. Providing an input form

[1190] The system provides an input form that users can access through a web browser. This form has multiple input fields where users enter the necessary information.

[1191] 2. Data Analysis

[1192] When a user enters data into an input form and presses the submit button, the device uses JavaScript to collect the form data and convert it into JSON format. This ensures that the data is sent to the server efficiently.

[1193] 3. Sending data and receiving responses

[1194] The server receives data in JSON format sent from the terminal. The server analyzes the received data and sends a request to the generative AI's API. The generative AI generates a response based on the input data. The server receives this response, converts it back to JSON format, and sends it back to the terminal.

[1195] 4. Display of response

[1196] The terminal receives response data sent back from the server. The received data is analyzed and displayed in the browser in a format that is easy for the user to understand. This allows the user to obtain specific advice and suggestions for efficiently carrying out their work.

[1197] Specific example

[1198] 1. Using the input form

[1199] A user (for example, an employee of the General Affairs Department of the city hall) enters "General Affairs Department of City Hall" in the "Role" field, enters "I would like to know the progress of paperless operations" in the "Request" field, and presses the submit button.

[1200] 2. Data transmission and analysis

[1201] The terminal converts the input data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI asking "How should the General Affairs Department of the city hall proceed with paperless operations?"

[1202] 3. Response from a generative AI

[1203] The generative AI generates advice on specific methods for implementing a paperless system based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal.

[1204] 4. Display of response

[1205] The terminal analyzes the response data it receives and displays it to the user as specific advice, such as "To proceed with paperless operations, please take the following steps..." The user can then take concrete action based on this information.

[1206] This system can streamline inefficient processes caused by outdated practices in government offices, thereby improving productivity. Users can easily input information and receive specific suggestions from generative AI, leading to improved work processes, reduced overtime, and a move towards a paperless environment.

[1207] The following describes the processing flow.

[1208] Step 1:

[1209] The user opens a web browser and accesses the specified URL. This displays an input form.

[1210] Step 2:

[1211] The user enters the necessary information into each field of the input form (e.g., "Role", "Request").

[1212] Step 3:

[1213] The user clicks the "Submit" button. This collects the input data.

[1214] Step 4:

[1215] The JavaScript on the device converts the collected data into JSON format.

[1216] Step 5:

[1217] The terminal sends the converted JSON data to the server as a POST request.

[1218] Step 6:

[1219] The server receives the POST request and extracts JSON data from the request body.

[1220] Step 7:

[1221] The server parses the extracted JSON data and splits it into individual fields (e.g., role, request).

[1222] Step 8:

[1223] The server sends the analyzed data as a request to the generative AI's API.

[1224] Step 9:

[1225] The generative AI receives the request and generates a response based on the input data.

[1226] Step 10:

[1227] The generative AI generates a response which is then sent back to the server.

[1228] Step 11:

[1229] The server receives the response from the generative AI and converts it into the required format.

[1230] Step 12:

[1231] The server returns the converted response data to the terminal in JSON format.

[1232] Step 13:

[1233] The terminal receives and parses the JSON data sent back from the server.

[1234] Step 14:

[1235] The data analyzed by the device is embedded in an HTML element to be displayed visually to the user.

[1236] Step 15:

[1237] Users review the displayed response data and take specific actions to improve operations and increase productivity.

[1238] (Example 1)

[1239] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1240] In modern public institutions, productivity is low because many tasks are performed manually or on paper. Furthermore, this leads to delays in work progress, slow feedback and decision-making, and ultimately, a decrease in overall operational efficiency. A system is needed to address this challenge and efficiently improve productivity.

[1241] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1242] In this invention, the server includes a user interface means for the user to input information, a conversion means for collecting the information input through the user interface means and converting it into JSON format, a data transmission means for analyzing the JSON format information converted by the conversion means and sending a request to a generative AI model, and a display means for converting the response received from the generative AI model back into JSON format and displaying it to the user. This makes it possible to efficiently analyze the input data and provide appropriate advice to the user.

[1243] A "user interface means" is a tool that provides an interface (UI) for users to input information.

[1244] A "conversion tool" is a tool that collects information entered through a user interface and performs the process of converting it into a specific format (e.g., JSON format).

[1245] A "data transmission means" is a tool that performs the process of analyzing the information converted by the conversion means, sending a request to a generative AI model, and receiving a response.

[1246] A "display means" is a tool that transforms the response received from a generative AI model and displays it in a format that is easy for the user to understand.

[1247] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight data exchange format for structuring and exchanging data.

[1248] A "generative AI model" is an artificial intelligence model that analyzes data provided by users and generates appropriate responses based on that data.

[1249] An "input field" refers to a specific area in a user interface where a user enters information.

[1250] This invention aims to improve the productivity of civil servants by providing a system in which a user inputs information, which is then analyzed and sent to a generative AI model, and a response is received. Specific embodiments of this system are described below.

[1251] Providing an input form

[1252] This system utilizes an input form accessible through a web browser. Specifically, browsers such as Google Chrome and Mozilla Firefox can be used. This input form contains multiple fields where users enter the necessary information. For example, information can be entered in fields such as "Role" and "Requests." The entire input form is constructed using HTML and CSS, and a dynamic user interface is implemented using JavaScript.

[1253] Data analysis

[1254] When a user enters information into an input form and presses the submit button, the device uses JavaScript to collect the form data. Next, this collected data is converted into JSON format. JSON is a lightweight data exchange format for efficient data exchange, and the data is sent to the server in this format.

[1255] Sending data and receiving responses

[1256] The JSON data sent from the device is received by the server. The server receives this data via an HTTP request and first verifies its contents. Next, it sends the verified data to a generative AI model. The generative AI model generates an appropriate response based on the user's input data. The response generated by the generative AI model is sent back to the server. The server receives this response, converts it back to JSON format if necessary, and sends it back to the device.

[1257] Display of response

[1258] The response data sent from the server to the terminal is received again by the terminal. The terminal parses this response data and uses JavaScript to display it in a user-friendly format on the browser. This allows the user to obtain specific advice and suggestions to efficiently carry out their work.

[1259] Specific example

[1260] For example, an employee of the General Affairs Division of a city hall enters "General Affairs Division of City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field, then presses the submit button. The terminal converts the input data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI model asking "How will the General Affairs Division of City Hall proceed with paperless initiatives?" The generative AI model generates advice on specific methods for proceeding with paperless initiatives based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal. The terminal analyzes the received response data and displays it to the user as specific advice such as "To proceed with paperless initiatives, follow these steps..."

[1261] Example of a prompt

[1262] Examples of prompt statements include the following:

[1263] "I'm a staff member in the General Affairs Department. I'd like to promote a paperless system, but could you tell me what steps I should take?"

[1264] By inputting such prompt messages into the AI ​​model, advice can be obtained on specific methods for implementing a paperless system.

[1265] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1266] Step 1:

[1267] The user accesses the system using a web browser. The user enters information into the input form. For example, the user might enter "General Affairs Department, City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field. The user proceeds to the next step by pressing the "Submit" button. The input is the information entered by the user into the input form, and the output is the data entered into the input form converted into JSON format.

[1268] Step 2:

[1269] The device uses JavaScript to collect form data and convert it into JSON format. Specifically, JavaScript retrieves data from input fields and converts it into a JSON object. For example, it might be converted to a format like {"Role": "General Affairs Department, City Hall", "Request": "I want to know the progress of paperless initiatives"}. The input is information entered by the user, and the output is data generated in JSON format.

[1270] Step 3:

[1271] The device uses an HTTP POST request to send the generated JSON data to the server. Specifically, it uses JavaScript's fetch API or similar to send data to a specific endpoint on the server. For example, it sends JSON data to POST / api / request. The input is data in JSON format, and the output is the request that was sent to the server.

[1272] Step 4:

[1273] The server receives an HTTP request and parses the JSON data in the request body. Specifically, the server-side program (e.g., Node.js or Python) parses the request data. For example, it parses the JSON data and extracts the values ​​of each field. The input is the JSON data sent from the terminal, and the output is the parsed data.

[1274] Step 5:

[1275] The server sends the analyzed data to the generative AI model. Specifically, it issues an HTTP request to the generative AI model's API. For example, it sends data containing a prompt to the generative AI model's endpoint. The input is the analyzed data, and the output is the request that was sent to the generative AI model.

[1276] Step 6:

[1277] The generative AI model generates a response and sends it back to the server. Specifically, the generative AI model analyzes the received data and generates an appropriate response based on that analysis. For example, it might generate a response such as, "To proceed with paperless operations, please follow these steps..." The input is prompt data sent from the server, and the output is the generated response data.

[1278] Step 7:

[1279] The server converts the response data received from the generative AI model back into JSON format and sends it to the terminal. Specifically, it structures the received response data and converts it into JSON format. The input is the response data received from the generative AI model, and the output is the data converted into JSON format.

[1280] Step 8:

[1281] The terminal receives and parses response data from the server. Specifically, it uses JavaScript to parse JSON data and retrieve data for each item. The input is the JSON data sent from the server, and the output is the parsed response data.

[1282] Step 9:

[1283] The terminal analyzes response data and displays it to the user. Specifically, it inserts the response data into an HTML element and displays it in the browser. For example, it might display specific advice such as, "To proceed with paperless operations, follow these steps..." The input is the analyzed response data, and the output is the content displayed to the user.

[1284] (Application Example 1)

[1285] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1286] In virtual stores, there is a need to respond to customer inquiries quickly and appropriately. However, conventional systems lack sufficient automation to provide optimal answers tailored to the content of inquiries, making it difficult to improve customer satisfaction. Furthermore, even simple inquiries required human intervention, resulting in increased effort and cost. To solve this problem, there is a need to automate inquiry handling using generative AI.

[1287] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1288] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, a data transmission and reception means for sending the analyzed information to a generative AI and receiving a response, and an API integration means for calling the generative AI's API and automatically generating appropriate responses to customer inquiries in the virtual store. This makes it possible to automate customer inquiries in the virtual store quickly and accurately.

[1289] "User interface means" refers to an interface for users to input information, and specifically includes input forms, buttons, and the like.

[1290] "Analysis means" refers to means for analyzing information input through a user interface and handling it as data.

[1291] "Data transmission and reception means" refers to means for transmitting information analyzed by the analysis means to the generative AI and receiving a response from the generative AI.

[1292] "Display means" refers to means for displaying the response received by the data transmission means to the user.

[1293] "Generative AI" refers to artificial intelligence that automatically generates appropriate responses based on input data.

[1294] "API integration means" refers to a method for calling the API of a generative AI and exchanging data in conjunction with an external system.

[1295] A "virtual store" is an online store that provides goods and services via the internet.

[1296] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a method for representing data in a lightweight and structured format.

[1297] As a concrete example of implementing this invention, a customer service system for a virtual store is provided. The system includes a user interface means for users to input information. This user interface means includes an input form with multiple different input fields, allowing customers to easily input their inquiries.

[1298] The input information is analyzed by an analysis tool. This analysis tool uses a programming language such as JavaScript to convert the input data into JSON format. The converted data is then sent to the server via the analysis tool.

[1299] The server processes the received JSON data using a data transmission and reception mechanism and calls the generative AI API. This generative AI API automatically generates an appropriate response based on the input data, and can use, for example, the OpenAI API. The server converts the response from the generative AI back into JSON format and sends it back to the terminal.

[1300] The terminal displays the received response data to the customer via a display device. This series of processes enables quick and efficient customer service in the virtual store.

[1301] Hardware and software to use

[1302] 1. Hardware

[1303] Server: Cloud server (e.g., Amazon Web Services, Google Cloud Platform)

[1304] Device: The iOS or Android device used by the user.

[1305] 2. Software

[1306] Framework: Flask (a web framework written in Python)

[1307] HTTP Library: Requests (Python's HTTP library)

[1308] Generative AI: OpenAI API, etc.

[1309] The server receives the user's inquiry data, converts it to JSON format, and works with a generative AI API to generate an appropriate response. The generated response is converted back to JSON format and sent to the user's device. The device then displays this response to the user.

[1310] Specific example

[1311] The following example prompt will be used to generate a response from the generative AI.

[1312] Example of a prompt:

[1313] User inquiry: "What is the best way to use the specified product?"

[1314] This system enables the rapid and accurate automation of customer inquiries at virtual stores, leading to improved customer satisfaction. Furthermore, because responses are generated automatically without the need for human intervention, it also reduces labor and costs.

[1315] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1316] Step 1:

[1317] The user enters information.

[1318] Users enter their inquiries using the user interface of a smartphone application. Specifically, they enter information into multiple input fields in an inquiry form and press the submit button. The input data is in text format and may include questions such as, "What is the best way to use the specified product?"

[1319] Step 2:

[1320] The device converts the data to JSON format.

[1321] The terminal converts the data entered by the user into JSON format using methods such as JavaScript. This conversion structures the data, making it easier to send to the server. For example, the input data is converted into JSON format as follows:

[1322] json

[1323] {

[1324] "query": "What is the best way to use the specified product?"

[1325] }

[1326] Step 3:

[1327] Send data to the server

[1328] The terminal sends the converted JSON data to the server using an HTTP request. The transmitted data is then passed directly to the analysis tool.

[1329] Step 4:

[1330] The server analyzes the data.

[1331] The server analyzes the received JSON data using parsing tools. This analysis helps understand what information the query requires. Specifically, it extracts the contents of the "query" field of the data and prepares it for the next processing step.

[1332] Step 5:

[1333] Call the API of a generative AI.

[1334] The server sends the analyzed query content as a request to the generative AI's API. Using the API integration method, the query content is sent as a prompt message to the OpenAI API. An example of a prompt message sent is as follows:

[1335] User inquiry: "What is the best way to use the specified product?"

[1336] Step 6:

[1337] Generative AI generates responses.

[1338] The generative AI generates a response based on the received prompt. This response includes specific advice, such as "This product is best used in the following procedure." The generated response is sent to the server in JSON format.

[1339] Step 7:

[1340] The server converts the response back to JSON format.

[1341] The server converts the response received from the generative AI back into JSON format. This response data is also in a structured format, for example, as follows:

[1342] json

[1343] {

[1344] "Response": "This product is best used in the following way:"

[1345] }

[1346] Step 8:

[1347] The device receives a response.

[1348] The terminal receives response data in JSON format sent from the server. It then parses this data and prepares to display the necessary information on the screen.

[1349] Step 9:

[1350] The terminal displays a response to the user.

[1351] The terminal displays the received response data to the user through a display mechanism. The user can see specific advice on the screen, such as, "This product is best used in the following way."

[1352] This series of processes allows users to receive quick and appropriate responses, streamlining customer inquiries in virtual stores.

[1353] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1354] This invention relates to a system in which a user inputs information, and a response is provided by combining a generative AI and an emotion recognition means. The specific program processing of this system is described below in natural language, along with concrete examples.

[1355] System Overview

[1356] 1. Providing an input form

[1357] The system provides an input form that users can access through a web browser. This form has multiple input fields where users enter the necessary information.

[1358] 2. Data Analysis

[1359] When a user enters data into an input form and presses the submit button, the device uses JavaScript to collect the form data and convert it into JSON format. This ensures that the data is sent to the server efficiently.

[1360] 3. Emotion recognition

[1361] The device is equipped with emotion recognition capabilities to analyze the user's facial expressions, voice, or biosignals. The device collects and analyzes the user's emotion data. This emotion data is also converted to JSON format and sent to the server.

[1362] 4. Sending data and receiving responses

[1363] The server receives data in JSON format sent from the terminal. The server analyzes the received data and sends a request to the generative AI's API. The generative AI generates a response based on the input data. The server receives this response, converts it back to JSON format, and sends it back to the terminal.

[1364] 5. Display of response

[1365] The device receives response data sent back from the server. The received data is analyzed and displayed in the browser in a user-friendly format. Furthermore, customized feedback based on the user's emotions is also displayed.

[1366] Specific example

[1367] 1. Using the input form

[1368] A user (for example, an employee of the General Affairs Department of the city hall) enters "General Affairs Department of City Hall" in the "Role" field, enters "I would like to know the progress of paperless operations" in the "Request" field, and presses the submit button.

[1369] 2. Collection of emotional data

[1370] The emotion recognition system identifies emotions such as "interest" or "confusion" from the user's facial expressions and voice. This emotion data is also sent to the server.

[1371] 3. Data transmission and analysis

[1372] The terminal converts input data and sentiment data into JSON format and sends it to the server. The server analyzes the received data and sends a request to a generative AI asking "How should the general affairs department of the city hall proceed with paperless operations?"

[1373] 4. Response from a generative AI

[1374] The generative AI generates advice on specific methods for implementing a paperless system based on the analyzed data and sends the response back to the server. The server receives this response, converts it back into JSON format, and sends it back to the terminal.

[1375] 5. Display of response

[1376] The device analyzes the response data it receives and displays it to the user as specific advice, such as "To proceed with paperless operations, please take the following steps..." It also displays customized feedback, including additional explanations and links to resources, based on the user's emotions, such as "interest" or "confusion." The user can then take specific actions based on this feedback.

[1377] This system can streamline inefficient processes caused by outdated practices in government offices, thereby improving productivity. Users can easily input information and receive specific suggestions and emotion-based feedback from generative AI, leading to improved work processes, reduced overtime, and a shift towards a paperless environment.

[1378] The following describes the processing flow.

[1379] Step 1:

[1380] The user opens a web browser and accesses the specified URL. This displays an input form.

[1381] Step 2:

[1382] The user enters the necessary information into each field of the input form (e.g., "Role", "Request").

[1383] Step 3:

[1384] The user clicks the "Submit" button. This collects the input data.

[1385] Step 4:

[1386] The JavaScript on the device converts the collected data into JSON format.

[1387] Step 5:

[1388] The device sends JSON data to the server as a POST request.

[1389] Step 6:

[1390] The server receives the POST request and extracts JSON data from the request body.

[1391] Step 7:

[1392] The server parses the extracted JSON data and splits it into individual fields (e.g., role, request).

[1393] Step 8:

[1394] The user's webcam, microphone, and other sensors collect the user's facial expressions, voice, and biosignals.

[1395] Step 9:

[1396] The emotion recognition system analyzes the collected data to identify the user's emotions.

[1397] Step 10:

[1398] The device converts the emotional data into JSON format and sends it to the server.

[1399] Step 11:

[1400] The server receives and analyzes emotional data.

[1401] Step 12:

[1402] The server sends the analysis data and emotion data as requests to the generative AI's API.

[1403] Step 13:

[1404] A generative AI receives a request and generates a response based on the input data and sentiment data.

[1405] Step 14:

[1406] The generative AI generates a response which is then sent back to the server.

[1407] Step 15:

[1408] The server receives the response from the generative AI and converts it into the required format.

[1409] Step 16:

[1410] The server returns the converted response data to the terminal in JSON format.

[1411] Step 17:

[1412] The terminal receives and parses the JSON data sent back from the server.

[1413] Step 18:

[1414] The data analyzed by the device is embedded in an HTML element to be displayed visually to the user.

[1415] Step 19:

[1416] The device also displays customized feedback based on the user's emotions.

[1417] Step 20:

[1418] Users review the displayed response data and customized feedback to take concrete actions for business improvement and productivity enhancement.

[1419] (Example 2)

[1420] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1421] In modern business processes, there is a need for systems that generate appropriate responses to information entered by users. However, conventional systems cannot take user emotions into account, making it difficult to provide specific feedback tailored to individual needs and feelings. Furthermore, efficient and highly accurate processing is required in data analysis and exchange with generative artificial intelligence. To solve these problems, a response generation system that incorporates user emotion data is necessary.

[1422] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1423] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, and a data transmission and reception means for sending data to a generative artificial intelligence and receiving a response. This enables the provision of customized feedback based on the user's emotions by including an emotion recognition means for collecting and analyzing the user's emotion data, a data transmission and reception means for sending the emotion data to the generative artificial intelligence, and a display means for displaying the received response to the user.

[1424] "User interface means" refers to an interface for users to input information, and includes devices such as input forms and buttons.

[1425] "Analysis means" refers to software or hardware devices for analyzing information input through a user interface means.

[1426] "Generative artificial intelligence" refers to artificial intelligence technologies that generate responses based on user input, and specifically includes natural language processing models.

[1427] A "data transmission and reception means" is a device that has network communication capabilities for transmitting information analyzed by an analysis means to a generative artificial intelligence and receiving a response.

[1428] "Display means" refers to devices such as displays and monitors that visually display responses received by data transmission means to the user.

[1429] "Emotion recognition means" refers to software or hardware devices that analyze a user's facial expressions, voice, or biosignals to collect emotional data.

[1430] "Structured data format" is a general term for data formats that represent data in a standardized format such as JSON and allow for its exchange with other data.

[1431] An "input form" is a form with multiple different input fields, and is an interface that users use to enter specific information.

[1432] This invention is a system in which a user inputs information and a response is provided by combining generative artificial intelligence and emotion recognition means. In this system, the user can input information via a web browser and receive a response from generative artificial intelligence. Furthermore, by collecting the user's emotion data and reflecting it in the response, the system provides more personalized feedback.

[1433] Specifically, the following hardware and software will be used.

[1434] Hardware:

[1435] 1. Server - A computer used for high-speed and stable data processing and communication.

[1436] 2. Device - A device used by the user, such as a PC, smartphone, or tablet.

[1437] 3. Camera and microphone - Input devices for capturing the user's facial expressions and voice.

[1438] software:

[1439] 1. Web browser - Software used as a user interface.

[1440] 2. HTML, CSS, JavaScript - Languages ​​used to build input forms and user interfaces.

[1441] 3. OpenCV, TensorFlow - Libraries used for emotion recognition.

[1442] 4. Generative AI APIs - AI interfaces used to generate responses (e.g., GPT-3).

[1443] Here are some specific examples:

[1444] 1. User information input:

[1445] A user (for example, an employee of the general affairs department of a city hall) enters "General Affairs Department of City Hall" in the "Role" field and "I would like to know the progress of paperless initiatives" in the "Request" field of an input form on a web browser. Then they press the "Submit" button.

[1446] 2. Data analysis and transmission:

[1447] The device uses JavaScript to collect the entered information and converts it to JSON format using the JSON.stringify() function. The converted data is then sent to the server as an HTTP POST request.

[1448] 3. Collection and analysis of emotional data:

[1449] The device uses its camera and microphone to capture the user's facial expressions and voice, and performs emotion analysis using OpenCV and TensorFlow. The analysis results (e.g., "interested" or "confused") are converted to JSON format and sent to the server.

[1450] 4. Data transmission to generative artificial intelligence:

[1451] The server analyzes the user's input data and sentiment data, and sends requests to a generative artificial intelligence API (e.g., GPT-3). The following is an example of a prompt:

[1452] Text format: "How should the general affairs department of the city hall proceed with paperless operations?"

[1453] Text format: "How can we provide user feedback based on their emotions?"

[1454] 5. Receiving and displaying responses:

[1455] The server receives the response from the generative artificial intelligence, converts it to JSON format, and sends it to the terminal. The terminal parses this and displays it in a user-friendly format in the browser. For example, it might display advice such as, "To proceed with paperless operations, follow these steps..." It also displays additional customized feedback based on the user's interests and concerns.

[1456] In this way, users can easily input information and receive specific suggestions and emotion-based feedback from generative artificial intelligence. This can lead to improvements and increased efficiency in their work.

[1457] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1458] Step 1:

[1459] Providing an input form

[1460] The server provides an input form for the user to enter information. The server generates the input form using HTML, CSS, and JavaScript, and sends it to the user's web browser.

[1461] Input: HTML, CSS, and JavaScript code from the server.

[1462] Output: An input form displayed in the user's web browser.

[1463] Specific actions:

[1464] Users enter information such as their "role" and "requests" into this form. For example, they might enter "General Affairs Department, City Hall" for "role" and "I would like to know the progress of the paperless initiative" for "request."

[1465] Step 2:

[1466] Data collection and transmission

[1467] When the user presses the "Submit" button, the device uses JavaScript to collect the entered information. The collected data is converted to JSON format and sent to the server as an HTTP POST request.

[1468] Input: Information entered by the user.

[1469] Output: The data, converted to JSON format, is sent to the server.

[1470] Specific actions:

[1471] The device uses an event listener to detect a click of the "Submit" button, retrieves the form data using JavaScript, and converts it to JSON format using the JSON.stringify() function. This is then sent to the server as an HTTP POST request.

[1472] Step 3:

[1473] Collection and analysis of emotional data

[1474] The device uses its camera and microphone to capture the user's facial expressions and voice, and analyzes them using emotion recognition technology. The analysis results are also converted into JSON format and sent to the server.

[1475] Input: Real-time data from camera and microphone.

[1476] Output: The analyzed emotion data is converted to JSON format and sent to the server.

[1477] Specific actions:

[1478] The device uses OpenCV and TensorFlow to analyze the user's facial expressions and voice, collecting emotional data such as "interest" and "confusion." This data is then converted to JSON format and sent to the server as an additional HTTP POST request.

[1479] Step 4:

[1480] Sending data to generative artificial intelligence

[1481] The server analyzes user input data and sentiment data and sends a request to a generative artificial intelligence. It generates a request that includes a prompt and sentiment data.

[1482] Input: User input data and sentiment data.

[1483] Output: Request sent to the generative artificial intelligence.

[1484] Specific actions:

[1485] The server analyzes the received data using Python, Node.js, etc., and generates appropriate prompts for a generative artificial intelligence (e.g., GPT-3). For example, it might generate a prompt such as "How should the general affairs department of the city hall proceed with paperless operations?" and send it to the generative AI.

[1486] Step 5:

[1487] Receiving responses and sending data from generative artificial intelligence.

[1488] The server receives the response from the generative artificial intelligence and converts it into JSON format. It then sends the converted data to the terminal.

[1489] Input: Response data from a generative artificial intelligence.

[1490] Output: JSON-formatted response data sent to the terminal.

[1491] Specific actions:

[1492] The server receives the response from the generative artificial intelligence and converts it into JSON format. This converted data is then sent to the terminal as an HTTP response.

[1493] Step 6:

[1494] Display of response and feedback

[1495] The device parses the received JSON response data and displays it in a user-friendly format on the browser. It also provides customized feedback based on the user's emotions.

[1496] Input: JSON-formatted response data received from the server.

[1497] Output: Analyzed response data and sentiment-based feedback displayed in the user's browser.

[1498] Specific actions:

[1499] The JSON data received by the device is displayed in the browser using HTML / CSS. For example, it might display specific advice such as, "To proceed with paperless operations, follow these steps..." Furthermore, it displays links to additional information and resources based on the user's "interests" and "concerns."

[1500] This system allows users to easily input information and receive specific suggestions and emotion-based feedback from generative artificial intelligence. This can lead to improvements and increased efficiency in work processes.

[1501] (Application Example 2)

[1502] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1503] Traditional in-store customer service often suffers from insufficient personalization, making it difficult to enhance customer satisfaction. Furthermore, store staff struggle to accurately understand customers' emotions and interests, hindering their ability to provide optimal product recommendations. In addition, inconsistent staff service can often degrade the quality of the customer experience.

[1504] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1505] In this invention, the server includes a user interface means for the user to input information, an analysis means for analyzing the input information, a data transmission and reception means for transmitting the analyzed information to a generative AI and receiving a response, a display means for displaying the received response to the user, an emotion recognition means for analyzing the user's emotions using an facial recognition camera, and a data transmission and reception means for transmitting emotion data to the generative AI. This enables personalized product suggestions based on customer emotions and interests in physical stores. Furthermore, it allows store staff to provide consistently high-quality customer service, thereby improving customer satisfaction.

[1506] "User interface means" refers to an interface that a user uses to input information.

[1507] "Analysis means" refers to means for analyzing information input through a user interface.

[1508] "Data transmission and reception means" refers to means for transmitting information analyzed by the analysis means to the generative AI and receiving a response from the generative AI.

[1509] "Display means" refers to means for displaying the response received by the data transmission means to the user.

[1510] An "input form" refers to a web form that has multiple different input fields for users to enter information.

[1511] A "facial recognition camera" refers to a camera device that captures a user's facial expressions and analyzes emotional data.

[1512] "Emotion recognition means" refers to a method for analyzing a user's emotions using a facial recognition camera.

[1513] "Generative AI" refers to an artificial intelligence system that generates responses based on input data.

[1514] A "custom support app for physical stores" refers to a mobile application used by store staff in physical stores to provide optimal product recommendations through dialogue with customers.

[1515] One embodiment of this invention is a system for personalizing customer service in physical stores and improving customer satisfaction. This system takes user input and provides a response by combining generative AI and emotion recognition means.

[1516] First, the system provides a user interface. This interface is for use by in-store staff via smartphones or tablets and provides an input form with multiple input fields for entering customer information, purchase history, interests, and more.

[1517] Next, the information entered on the terminal is analyzed using an analysis tool. This analysis tool primarily analyzes text data to understand customer needs and interests. This data is converted to JSON format and sent to the server. The server is equipped with a data transmission and reception mechanism, which sends the analyzed information to a generative AI and receives the response.

[1518] This system includes a facial recognition camera that captures the user's facial expressions. The facial expression data is analyzed using emotion recognition technology to understand the user's emotions. This emotion data is also converted to JSON format and sent to the server. All information, including the emotion data, is processed by generative AI to generate appropriate product suggestions and customized feedback.

[1519] As a concrete example, consider a scenario where a user shows interest in "casual fashion" in a physical store. Based on the user's purchase history, interests, and the emotion of "interest" inferred from their facial expressions, the generative AI makes optimal product suggestions. The prompt text in this case would be as follows:

[1520] The customer's name is Taro Yamada. His purchase history includes shirts and shoes, and his interest lies in casual fashion. He appears interested based on his expression. Please provide him with the most suitable product suggestions and additional information.

[1521] The server converts the response received from the generative AI into JSON format and sends it back to the terminal. The terminal analyzes the received data and displays it as information for in-store staff to make suggestions to customers. In addition, more personalized feedback is provided based on sentiment data. This system not only improves customer satisfaction but also ensures consistent service quality from store staff.

[1522] The main hardware and software used to implement this system include smartphones, tablets, facial recognition cameras, and generative AI models (e.g., GPT-3). Additionally, server-side APIs and associated analysis software are required for data transmission, reception, and analysis.

[1523] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1524] Step 1:

[1525] Users use their smartphones in physical stores to input customer information through a user interface. Specifically, they enter their purchase history and interests into input fields. An example of input data is "Name: Taro Yamada," "Purchase History: Shirts, Shoes," and "Interests: Casual Fashion." The entered data is temporarily stored internally.

[1526] Step 2:

[1527] The device uses a facial recognition camera to capture customer facial data. The facial data captured by the camera is transmitted in real time to an emotion recognition system for analysis. For example, the emotion of "interested" is analyzed. This emotion data is also temporarily stored internally.

[1528] Step 3:

[1529] The terminal converts the analyzed input data and sentiment data into JSON format. The converted data is in the following format as an example:

[1530] json

[1531] {

[1532] "name": "Yamada Taro",

[1533] "purchase_history": ["shirt", "shoes"],

[1534] "interests": "casual fashion",

[1535] "emotion": "Interested"

[1536] }

[1537] Step 4:

[1538] The terminal sends the data, converted to JSON format, to the server. A data transmission method is used for this process. The data is sent as an HTTP POST request.

[1539] Step 5:

[1540] The server analyzes the received data and sends a request to a generative AI model (e.g., GPT-3). This request includes data and a corresponding prompt. Specifically, the prompt used might be: "The customer's name is Yamada Taro. His purchase history includes shirts and shoes, and his interest is in casual fashion. His facial expression suggests he is interested. Please provide him with the most suitable product suggestions and additional information."

[1541] Step 6:

[1542] The generative AI model generates a response based on the prompt and data. An example of the generated response might be, "We recommend these new jackets and sneakers, perfect for casual fashion. For more information, please refer to this link." This response data is then sent back to the server.

[1543] Step 7:

[1544] The server receives the response from the generative AI model and converts it into JSON format. The converted response data is in the following format:

[1545] json

[1546] {

[1547] "recommendations": ["New jacket", "Sneakers"],

[1548] "additional_info": "For more detailed information, please refer to this link."

[1549] }

[1550] Step 8:

[1551] The server sends response data in JSON format back to the terminal. The returned data is then sent again as an HTTP POST request.

[1552] Step 9:

[1553] The device analyzes the received response data and displays it to the user. The displayed content may include customized feedback based on the customer's sentiment. For example, it might display: "For those interested in casual fashion, we recommend our new jackets and sneakers. For more information, please see this link."

[1554] The above steps enable personalized customer service in physical stores.

[1555] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1556] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1557] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1558] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1559] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1560] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1561] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1562] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1563] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1564] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1565] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1566] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1567] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1568] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1569] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1570] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1571] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1572] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1573] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1574] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1575] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1576] The following is further disclosed regarding the embodiments described above.

[1577] (Claim 1)

[1578] A user interface means for the user to input information,

[1579] Analysis means for analyzing information input through the user interface means,

[1580] A data transmission and reception means for transmitting the information analyzed by the aforementioned analysis means to a generative AI and receiving a response,

[1581] A display means for displaying the response received by the data transmission and reception means to the user,

[1582] A system that includes this.

[1583] (Claim 2)

[1584] The system according to claim 1, wherein the data transmission / reception means transmits and receives data in JSON format with the generative AI.

[1585] (Claim 3)

[1586] The system according to claim 1, wherein the user interface means provides an input form having a plurality of different input fields.

[1587] "Example 1"

[1588] (Claim 1)

[1589] A user interface means for the user to input information,

[1590] A conversion means for collecting information input through the user interface means and converting it into JSON format,

[1591] A data transmission means for analyzing the JSON format information converted by the aforementioned conversion means and sending a request to a generative AI model,

[1592] A means for converting the response received from the aforementioned generative AI model back into JSON format and displaying it to the user,

[1593] A system that includes this.

[1594] (Claim 2)

[1595] The system according to claim 1, wherein the data transmission means sends and receives data in JSON format with the generative AI model.

[1596] (Claim 3)

[1597] The system according to claim 1, wherein the user interface means provides an input form having a plurality of different input fields.

[1598] "Application Example 1"

[1599] (Claim 1)

[1600] A user interface means for the user to input information,

[1601] Analysis means for analyzing information input through the user interface means,

[1602] A data transmission and reception means for transmitting the information analyzed by the aforementioned analysis means to a generative AI and receiving a response,

[1603] A display means for displaying the response received by the data transmission and reception means to the user,

[1604] An API integration method for calling a generative AI API to automatically generate appropriate responses to customer inquiries in a virtual store,

[1605] A system that includes this.

[1606] (Claim 2)

[1607] The system according to claim 1, wherein the data transmission / reception means transmits and receives data in JSON format with the generative AI.

[1608] (Claim 3)

[1609] The system according to claim 1, wherein the user interface means provides an input form having a plurality of different input fields.

[1610] "Example 2 of combining an emotion engine"

[1611] (Claim 1)

[1612] A user interface means for the user to input information,

[1613] Analysis means for analyzing information input through the user interface means,

[1614] A data transmission and reception means for transmitting the information analyzed by the aforementioned analysis means to a generative artificial intelligence and receiving a response,

[1615] A display means for displaying the response received by the data transmission and reception means to the user,

[1616] A means of emotion recognition that collects and analyzes user emotion data,

[1617] A data transmission and reception means for transmitting emotional data collected by the aforementioned emotion recognition means to a generative artificial intelligence,

[1618] A system that includes this.

[1619] (Claim 2)

[1620] The system according to claim 1, wherein the data transmission and reception means transmits and receives data in structured data format with the generative artificial intelligence.

[1621] (Claim 3)

[1622] The system according to claim 1, wherein the user interface means provides an input form having a plurality of different input fields.

[1623] "Application example 2 when combining with an emotional engine"

[1624] (Claim 1)

[1625] A user interface means for the user to input information,

[1626] Analysis means for analyzing information input through the user interface means,

[1627] A data transmission and reception means for transmitting the information analyzed by the aforementioned analysis means to a generative AI and receiving a response,

[1628] A display means for displaying the response received by the data transmission and reception means to the user,

[1629] The user interface means includes means for providing an input form having a plurality of different input fields,

[1630] An emotion recognition means for analyzing a user's emotions using a facial recognition camera,

[1631] A data transmission and reception means for transmitting emotional data received by the emotion recognition means to a generative AI,

[1632] A system that includes this.

[1633] (Claim 2)

[1634] The system according to claim 1, wherein the data transmission / reception means transmits and receives data in JSON format with the generative AI.

[1635] (Claim 3)

[1636] The system according to claim 1, which provides a custom support app for physical stores in which store staff collect purchase history and interests through conversations with customers in physical stores and make optimal product suggestions using generative AI and emotion recognition technology. [Explanation of Symbols]

[1637] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A user interface means for the user to input information, Analysis means for analyzing information input through the user interface means, A data transmission and reception means for transmitting the information analyzed by the aforementioned analysis means to a generative AI and receiving a response, A display means for displaying the response received by the data transmission and reception means to the user, A system that includes this.

2. The system according to claim 1, wherein the data transmission and reception means transmits and receives data in JSON format with the generative AI.

3. The system according to claim 1, wherein the user interface means provides an input form having a plurality of different input fields.

Images