system

Abstract

We provide the system. [Solution] An input method for users to enter questions or doubts, A means for sending the entered question to the server, A natural language processing means for analyzing submitted questions and retrieving related information, A means for generating answers based on searched information, A means for sending the generated response to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In operations that require specialized knowledge such as legal affairs, finance, and accounting departments, even though manuals are publicly available, there are many cases where it is not clear what questions should be asked. Especially in accounting-related matters, since they are closely related to operations, many doubts arise. To address such doubts, conventionally, it is often necessary to directly inquire with the accounting staff. However, as the number of inquiries increases, the burden on the accounting staff increases, causing delays in operations. An object of the present invention is to provide a system that improves the efficiency of confirming and acquiring such specialized knowledge and prevents delays in operations.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides a system that includes an input means for a user to input questions or doubts, a transmission means for sending the input questions to a server, a natural language processing means for analyzing the transmitted questions and searching for related information, an answer generation means for generating answers based on the searched information, and a transmission means for sending the generated answers to the user. With this system, users can efficiently acquire knowledge in an interactive format without having specialized knowledge, and inquiries to accounting staff can be reduced, thereby preventing delays in operations.

[0006] A "user" refers to a person who uses the system to input questions or inquiries.

[0007] "Input means" refers to devices or software that provide an interface for users to input questions or doubts.

[0008] "Transmission means" refers to devices or software that have the function of sending questions entered by the user to the server.

[0009] A "server" refers to a computer system that receives questions submitted by users, analyzes them, retrieves information, and generates answers.

[0010] "Natural language processing means" refers to algorithms and software used to analyze submitted questions and extract and retrieve relevant information.

[0011] "Category" refers to a classification criterion used to categorize questions into specific fields of expertise or themes.

[0012] "Answer generation means" refers to devices or software that have the function of constructing answers to user questions based on retrieved information.

[0013] "Answer" or "generated answer" refers to the information or explanation provided in response to a user's question.

[0014] "Search means" refers to a device or software having a function to search for relevant information from a database or a manual based on a question.

[0015] "Database" refers to a digital storage device for storing information and data in an organized manner.

[0016] "FAQ" refers to a list summarizing frequently asked questions and their answers.

[0017] "Screen display means" refers to a device or software for displaying an answer to a question sent by a consumer on the screen of a terminal.

Brief Description of Drawings

[0018] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing disruptions to their work. This system consists of user, terminal, and server components. The system's program processing and specific implementation examples are described below.

[0040] 1. User input of question

[0041] Users access the system using devices such as PCs and smartphones. Users enter their questions through an interface for entering doubts and inquiries (e.g., a text box). For example, a user might enter, "Please tell me how accounts receivable are accounted for."

[0042] 2. Sending questions via device

[0043] The questions entered by the user are sent from the terminal to the server. In this process, data is sent to the server as an HTTP request (e.g., a POST request).

[0044] 3. Server analysis of the question

[0045] The server analyzes the received question. Specifically, it uses natural language processing (NLP) techniques to analyze the question and extract relevant keywords. For example, keywords such as "accounts receivable" and "accounting method" may be extracted.

[0046] 4. Server-side determination of question category

[0047] The server determines the category of the question based on the extracted keywords. For example, since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[0048] 5. Searching for information on the server

[0049] The server searches for relevant information from internal databases, manuals, FAQ databases, etc., based on the question category. For example, if information on "how to account for accounts receivable" is found in Chapter 3, Section 3.2 of the manual, it will locate it.

[0050] 6. Server-driven response generation

[0051] Based on the searched information, the server generates an appropriate answer to the user's question. For example, it might generate an answer such as, "I will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided. Accounting is done in the following steps: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable."

[0052] 7. Server sends response

[0053] The generated response is sent from the server to the terminal. This is also sent as an HTTP response and received by the terminal.

[0054] 8. Displaying responses via device

[0055] The terminal receives the response sent from the server and displays it on the user's screen. The user can then review this and resolve their question.

[0056] 9. User input of additional questions

[0057] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[0058] Specific example:

[0059] If a user enters "Please tell me how to record accounts receivable," the server identifies this question as belonging to the "Accounting" category and searches Chapter 3, Section 3.2 of the manual to generate an appropriate answer. The answer is: "We will explain how to record accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The recording process is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match sales with accounts receivable." This answer is displayed on the user's terminal, allowing the user to resolve their question.

[0060] This invention allows users to efficiently acquire knowledge through an interactive format, even without specialized knowledge, and prevents delays in their work.

[0061] The following describes the processing flow.

[0062] Step 1:

[0063] The user accesses the system's web application or dedicated app. The user enters "Please tell me how accounts receivable are accounted for" into the interface (text box) for entering questions or inquiries.

[0064] Step 2:

[0065] The terminal sends the user's entered question to the server. Specifically, it sends the question's text data to the server as an HTTP POST request.

[0066] Step 3:

[0067] The server receives the sent HTTP request and retrieves the question text. The question text is "Please tell me how to account for accounts receivable."

[0068] Step 4:

[0069] The server analyzes the question text using a natural language processing (NLP) library. Specifically, it performs keyword extraction, identifying important keywords such as "accounts receivable" and "accounting method."

[0070] Step 5:

[0071] The server determines the category of the question based on the extracted keywords. In this case, the keyword "accounts receivable" indicates that the question belongs to the "accounting" category.

[0072] Step 6:

[0073] The server searches for relevant information from its internal database or manual based on the category. For example, it might identify information on "Accounts Receivable Accounting Methods" from Chapter 3, Section 3.2 of the manual.

[0074] Step 7:

[0075] The server generates an answer based on the searched information. It generates an answer such as: "We will explain how to record accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. The recording procedure is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match the sales with the accounts receivable."

[0076] Step 8:

[0077] The server sends the generated response to the user's device. Specifically, it returns the response text to the device as an HTTP response.

[0078] Step 9:

[0079] The device displays the received response on the user's screen. The user reviews the response displayed on the screen and resolves their question.

[0080] Step 10:

[0081] If the user has any further questions, they can enter them again in the text box and click the submit button. This process is repeated, allowing the user to obtain the necessary information step by step.

[0082] The above outlines the specific flow of program processing in this system.

[0083] (Example 1)

[0084] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0085] Many modern jobs require specialized knowledge, but it is extremely difficult for users to efficiently acquire this knowledge and smoothly carry out their work. Furthermore, a lack of appropriate guidance and learning materials can cause work to stall. To solve these problems, there is a need for a system that allows users to easily access and deepen their understanding of specialized knowledge.

[0086] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0087] In this invention, the server includes an input means for the user to input questions or doubts, a transmission means for sending the input questions to the server, a natural language processing means for analyzing the transmitted questions and searching for related information, a generative AI model means for generating answers based on the searched information, and a transmission means for sending the generated answers to the user. This makes it possible for users to efficiently acquire knowledge in an interactive format without having specialized knowledge, and to prevent delays in their work.

[0088] An "input method" is a means of providing an interface for users to input questions or doubts.

[0089] "Transmission method" refers to the means by which questions and data entered by the user are sent to the server.

[0090] "Natural language processing means" refers to technologies and methods used to analyze submitted questions and retrieve relevant information.

[0091] "Generative AI model means" refers to a means of using an artificial intelligence model to generate answers based on searched information.

[0092] An "answer generation means" is a means of generating an appropriate answer to a user's question.

[0093] A "category classification method" is a means of classifying questions analyzed using natural language processing into specific categories.

[0094] A "past questions and FAQ database" refers to a database that compiles questions and frequently asked questions accumulated to date.

[0095] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing disruptions to their work. This system consists of user, terminal, and server components.

[0096] 1. User input of question

[0097] Users access the system using devices such as PCs and smartphones. A text box appears on the input screen, into which the user enters their question. For example, they might enter, "Please tell me how accounts receivable are accounted for."

[0098] 2. Sending questions via device

[0099] The question entered by the user is sent from the terminal to the server as an HTTP POST request. The POST request contains the user's question.

[0100] 3. Server analysis of the question

[0101] The server uses a natural language processing (NLP) library (e.g., SpaCy) to analyze the received data. It divides the question into tokens, analyzes their meaning, and extracts keywords.

[0102] 4. Server-side determination of question category

[0103] The server uses a machine learning model (e.g., a Scikit-learn classifier) ​​to determine the category of the question based on the extracted keywords. The model is pre-trained on data related to categories such as accounting and legal affairs.

[0104] 5. Searching for information on the server

[0105] Based on the question category, the server searches for relevant information from its internal databases and manuals (e.g., the MySQL® database). It then identifies the most relevant information from the search results.

[0106] 6. Server-driven response generation

[0107] Based on the searched information, the server uses a generative AI model (e.g., GPT-3®) to generate appropriate answers. The answers are then formatted in a user-friendly manner.

[0108] 7. Server sends response

[0109] The generated response is sent from the server to the terminal as an HTTP response. This includes data in JSON format.

[0110] 8. Displaying responses via device

[0111] The terminal displays the response data received from the server on the screen. The user reviews this and resolves their questions.

[0112] 9. User input of additional questions

[0113] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[0114] Specific example:

[0115] If a user enters "Please tell me how to record accounts receivable," the server identifies this question as belonging to the "Accounting" category and searches Chapter 3, Section 3.2 of the accounting manual to generate an appropriate answer. The answer is: "We will explain how to record accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. The recording procedure is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match sales with accounts receivable." This answer is displayed on the user's terminal, allowing the user to resolve their question.

[0116] The above is one specific example of the present invention. Through the processing procedure described above, users can effectively proceed with learning and prevent delays in their work, even without specialized knowledge.

[0117] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0118] Step 1:

[0119] User input of question

[0120] Processing details:

[0121] Users access the system using a PC or smartphone and enter questions into text boxes displayed on the input screen. The entered information is saved in text format.

[0122] input:

[0123] The question entered by the user in the text box (e.g., "Please tell me how accounts receivable are accounted for.").

[0124] output:

[0125] Text data entered by the user.

[0126] Specific actions:

[0127] The user launches a browser, accesses the system's webpage, and enters the question into the text box.

[0128] Step 2:

[0129] Sending questions via device

[0130] Processing details:

[0131] The question entered by the user is sent from the terminal to the server as an HTTP POST request. This request contains the question content.

[0132] input:

[0133] Text data entered by the user.

[0134] output:

[0135] An HTTP POST request sent to the server.

[0136] Specific actions:

[0137] The device sends data to the / api / question endpoint requesting information on how accounts receivable are accounted for.

[0138] Step 3:

[0139] Server-based analysis of the question

[0140] Processing details:

[0141] The server uses a natural language processing (NLP) library (e.g., SpaCy) to analyze the received question data. It splits the text data into tokens, analyzes their meaning, and extracts relevant keywords.

[0142] input:

[0143] Question data sent to the server.

[0144] output:

[0145] Extracted keywords (e.g., "accounts receivable", "accounting method").

[0146] Specific actions:

[0147] The server tokenizes the received text data using the SpaCy library and extracts important keywords.

[0148] Step 4:

[0149] Server-based question category determination

[0150] Processing details:

[0151] The server uses a machine learning model (e.g., a Scikit-learn classifier) ​​to determine the category of the question based on the extracted keywords. The model is trained on data related to categories such as accounting and legal affairs.

[0152] input:

[0153] Extracted keywords (e.g., "accounts receivable", "accounting method").

[0154] output:

[0155] The category of the question (e.g., "Accounting").

[0156] Specific actions:

[0157] The server uses a Scikit-learn classifier to analyze keywords and categorize the question into the appropriate category (e.g., "accounting").

[0158] Step 5:

[0159] Searching for information via server

[0160] Processing details:

[0161] Based on the question category, the server searches for relevant information in its internal databases and manuals (e.g., MySQL database). It then identifies the most relevant information from the search results.

[0162] input:

[0163] Categorized questions.

[0164] output:

[0165] Related information (e.g., Accounting Manual, Chapter 3, Section 3.2).

[0166] Specific actions:

[0167] The server uses an SQL query to retrieve information about "accounts receivable accounting methods" from accounting-related tables in the MySQL database.

[0168] Step 6:

[0169] Server-driven response generation

[0170] Processing details:

[0171] Based on the searched information, the server uses a generative AI model (e.g., GPT-3) to generate appropriate answers. The generated answers are then formatted in a user-friendly manner.

[0172] input:

[0173] Related information found through searches.

[0174] output:

[0175] The generated response text (example: "We will explain how to account for accounts receivable. Accounts receivable are...").

[0176] Specific actions:

[0177] Enter "Please tell me how to account for accounts receivable" as the GPT-3 prompt and retrieve the generated response.

[0178] Step 7:

[0179] Server sends response

[0180] Processing details:

[0181] The generated response is sent from the server to the terminal as an HTTP response. The response contains data in JSON format.

[0182] input:

[0183] The generated response text.

[0184] output:

[0185] JSON data sent to the terminal as an HTTP response.

[0186] Specific actions:

[0187] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0188] Step 8:

[0189] Displaying responses via device

[0190] Processing details:

[0191] The terminal displays the response data received from the server on the screen. The user can then review this and resolve their questions.

[0192] input:

[0193] JSON data sent from the server.

[0194] output:

[0195] The answer text displayed on the screen.

[0196] Specific actions:

[0197] The device displays the following response in its browser: "We will explain how to account for accounts receivable..."

[0198] Step 9:

[0199] User input of additional questions

[0200] Processing details:

[0201] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[0202] input:

[0203] Additional questions from the user.

[0204] output:

[0205] Additional questions resent to the server.

[0206] Specific actions:

[0207] The user types, "Could you please explain the accounting method in more detail?" and presses the submit button again.

[0208] (Application Example 1)

[0209] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0210] Currently, many virtual stores have customers who frequently have specialized questions regarding finance and legal matters, but there is a lack of mechanisms to answer these questions quickly and appropriately. As a result, transactions may stall because customer questions remain unresolved, potentially leading to lost business opportunities. This invention aims to solve this problem by providing a system that utilizes a generative AI model to provide appropriate answers in real time.

[0211] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0212] In this invention, the server includes an input means for the user to input questions or doubts; a transmission means for sending the input questions to the server; a natural language processing means for analyzing the transmitted questions and searching for relevant information; an answer generation means for generating answers based on the searched information; a transmission means for sending the generated answers to the user; and a means for generating answers to user-inputted questions using a generation AI model. This makes it possible to provide appropriate answers to user questions immediately.

[0213] A "user" is someone who uses the system to input questions and receive answers.

[0214] "Input methods for entering questions or doubts" refers to interfaces or devices that users use to enter questions or doubts, such as text boxes or voice input devices.

[0215] "Means of transmission for sending entered questions to the server" refers to communication means for sending questions entered by the user to the server, such as software functions for sending HTTP requests.

[0216] "Natural language processing means" refers to software that uses natural language processing techniques to analyze input questions and extract relevant information.

[0217] "Answer generation means" refers to algorithms or programs that generate appropriate answers to user questions based on retrieved information.

[0218] "Means of transmission for sending generated responses to users" refers to communication means for sending generated responses to users, such as software functions for sending HTTP responses.

[0219] A "generative AI model" refers to an artificial intelligence model used to generate answers to questions, such as a deep learning model used for natural language processing.

[0220] A "prompt statement" refers to the input text used when generating an answer for a generative AI model.

[0221] This invention relates to a system that enables users to efficiently acquire specialized knowledge in finance and law, thereby preventing disruptions to their work. This system consists of user, terminal, and server components. A detailed example of the system's operation is described below.

[0222] 1. User input of question

[0223] Users access the system using devices such as smartphones or head-mounted displays. Users input their questions through interfaces for entering doubts and inquiries, such as text boxes or voice input devices. For example, a user might input, "Please tell me how accounts receivable are accounted for."

[0224] 2. Submit the entered question.

[0225] The terminal uses a communication method to send the user-entered questions to the server, sending the data as an HTTP request.

[0226] 3. Question Analysis and Search

[0227] The server analyzes the received question using natural language processing techniques and searches for relevant information. This analysis utilizes a generative AI model. For example, it extracts keywords such as "accounts receivable" and "accounting method" and searches for relevant information from past questions and FAQ databases.

[0228] 4. Generating the answer

[0229] The server uses a generative AI model based on the retrieved information to generate appropriate answers to the user's questions. In this process, the prompt is used as input to the generative AI model. An example of a prompt is, "Please tell me how accounts receivable are accounted for."

[0230] 5. Submit the generated response.

[0231] The generated response is sent from the server to the terminal. It is sent as an HTTP response and received by the terminal.

[0232] 6. Display the answer

[0233] The terminal receives the answer sent from the server and displays it on the user's screen. The user can then review this and resolve their questions. For example, an answer might be generated stating, "We will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. Accounting is done using the following steps: 1. Process the transaction when the sale is finalized. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable."

[0234] This invention allows users to efficiently acquire knowledge through dialogue, even without specialized knowledge, and prevents delays in their work. This system utilizes a generated AI model and prompt text to provide quick and appropriate answers. Furthermore, because users can resolve financial and legal questions through dialogue, it enables the provision of high-quality customer support even in virtual stores.

[0235] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0236] Step 1:

[0237] The user inputs a question or inquiry. The user uses a smartphone or head-mounted display to input a question into the interface via text or voice input, such as "Please tell me how accounts receivable are accounted for." The input in this step is the user's question, and the output is the question data on the device.

[0238] Step 2:

[0239] The terminal sends the entered question to the server. The terminal sends the user-entered question to the server as an HTTP request (POST request). The input in this step is the question data, and the output is the request data sent to the server.

[0240] Step 3:

[0241] The server analyzes the question and searches for relevant information. The server uses natural language processing techniques to analyze the question and extract keywords. For example, it might extract "accounts receivable" or "accounting method" and search for relevant information in its internal database and FAQ database. The input for this step is the request data, and the output is the analyzed keywords and search results.

[0242] Step 4:

[0243] The server generates an answer based on the search results. The server uses a generative AI model to generate a prompt sentence from the searched information, and then uses this prompt sentence to generate an appropriate answer to the user's question. For example, it might generate an answer such as, "Let me explain how accounts receivable are accounted for. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided..." The inputs to this step are the search results and the prompt sentence, and the output is the generated answer.

[0244] Step 5:

[0245] The server sends the generated response to the terminal. The server sends the generated response to the terminal as an HTTP response. The input for this step is the generated response, and the output is the response data sent to the terminal.

[0246] Step 6:

[0247] The terminal displays the answer on the user's screen. The terminal displays the answer received from the server in the user interface. For example, it might display something like, "We will explain how accounts receivable are accounted for. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services..." The input for this step is response data, and the output is the answer displayed to the user.

[0248] This allows users to resolve their questions and perform their tasks smoothly.

[0249] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0250] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing delays in work. By incorporating an emotion engine that recognizes user emotions, this system can improve the user experience and provide more appropriate responses. The system's program processing and specific implementation examples are described below.

[0251] 1. User input of question

[0252] Users access the system using devices such as PCs and smartphones. They enter a question or inquiry into a text box interface, such as "Please tell me how accounts receivable are accounted for." During this process, an emotion engine recognizes the user's emotions (e.g., anxiety, impatience) in real time.

[0253] 2. Transmission of questions and sentiment data via the device

[0254] Along with the question entered by the user, the sentiment data recognized by the sentiment engine is sent to the server. Specifically, the text data and sentiment data are sent to the server as an HTTP POST request.

[0255] 3. Server analysis of the question

[0256] The server analyzes the submitted question text using a natural language processing (NLP) library. It extracts important keywords from the question text (e.g., "accounts receivable," "accounting method").

[0257] 4. Server-side determination of question category

[0258] The server determines the category of the question based on the extracted keywords. For example, since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[0259] 5. Use of emotional data by servers

[0260] The server also takes into account the user's emotional data as recognized by the emotion engine. For example, if the user indicates an "anxious" emotion, it adjusts its response to be more helpful and detailed.

[0261] 6. Searching for information on the server

[0262] The server searches its internal database and manuals for relevant information based on the question category and sentiment data. For example, it might identify information about "accounts receivable accounting methods" from Chapter 3, Section 3.2 of the manual.

[0263] 7. Server-driven response generation

[0264] Based on the searched information, the server generates an appropriate response tailored to the user's emotional state. For example, it might generate a response such as, "Let me explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. Accounting is done using the following steps: 1. Process the transaction as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match sales with accounts receivable. If you have any questions, please do not hesitate to contact us."

[0265] 8. Server sends response

[0266] The generated response is sent from the server to the terminal. Specifically, the response text is returned to the terminal as an HTTP response.

[0267] 9. Displaying responses via the device

[0268] The device displays the received response on the user's screen. The user reviews the displayed response and resolves their question. They can also re-enter their question if necessary.

[0269] Specific example:

[0270] If a user types "Please tell me how to account for accounts receivable" and the sentiment engine recognizes that the user is feeling "anxious," the server categorizes this question under "Accounting." It then searches for relevant manual information and generates a helpful answer that includes detailed instructions on "how to account for accounts receivable." The generated answer might say something like, "We will explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The process involves the following steps: 1. Processing the transaction as soon as the sale is confirmed. 2. Adding an accounts receivable account to your accounting software. 3. Matching sales with accounts receivable. Please feel free to contact us if you have any questions." This answer is then displayed on the user's device, resolving their concerns.

[0271] The above describes the program processing and specific embodiments of the present invention system. This system allows users to efficiently acquire knowledge through dialogue without requiring specialized knowledge, reducing inquiries to accounting staff and preventing disruptions to operations. Furthermore, by combining it with an emotion engine, it becomes possible to provide appropriate responses in accordance with the user's emotions, thereby improving the user experience.

[0272] The following describes the processing flow.

[0273] Step 1:

[0274] The user accesses the system's web application or dedicated app. The user enters "Please tell me how accounts receivable are accounted for" into the interface (text box) for entering questions or inquiries.

[0275] Step 2:

[0276] The device's emotion engine analyzes the user's facial expressions, typing speed, and text content during input to recognize the user's emotions. For example, if the user is feeling anxious or worried, the emotion engine identifies that emotion as "anxiety."

[0277] Step 3:

[0278] The device sends the user-entered question and recognized sentiment data to the server. Specifically, it sends text data and sentiment data to the server as an HTTP POST request.

[0279] Step 4:

[0280] The server receives the sent HTTP request and retrieves the question text and sentiment data. The question text is "Please tell me how to account for accounts receivable," and the sentiment data is "anxious."

[0281] Step 5:

[0282] The server analyzes the question text using a natural language processing (NLP) library. It extracts important keywords such as "accounts receivable" and "recording method" from the question text.

[0283] Step 6:

[0284] The server determines the category of the question based on the extracted keywords. Since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[0285] Step 7:

[0286] The server searches for relevant information from an internal database or manual. For example, it identifies information on the "recording method of accounts receivable" from Section 3.2, Chapter 3 of the manual.

[0287] Step 8:

[0288] The server generates an answer according to the user's emotional state considering emotional data. For example, if the recognized emotion is "uneasy", the answer will be adjusted to be more friendly and detailed. The specific answer is "I will explain the recording method of accounts receivable. Accounts receivable represents the amount that has not yet been paid despite providing goods or services. The recording is done in the following steps: 1. Process when the sales are confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match the sales with the accounts receivable. If you have any questions, please feel free to contact us."

[0289] Step 9:

[0290] The server sends the generated answer to the user's terminal. Specifically, it returns the answer text to the terminal as an HTTP response.

[0291] Step 10:

[0292] The device displays the received response on the user's screen. The user can review the displayed response and resolve their questions. They can also re-enter their questions if necessary.

[0293] The above outlines the specific flow of program processing in this system.

[0294] (Example 2)

[0295] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0296] In today's complex business environment, it is crucial for users to acquire specialized knowledge quickly and accurately, preventing disruptions to their work. Furthermore, traditional systems often generate uniform answers without considering the user's emotional state, resulting in a poor user experience. This is particularly true in legal, finance, and accounting departments, where high levels of expertise are required, but at the same time, responses that consider the user's emotions are also necessary. Traditional systems struggled to meet these needs, leading to increased user stress and inquiries.

[0297] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means for the user to input questions or doubts, a transmission means for sending the input questions and sentiment data to the server, a natural language processing means for analyzing the transmitted questions and searching for related information, an answer generation means for generating answers based on the user's sentiment data, and a transmission means for sending the generated answers to the user. This makes it possible to provide quick and accurate answers based on expert knowledge while taking into account the user's emotional state. The user can efficiently acquire knowledge in an interactive format and perform tasks without stress, thus improving overall work efficiency. Furthermore, by using a generation AI model, flexible and appropriate answers can be automatically generated for the user, reducing the number of inquiries and preventing delays in operations.

[0298] A "user" is an entity that uses a system to input questions and obtain answers.

[0299] "Questions and inquiries" refer to text information related to the information users want to know or the problems they want to solve.

[0300] An "input method" refers to an interface or device used by a user to input questions or doubts.

[0301] "Transmission means" refers to the mechanism or function for sending the entered question and sentiment data to the server.

[0302] A "server" is a computer system that analyzes input questions, searches for relevant information, and generates and transmits answers.

[0303] "Natural language processing tools" are algorithms and technologies used to analyze input questions and understand important keywords and context.

[0304] "Emotional data" refers to data that indicates the emotional state of a user when they enter a question or inquiry.

[0305] The "answer generation means" is a function that generates an answer to be provided to the user based on the analyzed question and sentiment data.

[0306] The "generated AI model" is a trained model for generating appropriate answers using artificial intelligence.

[0307] The present invention is a system for enabling a user to efficiently acquire specialized knowledge in the legal field, finance, accounting departments, etc., and prevent business disruptions. Specifically, by combining a sentiment engine that recognizes the user's sentiment, the user experience can be improved and more appropriate answers can be provided. The specific embodiments of this system will be described below.

[0308] First, the user accesses the system using a terminal such as a PC or smartphone. The user inputs a question or query into the text box provided by the system interface. At this time, the sentiment engine recognizes the user's sentiment (e.g., anxiety, impatience) in real time during the user's input.

[0309] For example, when the user inputs "Please teach me about the method of recording accounts receivable", the sentiment engine recognizes that the user is feeling "anxious". This information is sent from the terminal to the server. Specifically, as an HTTP POST request, text data (the content of the user's question) and sentiment data (the sentiment recognized by the sentiment engine) are sent to the server.

[0310] Next, the server analyzes the received data. The server uses a natural language processing (NLP) library (e.g., spaCy or NLTK, etc.) to tokenize the question text and extract important keywords (e.g., "accounts receivable", "recording method"). Then, based on the extracted keywords, the category of the question is determined. For example, since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[0311] The server adjusts its response generation algorithm, taking emotional data into consideration. If the user is feeling "anxious," an algorithm is activated to make the generated response more helpful and detailed. The server then searches its internal database and documentation manuals for relevant information. Search engines such as ElasticSearch® are used for this purpose. For example, it might identify relevant information from "Chapter 3, Section 3.2 of the manual."

[0312] Based on the searched information, the server generates an appropriate response that corresponds to the user's emotional state. In this process, a generative AI model (e.g., GPT-3 or ChatGPT®) is used to create an appropriate natural language response. The generated response is sent from the server to the terminal, returning the response text as an HTTP response. The terminal displays the received response on the user's screen, allowing the user to review the displayed response and resolve their questions.

[0313] For example, if a user enters "Please tell me how to record accounts receivable," the server will categorize this question under accounting, search for relevant manuals, and generate a helpful answer with detailed instructions. The generated answer might say something like, "We will explain how to record accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The recording process is as follows: 1. Process the payment as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match sales with accounts receivable. If you have any questions, please do not hesitate to contact us." By displaying this answer on the terminal, the user can resolve their questions and proceed with their work more efficiently.

[0314] Examples of prompt statements include the following:

[0315] "Please explain how accounts receivable are accounted for. Please take into consideration that the emotional engine is experiencing 'anxiety'."

[0316] "Generate appropriate answers when users ask questions about accounting and finance."

[0317] The above describes the specific implementation of the system of the present invention. This system allows users to efficiently acquire knowledge through dialogue without requiring specialized knowledge, reducing inquiries to accounting staff and preventing disruptions to operations. Furthermore, by combining it with an emotion engine, it becomes possible to provide appropriate responses in accordance with the user's emotions, thereby improving the user experience.

[0318] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0319] Step 1:

[0320] Users access the system using devices such as PCs and smartphones. Next, users enter questions or concerns into text boxes located on the system's interface. For example, they might enter, "Please tell me how accounts receivable are accounted for." At this time, the emotion engine recognizes emotional data in real time from the user's facial expressions and tone of voice, and collects emotional data indicating "anxiety" or "impatience."

[0321] Input: User's question text and sentiment data

[0322] Output: Combined information of user question text and sentiment data

[0323] Step 2:

[0324] The terminal sends the user-entered question text and sentiment data to the server. This combined information is sent in the form of an HTTP POST request, for example, as data in JSON format.

[0325] Input: User's question text and sentiment data

[0326] Output: JSON format data passed to the server

[0327] Step 3:

[0328] The server analyzes the received data. First, the server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to tokenize the question text and extract important keywords. For example, "accounts receivable" and "accounting method" might be extracted.

[0329] Input: User's question text and sentiment data

[0330] Output: Key keywords (e.g., "accounts receivable", "accounting method")

[0331] Step 4:

[0332] Based on the extracted keywords, the server determines the category of the question. Using a pre-trained category classification model such as BERT, it determines, for example, that the question belongs to the "accounting" category.

[0333] Input: Important keywords

[0334] Output: Question category (e.g., "Accounting")

[0335] Step 5:

[0336] The server uses sentiment data to adjust the response generation algorithm. If the user indicates "anxiety," an algorithm is activated to make the response more helpful and detailed.

[0337] Input: Sentiment data

[0338] Output: Adjustment of the response generation algorithm based on emotion.

[0339] Step 6:

[0340] The server searches for relevant information based on the question category and sentiment data. It uses search engines such as Elasticsearch to identify relevant information from internal databases and documentation manuals. For example, information might be identified from "Chapter 3, Section 3.2 of the manual."

[0341] Input: Question category and sentiment data

[0342] Output: Related information (e.g., Chapter 3, Section 3.2 of the manual)

[0343] Step 7:

[0344] The server generates answers based on relevant information. It uses a generative AI model (e.g., GPT-3 or ChatGPT) to generate appropriate natural language responses. The generated answers include specific details such as, "We will explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. Accounting is done using the following procedure."

[0345] Input: Related information and sentiment data

[0346] Output: Generated answer text

[0347] Step 8:

[0348] The server sends the generated response to the terminal. The response text is returned to the terminal as an HTTP response.

[0349] Input: Generated response text

[0350] Output: Response text sent to the terminal

[0351] Step 9:

[0352] The terminal displays the answer received from the server on the user's screen. The user reviews the displayed answer and resolves their question. They can also re-enter their question if necessary.

[0353] Input: Response text received from the server

[0354] Output: The answer displayed on the user's screen.

[0355] The above describes the specific processing flow of this system's program.

[0356] (Application Example 2)

[0357] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0358] Traditional systems have faced challenges in providing users with timely and appropriate information when they are efficiently acquiring legal, financial, and accounting-related expertise. Furthermore, a lack of consideration for user emotions leads to a poor user experience and difficulties in effective communication. Additionally, insufficient answers to user questions can trigger further inquiries, disrupting business operations.

[0359] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion recognition means for recognizing the user's emotions at the time of input, natural language processing means for analyzing the transmitted question and retrieving related information, and answer generation means for generating an answer based on the retrieved information and the user's emotional state. This makes it possible to quickly generate an appropriate answer that takes the user's emotions into consideration.

[0360] A "user" is someone who uses this system to input questions or inquiries.

[0361] An "input method" is an interface for users to input questions or doubts.

[0362] "Emotion recognition means" refers to technology for recognizing the emotions a user expresses during input.

[0363] "Transmission means" refers to the function for sending the entered question and recognized sentiment data to the server.

[0364] "Natural language processing means" refers to technologies for analyzing submitted questions and retrieving related information.

[0365] "Answer generation means" refers to technology for generating answers based on searched information and the user's emotional state.

[0366] A "server" is a computing system that analyzes input questions and sentiment data to provide appropriate information.

[0367] A "category" is a field or theme that is classified according to the content of the question.

[0368] "Information" refers to data used to resolve user questions, including past questions and FAQ databases.

[0369] An "answer" is the information and instructions generated in response to a user's question.

[0370] "Additional support information" refers to supplementary information provided in addition to basic answers, based on user sentiment data.

[0371] Modes for carrying out the invention

[0372] This invention is a support system for users to resolve questions and problems related to electronic payments. This system incorporates an emotion recognition mechanism to recognize the user's emotions, thereby improving the user experience. The system's program processing and specific embodiments are described below.

[0373] Hardware and software to be used

[0374] Hardware:

[0375] 1. Smartphone: A device used by the user for input.

[0376] 2. Server: A computer system that analyzes the entered questions and sentiment data and provides appropriate information.

[0377] software:

[0378] 1. EmotionEngine: An emotion recognition library that recognizes user emotions in real time.

[0379] 2. NLPProcessor: A library for natural language processing.

[0380] 3. Python and Flask: Languages ​​and frameworks for implementing server-side programs

[0381] Data processing and data calculation

[0382] 1. User input:

[0383] The device (smartphone) provides an interface for the user to input questions or concerns. The entered text is analyzed by sentiment recognition technology to identify the user's emotions. For example, a question such as "I can't find my recent transaction history, what should I do?" might be entered.

[0384] 2. Emotion recognition:

[0385] The emotion recognition system (EmotionEngine) identifies the user's emotions from the input text. In this case, it recognizes that the user is feeling "anxiety" or "unease."

[0386] 3. Submission of questions and sentiment data:

[0387] The user's question data, along with the identified sentiment data, is sent to the server. This transmission is an HTTP POST request.

[0388] 4. Question analysis and information retrieval:

[0389] The server uses an NLP processor to analyze text data and extract relevant keywords (e.g., "transaction history," "not displayed"). These keywords are then used to classify the data into appropriate categories (e.g., "electronic payment").

[0390] 5. Response generation and emotional response:

[0391] The server searches its internal database and FAQs for relevant information based on the extracted keywords and recognized sentiment data. At this time, it generates an appropriate response based on the sentiment state. For example, it might generate a response such as, "If your recent transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Refresh the data from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link."

[0392] 6. Submitting and viewing responses:

[0393] The generated answer is sent from the server to the terminal and displayed on the user's screen. The user can then review this answer and resolve their question.

[0394] Examples of specific cases and prompt statements

[0395] If a user enters the question "I can't find my recent transaction history, what should I do?" on their smartphone, and the emotion recognition system identifies the user as "anxious," the server categorizes the question under "electronic payments" and provides an appropriate answer from the relevant FAQ database. This answer might be something like, "If your recent transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Perform a data refresh from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link." This answer is then displayed on the user's device, quickly resolving their question.

[0396] The above describes a specific embodiment of the present invention system. This system enables users to efficiently resolve problems related to electronic payments and receive appropriate support that takes their feelings into consideration.

[0397] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0398] Step 1:

[0399] User input:

[0400] The user enters questions or inquiries using a smartphone interface. For example, they might enter text like, "I can't find my recent transaction history, what should I do?" This input generates text data for the question. Simultaneously, the user's emotions are recognized in real time.

[0401] Step 2:

[0402] Emotion recognition:

[0403] The smartphone device uses emotion recognition technology (EmotionEngine) to analyze the user's emotions at the time of input. The analysis identifies the emotion the user is feeling (e.g., anxiety). Input data consists of a question text and emotion data. Output data consists of text data and the recognized emotion data.

[0404] Step 3:

[0405] Transmission method:

[0406] The smartphone generates and sends an HTTP POST request to the server to send text data entered by the user and recognized sentiment data. The input data at this time consists of text data and sentiment data, and the output after transmission is the status that the data has been successfully sent to the server.

[0407] Step 4:

[0408] Natural language processing:

[0409] The server analyzes the transmitted text data using an NLP processor and extracts key keywords from the text (e.g., "transaction history", "not displayed"). The input data is text data, and the output data is a list of the extracted keywords.

[0410] Step 5:

[0411] Category classification:

[0412] The server classifies user questions into specific categories (e.g., electronic payments) based on extracted keywords. The input data is a list of keywords, and the output data is category information.

[0413] Step 6:

[0414] Information retrieval and answer generation:

[0415] The server searches for relevant information from its internal database and FAQs, taking into account category and user sentiment data. Based on the retrieved data, it generates an answer. For example, it retrieves information on how to view transaction history from the relevant FAQ database and generates a detailed and helpful answer when the user is anxious. Input data consists of category information and sentiment data, while output data is the answer text.

[0416] Step 7:

[0417] Transmission method:

[0418] The server sends the generated response to the smartphone as an HTTP response. The input data is the response text, and the output data is the status of the successful transmission of the response data to the smartphone.

[0419] Step 8:

[0420] Display answer:

[0421] The smartphone device displays the response sent from the server on the user's screen. For example, it might display: "If your transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Refresh the data from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link." The input data is the response text, and the output data is the displayed response.

[0422] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0423] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0424] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0425] [Second Embodiment]

[0426] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0427] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0428] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0429] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0430] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0431] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0432] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0433] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0434] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0435] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0436] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0437] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0438] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing disruptions to their work. This system consists of user, terminal, and server components. The system's program processing and specific implementation examples are described below.

[0439] 1. User input of question

[0440] Users access the system using devices such as PCs and smartphones. Users enter their questions through an interface for entering doubts and inquiries (e.g., a text box). For example, a user might enter, "Please tell me how accounts receivable are accounted for."

[0441] 2. Sending questions via device

[0442] The questions entered by the user are sent from the terminal to the server. In this process, data is sent to the server as an HTTP request (e.g., a POST request).

[0443] 3. Server analysis of the question

[0444] The server analyzes the received question. Specifically, it uses natural language processing (NLP) techniques to analyze the question and extract relevant keywords. For example, keywords such as "accounts receivable" and "accounting method" may be extracted.

[0445] 4. Server-side determination of question category

[0446] The server determines the category of the question based on the extracted keywords. For example, since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[0447] 5. Searching for information on the server

[0448] The server searches for relevant information from internal databases, manuals, FAQ databases, etc., based on the question category. For example, if information on "how to account for accounts receivable" is found in Chapter 3, Section 3.2 of the manual, it will locate it.

[0449] 6. Server-driven response generation

[0450] Based on the searched information, the server generates an appropriate answer to the user's question. For example, it might generate an answer such as, "I will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided. Accounting is done in the following steps: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable."

[0451] 7. Server sends response

[0452] The generated response is sent from the server to the terminal. This is also sent as an HTTP response and received by the terminal.

[0453] 8. Displaying responses via device

[0454] The terminal receives the response sent from the server and displays it on the user's screen. The user can then review this and resolve their question.

[0455] 9. User input of additional questions

[0456] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[0457] Specific example:

[0458] If a user enters "Please tell me how to record accounts receivable," the server identifies this question as belonging to the "Accounting" category and searches Chapter 3, Section 3.2 of the manual to generate an appropriate answer. The answer is: "We will explain how to record accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The recording process is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match sales with accounts receivable." This answer is displayed on the user's terminal, allowing the user to resolve their question.

[0459] This invention allows users to efficiently acquire knowledge through an interactive format, even without specialized knowledge, and prevents delays in their work.

[0460] The following describes the processing flow.

[0461] Step 1:

[0462] The user accesses the system's web application or dedicated app. The user enters "Please tell me how accounts receivable are accounted for" into the interface (text box) for entering questions or inquiries.

[0463] Step 2:

[0464] The terminal sends the user's entered question to the server. Specifically, it sends the question's text data to the server as an HTTP POST request.

[0465] Step 3:

[0466] The server receives the sent HTTP request and retrieves the question text. The question text is "Please tell me how to account for accounts receivable."

[0467] Step 4:

[0468] The server analyzes the question text using a natural language processing (NLP) library. Specifically, it performs keyword extraction, identifying important keywords such as "accounts receivable" and "accounting method."

[0469] Step 5:

[0470] The server determines the category of the question based on the extracted keywords. In this case, the keyword "accounts receivable" indicates that the question belongs to the "accounting" category.

[0471] Step 6:

[0472] The server searches for relevant information from its internal database or manual based on the category. For example, it might identify information on "Accounts Receivable Accounting Methods" from Chapter 3, Section 3.2 of the manual.

[0473] Step 7:

[0474] The server generates an answer based on the searched information. It generates an answer such as: "We will explain how to record accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. The recording procedure is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match the sales with the accounts receivable."

[0475] Step 8:

[0476] The server sends the generated response to the user's device. Specifically, it returns the response text to the device as an HTTP response.

[0477] Step 9:

[0478] The device displays the received response on the user's screen. The user reviews the response displayed on the screen and resolves their question.

[0479] Step 10:

[0480] If the user has any further questions, they can enter them again in the text box and click the submit button. This process is repeated, allowing the user to obtain the necessary information step by step.

[0481] The above outlines the specific flow of program processing in this system.

[0482] (Example 1)

[0483] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0484] Many modern jobs require specialized knowledge, but it is extremely difficult for users to efficiently acquire this knowledge and smoothly carry out their work. Furthermore, a lack of appropriate guidance and learning materials can cause work to stall. To solve these problems, there is a need for a system that allows users to easily access and deepen their understanding of specialized knowledge.

[0485] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0486] In this invention, the server includes an input means for the user to input questions or doubts, a transmission means for sending the input questions to the server, a natural language processing means for analyzing the transmitted questions and searching for related information, a generative AI model means for generating answers based on the searched information, and a transmission means for sending the generated answers to the user. This makes it possible for users to efficiently acquire knowledge in an interactive format without having specialized knowledge, and to prevent delays in their work.

[0487] An "input method" is a means of providing an interface for users to input questions or doubts.

[0488] "Transmission method" refers to the means by which questions and data entered by the user are sent to the server.

[0489] "Natural language processing means" refers to technologies and methods used to analyze submitted questions and retrieve relevant information.

[0490] "Generative AI model means" refers to a means of using an artificial intelligence model to generate answers based on searched information.

[0491] An "answer generation means" is a means of generating an appropriate answer to a user's question.

[0492] A "category classification method" is a means of classifying questions analyzed using natural language processing into specific categories.

[0493] A "past questions and FAQ database" refers to a database that compiles questions and frequently asked questions accumulated to date.

[0494] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing disruptions to their work. This system consists of user, terminal, and server components.

[0495] 1. User input of question

[0496] Users access the system using devices such as PCs and smartphones. A text box appears on the input screen, into which the user enters their question. For example, they might enter, "Please tell me how accounts receivable are accounted for."

[0497] 2. Sending questions via device

[0498] The question entered by the user is sent from the terminal to the server as an HTTP POST request. The POST request contains the user's question.

[0499] 3. Server analysis of the question

[0500] The server uses a natural language processing (NLP) library (e.g., SpaCy) to analyze the received data. It divides the question into tokens, analyzes their meaning, and extracts keywords.

[0501] 4. Server-side determination of question category

[0502] The server uses a machine learning model (e.g., a Scikit-learn classifier) ​​to determine the category of the question based on the extracted keywords. The model is pre-trained on data related to categories such as accounting and legal affairs.

[0503] 5. Searching for information on the server

[0504] Based on the question category, the server searches for relevant information from its internal databases and manuals (e.g., the MySQL database). It then identifies the most relevant information from the search results.

[0505] 6. Server-driven response generation

[0506] Based on the searched information, the server uses a generative AI model (e.g., GPT-3) to generate appropriate answers. The answers are then formatted in a user-friendly manner.

[0507] 7. Server sends response

[0508] The generated response is sent from the server to the terminal as an HTTP response. This includes data in JSON format.

[0509] 8. Displaying responses via device

[0510] The terminal displays the response data received from the server on the screen. The user reviews this and resolves their questions.

[0511] 9. User input of additional questions

[0512] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[0513] Specific example:

[0514] If a user enters "Please tell me how to record accounts receivable," the server identifies this question as belonging to the "Accounting" category and searches Chapter 3, Section 3.2 of the accounting manual to generate an appropriate answer. The answer is: "We will explain how to record accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. The recording procedure is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match sales with accounts receivable." This answer is displayed on the user's terminal, allowing the user to resolve their question.

[0515] The above is one specific example of the present invention. Through the processing procedure described above, users can effectively proceed with learning and prevent delays in their work, even without specialized knowledge.

[0516] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0517] Step 1:

[0518] User input of question

[0519] Processing details:

[0520] Users access the system using a PC or smartphone and enter questions into text boxes displayed on the input screen. The entered information is saved in text format.

[0521] input:

[0522] The question entered by the user in the text box (e.g., "Please tell me how accounts receivable are accounted for.").

[0523] output:

[0524] Text data entered by the user.

[0525] Specific actions:

[0526] The user launches a browser, accesses the system's webpage, and enters the question into the text box.

[0527] Step 2:

[0528] Sending questions via device

[0529] Processing details:

[0530] The question entered by the user is sent from the terminal to the server as an HTTP POST request. This request contains the question content.

[0531] input:

[0532] Text data entered by the user.

[0533] output:

[0534] An HTTP POST request sent to the server.

[0535] Specific actions:

[0536] The device sends data to the / api / question endpoint requesting information on how accounts receivable are accounted for.

[0537] Step 3:

[0538] Server-based analysis of the question

[0539] Processing details:

[0540] The server uses a natural language processing (NLP) library (e.g., SpaCy) to analyze the received question data. It splits the text data into tokens, analyzes their meaning, and extracts relevant keywords.

[0541] input:

[0542] Question data sent to the server.

[0543] output:

[0544] Extracted keywords (e.g., "accounts receivable", "accounting method").

[0545] Specific actions:

[0546] The server tokenizes the received text data using the SpaCy library and extracts important keywords.

[0547] Step 4:

[0548] Server-based question category determination

[0549] Processing details:

[0550] The server uses a machine learning model (e.g., a Scikit-learn classifier) ​​to determine the category of the question based on the extracted keywords. The model is trained on data related to categories such as accounting and legal affairs.

[0551] input:

[0552] Extracted keywords (e.g., "accounts receivable", "accounting method").

[0553] output:

[0554] The category of the question (e.g., "Accounting").

[0555] Specific actions:

[0556] The server uses a Scikit-learn classifier to analyze keywords and categorize the question into the appropriate category (e.g., "accounting").

[0557] Step 5:

[0558] Searching for information via server

[0559] Processing details:

[0560] Based on the question category, the server searches for relevant information in its internal databases and manuals (e.g., MySQL database). It then identifies the most relevant information from the search results.

[0561] input:

[0562] Categorized questions.

[0563] output:

[0564] Related information (e.g., Accounting Manual, Chapter 3, Section 3.2).

[0565] Specific actions:

[0566] The server uses an SQL query to retrieve information about "accounts receivable accounting methods" from accounting-related tables in the MySQL database.

[0567] Step 6:

[0568] Server-driven response generation

[0569] Processing details:

[0570] Based on the searched information, the server uses a generative AI model (e.g., GPT-3) to generate appropriate answers. The generated answers are then formatted in a user-friendly manner.

[0571] input:

[0572] Related information found through searches.

[0573] output:

[0574] The generated response text (example: "We will explain how to account for accounts receivable. Accounts receivable are...").

[0575] Specific actions:

[0576] Enter "Please tell me how to account for accounts receivable" as the GPT-3 prompt and retrieve the generated response.

[0577] Step 7:

[0578] Server sends response

[0579] Processing details:

[0580] The generated response is sent from the server to the terminal as an HTTP response. The response contains data in JSON format.

[0581] input:

[0582] The generated response text.

[0583] output:

[0584] JSON data sent to the terminal as an HTTP response.

[0585] Specific actions:

[0586] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0587] Step 8:

[0588] Displaying responses via device

[0589] Processing details:

[0590] The terminal displays the response data received from the server on the screen. The user can then review this and resolve their questions.

[0591] input:

[0592] JSON data sent from the server.

[0593] output:

[0594] The answer text displayed on the screen.

[0595] Specific actions:

[0596] The device displays the following response in its browser: "We will explain how to account for accounts receivable..."

[0597] Step 9:

[0598] User input of additional questions

[0599] Processing details:

[0600] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[0601] input:

[0602] Additional questions from the user.

[0603] output:

[0604] Additional questions resent to the server.

[0605] Specific actions:

[0606] The user types, "Could you please explain the accounting method in more detail?" and presses the submit button again.

[0607] (Application Example 1)

[0608] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0609] Currently, many virtual stores have customers who frequently have specialized questions regarding finance and legal matters, but there is a lack of mechanisms to answer these questions quickly and appropriately. As a result, transactions may stall because customer questions remain unresolved, potentially leading to lost business opportunities. This invention aims to solve this problem by providing a system that utilizes a generative AI model to provide appropriate answers in real time.

[0610] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0611] In this invention, the server includes an input means for the user to input questions or doubts; a transmission means for sending the input questions to the server; a natural language processing means for analyzing the transmitted questions and searching for relevant information; an answer generation means for generating answers based on the searched information; a transmission means for sending the generated answers to the user; and a means for generating answers to user-inputted questions using a generation AI model. This makes it possible to provide appropriate answers to user questions immediately.

[0612] A "user" is someone who uses the system to input questions and receive answers.

[0613] "Input methods for entering questions or doubts" refers to interfaces or devices that users use to enter questions or doubts, such as text boxes or voice input devices.

[0614] "Means of transmission for sending entered questions to the server" refers to communication means for sending questions entered by the user to the server, such as software functions for sending HTTP requests.

[0615] "Natural language processing means" refers to software that uses natural language processing techniques to analyze input questions and extract relevant information.

[0616] "Answer generation means" refers to algorithms or programs that generate appropriate answers to user questions based on retrieved information.

[0617] "Means of transmission for sending generated responses to users" refers to communication means for sending generated responses to users, such as software functions for sending HTTP responses.

[0618] A "generative AI model" refers to an artificial intelligence model used to generate answers to questions, such as a deep learning model used for natural language processing.

[0619] A "prompt statement" refers to the input text used when generating an answer for a generative AI model.

[0620] This invention relates to a system that enables users to efficiently acquire specialized knowledge in finance and law, thereby preventing disruptions to their work. This system consists of user, terminal, and server components. A detailed example of the system's operation is described below.

[0621] 1. User input of question

[0622] Users access the system using devices such as smartphones or head-mounted displays. Users input their questions through interfaces for entering doubts and inquiries, such as text boxes or voice input devices. For example, a user might input, "Please tell me how accounts receivable are accounted for."

[0623] 2. Submit the entered question.

[0624] The terminal uses a communication method to send the user-entered questions to the server, sending the data as an HTTP request.

[0625] 3. Question Analysis and Search

[0626] The server analyzes the received question using natural language processing techniques and searches for relevant information. This analysis utilizes a generative AI model. For example, it extracts keywords such as "accounts receivable" and "accounting method" and searches for relevant information from past questions and FAQ databases.

[0627] 4. Generating the answer

[0628] The server uses a generative AI model based on the retrieved information to generate appropriate answers to the user's questions. In this process, the prompt is used as input to the generative AI model. An example of a prompt is, "Please tell me how accounts receivable are accounted for."

[0629] 5. Submit the generated response.

[0630] The generated response is sent from the server to the terminal. It is sent as an HTTP response and received by the terminal.

[0631] 6. Display the answer

[0632] The terminal receives the answer sent from the server and displays it on the user's screen. The user can then review this and resolve their questions. For example, an answer might be generated stating, "We will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. Accounting is done using the following steps: 1. Process the transaction when the sale is finalized. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable."

[0633] This invention allows users to efficiently acquire knowledge through dialogue, even without specialized knowledge, and prevents delays in their work. This system utilizes a generated AI model and prompt text to provide quick and appropriate answers. Furthermore, because users can resolve financial and legal questions through dialogue, it enables the provision of high-quality customer support even in virtual stores.

[0634] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0635] Step 1:

[0636] The user inputs a question or inquiry. The user uses a smartphone or head-mounted display to input a question into the interface via text or voice input, such as "Please tell me how accounts receivable are accounted for." The input in this step is the user's question, and the output is the question data on the device.

[0637] Step 2:

[0638] The terminal sends the entered question to the server. The terminal sends the user-entered question to the server as an HTTP request (POST request). The input in this step is the question data, and the output is the request data sent to the server.

[0639] Step 3:

[0640] The server analyzes the question and searches for relevant information. The server uses natural language processing techniques to analyze the question and extract keywords. For example, it might extract "accounts receivable" or "accounting method" and search for relevant information in its internal database and FAQ database. The input for this step is the request data, and the output is the analyzed keywords and search results.

[0641] Step 4:

[0642] The server generates an answer based on the search results. The server uses a generative AI model to generate a prompt sentence from the searched information, and then uses this prompt sentence to generate an appropriate answer to the user's question. For example, it might generate an answer such as, "Let me explain how accounts receivable are accounted for. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided..." The inputs to this step are the search results and the prompt sentence, and the output is the generated answer.

[0643] Step 5:

[0644] The server sends the generated response to the terminal. The server sends the generated response to the terminal as an HTTP response. The input for this step is the generated response, and the output is the response data sent to the terminal.

[0645] Step 6:

[0646] The terminal displays the answer on the user's screen. The terminal displays the answer received from the server in the user interface. For example, it might display something like, "We will explain how accounts receivable are accounted for. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services..." The input for this step is response data, and the output is the answer displayed to the user.

[0647] This allows users to resolve their questions and perform their tasks smoothly.

[0648] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0649] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing delays in work. By incorporating an emotion engine that recognizes user emotions, this system can improve the user experience and provide more appropriate responses. The system's program processing and specific implementation examples are described below.

[0650] 1. User input of question

[0651] Users access the system using devices such as PCs and smartphones. They enter a question or inquiry into a text box interface, such as "Please tell me how accounts receivable are accounted for." During this process, an emotion engine recognizes the user's emotions (e.g., anxiety, impatience) in real time.

[0652] 2. Transmission of questions and sentiment data via the device

[0653] Along with the question entered by the user, the sentiment data recognized by the sentiment engine is sent to the server. Specifically, the text data and sentiment data are sent to the server as an HTTP POST request.

[0654] 3. Server analysis of the question

[0655] The server analyzes the submitted question text using a natural language processing (NLP) library. It extracts important keywords from the question text (e.g., "accounts receivable," "accounting method").

[0656] 4. Server-side determination of question category

[0657] The server determines the category of the question based on the extracted keywords. For example, since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[0658] 5. Use of emotional data by servers

[0659] The server also takes into account the user's emotional data as recognized by the emotion engine. For example, if the user indicates an "anxious" emotion, it adjusts its response to be more helpful and detailed.

[0660] 6. Searching for information on the server

[0661] The server searches its internal database and manuals for relevant information based on the question category and sentiment data. For example, it might identify information about "accounts receivable accounting methods" from Chapter 3, Section 3.2 of the manual.

[0662] 7. Server-driven response generation

[0663] Based on the searched information, the server generates an appropriate response tailored to the user's emotional state. For example, it might generate a response such as, "Let me explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. Accounting is done using the following steps: 1. Process the transaction as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match sales with accounts receivable. If you have any questions, please do not hesitate to contact us."

[0664] 8. Server sends response

[0665] The generated response is sent from the server to the terminal. Specifically, the response text is returned to the terminal as an HTTP response.

[0666] 9. Displaying responses via the device

[0667] The device displays the received response on the user's screen. The user reviews the displayed response and resolves their question. They can also re-enter their question if necessary.

[0668] Specific example:

[0669] If a user types "Please tell me how to account for accounts receivable" and the sentiment engine recognizes that the user is feeling "anxious," the server categorizes this question under "Accounting." It then searches for relevant manual information and generates a helpful answer that includes detailed instructions on "how to account for accounts receivable." The generated answer might say something like, "We will explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The process involves the following steps: 1. Processing the transaction as soon as the sale is confirmed. 2. Adding an accounts receivable account to your accounting software. 3. Matching sales with accounts receivable. Please feel free to contact us if you have any questions." This answer is then displayed on the user's device, resolving their concerns.

[0670] The above describes the program processing and specific embodiments of the present invention system. This system allows users to efficiently acquire knowledge through dialogue without requiring specialized knowledge, reducing inquiries to accounting staff and preventing disruptions to operations. Furthermore, by combining it with an emotion engine, it becomes possible to provide appropriate responses in accordance with the user's emotions, thereby improving the user experience.

[0671] The following describes the processing flow.

[0672] Step 1:

[0673] The user accesses the system's web application or dedicated app. The user enters "Please tell me how accounts receivable are accounted for" into the interface (text box) for entering questions or inquiries.

[0674] Step 2:

[0675] The device's emotion engine analyzes the user's facial expressions, typing speed, and text content during input to recognize the user's emotions. For example, if the user is feeling anxious or worried, the emotion engine identifies that emotion as "anxiety."

[0676] Step 3:

[0677] The device sends the user-entered question and recognized sentiment data to the server. Specifically, it sends text data and sentiment data to the server as an HTTP POST request.

[0678] Step 4:

[0679] The server receives the sent HTTP request and retrieves the question text and sentiment data. The question text is "Please tell me how to account for accounts receivable," and the sentiment data is "anxious."

[0680] Step 5:

[0681] The server analyzes the question text using a natural language processing (NLP) library. It extracts important keywords such as "accounts receivable" and "accounting method" from the question text.

[0682] Step 6:

[0683] The server determines the category of the question based on the extracted keywords. Since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[0684] Step 7:

[0685] The server searches for relevant information in its internal database and manuals. For example, it might identify information regarding "Accounts Receivable Accounting Methods" from Chapter 3, Section 3.2 of the manual.

[0686] Step 8:

[0687] The server takes emotional data into account and generates responses that correspond to the user's emotional state. For example, if the recognized emotion is "anxiety," the response is adjusted to be more helpful and detailed. A specific response might be: "We will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided. Accounting is done using the following steps: 1. Process the receivable as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable. If you have any questions, please do not hesitate to contact us."

[0688] Step 9:

[0689] The server sends the generated response to the user's device. Specifically, it returns the response text to the device as an HTTP response.

[0690] Step 10:

[0691] The device displays the received response on the user's screen. The user can review the displayed response and resolve their questions. They can also re-enter their questions if necessary.

[0692] The above outlines the specific flow of program processing in this system.

[0693] (Example 2)

[0694] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0695] In today's complex business environment, it is crucial for users to acquire specialized knowledge quickly and accurately, preventing disruptions to their work. Furthermore, traditional systems often generate uniform answers without considering the user's emotional state, resulting in a poor user experience. This is particularly true in legal, finance, and accounting departments, where high levels of expertise are required, but at the same time, responses that consider the user's emotions are also necessary. Traditional systems struggled to meet these needs, leading to increased user stress and inquiries.

[0696] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means for the user to input questions or doubts, a transmission means for sending the input questions and sentiment data to the server, a natural language processing means for analyzing the transmitted questions and searching for related information, an answer generation means for generating answers based on the user's sentiment data, and a transmission means for sending the generated answers to the user. This makes it possible to provide quick and accurate answers based on expert knowledge while taking into account the user's emotional state. The user can efficiently acquire knowledge in an interactive format and perform tasks without stress, thus improving overall work efficiency. Furthermore, by using a generation AI model, flexible and appropriate answers can be automatically generated for the user, reducing the number of inquiries and preventing delays in operations.

[0697] A "user" is an entity that uses a system to input questions and obtain answers.

[0698] "Questions and inquiries" refer to text information related to the information users want to know or the problems they want to solve.

[0699] An "input method" refers to an interface or device used by a user to input questions or doubts.

[0700] "Transmission means" refers to the mechanism or function for sending the entered question and sentiment data to the server.

[0701] A "server" is a computer system that analyzes input questions, searches for relevant information, and generates and transmits answers.

[0702] "Natural language processing tools" are algorithms and technologies used to analyze input questions and understand important keywords and context.

[0703] "Emotional data" refers to data that indicates the emotional state of a user when they enter a question or inquiry.

[0704] "Answer generation means" refers to a function that generates answers to be provided to the user based on analyzed questions and sentiment data.

[0705] A "generative AI model" is a pre-trained model that uses artificial intelligence to generate appropriate answers.

[0706] This invention is a system designed to enable users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing delays in work. Specifically, by incorporating an emotion engine that recognizes user emotions, it can improve the user experience and provide more appropriate answers. The following describes a specific embodiment of this system.

[0707] First, the user accesses the system using a device such as a PC or smartphone. The user enters questions or doubts into text boxes provided on the system's interface. At this time, the emotion engine recognizes the user's emotions (e.g., anxiety, impatience) in real time.

[0708] For example, if a user enters "Please tell me how accounts receivable are accounted for," the emotion engine recognizes that the user is feeling "anxious." This information is sent from the terminal to the server. Specifically, text data (the user's question) and emotion data (the emotion recognized by the emotion engine) are sent to the server as an HTTP POST request.

[0709] Next, the server parses the received data. The server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to tokenize the question text and extract key keywords (e.g., "accounts receivable," "accounting method"). Then, based on the extracted keywords, it determines the category of the question. For example, since "accounts receivable" is an accounting term, the server recognizes that this question belongs to the "accounting" category.

[0710] The server adjusts its response generation algorithm, taking sentiment data into account. If the user is feeling "anxious," an algorithm is activated to make the generated response more helpful and detailed. The server then searches for relevant information in its internal database and documentation manuals. Search engines such as Elasticsearch are used for this purpose. For example, it might identify relevant information from "Chapter 3, Section 3.2 of the manual."

[0711] Based on the searched information, the server generates an appropriate response tailored to the user's emotional state. This is done using a generative AI model (e.g., GPT-3 or ChatGPT) to create a suitable natural language response. The generated response is sent from the server to the terminal, returning the response text as an HTTP response. The terminal displays the received response on the user's screen, allowing the user to review the displayed response and resolve their questions.

[0712] For example, if a user enters "Please tell me how to record accounts receivable," the server will categorize this question under accounting, search for relevant manuals, and generate a helpful answer with detailed instructions. The generated answer might say something like, "We will explain how to record accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The recording process is as follows: 1. Process the payment as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match sales with accounts receivable. If you have any questions, please do not hesitate to contact us." By displaying this answer on the terminal, the user can resolve their questions and proceed with their work more efficiently.

[0713] Examples of prompt statements include the following:

[0714] "Please explain how accounts receivable are accounted for. Please take into consideration that the emotional engine is experiencing 'anxiety'."

[0715] "Generate appropriate answers when users ask questions about accounting and finance."

[0716] The above describes the specific implementation of the system of the present invention. This system allows users to efficiently acquire knowledge through dialogue without requiring specialized knowledge, reducing inquiries to accounting staff and preventing disruptions to operations. Furthermore, by combining it with an emotion engine, it becomes possible to provide appropriate responses in accordance with the user's emotions, thereby improving the user experience.

[0717] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0718] Step 1:

[0719] Users access the system using devices such as PCs and smartphones. Next, users enter questions or concerns into text boxes located on the system's interface. For example, they might enter, "Please tell me how accounts receivable are accounted for." At this time, the emotion engine recognizes emotional data in real time from the user's facial expressions and tone of voice, and collects emotional data indicating "anxiety" or "impatience."

[0720] Input: User's question text and sentiment data

[0721] Output: Combined information of user question text and sentiment data

[0722] Step 2:

[0723] The terminal sends the user-entered question text and sentiment data to the server. This combined information is sent in the form of an HTTP POST request, for example, as data in JSON format.

[0724] Input: User's question text and sentiment data

[0725] Output: JSON format data passed to the server

[0726] Step 3:

[0727] The server analyzes the received data. First, the server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to tokenize the question text and extract important keywords. For example, "accounts receivable" and "accounting method" might be extracted.

[0728] Input: User's question text and sentiment data

[0729] Output: Key keywords (e.g., "accounts receivable", "accounting method")

[0730] Step 4:

[0731] Based on the extracted keywords, the server determines the category of the question. Using a pre-trained category classification model such as BERT, it determines, for example, that the question belongs to the "accounting" category.

[0732] Input: Important keywords

[0733] Output: Question category (e.g., "Accounting")

[0734] Step 5:

[0735] The server uses sentiment data to adjust the response generation algorithm. If the user indicates "anxiety," an algorithm is activated to make the response more helpful and detailed.

[0736] Input: Sentiment data

[0737] Output: Adjustment of the response generation algorithm based on emotion.

[0738] Step 6:

[0739] The server searches for relevant information based on the question category and sentiment data. It uses search engines such as Elasticsearch to identify relevant information from internal databases and documentation manuals. For example, information might be identified from "Chapter 3, Section 3.2 of the manual."

[0740] Input: Question category and sentiment data

[0741] Output: Related information (e.g., Chapter 3, Section 3.2 of the manual)

[0742] Step 7:

[0743] The server generates answers based on relevant information. It uses a generative AI model (e.g., GPT-3 or ChatGPT) to generate appropriate natural language responses. The generated answers include specific details such as, "We will explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. Accounting is done using the following procedure."

[0744] Input: Related information and sentiment data

[0745] Output: Generated answer text

[0746] Step 8:

[0747] The server sends the generated response to the terminal. The response text is returned to the terminal as an HTTP response.

[0748] Input: Generated response text

[0749] Output: Response text sent to the terminal

[0750] Step 9:

[0751] The terminal displays the answer received from the server on the user's screen. The user reviews the displayed answer and resolves their question. They can also re-enter their question if necessary.

[0752] Input: Response text received from the server

[0753] Output: The answer displayed on the user's screen.

[0754] The above describes the specific processing flow of this system's program.

[0755] (Application Example 2)

[0756] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0757] Traditional systems have faced challenges in providing users with timely and appropriate information when they are efficiently acquiring legal, financial, and accounting-related expertise. Furthermore, a lack of consideration for user emotions leads to a poor user experience and difficulties in effective communication. Additionally, insufficient answers to user questions can trigger further inquiries, disrupting business operations.

[0758] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion recognition means for recognizing the user's emotions at the time of input, natural language processing means for analyzing the transmitted question and retrieving related information, and answer generation means for generating an answer based on the retrieved information and the user's emotional state. This makes it possible to quickly generate an appropriate answer that takes the user's emotions into consideration.

[0759] A "user" is someone who uses this system to input questions or inquiries.

[0760] An "input method" is an interface for users to input questions or doubts.

[0761] "Emotion recognition means" refers to technology for recognizing the emotions a user expresses during input.

[0762] "Transmission means" refers to the function for sending the entered question and recognized sentiment data to the server.

[0763] "Natural language processing means" refers to technologies for analyzing submitted questions and retrieving related information.

[0764] "Answer generation means" refers to technology for generating answers based on searched information and the user's emotional state.

[0765] A "server" is a computing system that analyzes input questions and sentiment data to provide appropriate information.

[0766] A "category" is a field or theme that is classified according to the content of the question.

[0767] "Information" refers to data used to resolve user questions, including past questions and FAQ databases.

[0768] An "answer" is the information and instructions generated in response to a user's question.

[0769] "Additional support information" refers to supplementary information provided in addition to basic answers, based on user sentiment data.

[0770] Modes for carrying out the invention

[0771] This invention is a support system for users to resolve questions and problems related to electronic payments. This system incorporates an emotion recognition mechanism to recognize the user's emotions, thereby improving the user experience. The system's program processing and specific embodiments are described below.

[0772] Hardware and software to be used

[0773] Hardware:

[0774] 1. Smartphone: A device used by the user for input.

[0775] 2. Server: A computer system that analyzes the entered questions and sentiment data and provides appropriate information.

[0776] software:

[0777] 1. EmotionEngine: An emotion recognition library that recognizes user emotions in real time.

[0778] 2. NLPProcessor: A library for natural language processing.

[0779] 3. Python and Flask: Languages ​​and frameworks for implementing server-side programs

[0780] Data processing and data calculation

[0781] 1. User input:

[0782] The device (smartphone) provides an interface for the user to input questions or concerns. The entered text is analyzed by sentiment recognition technology to identify the user's emotions. For example, a question such as "I can't find my recent transaction history, what should I do?" might be entered.

[0783] 2. Emotion recognition:

[0784] The emotion recognition system (EmotionEngine) identifies the user's emotions from the input text. In this case, it recognizes that the user is feeling "anxiety" or "unease."

[0785] 3. Submission of questions and sentiment data:

[0786] The user's question data, along with the identified sentiment data, is sent to the server. This transmission is an HTTP POST request.

[0787] 4. Question analysis and information retrieval:

[0788] The server uses an NLP processor to analyze text data and extract relevant keywords (e.g., "transaction history," "not displayed"). These keywords are then used to classify the data into appropriate categories (e.g., "electronic payment").

[0789] 5. Response generation and emotional response:

[0790] The server searches its internal database and FAQs for relevant information based on the extracted keywords and recognized sentiment data. At this time, it generates an appropriate response based on the sentiment state. For example, it might generate a response such as, "If your recent transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Refresh the data from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link."

[0791] 6. Submitting and viewing responses:

[0792] The generated answer is sent from the server to the terminal and displayed on the user's screen. The user can then review this answer and resolve their question.

[0793] Examples of specific cases and prompt statements

[0794] If a user enters the question "I can't find my recent transaction history, what should I do?" on their smartphone, and the emotion recognition system identifies the user as "anxious," the server categorizes the question under "electronic payments" and provides an appropriate answer from the relevant FAQ database. This answer might be something like, "If your recent transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Perform a data refresh from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link." This answer is then displayed on the user's device, quickly resolving their question.

[0795] The above describes a specific embodiment of the present invention system. This system enables users to efficiently resolve problems related to electronic payments and receive appropriate support that takes their feelings into consideration.

[0796] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0797] Step 1:

[0798] User input:

[0799] The user enters questions or inquiries using a smartphone interface. For example, they might enter text like, "I can't find my recent transaction history, what should I do?" This input generates text data for the question. Simultaneously, the user's emotions are recognized in real time.

[0800] Step 2:

[0801] Emotion recognition:

[0802] The smartphone device uses emotion recognition technology (EmotionEngine) to analyze the user's emotions at the time of input. The analysis identifies the emotion the user is feeling (e.g., anxiety). Input data consists of a question text and emotion data. Output data consists of text data and the recognized emotion data.

[0803] Step 3:

[0804] Transmission method:

[0805] The smartphone generates and sends an HTTP POST request to the server to send text data entered by the user and recognized sentiment data. The input data at this time consists of text data and sentiment data, and the output after transmission is the status that the data has been successfully sent to the server.

[0806] Step 4:

[0807] Natural language processing:

[0808] The server analyzes the transmitted text data using an NLP processor and extracts key keywords from the text (e.g., "transaction history", "not displayed"). The input data is text data, and the output data is a list of the extracted keywords.

[0809] Step 5:

[0810] Category classification:

[0811] The server classifies user questions into specific categories (e.g., electronic payments) based on extracted keywords. The input data is a list of keywords, and the output data is category information.

[0812] Step 6:

[0813] Information retrieval and answer generation:

[0814] The server searches for relevant information from its internal database and FAQs, taking into account category and user sentiment data. Based on the retrieved data, it generates an answer. For example, it retrieves information on how to view transaction history from the relevant FAQ database and generates a detailed and helpful answer when the user is anxious. Input data consists of category information and sentiment data, while output data is the answer text.

[0815] Step 7:

[0816] Transmission method:

[0817] The server sends the generated response to the smartphone as an HTTP response. The input data is the response text, and the output data is the status of the successful transmission of the response data to the smartphone.

[0818] Step 8:

[0819] Display answer:

[0820] The smartphone device displays the response sent from the server on the user's screen. For example, it might display: "If your transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Refresh the data from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link." The input data is the response text, and the output data is the displayed response.

[0821] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0822] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0823] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0824] [Third Embodiment]

[0825] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0826] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0827] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0828] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0829] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0830] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0831] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0832] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0833] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0834] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0835] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0836] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0837] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing disruptions to their work. This system consists of user, terminal, and server components. The system's program processing and specific implementation examples are described below.

[0838] 1. User input of question

[0839] Users access the system using devices such as PCs and smartphones. Users enter their questions through an interface for entering doubts and inquiries (e.g., a text box). For example, a user might enter, "Please tell me how accounts receivable are accounted for."

[0840] 2. Sending questions via device

[0841] The questions entered by the user are sent from the terminal to the server. In this process, data is sent to the server as an HTTP request (e.g., a POST request).

[0842] 3. Server analysis of the question

[0843] The server analyzes the received question. Specifically, it uses natural language processing (NLP) techniques to analyze the question and extract relevant keywords. For example, keywords such as "accounts receivable" and "accounting method" may be extracted.

[0844] 4. Server-side determination of question category

[0845] The server determines the category of the question based on the extracted keywords. For example, since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[0846] 5. Searching for information on the server

[0847] The server searches for relevant information from internal databases, manuals, FAQ databases, etc., based on the question category. For example, if information on "how to account for accounts receivable" is found in Chapter 3, Section 3.2 of the manual, it will locate it.

[0848] 6. Server-driven response generation

[0849] Based on the searched information, the server generates an appropriate answer to the user's question. For example, it might generate an answer such as, "I will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided. Accounting is done in the following steps: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable."

[0850] 7. Server sends response

[0851] The generated response is sent from the server to the terminal. This is also sent as an HTTP response and received by the terminal.

[0852] 8. Displaying responses via device

[0853] The terminal receives the response sent from the server and displays it on the user's screen. The user can then review this and resolve their question.

[0854] 9. User input of additional questions

[0855] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[0856] Specific example:

[0857] If a user enters "Please tell me how to record accounts receivable," the server identifies this question as belonging to the "Accounting" category and searches Chapter 3, Section 3.2 of the manual to generate an appropriate answer. The answer is: "We will explain how to record accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The recording process is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match sales with accounts receivable." This answer is displayed on the user's terminal, allowing the user to resolve their question.

[0858] This invention allows users to efficiently acquire knowledge through an interactive format, even without specialized knowledge, and prevents delays in their work.

[0859] The following describes the processing flow.

[0860] Step 1:

[0861] The user accesses the system's web application or dedicated app. The user enters "Please tell me how accounts receivable are accounted for" into the interface (text box) for entering questions or inquiries.

[0862] Step 2:

[0863] The terminal sends the user's entered question to the server. Specifically, it sends the question's text data to the server as an HTTP POST request.

[0864] Step 3:

[0865] The server receives the sent HTTP request and retrieves the question text. The question text is "Please tell me how to account for accounts receivable."

[0866] Step 4:

[0867] The server analyzes the question text using a natural language processing (NLP) library. Specifically, it performs keyword extraction, identifying important keywords such as "accounts receivable" and "accounting method."

[0868] Step 5:

[0869] The server determines the category of the question based on the extracted keywords. In this case, the keyword "accounts receivable" indicates that the question belongs to the "accounting" category.

[0870] Step 6:

[0871] The server searches for relevant information from its internal database or manual based on the category. For example, it might identify information on "Accounts Receivable Accounting Methods" from Chapter 3, Section 3.2 of the manual.

[0872] Step 7:

[0873] The server generates an answer based on the searched information. It generates an answer such as: "We will explain how to record accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. The recording procedure is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match the sales with the accounts receivable."

[0874] Step 8:

[0875] The server sends the generated response to the user's device. Specifically, it returns the response text to the device as an HTTP response.

[0876] Step 9:

[0877] The device displays the received response on the user's screen. The user reviews the response displayed on the screen and resolves their question.

[0878] Step 10:

[0879] If the user has any further questions, they can enter them again in the text box and click the submit button. This process is repeated, allowing the user to obtain the necessary information step by step.

[0880] The above outlines the specific flow of program processing in this system.

[0881] (Example 1)

[0882] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0883] Many modern jobs require specialized knowledge, but it is extremely difficult for users to efficiently acquire this knowledge and smoothly carry out their work. Furthermore, a lack of appropriate guidance and learning materials can cause work to stall. To solve these problems, there is a need for a system that allows users to easily access and deepen their understanding of specialized knowledge.

[0884] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0885] In this invention, the server includes an input means for the user to input questions or doubts, a transmission means for sending the input questions to the server, a natural language processing means for analyzing the transmitted questions and searching for related information, a generative AI model means for generating answers based on the searched information, and a transmission means for sending the generated answers to the user. This makes it possible for users to efficiently acquire knowledge in an interactive format without having specialized knowledge, and to prevent delays in their work.

[0886] An "input method" is a means of providing an interface for users to input questions or doubts.

[0887] "Transmission method" refers to the means by which questions and data entered by the user are sent to the server.

[0888] "Natural language processing means" refers to technologies and methods used to analyze submitted questions and retrieve relevant information.

[0889] "Generative AI model means" refers to a means of using an artificial intelligence model to generate answers based on searched information.

[0890] An "answer generation means" is a means of generating an appropriate answer to a user's question.

[0891] A "category classification method" is a means of classifying questions analyzed using natural language processing into specific categories.

[0892] A "past questions and FAQ database" refers to a database that compiles questions and frequently asked questions accumulated to date.

[0893] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing disruptions to their work. This system consists of user, terminal, and server components.

[0894] 1. User input of question

[0895] Users access the system using devices such as PCs and smartphones. A text box appears on the input screen, into which the user enters their question. For example, they might enter, "Please tell me how accounts receivable are accounted for."

[0896] 2. Sending questions via device

[0897] The question entered by the user is sent from the terminal to the server as an HTTP POST request. The POST request contains the user's question.

[0898] 3. Server analysis of the question

[0899] The server uses a natural language processing (NLP) library (e.g., SpaCy) to analyze the received data. It divides the question into tokens, analyzes their meaning, and extracts keywords.

[0900] 4. Server-side determination of question category

[0901] The server uses a machine learning model (e.g., a Scikit-learn classifier) ​​to determine the category of the question based on the extracted keywords. The model is pre-trained on data related to categories such as accounting and legal affairs.

[0902] 5. Searching for information on the server

[0903] Based on the question category, the server searches for relevant information from its internal databases and manuals (e.g., the MySQL database). It then identifies the most relevant information from the search results.

[0904] 6. Server-driven response generation

[0905] Based on the searched information, the server uses a generative AI model (e.g., GPT-3) to generate appropriate answers. The answers are then formatted in a user-friendly manner.

[0906] 7. Server sends response

[0907] The generated response is sent from the server to the terminal as an HTTP response. This includes data in JSON format.

[0908] 8. Displaying responses via device

[0909] The terminal displays the response data received from the server on the screen. The user reviews this and resolves their questions.

[0910] 9. User input of additional questions

[0911] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[0912] Specific example:

[0913] If a user enters "Please tell me how to record accounts receivable," the server identifies this question as belonging to the "Accounting" category and searches Chapter 3, Section 3.2 of the accounting manual to generate an appropriate answer. The answer is: "We will explain how to record accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. The recording procedure is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match sales with accounts receivable." This answer is displayed on the user's terminal, allowing the user to resolve their question.

[0914] The above is one specific example of the present invention. Through the processing procedure described above, users can effectively proceed with learning and prevent delays in their work, even without specialized knowledge.

[0915] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0916] Step 1:

[0917] User input of question

[0918] Processing details:

[0919] Users access the system using a PC or smartphone and enter questions into text boxes displayed on the input screen. The entered information is saved in text format.

[0920] input:

[0921] The question entered by the user in the text box (e.g., "Please tell me how accounts receivable are accounted for.").

[0922] output:

[0923] Text data entered by the user.

[0924] Specific actions:

[0925] The user launches a browser, accesses the system's webpage, and enters the question into the text box.

[0926] Step 2:

[0927] Sending questions via device

[0928] Processing details:

[0929] The question entered by the user is sent from the terminal to the server as an HTTP POST request. This request contains the question content.

[0930] input:

[0931] Text data entered by the user.

[0932] output:

[0933] An HTTP POST request sent to the server.

[0934] Specific actions:

[0935] The device sends data to the / api / question endpoint requesting information on how accounts receivable are accounted for.

[0936] Step 3:

[0937] Server-based analysis of the question

[0938] Processing details:

[0939] The server uses a natural language processing (NLP) library (e.g., SpaCy) to analyze the received question data. It splits the text data into tokens, analyzes their meaning, and extracts relevant keywords.

[0940] input:

[0941] Question data sent to the server.

[0942] output:

[0943] Extracted keywords (e.g., "accounts receivable", "accounting method").

[0944] Specific actions:

[0945] The server tokenizes the received text data using the SpaCy library and extracts important keywords.

[0946] Step 4:

[0947] Server-based question category determination

[0948] Processing details:

[0949] The server uses a machine learning model (e.g., a Scikit-learn classifier) ​​to determine the category of the question based on the extracted keywords. The model is trained on data related to categories such as accounting and legal affairs.

[0950] input:

[0951] Extracted keywords (e.g., "accounts receivable", "accounting method").

[0952] output:

[0953] The category of the question (e.g., "Accounting").

[0954] Specific actions:

[0955] The server uses a Scikit-learn classifier to analyze keywords and categorize the question into the appropriate category (e.g., "accounting").

[0956] Step 5:

[0957] Searching for information via server

[0958] Processing details:

[0959] Based on the question category, the server searches for relevant information in its internal databases and manuals (e.g., MySQL database). It then identifies the most relevant information from the search results.

[0960] input:

[0961] Categorized questions.

[0962] output:

[0963] Related information (e.g., Accounting Manual, Chapter 3, Section 3.2).

[0964] Specific actions:

[0965] The server uses an SQL query to retrieve information about "accounts receivable accounting methods" from accounting-related tables in the MySQL database.

[0966] Step 6:

[0967] Server-driven response generation

[0968] Processing details:

[0969] Based on the searched information, the server uses a generative AI model (e.g., GPT-3) to generate appropriate answers. The generated answers are then formatted in a user-friendly manner.

[0970] input:

[0971] Related information found through searches.

[0972] output:

[0973] The generated response text (example: "We will explain how to account for accounts receivable. Accounts receivable are...").

[0974] Specific actions:

[0975] Enter "Please tell me how to account for accounts receivable" as the GPT-3 prompt and retrieve the generated response.

[0976] Step 7:

[0977] Server sends response

[0978] Processing details:

[0979] The generated response is sent from the server to the terminal as an HTTP response. The response contains data in JSON format.

[0980] input:

[0981] The generated response text.

[0982] output:

[0983] JSON data sent to the terminal as an HTTP response.

[0984] Specific actions:

[0985] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0986] Step 8:

[0987] Displaying responses via device

[0988] Processing details:

[0989] The terminal displays the response data received from the server on the screen. The user can then review this and resolve their questions.

[0990] input:

[0991] JSON data sent from the server.

[0992] output:

[0993] The answer text displayed on the screen.

[0994] Specific actions:

[0995] The device displays the following response in its browser: "We will explain how to account for accounts receivable..."

[0996] Step 9:

[0997] User input of additional questions

[0998] Processing details:

[0999] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[1000] input:

[1001] Additional questions from the user.

[1002] output:

[1003] Additional questions resent to the server.

[1004] Specific actions:

[1005] The user types, "Could you please explain the accounting method in more detail?" and presses the submit button again.

[1006] (Application Example 1)

[1007] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1008] Currently, many virtual stores have customers who frequently have specialized questions regarding finance and legal matters, but there is a lack of mechanisms to answer these questions quickly and appropriately. As a result, transactions may stall because customer questions remain unresolved, potentially leading to lost business opportunities. This invention aims to solve this problem by providing a system that utilizes a generative AI model to provide appropriate answers in real time.

[1009] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1010] In this invention, the server includes an input means for the user to input questions or doubts; a transmission means for sending the input questions to the server; a natural language processing means for analyzing the transmitted questions and searching for relevant information; an answer generation means for generating answers based on the searched information; a transmission means for sending the generated answers to the user; and a means for generating answers to user-inputted questions using a generation AI model. This makes it possible to provide appropriate answers to user questions immediately.

[1011] A "user" is someone who uses the system to input questions and receive answers.

[1012] "Input methods for entering questions or doubts" refers to interfaces or devices that users use to enter questions or doubts, such as text boxes or voice input devices.

[1013] "Means of transmission for sending entered questions to the server" refers to communication means for sending questions entered by the user to the server, such as software functions for sending HTTP requests.

[1014] "Natural language processing means" refers to software that uses natural language processing techniques to analyze input questions and extract relevant information.

[1015] "Answer generation means" refers to algorithms or programs that generate appropriate answers to user questions based on retrieved information.

[1016] "Means of transmission for sending generated responses to users" refers to communication means for sending generated responses to users, such as software functions for sending HTTP responses.

[1017] A "generative AI model" refers to an artificial intelligence model used to generate answers to questions, such as a deep learning model used for natural language processing.

[1018] A "prompt statement" refers to the input text used when generating an answer for a generative AI model.

[1019] This invention relates to a system that enables users to efficiently acquire specialized knowledge in finance and law, thereby preventing disruptions to their work. This system consists of user, terminal, and server components. A detailed example of the system's operation is described below.

[1020] 1. User input of question

[1021] Users access the system using devices such as smartphones or head-mounted displays. Users input their questions through interfaces for entering doubts and inquiries, such as text boxes or voice input devices. For example, a user might input, "Please tell me how accounts receivable are accounted for."

[1022] 2. Submit the entered question.

[1023] The terminal uses a communication method to send the user-entered questions to the server, sending the data as an HTTP request.

[1024] 3. Question Analysis and Search

[1025] The server analyzes the received question using natural language processing techniques and searches for relevant information. This analysis utilizes a generative AI model. For example, it extracts keywords such as "accounts receivable" and "accounting method" and searches for relevant information from past questions and FAQ databases.

[1026] 4. Generating the answer

[1027] The server uses a generative AI model based on the retrieved information to generate appropriate answers to the user's questions. In this process, the prompt is used as input to the generative AI model. An example of a prompt is, "Please tell me how accounts receivable are accounted for."

[1028] 5. Submit the generated response.

[1029] The generated response is sent from the server to the terminal. It is sent as an HTTP response and received by the terminal.

[1030] 6. Display the answer

[1031] The terminal receives the answer sent from the server and displays it on the user's screen. The user can then review this and resolve their questions. For example, an answer might be generated stating, "We will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. Accounting is done using the following steps: 1. Process the transaction when the sale is finalized. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable."

[1032] This invention allows users to efficiently acquire knowledge through dialogue, even without specialized knowledge, and prevents delays in their work. This system utilizes a generated AI model and prompt text to provide quick and appropriate answers. Furthermore, because users can resolve financial and legal questions through dialogue, it enables the provision of high-quality customer support even in virtual stores.

[1033] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1034] Step 1:

[1035] The user inputs a question or inquiry. The user uses a smartphone or head-mounted display to input a question into the interface via text or voice input, such as "Please tell me how accounts receivable are accounted for." The input in this step is the user's question, and the output is the question data on the device.

[1036] Step 2:

[1037] The terminal sends the entered question to the server. The terminal sends the user-entered question to the server as an HTTP request (POST request). The input in this step is the question data, and the output is the request data sent to the server.

[1038] Step 3:

[1039] The server analyzes the question and searches for relevant information. The server uses natural language processing techniques to analyze the question and extract keywords. For example, it might extract "accounts receivable" or "accounting method" and search for relevant information in its internal database and FAQ database. The input for this step is the request data, and the output is the analyzed keywords and search results.

[1040] Step 4:

[1041] The server generates an answer based on the search results. The server uses a generative AI model to generate a prompt sentence from the searched information, and then uses this prompt sentence to generate an appropriate answer to the user's question. For example, it might generate an answer such as, "Let me explain how accounts receivable are accounted for. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided..." The inputs to this step are the search results and the prompt sentence, and the output is the generated answer.

[1042] Step 5:

[1043] The server sends the generated response to the terminal. The server sends the generated response to the terminal as an HTTP response. The input for this step is the generated response, and the output is the response data sent to the terminal.

[1044] Step 6:

[1045] The terminal displays the answer on the user's screen. The terminal displays the answer received from the server in the user interface. For example, it might display something like, "We will explain how accounts receivable are accounted for. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services..." The input for this step is response data, and the output is the answer displayed to the user.

[1046] This allows users to resolve their questions and perform their tasks smoothly.

[1047] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1048] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing delays in work. By incorporating an emotion engine that recognizes user emotions, this system can improve the user experience and provide more appropriate responses. The system's program processing and specific implementation examples are described below.

[1049] 1. User input of question

[1050] Users access the system using devices such as PCs and smartphones. They enter a question or inquiry into a text box interface, such as "Please tell me how accounts receivable are accounted for." During this process, an emotion engine recognizes the user's emotions (e.g., anxiety, impatience) in real time.

[1051] 2. Transmission of questions and sentiment data via the device

[1052] Along with the question entered by the user, the sentiment data recognized by the sentiment engine is sent to the server. Specifically, the text data and sentiment data are sent to the server as an HTTP POST request.

[1053] 3. Server analysis of the question

[1054] The server analyzes the submitted question text using a natural language processing (NLP) library. It extracts important keywords from the question text (e.g., "accounts receivable," "accounting method").

[1055] 4. Server-side determination of question category

[1056] The server determines the category of the question based on the extracted keywords. For example, since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[1057] 5. Use of emotional data by servers

[1058] The server also takes into account the user's emotional data as recognized by the emotion engine. For example, if the user indicates an "anxious" emotion, it adjusts its response to be more helpful and detailed.

[1059] 6. Searching for information on the server

[1060] The server searches its internal database and manuals for relevant information based on the question category and sentiment data. For example, it might identify information about "accounts receivable accounting methods" from Chapter 3, Section 3.2 of the manual.

[1061] 7. Server-driven response generation

[1062] Based on the searched information, the server generates an appropriate response tailored to the user's emotional state. For example, it might generate a response such as, "Let me explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. Accounting is done using the following steps: 1. Process the transaction as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match sales with accounts receivable. If you have any questions, please do not hesitate to contact us."

[1063] 8. Server sends response

[1064] The generated response is sent from the server to the terminal. Specifically, the response text is returned to the terminal as an HTTP response.

[1065] 9. Displaying responses via the device

[1066] The device displays the received response on the user's screen. The user reviews the displayed response and resolves their question. They can also re-enter their question if necessary.

[1067] Specific example:

[1068] If a user types "Please tell me how to account for accounts receivable" and the sentiment engine recognizes that the user is feeling "anxious," the server categorizes this question under "Accounting." It then searches for relevant manual information and generates a helpful answer that includes detailed instructions on "how to account for accounts receivable." The generated answer might say something like, "We will explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The process involves the following steps: 1. Processing the transaction as soon as the sale is confirmed. 2. Adding an accounts receivable account to your accounting software. 3. Matching sales with accounts receivable. Please feel free to contact us if you have any questions." This answer is then displayed on the user's device, resolving their concerns.

[1069] The above describes the program processing and specific embodiments of the present invention system. This system allows users to efficiently acquire knowledge through dialogue without requiring specialized knowledge, reducing inquiries to accounting staff and preventing disruptions to operations. Furthermore, by combining it with an emotion engine, it becomes possible to provide appropriate responses in accordance with the user's emotions, thereby improving the user experience.

[1070] The following describes the processing flow.

[1071] Step 1:

[1072] The user accesses the system's web application or dedicated app. The user enters "Please tell me how accounts receivable are accounted for" into the interface (text box) for entering questions or inquiries.

[1073] Step 2:

[1074] The device's emotion engine analyzes the user's facial expressions, typing speed, and text content during input to recognize the user's emotions. For example, if the user is feeling anxious or worried, the emotion engine identifies that emotion as "anxiety."

[1075] Step 3:

[1076] The device sends the user-entered question and recognized sentiment data to the server. Specifically, it sends text data and sentiment data to the server as an HTTP POST request.

[1077] Step 4:

[1078] The server receives the sent HTTP request and retrieves the question text and sentiment data. The question text is "Please tell me how to account for accounts receivable," and the sentiment data is "anxious."

[1079] Step 5:

[1080] The server analyzes the question text using a natural language processing (NLP) library. It extracts important keywords such as "accounts receivable" and "accounting method" from the question text.

[1081] Step 6:

[1082] The server determines the category of the question based on the extracted keywords. Since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[1083] Step 7:

[1084] The server searches for relevant information in its internal database and manuals. For example, it might identify information regarding "Accounts Receivable Accounting Methods" from Chapter 3, Section 3.2 of the manual.

[1085] Step 8:

[1086] The server takes emotional data into account and generates responses that correspond to the user's emotional state. For example, if the recognized emotion is "anxiety," the response is adjusted to be more helpful and detailed. A specific response might be: "We will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided. Accounting is done using the following steps: 1. Process the receivable as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable. If you have any questions, please do not hesitate to contact us."

[1087] Step 9:

[1088] The server sends the generated response to the user's device. Specifically, it returns the response text to the device as an HTTP response.

[1089] Step 10:

[1090] The device displays the received response on the user's screen. The user can review the displayed response and resolve their questions. They can also re-enter their questions if necessary.

[1091] The above outlines the specific flow of program processing in this system.

[1092] (Example 2)

[1093] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1094] In today's complex business environment, it is crucial for users to acquire specialized knowledge quickly and accurately, preventing disruptions to their work. Furthermore, traditional systems often generate uniform answers without considering the user's emotional state, resulting in a poor user experience. This is particularly true in legal, finance, and accounting departments, where high levels of expertise are required, but at the same time, responses that consider the user's emotions are also necessary. Traditional systems struggled to meet these needs, leading to increased user stress and inquiries.

[1095] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means for the user to input questions or doubts, a transmission means for sending the input questions and sentiment data to the server, a natural language processing means for analyzing the transmitted questions and searching for related information, an answer generation means for generating answers based on the user's sentiment data, and a transmission means for sending the generated answers to the user. This makes it possible to provide quick and accurate answers based on expert knowledge while taking into account the user's emotional state. The user can efficiently acquire knowledge in an interactive format and perform tasks without stress, thus improving overall work efficiency. Furthermore, by using a generation AI model, flexible and appropriate answers can be automatically generated for the user, reducing the number of inquiries and preventing delays in operations.

[1096] A "user" is an entity that uses a system to input questions and obtain answers.

[1097] "Questions and inquiries" refer to text information related to the information users want to know or the problems they want to solve.

[1098] An "input method" refers to an interface or device used by a user to input questions or doubts.

[1099] "Transmission means" refers to the mechanism or function for sending the entered question and sentiment data to the server.

[1100] A "server" is a computer system that analyzes input questions, searches for relevant information, and generates and transmits answers.

[1101] "Natural language processing tools" are algorithms and technologies used to analyze input questions and understand important keywords and context.

[1102] "Emotional data" refers to data that indicates the emotional state of a user when they enter a question or inquiry.

[1103] "Answer generation means" refers to a function that generates answers to be provided to the user based on analyzed questions and sentiment data.

[1104] A "generative AI model" is a pre-trained model that uses artificial intelligence to generate appropriate answers.

[1105] This invention is a system designed to enable users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing delays in work. Specifically, by incorporating an emotion engine that recognizes user emotions, it can improve the user experience and provide more appropriate answers. The following describes a specific embodiment of this system.

[1106] First, the user accesses the system using a device such as a PC or smartphone. The user enters questions or doubts into text boxes provided on the system's interface. At this time, the emotion engine recognizes the user's emotions (e.g., anxiety, impatience) in real time.

[1107] For example, if a user enters "Please tell me how accounts receivable are accounted for," the emotion engine recognizes that the user is feeling "anxious." This information is sent from the terminal to the server. Specifically, text data (the user's question) and emotion data (the emotion recognized by the emotion engine) are sent to the server as an HTTP POST request.

[1108] Next, the server parses the received data. The server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to tokenize the question text and extract key keywords (e.g., "accounts receivable," "accounting method"). Then, based on the extracted keywords, it determines the category of the question. For example, since "accounts receivable" is an accounting term, the server recognizes that this question belongs to the "accounting" category.

[1109] The server adjusts its response generation algorithm, taking sentiment data into account. If the user is feeling "anxious," an algorithm is activated to make the generated response more helpful and detailed. The server then searches for relevant information in its internal database and documentation manuals. Search engines such as Elasticsearch are used for this purpose. For example, it might identify relevant information from "Chapter 3, Section 3.2 of the manual."

[1110] Based on the searched information, the server generates an appropriate response tailored to the user's emotional state. This is done using a generative AI model (e.g., GPT-3 or ChatGPT) to create a suitable natural language response. The generated response is sent from the server to the terminal, returning the response text as an HTTP response. The terminal displays the received response on the user's screen, allowing the user to review the displayed response and resolve their questions.

[1111] For example, if a user enters "Please tell me how to record accounts receivable," the server will categorize this question under accounting, search for relevant manuals, and generate a helpful answer with detailed instructions. The generated answer might say something like, "We will explain how to record accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The recording process is as follows: 1. Process the payment as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match sales with accounts receivable. If you have any questions, please do not hesitate to contact us." By displaying this answer on the terminal, the user can resolve their questions and proceed with their work more efficiently.

[1112] Examples of prompt statements include the following:

[1113] "Please explain how accounts receivable are accounted for. Please take into consideration that the emotional engine is experiencing 'anxiety'."

[1114] "Generate appropriate answers when users ask questions about accounting and finance."

[1115] The above describes the specific implementation of the system of the present invention. This system allows users to efficiently acquire knowledge through dialogue without requiring specialized knowledge, reducing inquiries to accounting staff and preventing disruptions to operations. Furthermore, by combining it with an emotion engine, it becomes possible to provide appropriate responses in accordance with the user's emotions, thereby improving the user experience.

[1116] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1117] Step 1:

[1118] Users access the system using devices such as PCs and smartphones. Next, users enter questions or concerns into text boxes located on the system's interface. For example, they might enter, "Please tell me how accounts receivable are accounted for." At this time, the emotion engine recognizes emotional data in real time from the user's facial expressions and tone of voice, and collects emotional data indicating "anxiety" or "impatience."

[1119] Input: User's question text and sentiment data

[1120] Output: Combined information of user question text and sentiment data

[1121] Step 2:

[1122] The terminal sends the user-entered question text and sentiment data to the server. This combined information is sent in the form of an HTTP POST request, for example, as data in JSON format.

[1123] Input: User's question text and sentiment data

[1124] Output: JSON format data passed to the server

[1125] Step 3:

[1126] The server analyzes the received data. First, the server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to tokenize the question text and extract important keywords. For example, "accounts receivable" and "accounting method" might be extracted.

[1127] Input: User's question text and sentiment data

[1128] Output: Key keywords (e.g., "accounts receivable", "accounting method")

[1129] Step 4:

[1130] Based on the extracted keywords, the server determines the category of the question. Using a pre-trained category classification model such as BERT, it determines, for example, that the question belongs to the "accounting" category.

[1131] Input: Important keywords

[1132] Output: Question category (e.g., "Accounting")

[1133] Step 5:

[1134] The server uses sentiment data to adjust the response generation algorithm. If the user indicates "anxiety," an algorithm is activated to make the response more helpful and detailed.

[1135] Input: Sentiment data

[1136] Output: Adjustment of the response generation algorithm based on emotion.

[1137] Step 6:

[1138] The server searches for relevant information based on the question category and sentiment data. It uses search engines such as Elasticsearch to identify relevant information from internal databases and documentation manuals. For example, information might be identified from "Chapter 3, Section 3.2 of the manual."

[1139] Input: Question category and sentiment data

[1140] Output: Related information (e.g., Chapter 3, Section 3.2 of the manual)

[1141] Step 7:

[1142] The server generates answers based on relevant information. It uses a generative AI model (e.g., GPT-3 or ChatGPT) to generate appropriate natural language responses. The generated answers include specific details such as, "We will explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. Accounting is done using the following procedure."

[1143] Input: Related information and sentiment data

[1144] Output: Generated answer text

[1145] Step 8:

[1146] The server sends the generated response to the terminal. The response text is returned to the terminal as an HTTP response.

[1147] Input: Generated response text

[1148] Output: Response text sent to the terminal

[1149] Step 9:

[1150] The terminal displays the answer received from the server on the user's screen. The user reviews the displayed answer and resolves their question. They can also re-enter their question if necessary.

[1151] Input: Response text received from the server

[1152] Output: The answer displayed on the user's screen.

[1153] The above describes the specific processing flow of this system's program.

[1154] (Application Example 2)

[1155] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1156] Traditional systems have faced challenges in providing users with timely and appropriate information when they are efficiently acquiring legal, financial, and accounting-related expertise. Furthermore, a lack of consideration for user emotions leads to a poor user experience and difficulties in effective communication. Additionally, insufficient answers to user questions can trigger further inquiries, disrupting business operations.

[1157] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion recognition means for recognizing the user's emotions at the time of input, natural language processing means for analyzing the transmitted question and retrieving related information, and answer generation means for generating an answer based on the retrieved information and the user's emotional state. This makes it possible to quickly generate an appropriate answer that takes the user's emotions into consideration.

[1158] A "user" is someone who uses this system to input questions or inquiries.

[1159] An "input method" is an interface for users to input questions or doubts.

[1160] "Emotion recognition means" refers to technology for recognizing the emotions a user expresses during input.

[1161] "Transmission means" refers to the function for sending the entered question and recognized sentiment data to the server.

[1162] "Natural language processing means" refers to technologies for analyzing submitted questions and retrieving related information.

[1163] "Answer generation means" refers to technology for generating answers based on searched information and the user's emotional state.

[1164] A "server" is a computing system that analyzes input questions and sentiment data to provide appropriate information.

[1165] A "category" is a field or theme that is classified according to the content of the question.

[1166] "Information" refers to data used to resolve user questions, including past questions and FAQ databases.

[1167] An "answer" is the information and instructions generated in response to a user's question.

[1168] "Additional support information" refers to supplementary information provided in addition to basic answers, based on user sentiment data.

[1169] Modes for carrying out the invention

[1170] This invention is a support system for users to resolve questions and problems related to electronic payments. This system incorporates an emotion recognition mechanism to recognize the user's emotions, thereby improving the user experience. The system's program processing and specific embodiments are described below.

[1171] Hardware and software to be used

[1172] Hardware:

[1173] 1. Smartphone: A device used by the user for input.

[1174] 2. Server: A computer system that analyzes the entered questions and sentiment data and provides appropriate information.

[1175] software:

[1176] 1. EmotionEngine: An emotion recognition library that recognizes user emotions in real time.

[1177] 2. NLPProcessor: A library for natural language processing.

[1178] 3. Python and Flask: Languages ​​and frameworks for implementing server-side programs

[1179] Data processing and data calculation

[1180] 1. User input:

[1181] The device (smartphone) provides an interface for the user to input questions or concerns. The entered text is analyzed by sentiment recognition technology to identify the user's emotions. For example, a question such as "I can't find my recent transaction history, what should I do?" might be entered.

[1182] 2. Emotion recognition:

[1183] The emotion recognition system (EmotionEngine) identifies the user's emotions from the input text. In this case, it recognizes that the user is feeling "anxiety" or "unease."

[1184] 3. Submission of questions and sentiment data:

[1185] The user's question data, along with the identified sentiment data, is sent to the server. This transmission is an HTTP POST request.

[1186] 4. Question analysis and information retrieval:

[1187] The server uses an NLP processor to analyze text data and extract relevant keywords (e.g., "transaction history," "not displayed"). These keywords are then used to classify the data into appropriate categories (e.g., "electronic payment").

[1188] 5. Response generation and emotional response:

[1189] The server searches its internal database and FAQs for relevant information based on the extracted keywords and recognized sentiment data. At this time, it generates an appropriate response based on the sentiment state. For example, it might generate a response such as, "If your recent transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Refresh the data from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link."

[1190] 6. Submitting and viewing responses:

[1191] The generated answer is sent from the server to the terminal and displayed on the user's screen. The user can then review this answer and resolve their question.

[1192] Examples of specific cases and prompt statements

[1193] If a user enters the question "I can't find my recent transaction history, what should I do?" on their smartphone, and the emotion recognition system identifies the user as "anxious," the server categorizes the question under "electronic payments" and provides an appropriate answer from the relevant FAQ database. This answer might be something like, "If your recent transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Perform a data refresh from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link." This answer is then displayed on the user's device, quickly resolving their question.

[1194] The above describes a specific embodiment of the present invention system. This system enables users to efficiently resolve problems related to electronic payments and receive appropriate support that takes their feelings into consideration.

[1195] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1196] Step 1:

[1197] User input:

[1198] The user enters questions or inquiries using a smartphone interface. For example, they might enter text like, "I can't find my recent transaction history, what should I do?" This input generates text data for the question. Simultaneously, the user's emotions are recognized in real time.

[1199] Step 2:

[1200] Emotion recognition:

[1201] The smartphone device uses emotion recognition technology (EmotionEngine) to analyze the user's emotions at the time of input. The analysis identifies the emotion the user is feeling (e.g., anxiety). Input data consists of a question text and emotion data. Output data consists of text data and the recognized emotion data.

[1202] Step 3:

[1203] Transmission method:

[1204] The smartphone generates and sends an HTTP POST request to the server to send text data entered by the user and recognized sentiment data. The input data at this time consists of text data and sentiment data, and the output after transmission is the status that the data has been successfully sent to the server.

[1205] Step 4:

[1206] Natural language processing:

[1207] The server analyzes the transmitted text data using an NLP processor and extracts key keywords from the text (e.g., "transaction history", "not displayed"). The input data is text data, and the output data is a list of the extracted keywords.

[1208] Step 5:

[1209] Category classification:

[1210] The server classifies user questions into specific categories (e.g., electronic payments) based on extracted keywords. The input data is a list of keywords, and the output data is category information.

[1211] Step 6:

[1212] Information retrieval and answer generation:

[1213] The server searches for relevant information from its internal database and FAQs, taking into account category and user sentiment data. Based on the retrieved data, it generates an answer. For example, it retrieves information on how to view transaction history from the relevant FAQ database and generates a detailed and helpful answer when the user is anxious. Input data consists of category information and sentiment data, while output data is the answer text.

[1214] Step 7:

[1215] Transmission method:

[1216] The server sends the generated response to the smartphone as an HTTP response. The input data is the response text, and the output data is the status of the successful transmission of the response data to the smartphone.

[1217] Step 8:

[1218] Display answer:

[1219] The smartphone device displays the response sent from the server on the user's screen. For example, it might display: "If your transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Refresh the data from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link." The input data is the response text, and the output data is the displayed response.

[1220] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1221] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1222] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1223] [Fourth Embodiment]

[1224] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1225] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1226] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1227] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1228] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1229] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1230] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1231] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1232] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1233] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1234] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1235] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1236] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1237] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing disruptions to their work. This system consists of user, terminal, and server components. The system's program processing and specific implementation examples are described below.

[1238] 1. User input of question

[1239] Users access the system using devices such as PCs and smartphones. Users enter their questions through an interface for entering doubts and inquiries (e.g., a text box). For example, a user might enter, "Please tell me how accounts receivable are accounted for."

[1240] 2. Sending questions via device

[1241] The questions entered by the user are sent from the terminal to the server. In this process, data is sent to the server as an HTTP request (e.g., a POST request).

[1242] 3. Server analysis of the question

[1243] The server analyzes the received question. Specifically, it uses natural language processing (NLP) techniques to analyze the question and extract relevant keywords. For example, keywords such as "accounts receivable" and "accounting method" may be extracted.

[1244] 4. Server-side determination of question category

[1245] The server determines the category of the question based on the extracted keywords. For example, since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[1246] 5. Searching for information on the server

[1247] The server searches for relevant information from internal databases, manuals, FAQ databases, etc., based on the question category. For example, if information on "how to account for accounts receivable" is found in Chapter 3, Section 3.2 of the manual, it will locate it.

[1248] 6. Server-driven response generation

[1249] Based on the searched information, the server generates an appropriate answer to the user's question. For example, it might generate an answer such as, "I will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided. Accounting is done in the following steps: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable."

[1250] 7. Server sends response

[1251] The generated response is sent from the server to the terminal. This is also sent as an HTTP response and received by the terminal.

[1252] 8. Displaying responses via device

[1253] The terminal receives the response sent from the server and displays it on the user's screen. The user can then review this and resolve their question.

[1254] 9. User input of additional questions

[1255] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[1256] Specific example:

[1257] If a user enters "Please tell me how to record accounts receivable," the server identifies this question as belonging to the "Accounting" category and searches Chapter 3, Section 3.2 of the manual to generate an appropriate answer. The answer is: "We will explain how to record accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The recording process is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match sales with accounts receivable." This answer is displayed on the user's terminal, allowing the user to resolve their question.

[1258] This invention allows users to efficiently acquire knowledge through an interactive format, even without specialized knowledge, and prevents delays in their work.

[1259] The following describes the processing flow.

[1260] Step 1:

[1261] The user accesses the system's web application or dedicated app. The user enters "Please tell me how accounts receivable are accounted for" into the interface (text box) for entering questions or inquiries.

[1262] Step 2:

[1263] The terminal sends the user's entered question to the server. Specifically, it sends the question's text data to the server as an HTTP POST request.

[1264] Step 3:

[1265] The server receives the sent HTTP request and retrieves the question text. The question text is "Please tell me how to account for accounts receivable."

[1266] Step 4:

[1267] The server analyzes the question text using a natural language processing (NLP) library. Specifically, it performs keyword extraction, identifying important keywords such as "accounts receivable" and "accounting method."

[1268] Step 5:

[1269] The server determines the category of the question based on the extracted keywords. In this case, the keyword "accounts receivable" indicates that the question belongs to the "accounting" category.

[1270] Step 6:

[1271] The server searches for relevant information from its internal database or manual based on the category. For example, it might identify information on "Accounts Receivable Accounting Methods" from Chapter 3, Section 3.2 of the manual.

[1272] Step 7:

[1273] The server generates an answer based on the searched information. It generates an answer such as: "We will explain how to record accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. The recording procedure is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match the sales with the accounts receivable."

[1274] Step 8:

[1275] The server sends the generated response to the user's device. Specifically, it returns the response text to the device as an HTTP response.

[1276] Step 9:

[1277] The device displays the received response on the user's screen. The user reviews the response displayed on the screen and resolves their question.

[1278] Step 10:

[1279] If the user has any further questions, they can enter them again in the text box and click the submit button. This process is repeated, allowing the user to obtain the necessary information step by step.

[1280] The above outlines the specific flow of program processing in this system.

[1281] (Example 1)

[1282] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1283] Many modern jobs require specialized knowledge, but it is extremely difficult for users to efficiently acquire this knowledge and smoothly carry out their work. Furthermore, a lack of appropriate guidance and learning materials can cause work to stall. To solve these problems, there is a need for a system that allows users to easily access and deepen their understanding of specialized knowledge.

[1284] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1285] In this invention, the server includes an input means for the user to input questions or doubts, a transmission means for sending the input questions to the server, a natural language processing means for analyzing the transmitted questions and searching for related information, a generative AI model means for generating answers based on the searched information, and a transmission means for sending the generated answers to the user. This makes it possible for users to efficiently acquire knowledge in an interactive format without having specialized knowledge, and to prevent delays in their work.

[1286] An "input method" is a means of providing an interface for users to input questions or doubts.

[1287] "Transmission method" refers to the means by which questions and data entered by the user are sent to the server.

[1288] "Natural language processing means" refers to technologies and methods used to analyze submitted questions and retrieve relevant information.

[1289] "Generative AI model means" refers to a means of using an artificial intelligence model to generate answers based on searched information.

[1290] An "answer generation means" is a means of generating an appropriate answer to a user's question.

[1291] A "category classification method" is a means of classifying questions analyzed using natural language processing into specific categories.

[1292] A "past questions and FAQ database" refers to a database that compiles questions and frequently asked questions accumulated to date.

[1293] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing disruptions to their work. This system consists of user, terminal, and server components.

[1294] 1. User input of question

[1295] Users access the system using devices such as PCs and smartphones. A text box appears on the input screen, into which the user enters their question. For example, they might enter, "Please tell me how accounts receivable are accounted for."

[1296] 2. Sending questions via device

[1297] The question entered by the user is sent from the terminal to the server as an HTTP POST request. The POST request contains the user's question.

[1298] 3. Server analysis of the question

[1299] The server uses a natural language processing (NLP) library (e.g., SpaCy) to analyze the received data. It divides the question into tokens, analyzes their meaning, and extracts keywords.

[1300] 4. Server-side determination of question category

[1301] The server uses a machine learning model (e.g., a Scikit-learn classifier) ​​to determine the category of the question based on the extracted keywords. The model is pre-trained on data related to categories such as accounting and legal affairs.

[1302] 5. Searching for information on the server

[1303] Based on the question category, the server searches for relevant information from its internal databases and manuals (e.g., the MySQL database). It then identifies the most relevant information from the search results.

[1304] 6. Server-driven response generation

[1305] Based on the searched information, the server uses a generative AI model (e.g., GPT-3) to generate appropriate answers. The answers are then formatted in a user-friendly manner.

[1306] 7. Server sends response

[1307] The generated response is sent from the server to the terminal as an HTTP response. This includes data in JSON format.

[1308] 8. Displaying responses via device

[1309] The terminal displays the response data received from the server on the screen. The user reviews this and resolves their questions.

[1310] 9. User input of additional questions

[1311] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[1312] Specific example:

[1313] If a user enters "Please tell me how to record accounts receivable," the server identifies this question as belonging to the "Accounting" category and searches Chapter 3, Section 3.2 of the accounting manual to generate an appropriate answer. The answer is: "We will explain how to record accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. The recording procedure is as follows: 1. Process the transaction when the sale is confirmed. 2. Add an accounts receivable account to the accounting software. 3. Match sales with accounts receivable." This answer is displayed on the user's terminal, allowing the user to resolve their question.

[1314] The above is one specific example of the present invention. Through the processing procedure described above, users can effectively proceed with learning and prevent delays in their work, even without specialized knowledge.

[1315] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1316] Step 1:

[1317] User input of question

[1318] Processing details:

[1319] Users access the system using a PC or smartphone and enter questions into text boxes displayed on the input screen. The entered information is saved in text format.

[1320] input:

[1321] The question entered by the user in the text box (e.g., "Please tell me how accounts receivable are accounted for.").

[1322] output:

[1323] Text data entered by the user.

[1324] Specific actions:

[1325] The user launches a browser, accesses the system's webpage, and enters the question into the text box.

[1326] Step 2:

[1327] Sending questions via device

[1328] Processing details:

[1329] The question entered by the user is sent from the terminal to the server as an HTTP POST request. This request contains the question content.

[1330] input:

[1331] Text data entered by the user.

[1332] output:

[1333] An HTTP POST request sent to the server.

[1334] Specific actions:

[1335] The device sends data to the / api / question endpoint requesting information on how accounts receivable are accounted for.

[1336] Step 3:

[1337] Server-based analysis of the question

[1338] Processing details:

[1339] The server uses a natural language processing (NLP) library (e.g., SpaCy) to analyze the received question data. It splits the text data into tokens, analyzes their meaning, and extracts relevant keywords.

[1340] input:

[1341] Question data sent to the server.

[1342] output:

[1343] Extracted keywords (e.g., "accounts receivable", "accounting method").

[1344] Specific actions:

[1345] The server tokenizes the received text data using the SpaCy library and extracts important keywords.

[1346] Step 4:

[1347] Server-based question category determination

[1348] Processing details:

[1349] The server uses a machine learning model (e.g., a Scikit-learn classifier) ​​to determine the category of the question based on the extracted keywords. The model is trained on data related to categories such as accounting and legal affairs.

[1350] input:

[1351] Extracted keywords (e.g., "accounts receivable", "accounting method").

[1352] output:

[1353] The category of the question (e.g., "Accounting").

[1354] Specific actions:

[1355] The server uses a Scikit-learn classifier to analyze keywords and categorize the question into the appropriate category (e.g., "accounting").

[1356] Step 5:

[1357] Searching for information via server

[1358] Processing details:

[1359] Based on the question category, the server searches for relevant information in its internal databases and manuals (e.g., MySQL database). It then identifies the most relevant information from the search results.

[1360] input:

[1361] Categorized questions.

[1362] output:

[1363] Related information (e.g., Accounting Manual, Chapter 3, Section 3.2).

[1364] Specific actions:

[1365] The server uses an SQL query to retrieve information about "accounts receivable accounting methods" from accounting-related tables in the MySQL database.

[1366] Step 6:

[1367] Server-driven response generation

[1368] Processing details:

[1369] Based on the searched information, the server uses a generative AI model (e.g., GPT-3) to generate appropriate answers. The generated answers are then formatted in a user-friendly manner.

[1370] input:

[1371] Related information found through searches.

[1372] output:

[1373] The generated response text (example: "We will explain how to account for accounts receivable. Accounts receivable are...").

[1374] Specific actions:

[1375] Enter "Please tell me how to account for accounts receivable" as the GPT-3 prompt and retrieve the generated response.

[1376] Step 7:

[1377] Server sends response

[1378] Processing details:

[1379] The generated response is sent from the server to the terminal as an HTTP response. The response contains data in JSON format.

[1380] input:

[1381] The generated response text.

[1382] output:

[1383] JSON data sent to the terminal as an HTTP response.

[1384] Specific actions:

[1385] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1386] Step 8:

[1387] Displaying responses via device

[1388] Processing details:

[1389] The terminal displays the response data received from the server on the screen. The user can then review this and resolve their questions.

[1390] input:

[1391] JSON data sent from the server.

[1392] output:

[1393] The answer text displayed on the screen.

[1394] Specific actions:

[1395] The device displays the following response in its browser: "We will explain how to account for accounts receivable..."

[1396] Step 9:

[1397] User input of additional questions

[1398] Processing details:

[1399] If the user has further questions, they can enter additional questions through the same interface and submit them again. This process is repeated.

[1400] input:

[1401] Additional questions from the user.

[1402] output:

[1403] Additional questions resent to the server.

[1404] Specific actions:

[1405] The user types, "Could you please explain the accounting method in more detail?" and presses the submit button again.

[1406] (Application Example 1)

[1407] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1408] Currently, many virtual stores have customers who frequently have specialized questions regarding finance and legal matters, but there is a lack of mechanisms to answer these questions quickly and appropriately. As a result, transactions may stall because customer questions remain unresolved, potentially leading to lost business opportunities. This invention aims to solve this problem by providing a system that utilizes a generative AI model to provide appropriate answers in real time.

[1409] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1410] In this invention, the server includes an input means for the user to input questions or doubts; a transmission means for sending the input questions to the server; a natural language processing means for analyzing the transmitted questions and searching for relevant information; an answer generation means for generating answers based on the searched information; a transmission means for sending the generated answers to the user; and a means for generating answers to user-inputted questions using a generation AI model. This makes it possible to provide appropriate answers to user questions immediately.

[1411] A "user" is someone who uses the system to input questions and receive answers.

[1412] "Input methods for entering questions or doubts" refers to interfaces or devices that users use to enter questions or doubts, such as text boxes or voice input devices.

[1413] "Means of transmission for sending entered questions to the server" refers to communication means for sending questions entered by the user to the server, such as software functions for sending HTTP requests.

[1414] "Natural language processing means" refers to software that uses natural language processing techniques to analyze input questions and extract relevant information.

[1415] "Answer generation means" refers to algorithms or programs that generate appropriate answers to user questions based on retrieved information.

[1416] "Means of transmission for sending generated responses to users" refers to communication means for sending generated responses to users, such as software functions for sending HTTP responses.

[1417] A "generative AI model" refers to an artificial intelligence model used to generate answers to questions, such as a deep learning model used for natural language processing.

[1418] A "prompt statement" refers to the input text used when generating an answer for a generative AI model.

[1419] This invention relates to a system that enables users to efficiently acquire specialized knowledge in finance and law, thereby preventing disruptions to their work. This system consists of user, terminal, and server components. A detailed example of the system's operation is described below.

[1420] 1. User input of question

[1421] Users access the system using devices such as smartphones or head-mounted displays. Users input their questions through interfaces for entering doubts and inquiries, such as text boxes or voice input devices. For example, a user might input, "Please tell me how accounts receivable are accounted for."

[1422] 2. Submit the entered question.

[1423] The terminal uses a communication method to send the user-entered questions to the server, sending the data as an HTTP request.

[1424] 3. Question Analysis and Search

[1425] The server analyzes the received question using natural language processing techniques and searches for relevant information. This analysis utilizes a generative AI model. For example, it extracts keywords such as "accounts receivable" and "accounting method" and searches for relevant information from past questions and FAQ databases.

[1426] 4. Generating the answer

[1427] The server uses a generative AI model based on the retrieved information to generate appropriate answers to the user's questions. In this process, the prompt is used as input to the generative AI model. An example of a prompt is, "Please tell me how accounts receivable are accounted for."

[1428] 5. Submit the generated response.

[1429] The generated response is sent from the server to the terminal. It is sent as an HTTP response and received by the terminal.

[1430] 6. Display the answer

[1431] The terminal receives the answer sent from the server and displays it on the user's screen. The user can then review this and resolve their questions. For example, an answer might be generated stating, "We will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services. Accounting is done using the following steps: 1. Process the transaction when the sale is finalized. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable."

[1432] This invention allows users to efficiently acquire knowledge through dialogue, even without specialized knowledge, and prevents delays in their work. This system utilizes a generated AI model and prompt text to provide quick and appropriate answers. Furthermore, because users can resolve financial and legal questions through dialogue, it enables the provision of high-quality customer support even in virtual stores.

[1433] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1434] Step 1:

[1435] The user inputs a question or inquiry. The user uses a smartphone or head-mounted display to input a question into the interface via text or voice input, such as "Please tell me how accounts receivable are accounted for." The input in this step is the user's question, and the output is the question data on the device.

[1436] Step 2:

[1437] The terminal sends the entered question to the server. The terminal sends the user-entered question to the server as an HTTP request (POST request). The input in this step is the question data, and the output is the request data sent to the server.

[1438] Step 3:

[1439] The server analyzes the question and searches for relevant information. The server uses natural language processing techniques to analyze the question and extract keywords. For example, it might extract "accounts receivable" or "accounting method" and search for relevant information in its internal database and FAQ database. The input for this step is the request data, and the output is the analyzed keywords and search results.

[1440] Step 4:

[1441] The server generates an answer based on the search results. The server uses a generative AI model to generate a prompt sentence from the searched information, and then uses this prompt sentence to generate an appropriate answer to the user's question. For example, it might generate an answer such as, "Let me explain how accounts receivable are accounted for. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided..." The inputs to this step are the search results and the prompt sentence, and the output is the generated answer.

[1442] Step 5:

[1443] The server sends the generated response to the terminal. The server sends the generated response to the terminal as an HTTP response. The input for this step is the generated response, and the output is the response data sent to the terminal.

[1444] Step 6:

[1445] The terminal displays the answer on the user's screen. The terminal displays the answer received from the server in the user interface. For example, it might display something like, "We will explain how accounts receivable are accounted for. Accounts receivable represent the amount of money that has not yet been paid despite the provision of goods or services..." The input for this step is response data, and the output is the answer displayed to the user.

[1446] This allows users to resolve their questions and perform their tasks smoothly.

[1447] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1448] This invention relates to a system that enables users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing delays in work. By incorporating an emotion engine that recognizes user emotions, this system can improve the user experience and provide more appropriate responses. The system's program processing and specific implementation examples are described below.

[1449] 1. User input of question

[1450] Users access the system using devices such as PCs and smartphones. They enter a question or inquiry into a text box interface, such as "Please tell me how accounts receivable are accounted for." During this process, an emotion engine recognizes the user's emotions (e.g., anxiety, impatience) in real time.

[1451] 2. Transmission of questions and sentiment data via the device

[1452] Along with the question entered by the user, the sentiment data recognized by the sentiment engine is sent to the server. Specifically, the text data and sentiment data are sent to the server as an HTTP POST request.

[1453] 3. Server analysis of the question

[1454] The server analyzes the submitted question text using a natural language processing (NLP) library. It extracts important keywords from the question text (e.g., "accounts receivable," "accounting method").

[1455] 4. Server-side determination of question category

[1456] The server determines the category of the question based on the extracted keywords. For example, since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[1457] 5. Use of emotional data by servers

[1458] The server also takes into account the user's emotional data as recognized by the emotion engine. For example, if the user indicates an "anxious" emotion, it adjusts its response to be more helpful and detailed.

[1459] 6. Searching for information on the server

[1460] The server searches its internal database and manuals for relevant information based on the question category and sentiment data. For example, it might identify information about "accounts receivable accounting methods" from Chapter 3, Section 3.2 of the manual.

[1461] 7. Server-driven response generation

[1462] Based on the searched information, the server generates an appropriate response tailored to the user's emotional state. For example, it might generate a response such as, "Let me explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. Accounting is done using the following steps: 1. Process the transaction as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match sales with accounts receivable. If you have any questions, please do not hesitate to contact us."

[1463] 8. Server sends response

[1464] The generated response is sent from the server to the terminal. Specifically, the response text is returned to the terminal as an HTTP response.

[1465] 9. Displaying responses via the device

[1466] The device displays the received response on the user's screen. The user reviews the displayed response and resolves their question. They can also re-enter their question if necessary.

[1467] Specific example:

[1468] If a user types "Please tell me how to account for accounts receivable" and the sentiment engine recognizes that the user is feeling "anxious," the server categorizes this question under "Accounting." It then searches for relevant manual information and generates a helpful answer that includes detailed instructions on "how to account for accounts receivable." The generated answer might say something like, "We will explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The process involves the following steps: 1. Processing the transaction as soon as the sale is confirmed. 2. Adding an accounts receivable account to your accounting software. 3. Matching sales with accounts receivable. Please feel free to contact us if you have any questions." This answer is then displayed on the user's device, resolving their concerns.

[1469] The above describes the program processing and specific embodiments of the present invention system. This system allows users to efficiently acquire knowledge through dialogue without requiring specialized knowledge, reducing inquiries to accounting staff and preventing disruptions to operations. Furthermore, by combining it with an emotion engine, it becomes possible to provide appropriate responses in accordance with the user's emotions, thereby improving the user experience.

[1470] The following describes the processing flow.

[1471] Step 1:

[1472] The user accesses the system's web application or dedicated app. The user enters "Please tell me how accounts receivable are accounted for" into the interface (text box) for entering questions or inquiries.

[1473] Step 2:

[1474] The device's emotion engine analyzes the user's facial expressions, typing speed, and text content during input to recognize the user's emotions. For example, if the user is feeling anxious or worried, the emotion engine identifies that emotion as "anxiety."

[1475] Step 3:

[1476] The device sends the user-entered question and recognized sentiment data to the server. Specifically, it sends text data and sentiment data to the server as an HTTP POST request.

[1477] Step 4:

[1478] The server receives the sent HTTP request and retrieves the question text and sentiment data. The question text is "Please tell me how to account for accounts receivable," and the sentiment data is "anxious."

[1479] Step 5:

[1480] The server analyzes the question text using a natural language processing (NLP) library. It extracts important keywords such as "accounts receivable" and "accounting method" from the question text.

[1481] Step 6:

[1482] The server determines the category of the question based on the extracted keywords. Since "accounts receivable" is an accounting-related term, the server recognizes that this question belongs to the "accounting" category.

[1483] Step 7:

[1484] The server searches for relevant information in its internal database and manuals. For example, it might identify information regarding "Accounts Receivable Accounting Methods" from Chapter 3, Section 3.2 of the manual.

[1485] Step 8:

[1486] The server takes emotional data into account and generates responses that correspond to the user's emotional state. For example, if the recognized emotion is "anxiety," the response is adjusted to be more helpful and detailed. A specific response might be: "We will explain how to account for accounts receivable. Accounts receivable represent the amount of money that has not yet been paid for goods or services that have been provided. Accounting is done using the following steps: 1. Process the receivable as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match the sales with the accounts receivable. If you have any questions, please do not hesitate to contact us."

[1487] Step 9:

[1488] The server sends the generated response to the user's device. Specifically, it returns the response text to the device as an HTTP response.

[1489] Step 10:

[1490] The device displays the received response on the user's screen. The user can review the displayed response and resolve their questions. They can also re-enter their questions if necessary.

[1491] The above outlines the specific flow of program processing in this system.

[1492] (Example 2)

[1493] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1494] In today's complex business environment, it is crucial for users to acquire specialized knowledge quickly and accurately, preventing disruptions to their work. Furthermore, traditional systems often generate uniform answers without considering the user's emotional state, resulting in a poor user experience. This is particularly true in legal, finance, and accounting departments, where high levels of expertise are required, but at the same time, responses that consider the user's emotions are also necessary. Traditional systems struggled to meet these needs, leading to increased user stress and inquiries.

[1495] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means for the user to input questions or doubts, a transmission means for sending the input questions and sentiment data to the server, a natural language processing means for analyzing the transmitted questions and searching for related information, an answer generation means for generating answers based on the user's sentiment data, and a transmission means for sending the generated answers to the user. This makes it possible to provide quick and accurate answers based on expert knowledge while taking into account the user's emotional state. The user can efficiently acquire knowledge in an interactive format and perform tasks without stress, thus improving overall work efficiency. Furthermore, by using a generation AI model, flexible and appropriate answers can be automatically generated for the user, reducing the number of inquiries and preventing delays in operations.

[1496] A "user" is an entity that uses a system to input questions and obtain answers.

[1497] "Questions and inquiries" refer to text information related to the information users want to know or the problems they want to solve.

[1498] An "input method" refers to an interface or device used by a user to input questions or doubts.

[1499] "Transmission means" refers to the mechanism or function for sending the entered question and sentiment data to the server.

[1500] A "server" is a computer system that analyzes input questions, searches for relevant information, and generates and transmits answers.

[1501] "Natural language processing tools" are algorithms and technologies used to analyze input questions and understand important keywords and context.

[1502] "Emotional data" refers to data that indicates the emotional state of a user when they enter a question or inquiry.

[1503] "Answer generation means" refers to a function that generates answers to be provided to the user based on analyzed questions and sentiment data.

[1504] A "generative AI model" is a pre-trained model that uses artificial intelligence to generate appropriate answers.

[1505] This invention is a system designed to enable users to efficiently acquire specialized knowledge in areas such as legal affairs and finance / accounting, thereby preventing delays in work. Specifically, by incorporating an emotion engine that recognizes user emotions, it can improve the user experience and provide more appropriate answers. The following describes a specific embodiment of this system.

[1506] First, the user accesses the system using a device such as a PC or smartphone. The user enters questions or doubts into text boxes provided on the system's interface. At this time, the emotion engine recognizes the user's emotions (e.g., anxiety, impatience) in real time.

[1507] For example, if a user enters "Please tell me how accounts receivable are accounted for," the emotion engine recognizes that the user is feeling "anxious." This information is sent from the terminal to the server. Specifically, text data (the user's question) and emotion data (the emotion recognized by the emotion engine) are sent to the server as an HTTP POST request.

[1508] Next, the server parses the received data. The server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to tokenize the question text and extract key keywords (e.g., "accounts receivable," "accounting method"). Then, based on the extracted keywords, it determines the category of the question. For example, since "accounts receivable" is an accounting term, the server recognizes that this question belongs to the "accounting" category.

[1509] The server adjusts its response generation algorithm, taking sentiment data into account. If the user is feeling "anxious," an algorithm is activated to make the generated response more helpful and detailed. The server then searches for relevant information in its internal database and documentation manuals. Search engines such as Elasticsearch are used for this purpose. For example, it might identify relevant information from "Chapter 3, Section 3.2 of the manual."

[1510] Based on the searched information, the server generates an appropriate response tailored to the user's emotional state. This is done using a generative AI model (e.g., GPT-3 or ChatGPT) to create a suitable natural language response. The generated response is sent from the server to the terminal, returning the response text as an HTTP response. The terminal displays the received response on the user's screen, allowing the user to review the displayed response and resolve their questions.

[1511] For example, if a user enters "Please tell me how to record accounts receivable," the server will categorize this question under accounting, search for relevant manuals, and generate a helpful answer with detailed instructions. The generated answer might say something like, "We will explain how to record accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. The recording process is as follows: 1. Process the payment as soon as the sale is confirmed. 2. Add an accounts receivable account to your accounting software. 3. Match sales with accounts receivable. If you have any questions, please do not hesitate to contact us." By displaying this answer on the terminal, the user can resolve their questions and proceed with their work more efficiently.

[1512] Examples of prompt statements include the following:

[1513] "Please explain how accounts receivable are accounted for. Please take into consideration that the emotional engine is experiencing 'anxiety'."

[1514] "Generate appropriate answers when users ask questions about accounting and finance."

[1515] The above describes the specific implementation of the system of the present invention. This system allows users to efficiently acquire knowledge through dialogue without requiring specialized knowledge, reducing inquiries to accounting staff and preventing disruptions to operations. Furthermore, by combining it with an emotion engine, it becomes possible to provide appropriate responses in accordance with the user's emotions, thereby improving the user experience.

[1516] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1517] Step 1:

[1518] Users access the system using devices such as PCs and smartphones. Next, users enter questions or concerns into text boxes located on the system's interface. For example, they might enter, "Please tell me how accounts receivable are accounted for." At this time, the emotion engine recognizes emotional data in real time from the user's facial expressions and tone of voice, and collects emotional data indicating "anxiety" or "impatience."

[1519] Input: User's question text and sentiment data

[1520] Output: Combined information of user question text and sentiment data

[1521] Step 2:

[1522] The terminal sends the user-entered question text and sentiment data to the server. This combined information is sent in the form of an HTTP POST request, for example, as data in JSON format.

[1523] Input: User's question text and sentiment data

[1524] Output: JSON format data passed to the server

[1525] Step 3:

[1526] The server analyzes the received data. First, the server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to tokenize the question text and extract important keywords. For example, "accounts receivable" and "accounting method" might be extracted.

[1527] Input: User's question text and sentiment data

[1528] Output: Key keywords (e.g., "accounts receivable", "accounting method")

[1529] Step 4:

[1530] Based on the extracted keywords, the server determines the category of the question. Using a pre-trained category classification model such as BERT, it determines, for example, that the question belongs to the "accounting" category.

[1531] Input: Important keywords

[1532] Output: Question category (e.g., "Accounting")

[1533] Step 5:

[1534] The server uses sentiment data to adjust the response generation algorithm. If the user indicates "anxiety," an algorithm is activated to make the response more helpful and detailed.

[1535] Input: Sentiment data

[1536] Output: Adjustment of the response generation algorithm based on emotion.

[1537] Step 6:

[1538] The server searches for relevant information based on the question category and sentiment data. It uses search engines such as Elasticsearch to identify relevant information from internal databases and documentation manuals. For example, information might be identified from "Chapter 3, Section 3.2 of the manual."

[1539] Input: Question category and sentiment data

[1540] Output: Related information (e.g., Chapter 3, Section 3.2 of the manual)

[1541] Step 7:

[1542] The server generates answers based on relevant information. It uses a generative AI model (e.g., GPT-3 or ChatGPT) to generate appropriate natural language responses. The generated answers include specific details such as, "We will explain how to account for accounts receivable. Accounts receivable represent payments for goods or services that have been provided but have not yet been paid. Accounting is done using the following procedure."

[1543] Input: Related information and sentiment data

[1544] Output: Generated answer text

[1545] Step 8:

[1546] The server sends the generated response to the terminal. The response text is returned to the terminal as an HTTP response.

[1547] Input: Generated response text

[1548] Output: Response text sent to the terminal

[1549] Step 9:

[1550] The terminal displays the answer received from the server on the user's screen. The user reviews the displayed answer and resolves their question. They can also re-enter their question if necessary.

[1551] Input: Response text received from the server

[1552] Output: The answer displayed on the user's screen.

[1553] The above describes the specific processing flow of this system's program.

[1554] (Application Example 2)

[1555] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1556] Traditional systems have faced challenges in providing users with timely and appropriate information when they are efficiently acquiring legal, financial, and accounting-related expertise. Furthermore, a lack of consideration for user emotions leads to a poor user experience and difficulties in effective communication. Additionally, insufficient answers to user questions can trigger further inquiries, disrupting business operations.

[1557] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion recognition means for recognizing the user's emotions at the time of input, natural language processing means for analyzing the transmitted question and retrieving related information, and answer generation means for generating an answer based on the retrieved information and the user's emotional state. This makes it possible to quickly generate an appropriate answer that takes the user's emotions into consideration.

[1558] A "user" is someone who uses this system to input questions or inquiries.

[1559] An "input method" is an interface for users to input questions or doubts.

[1560] "Emotion recognition means" refers to technology for recognizing the emotions a user expresses during input.

[1561] "Transmission means" refers to the function for sending the entered question and recognized sentiment data to the server.

[1562] "Natural language processing means" refers to technologies for analyzing submitted questions and retrieving related information.

[1563] "Answer generation means" refers to technology for generating answers based on searched information and the user's emotional state.

[1564] A "server" is a computing system that analyzes input questions and sentiment data to provide appropriate information.

[1565] A "category" is a field or theme that is classified according to the content of the question.

[1566] "Information" refers to data used to resolve user questions, including past questions and FAQ databases.

[1567] An "answer" is the information and instructions generated in response to a user's question.

[1568] "Additional support information" refers to supplementary information provided in addition to basic answers, based on user sentiment data.

[1569] Modes for carrying out the invention

[1570] This invention is a support system for users to resolve questions and problems related to electronic payments. This system incorporates an emotion recognition mechanism to recognize the user's emotions, thereby improving the user experience. The system's program processing and specific embodiments are described below.

[1571] Hardware and software to be used

[1572] Hardware:

[1573] 1. Smartphone: A device used by the user for input.

[1574] 2. Server: A computer system that analyzes the entered questions and sentiment data and provides appropriate information.

[1575] software:

[1576] 1. EmotionEngine: An emotion recognition library that recognizes user emotions in real time.

[1577] 2. NLPProcessor: A library for natural language processing.

[1578] 3. Python and Flask: Languages ​​and frameworks for implementing server-side programs

[1579] Data processing and data calculation

[1580] 1. User input:

[1581] The device (smartphone) provides an interface for the user to input questions or concerns. The entered text is analyzed by sentiment recognition technology to identify the user's emotions. For example, a question such as "I can't find my recent transaction history, what should I do?" might be entered.

[1582] 2. Emotion recognition:

[1583] The emotion recognition system (EmotionEngine) identifies the user's emotions from the input text. In this case, it recognizes that the user is feeling "anxiety" or "unease."

[1584] 3. Submission of questions and sentiment data:

[1585] The user's question data, along with the identified sentiment data, is sent to the server. This transmission is an HTTP POST request.

[1586] 4. Question analysis and information retrieval:

[1587] The server uses an NLP processor to analyze text data and extract relevant keywords (e.g., "transaction history," "not displayed"). These keywords are then used to classify the data into appropriate categories (e.g., "electronic payment").

[1588] 5. Response generation and emotional response:

[1589] The server searches its internal database and FAQs for relevant information based on the extracted keywords and recognized sentiment data. At this time, it generates an appropriate response based on the sentiment state. For example, it might generate a response such as, "If your recent transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Refresh the data from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link."

[1590] 6. Submitting and viewing responses:

[1591] The generated answer is sent from the server to the terminal and displayed on the user's screen. The user can then review this answer and resolve their question.

[1592] Examples of specific cases and prompt statements

[1593] If a user enters the question "I can't find my recent transaction history, what should I do?" on their smartphone, and the emotion recognition system identifies the user as "anxious," the server categorizes the question under "electronic payments" and provides an appropriate answer from the relevant FAQ database. This answer might be something like, "If your recent transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Perform a data refresh from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link." This answer is then displayed on the user's device, quickly resolving their question.

[1594] The above describes a specific embodiment of the present invention system. This system enables users to efficiently resolve problems related to electronic payments and receive appropriate support that takes their feelings into consideration.

[1595] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1596] Step 1:

[1597] User input:

[1598] The user enters questions or inquiries using a smartphone interface. For example, they might enter text like, "I can't find my recent transaction history, what should I do?" This input generates text data for the question. Simultaneously, the user's emotions are recognized in real time.

[1599] Step 2:

[1600] Emotion recognition:

[1601] The smartphone device uses emotion recognition technology (EmotionEngine) to analyze the user's emotions at the time of input. The analysis identifies the emotion the user is feeling (e.g., anxiety). Input data consists of a question text and emotion data. Output data consists of text data and the recognized emotion data.

[1602] Step 3:

[1603] Transmission method:

[1604] The smartphone generates and sends an HTTP POST request to the server to send text data entered by the user and recognized sentiment data. The input data at this time consists of text data and sentiment data, and the output after transmission is the status that the data has been successfully sent to the server.

[1605] Step 4:

[1606] Natural language processing:

[1607] The server analyzes the transmitted text data using an NLP processor and extracts key keywords from the text (e.g., "transaction history", "not displayed"). The input data is text data, and the output data is a list of the extracted keywords.

[1608] Step 5:

[1609] Category classification:

[1610] The server classifies user questions into specific categories (e.g., electronic payments) based on extracted keywords. The input data is a list of keywords, and the output data is category information.

[1611] Step 6:

[1612] Information retrieval and answer generation:

[1613] The server searches for relevant information from its internal database and FAQs, taking into account category and user sentiment data. Based on the retrieved data, it generates an answer. For example, it retrieves information on how to view transaction history from the relevant FAQ database and generates a detailed and helpful answer when the user is anxious. Input data consists of category information and sentiment data, while output data is the answer text.

[1614] Step 7:

[1615] Transmission method:

[1616] The server sends the generated response to the smartphone as an HTTP response. The input data is the response text, and the output data is the status of the successful transmission of the response data to the smartphone.

[1617] Step 8:

[1618] Display answer:

[1619] The smartphone device displays the response sent from the server on the user's screen. For example, it might display: "If your transaction history is not displayed, please try the following steps: 1. Restart the app. 2. Refresh the data from the settings menu. 3. If the problem persists, please contact customer support. If you have any concerns, you can get support via this link." The input data is the response text, and the output data is the displayed response.

[1620] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1621] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1622] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1623] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1624] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1625] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1626] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1627] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1628] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1629] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1630] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1631] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1632] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1633] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1634] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1635] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1636] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1637] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1638] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1639] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1640] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1641] The following is further disclosed regarding the embodiments described above.

[1642] (Claim 1)

[1643] An input method for users to enter questions or doubts,

[1644] A means for sending the entered question to the server,

[1645] A natural language processing means for analyzing submitted questions and retrieving related information,

[1646] A means for generating answers based on searched information,

[1647] A means for sending the generated response to the user,

[1648] A system that includes this.

[1649] (Claim 2)

[1650] The system according to claim 1, comprising means for analyzing a question entered by a user using natural language processing and classifying it into a specific category.

[1651] (Claim 3)

[1652] The system according to claim 1, wherein the retrieved information includes a database of past questions and FAQs.

[1653] "Example 1"

[1654] (Claim 1)

[1655] An input method for users to enter questions or doubts,

[1656] A means for sending the entered question to the server,

[1657] A natural language processing means for analyzing submitted questions and retrieving related information,

[1658] A generative AI model means for generating answers based on searched information,

[1659] A means for sending the generated response to the user,

[1660] A system that includes this.

[1661] (Claim 2)

[1662] The system according to claim 1, comprising means for analyzing a question entered by a user using natural language processing and classifying it into a specific category.

[1663] (Claim 3)

[1664] The system according to claim 1, wherein the retrieved information includes a database of past questions and FAQs.

[1665] "Application Example 1"

[1666] (Claim 1)

[1667] An input method for users to enter questions or doubts,

[1668] A means for sending the entered question to the server,

[1669] A natural language processing means for analyzing submitted questions and retrieving related information,

[1670] A means for generating answers based on searched information,

[1671] A means for sending the generated response to the user,

[1672] A means of generating answers to user-inputted questions using a generative AI model,

[1673] A system that includes this.

[1674] (Claim 2)

[1675] The system according to claim 1, comprising means for analyzing a question entered by a user using natural language processing and classifying it into a specific category.

[1676] (Claim 3)

[1677] The system according to claim 1, wherein the retrieved information includes a database of past questions and FAQs, and includes means for generating prompt sentences to be input to a generating AI model.

[1678] "Example 2 of combining an emotion engine"

[1679] (Claim 1)

[1680] An input method for users to enter questions or doubts,

[1681] A means for sending the entered question and sentiment data to the server,

[1682] A natural language processing means for analyzing submitted questions and retrieving related information,

[1683] A means for generating responses based on user sentiment data,

[1684] A means for sending the generated response to the user,

[1685] A system that includes this.

[1686] (Claim 2)

[1687] The system according to claim 1, comprising means for analyzing a question entered by a user using natural language processing and classifying it into a specific category.

[1688] (Claim 3)

[1689] The system according to claim 1, comprising means for generating an appropriate response based on information retrieved using a generative AI model.

[1690] "Application example 2 of combining emotional engines"

[1691] (Claim 1)

[1692] An input method for users to enter questions or doubts,

[1693] An emotion recognition method for recognizing the emotions of the user during input,

[1694] A transmission means for sending the entered question and recognized emotion data to a server,

[1695] A natural language processing means for analyzing submitted questions and retrieving related information,

[1696] A means for generating answers based on searched information and the user's emotional state,

[1697] A means for sending the generated response to the user,

[1698] A system that includes this.

[1699] (Claim 2)

[1700] The system according to claim 1, comprising means for analyzing a question entered by a user using natural language processing and classifying it into a specific category.

[1701] (Claim 3)

[1702] The system according to claim 1, wherein the retrieved information includes past questions and FAQ databases, and provides additional support information based on user sentiment data. [Explanation of symbols]

[1703] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An input method for users to enter questions or doubts, A means for sending the entered question to the server, A natural language processing means for analyzing submitted questions and retrieving related information, A means for generating answers based on searched information, A means for sending the generated response to the user, A system that includes this.

2. The system according to claim 1, comprising means for analyzing a question entered by a user using natural language processing and classifying it into a specific category.

3. The system according to claim 1, wherein the retrieved information includes a database of past questions and FAQs.

Images