system

Abstract

We provide the system. [Solution] Means for receiving user input, A means for analyzing the intent of the input using a natural language processing engine, A means for determining the necessary processing based on the analyzed intent and for executing that processing, A means of returning the processing result to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, chatbots have been widely used to efficiently communicate with customers. However, conventional chatbots mainly provide simple fixed-form responses and have a problem that they cannot adequately respond to complex user requests. For example, they often cannot appropriately provide functions such as providing information for smooth in-store reservations and procedures, guiding online procedures, and further providing detailed inquiries regarding unclear points in the fee details. The present invention aims to solve these problems and provide an advanced chatbot system capable of responding to various user requests.

Means for Solving the Problems

[0005] The present invention provides a system that includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for determining necessary processing based on the analyzed intent and executing that processing, and means for returning the processing results to the user. It also provides a system that includes means for acquiring and proposing information related to store visit reservations to the user, means for receiving the user's desired reservation date and time and storing the reservation information, means for acquiring and guiding the user to necessary document information for the procedure, and means for acquiring and introducing online procedure information to the user. Furthermore, it provides a system that includes means for verifying the user's identity, means for acquiring and providing the user's billing details after successful identity verification, and a system that includes identity verification by SMS authentication. This realizes an advanced chatbot system that can respond appropriately and quickly to a variety of user requests.

[0006] A "user" is a person who uses a system and enters data, or a user of a selected service.

[0007] "Input" refers to messages and commands that a user sends to the system, and is information expressed as text or audio.

[0008] A "natural language processing engine" is a technology and software component used to analyze human language and understand its intent.

[0009] "Intent" refers to the purpose or request that the user wants the system to achieve through their input.

[0010] "Processing" refers to a series of tasks performed by a system based on the user's intentions, such as specific operations, calculations, data acquisition, and storage.

[0011] A "response" refers to a response message that a system sends to a user, including providing information in response to the user's questions or requests.

[0012] "Store visit reservation" refers to the procedure for users to make a reservation to visit a physical store or service location.

[0013] "Document information required for the procedure" refers to information about the types of documents and certificates required to carry out a specific procedure, as well as details about them.

[0014] "Online procedures" refer to various procedures that can be performed via the internet, eliminating the need for physical visits.

[0015] "Identity verification" is the process by which a user proves their identity and qualifications, and is a means by which the system identifies the user.

[0016] "Billing details" refers to information about the breakdown and amount of charges for services used by the user.

[0017] "SMS authentication" is a process that uses the Short Message Service to send an authentication code to the user's phone number, and verifies the user's identity by having them enter that code. [Brief explanation of the drawing]

[0018] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the language used in the following description will be explained.

[0021] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0022] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] The system of the present invention is an advanced chatbot for streamlining customer service and can perform various practical procedures based on user input. The system of the present invention is realized through a series of processes that receive user input, analyze it, and generate an appropriate response.

[0040] Overall system configuration

[0041] The system is broadly composed of the following elements:

[0042] 1. User terminal

[0043] This refers to a device that users access using the LINE app or a web browser.

[0044] 2. LINE Server

[0045] Its role is to receive user input and forward it to the chatbot server.

[0046] 3. Chatbot Server

[0047] It is the central part of the system that analyzes user input and generates appropriate responses.

[0048] It has a built-in natural language processing engine that analyzes the user's intent.

[0049] Access, read from, and write to the database as needed.

[0050] 4. Database

[0051] This is a supplementary system for managing user information, reservation information, and necessary document information for procedures.

[0052] Processing flow

[0053] 1. Receive user input

[0054] For example, a user might type "I want to make a reservation to visit the store" in the chat screen of the LINE app.

[0055] The device sends this input to the LINE server.

[0056] 2. Analyze user intent using a natural language processing engine.

[0057] The LINE server receives the input and forwards it to the chatbot server.

[0058] The chatbot server uses a natural language processing engine to analyze the intent of the input and recognizes that the user "wants to make a reservation to visit the store."

[0059] 3. Executing processing in response to user requests

[0060] For store visit reservations, the chatbot server retrieves available dates and times from the database.

[0061] The database returns available dates and times, and the chatbot server suggests possible dates and times to the user.

[0062] 4. Process the user's response.

[0063] The user selects their preferred date and time and replies in the chat, "Please make it June 12th at 10:00."

[0064] The chatbot server then analyzes this again, saves the desired date and time to the database, and confirms the reservation.

[0065] 5. Send a confirmation notice to the user.

[0066] The chatbot server generates the message "Your reservation has been confirmed. Please come to the store on June 12th at 10:00" and sends it to the user's device via the LINE server.

[0067] Specific example

[0068] The following are some specific use cases.

[0069] 1. Information on the documents required for the procedure

[0070] The user's terminal sends the message, "Please tell me what documents are required for the relocation procedure."

[0071] The chatbot server uses a natural language processing engine to analyze intent.

[0072] The system queries a database for information on "documents required for moving procedures" and retrieves the results.

[0073] The chatbot server generates a response message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the user's terminal.

[0074] 2. Introduction to online procedures

[0075] A message is sent to the user's terminal saying, "Please tell me what procedures I can do online."

[0076] The chatbot server receives this input and retrieves the appropriate online procedure information from its database.

[0077] The response message reads: "The following procedures can be done online: change of address, review contract details, and review billing statements. You can find more details at this link [link]."

[0078] 3. Inquiry regarding unclear points in the billing statement.

[0079] The user enters, "My bill this month is high, please show me the details."

[0080] The chatbot server requests identity verification and sends an SMS authentication code to the user's device.

[0081] The user enters an SMS code into the chat, and the chatbot server verifies the code.

[0082] The chatbot server retrieves the billing details from the database and sends a response message saying, "Your billing details for this month are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen."

[0083] In this way, the present invention realizes a system that provides a quick and effective response to the diverse demands of users.

[0084] The following describes the processing flow.

[0085] Step 1:

[0086] The user types "I want to make a reservation to visit" in the LINE app's chat screen. The user then sends this message.

[0087] Step 2:

[0088] The device sends user input to the LINE server. The message is received by the LINE server.

[0089] Step 3:

[0090] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[0091] Step 4:

[0092] The chatbot server uses a natural language processing engine to analyze the intent of incoming messages. The natural language processing engine identifies the intent as "I would like to make a reservation to visit the store."

[0093] Step 5:

[0094] The chatbot server queries the database for available dates and times and retrieves them. The database then sends the available dates and times back to the chatbot server.

[0095] Step 6:

[0096] The chatbot server generates a message for the user saying, "Please tell us your preferred date and time. Suggestions: June 12th, 10:00 AM, June 12th, 2:00 PM," and sends it to the user's device via the LINE server.

[0097] Step 7:

[0098] The user enters their desired date and time, such as "June 12th, 10:00 AM, please," on their device and sends it to the LINE server.

[0099] Step 8:

[0100] The LINE server receives input from the user and forwards it to the chatbot server. The chatbot server receives the message for the requested date and time.

[0101] Step 9:

[0102] The chatbot server saves the date and time of the received message to the database and updates the reservation information. The database stores the reservation information.

[0103] Step 10:

[0104] The chatbot server generates a reservation confirmation message saying, "Your reservation has been confirmed. Please come to the store on June 12th at 10:00," and sends it to the user's device via the LINE server.

[0105] The following are example steps for obtaining the necessary documents and for inquiring about any unclear points regarding the fee breakdown.

[0106] Instructions on the documents required for the procedure

[0107] Step 1:

[0108] The user uses their device to type "Please tell me what documents are required for moving procedures" and then sends the message.

[0109] Step 2:

[0110] The device sends input to the LINE server. The message is received by the LINE server.

[0111] Step 3:

[0112] The LINE server forwards the incoming message to the chatbot server. The chatbot server receives the message.

[0113] Step 4:

[0114] The chatbot server uses a natural language processing engine to analyze the intent of the input. The natural language processing engine identifies the intent as "I want to know what documents are needed for moving procedures."

[0115] Step 5:

[0116] The chatbot server queries the database for relevant document information and retrieves it. The database then sends the relevant information back to the chatbot server.

[0117] Step 6:

[0118] Based on the information it has acquired, the chatbot server generates a message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the device via the LINE server.

[0119] Inquiry regarding unclear points in the billing statement

[0120] Step 1:

[0121] The user types, "My bill this month is high, please show me the details," and sends it via the LINE app.

[0122] Step 2:

[0123] The device sends the input to the LINE server. The message is received by the LINE server.

[0124] Step 3:

[0125] The LINE server forwards the incoming message to the chatbot server. The chatbot server receives the message.

[0126] Step 4:

[0127] The chatbot server uses a natural language processing engine to analyze the user's intent and recognizes that they "want to inquire about their billing details."

[0128] Step 5:

[0129] The chatbot server sends an SMS verification code to the user's registered phone number to verify their identity and generates a message for the verification process. It then sends the message, "We are verifying your identity. An SMS code has been sent to your registered phone number," to the user's device via the LINE server.

[0130] Step 6:

[0131] The user enters the code "123456" received via SMS into the LINE chat and sends it.

[0132] Step 7:

[0133] The device sends the input back to the LINE server. The message is received by the LINE server.

[0134] Step 8:

[0135] The LINE server forwards the entered SMS code to the chatbot server. The chatbot server verifies the code, and if it is correct, completes the identity verification process.

[0136] Step 9:

[0137] The chatbot server queries the database to retrieve the billing details. The database then sends the billing details information back to the chatbot server.

[0138] Step 10:

[0139] The chatbot server generates the message "This month's charges are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage fee 1000 yen" and sends it to the user's terminal via the LINE server.

[0140] (Example 1)

[0141] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0142] In today's business environment, providing prompt and effective customer service is essential. However, many companies still rely on manual processes and inefficient systems, leading to wasted resources and decreased customer satisfaction. Furthermore, while rapid and accurate information provision is crucial, particularly for appointment scheduling and procedural guidance, integrated and automated systems to achieve this are lacking. Additionally, efficient online identity verification processes are inadequate. These challenges need to be addressed.

[0143] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0144] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for determining necessary processing based on the analyzed intent and executing that processing, and means for returning the processing results to the user. This makes it possible to automatically analyze user input and generate an appropriate response.

[0145] Furthermore, the server includes means for acquiring information related to store visit reservations and proposing them to the user, means for receiving the user's preferred reservation date and time and saving the reservation information, and means for confirming and notifying the reservation in a conversational format via a chat server. This enables efficient store visit reservations and their confirmation and notification.

[0146] Furthermore, the server includes means for obtaining necessary document information for procedures and guiding users through them, means for obtaining online procedure information and presenting it to users, and means for receiving user authentication information and verifying their identity. This automates the procedure guidance and identity verification processes, significantly improving user convenience.

[0147] "Means for receiving user input" refers to the interface or protocol that allows a server to receive text data or voice data entered by a user through a terminal.

[0148] A "natural language processing engine" refers to algorithms and technologies that analyze user input and understand its intent and content, and examples include machine learning models and rule-based processing systems.

[0149] "Means for determining necessary processing based on analyzed intent and executing that processing" refers to software and hardware that determine and execute specific actions according to the user's intent analyzed by a natural language processing engine.

[0150] "Means of returning processing results to the user" refers to methods or devices for notifying the user of the results of processing performed by the server, such as a message generation engine or a communication interface.

[0151] "Means of obtaining and proposing information related to store visit reservations to users" refers to systems and methods for obtaining information related to store visit reservations from databases or APIs and presenting that information to users.

[0152] "A means of receiving a user's requested reservation date and time and saving the reservation information" refers to an interface or protocol for receiving a reservation date and time specified by a user and saving it to a database.

[0153] "A means of confirming and notifying reservations interactively via a chat server" refers to chat software and related infrastructure that allows users and servers to exchange information interactively, confirm the final reservation, and notify the user of the result.

[0154] "Means of obtaining and guiding users to the documents required for a procedure" refers to applications or systems that obtain information about the documents required for a procedure from databases or other sources and guide users to that information.

[0155] "Means of obtaining and introducing online procedural information to users" refers to methods and systems for obtaining information on services and procedures offered online and introducing them to users.

[0156] "Means of receiving user authentication information and verifying identity" refers to systems and methods for receiving authentication information provided by users (e.g., passwords or SMS codes) and using that information to verify the user's identity.

[0157] The system of the present invention is an advanced chatbot for streamlining customer service and can perform various practical procedures based on user input. This system is realized through a series of processes that receive user input, analyze it, and generate an appropriate response. A specific embodiment of this system is described below.

[0158] System Configuration

[0159] This system is broadly composed of the following elements.

[0160] 1. User terminal

[0161] This is a device that users access using the LINE app or a web browser. It has the function of receiving user input and sending it to the server.

[0162] 2. LINE Server

[0163] Its role is to receive user input and forward it to the chatbot server. It also handles protocol conversion and temporary storage of input data.

[0164] 3. Chatbot Server

[0165] This is the core of the system, analyzing user input and generating appropriate responses. Specifically, it incorporates a natural language processing engine to analyze user intent. This engine may include Google® NLP API, among others. It also accesses databases as needed to read and write information.

[0166] 4. Database

[0167] This is an auxiliary system for managing user information, reservation information, and necessary document information for procedures. For example, it uses a PostgreSQL database.

[0168] Program processing

[0169] 1. Receive user input

[0170] For example, a user might type "I want to make a reservation to visit the store" in the chat screen of the LINE app.

[0171] The device sends this input to the LINE server.

[0172] The LINE server forwards the received data to the chatbot server.

[0173] 2. Analyze user intent using a natural language processing engine.

[0174] The chatbot server uses a natural language processing engine to analyze the intent of the input and recognizes that the user "wants to make a reservation to visit the store."

[0175] 3. Executing processing in response to user requests

[0176] For store visit reservations, the chatbot server retrieves available dates and times from the database. For example, it might send an SQL query to a PostgreSQL database to retrieve the data.

[0177] The database returns available dates and times, and the chatbot server suggests possible dates and times to the user.

[0178] 4. Process the user's response.

[0179] The user selects their preferred date and time and replies via chat, for example, "June 12th at 10:00 AM, please."

[0180] The chatbot server then analyzes this again, saves the desired date and time to the database, and confirms the reservation.

[0181] 5. Send a confirmation notice to the user.

[0182] The chatbot server generates the message "Your reservation has been confirmed. Please come to the store on June 12th at 10:00" and sends it to the user's device via the LINE server.

[0183] Specific example

[0184] The following are some specific use cases.

[0185] 1. Information on the documents required for the procedure

[0186] The user's terminal sends the message, "Please tell me what documents are required for the moving procedure."

[0187] The chatbot server uses a natural language processing engine to analyze intent.

[0188] The system queries a database for information on "documents required for moving procedures" and retrieves the results.

[0189] The chatbot server generates a response message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the user's terminal.

[0190] 2. Introduction to online procedures

[0191] A message is sent to the user's terminal saying, "Please tell me what procedures I can do online."

[0192] The chatbot server receives this input and retrieves the appropriate online procedure information from its database.

[0193] The response message reads: "The following procedures can be done online: change of address, review contract details, and review billing statements. You can find more details at this link [link]."

[0194] 3. Inquiry regarding unclear points in the billing statement.

[0195] The user enters, "My bill this month is high, please show me the details."

[0196] The chatbot server requests identity verification and sends an SMS authentication code to the user's device.

[0197] The user enters an SMS code into the chat, and the chatbot server verifies the code.

[0198] The chatbot server retrieves the billing details from the database and sends a response message saying, "Your billing details for this month are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen."

[0199] Example of a prompt

[0200] 1. If the user wishes to make an appointment:

[0201] User: I want to make an appointment to visit the store.

[0202] System: Your appointment request has been received. Available dates and times are as follows: June 10th, 2:00 PM and June 12th, 10:00 AM. Please select your preferred date and time.

[0203] User: Please make it June 12th at 10:00.

[0204] System: Your reservation has been confirmed. Please come to the store on June 12th at 10:00.

[0205] 2. When the user asks about the documents required for the procedure:

[0206] User: What documents are required for moving procedures?

[0207] System: The following documents are required for the moving process: Resident registration certificate, utility bills, and identification.

[0208] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0209] Step 1:

[0210] Receive user input

[0211] The user types "I would like to make a reservation to visit the store" in the chat screen of the LINE app.

[0212] Input: Text data sent by the user via the LINE app.

[0213] The device sends input data to the LINE server via the LINE API. The data is transmitted encrypted.

[0214] The LINE server receives this input data and forwards it to the chatbot server using a specific protocol.

[0215] Output: User input data transferred to the chatbot server.

[0216] Step 2:

[0217] Analyze user intent using a natural language processing engine.

[0218] The chatbot server analyzes the received data.

[0219] Input: User input data transferred to the chatbot server.

[0220] The chatbot server calls a natural language processing engine such as the Google NLP API to analyze the input. Specifically, it extracts the intention "to make a store visit reservation" from the input "I want to make a store visit reservation."

[0221] Output: The analyzed user intent and related information are generated in JSON format.

[0222] Step 3:

[0223] Execute processing in response to user requests.

[0224] The chatbot server processes user requests based on the analysis results.

[0225] Input: Analyzed user intent and related information.

[0226] For example, in the case of a store visit reservation, the chatbot server connects to the database and retrieves information on available dates and times.

[0227] The chatbot server generates SQL queries and sends them to a database (e.g., PostgreSQL) to retrieve data.

[0228] The database searches for available date and time slots and sends the results back to the chatbot server.

[0229] Output: Retrieved reservation availability information.

[0230] Step 4:

[0231] Suggest possible dates and times to the user.

[0232] The chatbot server generates a message to notify the user of the available reservation date and time information it has retrieved.

[0233] Input: Retrieved information on available reservation dates and times.

[0234] The chatbot server generates a message and sends it to the LINE server.

[0235] The LINE server delivers notification messages to the user's device.

[0236] Output: The message reads: "The following dates and times are available for booking: June 10th at 2:00 PM, June 12th at 10:00 AM. Please choose your preferred date and time."

[0237] Step 5:

[0238] Processing user responses

[0239] The user selects their preferred date and time and replies, "Please make it June 12th at 10:00."

[0240] Input: The user's preferred date and time.

[0241] The user's terminal sends this response back to the LINE server.

[0242] The LINE server forwards the received response to the chatbot server.

[0243] The chatbot server then uses its natural language processing engine again to analyze the response and extract the user's preferred date and time.

[0244] Input: A response message containing the user's preferred date and time.

[0245] The chatbot server generates an SQL query to save the desired date and time to the database and sends it to the database.

[0246] The database stores the reservation information and sends the results back to the chatbot server.

[0247] Output: The desired date and time have been saved in the database.

[0248] Step 6:

[0249] Send a confirmation notice to the user.

[0250] The chatbot server generates the message, "Your reservation has been confirmed. Please come to the store on June 12th at 10:00."

[0251] Input: Desired date and time information stored in the database.

[0252] The chatbot server sends that message to the LINE server.

[0253] The LINE server delivers the message to the user's device.

[0254] The user receives a confirmation message.

[0255] Output: Message: "Your reservation has been confirmed. Please come to the store on June 12th at 10:00."

[0256] (Application Example 1)

[0257] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0258] To improve the work efficiency of factory workers, there is a need for a system that can provide work instructions and solve problems in real time. In particular, intuitive voice-based operation and rapid information delivery are required. However, conventional systems have difficulty analyzing voice input and making real-time decisions, which hinders efforts to improve work efficiency.

[0259] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0260] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for determining necessary processing based on the analyzed intent and executing that processing, means for returning the processing results to the user, means for analyzing voice input and converting the voice to text, and means for retrieving information from a database based on the analyzed intent and providing work instructions. This makes it possible for factory workers to obtain accurate instructions and information in real time by using voice input.

[0261] "Means for receiving user input" refers to devices or software used to incorporate information and commands provided by the user into the system.

[0262] A "natural language processing engine" is an algorithm and program that analyzes input text to understand its meaning and intent.

[0263] "Means for determining necessary processing based on analyzed intent" refers to the process by which a natural language processing engine selects the next action to take based on the results of its analysis.

[0264] "The means to carry out that process" refers to the system or device used to actually put the decided action into action.

[0265] "Means for returning processing results to the user" refer to communication means or display devices for providing feedback to the user regarding the results and information of the actions performed.

[0266] "Means for analyzing voice input and converting speech to text" refers to software or hardware that uses speech recognition technology to convert a user's voice into text information.

[0267] "Methods for retrieving information from a database" refers to the procedures for searching for and retrieving the required information from a database that stores the necessary data.

[0268] "Means for providing work instructions" refers to a mechanism for communicating specific work procedures and instructions to the user based on analysis and acquired information.

[0269] "Means of acquiring and proposing information related to store visit reservations to users" refers to the process of acquiring information such as available reservation dates, times, and locations, and presenting that information to the user.

[0270] "Means for saving reservation information" refers to a data management function that records reservation data selected by the user and allows it to be retrieved or updated as needed.

[0271] "Means of providing relevant business processes" refers to a system that provides the instructions and procedures necessary to perform specific tasks or procedures within a factory.

[0272] "Means of obtaining necessary document information for a procedure and informing the user" refers to a method of obtaining the documents and information necessary to perform a specific procedure and notifying the user of them.

[0273] "Means for obtaining and introducing online-procedure information to users" refers to the function of collecting information on procedures executable via the Internet and introducing it to users.

[0274] The system of the present invention is an advanced chatbot system for improving the work efficiency of workers in a factory, and can give practical work instructions and solve problems based on the voice input of users. The overall configuration of this system is as follows.

[0275] Overall configuration of the system

[0276] The system of the present invention is composed of the following elements.

[0277] 1. User terminal

[0278] It is a device accessed by workers using the microphone of smart glasses, a head-mounted display, or a PC.

[0279] 2. Server

[0280] It is a central processing unit for performing speech recognition and natural language processing.

[0281] It incorporates a natural language processing engine (e.g., Google Cloud NLP), converts voice input into text, and analyzes the user's intention.

[0282] 3. Database

[0283] It is a system for managing information on work instructions, assembly procedures, and work procedures.

[0284] Flow of processing

[0285] <b 1. Reception of voice input

[0286] The user terminal receives voice input (e.g., "Tell me the assembly procedure of the next product") when the operator speaks towards smart glasses or a head-mounted display.

[0287] 2. Voice Recognition and Text Conversion

[0288] The terminal sends the received voice to the server and converts the voice into text using voice recognition technology (e.g., SpeechRecognition library).

[0289] 3. Intent Analysis and Data Acquisition

[0290] The server analyzes the converted text with a natural language processing engine and queries the database for work instructions and necessary information.

[0291] 4. Provision of Work Instructions

[0292] The server organizes the information obtained from the database and provides work instructions to the user in text or voice.

[0293] As a specific example, when guiding the assembly procedure of the next product, instructions such as "First step: Attach part A. Second step: Tighten the screw…" are provided.

[0294] Hardware and Software Used

[0295] 1. Hardware

[0296] Devices for the user to access, such as smart glasses, head-mounted displays, and PC microphones.

[0297] Server (central processing unit for performing advanced processing)

[0298] 2. Software

[0299] Voice recognition library (e.g., SpeechRecognition)

[0300] Natural language processing engine (e.g., Google Cloud NLP)

[0301] Database management system (for managing work instructions and procedure information)

[0302] Specific example

[0303] A new worker in the factory uses smart glasses and gives an instruction by voice, saying "Tell me the assembly procedure for the next product". The server analyzes this voice, retrieves the procedure information from the database, displays it on the worker's display, and also guides by voice.

[0304] Example of prompt sentence:

[0305] "Tell me the assembly procedure for the next product"

[0306] <000^{}0965>With this mechanism, workers in the factory can obtain accurate instructions and information in real time by voice input, thus improving work efficiency.

[0307] This invention is a system that highly analyzes voice input and provides necessary information and instructions in real time to improve work efficiency.

[0308] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0309] Step 1:

[0310] The user performs voice input through smart glasses or a head-mounted display.

[0311] As a specific operation, the user speaks "Tell me the assembly procedure for the next product". [[ID=4^{}000982> The input is the user's voice, and the output is voice data.

[0313] Step 2:

[0314] The device that receives the audio data sends that audio data to the server.

[0315] Specifically, the device sends audio data to the server via the network.

[0316] The input is audio data, and the output is data sent to the server.

[0317] Step 3:

[0318] The server converts the received audio data into text using speech recognition software (e.g., the SpeechRecognition library).

[0319] Specifically, the server starts the speech recognition engine and converts the speech data into text data.

[0320] The input is audio data, and the output is converted text data.

[0321] Step 4:

[0322] The server analyzes the converted text data using a natural language processing engine (e.g., Google Cloud NLP) to understand the user's intent.

[0323] In terms of specific operations, the server invokes a natural language processing engine to analyze the text data and extract the intended meaning.

[0324] The input is text data, and the output is parsed intent data.

[0325] Step 5:

[0326] Based on the analyzed intent, the server retrieves relevant information from the database.

[0327] In terms of specific actions, the server queries the database to retrieve the necessary work procedure information.

[0328] The input is intent data, and the output is work instruction information retrieved from the database.

[0329] Step 6:

[0330] The server organizes the acquired work instruction information and generates text or voice messages to respond to the user.

[0331] In terms of specific operations, the server formats the information into a format that is easy to transmit to the user and generates a message.

[0332] The input is work instruction information retrieved from the database, and the output is the generated message.

[0333] Step 7:

[0334] The server sends the generated message to the terminal, and the terminal notifies the user.

[0335] In terms of specific operations, the server sends a message to the terminal over the network, and the terminal displays instructions to the user via voice and text.

[0336] The input is the generated message, and the output is the notification to the user.

[0337] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0338] The system of the present invention is an advanced chatbot that combines intent analysis of input text and emotion recognition to enable smooth communication with users. The system of the present invention has the function of recognizing the user's emotions and responding appropriately according to those emotions.

[0339] Overall system configuration

[0340] The system consists of the following elements:

[0341] 1. User terminal

[0342] This refers to a device that users access using the LINE app or a web browser.

[0343] 2. LINE Server

[0344] Its role is to receive user input and forward it to the chatbot server.

[0345] 3. Chatbot Server

[0346] This is the central part of the system that analyzes user input and generates responses.

[0347] It incorporates a natural language processing engine and an emotion engine to analyze the user's intentions and emotions.

[0348] Access the database and perform read and write operations as needed.

[0349] 4. Database

[0350] It manages user information, reservation information, required documents for procedures, and billing information.

[0351] Processing flow

[0352] 1. Receive user input

[0353] The user types "I would like to make a reservation to visit the store" into the LINE app and sends it.

[0354] 2. Perform natural language processing and sentiment recognition.

[0355] The LINE server forwards the input to the chatbot server.

[0356] The chatbot server uses a natural language processing engine to analyze the intent of the input.

[0357] At the same time, an emotion engine is used to recognize the user's emotions.

[0358] 3. Determine the necessary processing based on the analysis results.

[0359] The chatbot server determines the necessary actions based on the analyzed intent and emotions.

[0360] For example, in the case of a store visit reservation, available dates and times are retrieved from the database.

[0361] 4. Return the processing result to the user.

[0362] The system then suggests available dates and times to the user.

[0363] 5. Process the user's response.

[0364] The user selects their preferred date and time and replies.

[0365] The chatbot server receives the requested date and time and saves it to the database. The reservation is then confirmed.

[0366] 6. Send a reservation confirmation notice to the user.

[0367] Generate a confirmation message and send it to the user.

[0368] Specific example

[0369] The following are examples of specific use cases.

[0370] 1. Make an appointment to visit the store.

[0371] If a user includes anxious expressions when making a reservation (for example, "I'm worried because it's my first time"), the emotion engine recognizes this as "anxiety."

[0372] The chatbot server offers detailed explanations and support options to alleviate anxiety. "Don't worry if this is your first time visiting. We'll send you detailed instructions."

[0373] 2. Information on the documents required for the procedure

[0374] When a user requests procedural information, if a message containing an emotion such as "It's urgent" is sent, the emotion engine recognizes the "urgency."

[0375] The chatbot server quickly provides the necessary document information and responds appropriately with phrases like, "Thank you for your urgency. The required documents are as follows."

[0376] 3. Inquiry regarding unclear points in the billing statement.

[0377] If a user enters "The explanation about the fees is unclear," the sentiment engine recognizes this as "confusion."

[0378] The chatbot server adds a detailed and easy-to-understand explanation: "We apologize for any confusion regarding the billing details. Here is a detailed breakdown."

[0379] Thus, the present invention realizes a chatbot system that provides appropriate responses according to the user's intentions and emotions by combining natural language processing and emotion recognition.

[0380] The following describes the processing flow.

[0381] Specific processing steps for making a store visit reservation

[0382] Step 1:

[0383] The user types "I want to make a reservation to visit the store" in the LINE app's chat screen and sends it.

[0384] Step 2:

[0385] The device sends the user's message to the LINE server. The message is received by the LINE server.

[0386] Step 3:

[0387] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[0388] Step 4:

[0389] The chatbot server uses a natural language processing engine to analyze the intent of messages. Simultaneously, it uses an emotion engine to analyze the user's emotions.

[0390] Step 5:

[0391] Based on the analysis results, the chatbot server recognizes the user's intention to make an appointment and any feelings of anxiety.

[0392] Step 6:

[0393] The chatbot server connects to the database and queries it to retrieve available dates and times for booking. The database then returns the available dates and times.

[0394] Step 7:

[0395] The chatbot server generates a message suggesting possible dates and times to the user, saying, "Please let us know your preferred date and time. Suggestions: June 12th, 10:00 AM, June 12th, 2:00 PM. We also offer support to ensure a smooth experience even for first-time visitors." This message is then sent to the user's device via the LINE server.

[0396] Step 8:

[0397] The user replies, "Please make it June 12th at 10:00."

[0398] Step 9:

[0399] The device sends the user's reply to the LINE server. The LINE server receives the message and forwards it to the chatbot server.

[0400] Step 10:

[0401] The chatbot server saves the date and time of the received message to its database and updates the reservation information.

[0402] Step 11:

[0403] The database stores the reservation information. The chatbot server generates a reservation confirmation message and sends it to the user's device via the LINE server, stating, "Your reservation has been confirmed. Please come to our store on June 12th at 10:00. Please feel free to contact us if you have any questions."

[0404] Specific processing steps for the documents required for the procedure

[0405] Step 1:

[0406] The user types "Please tell me what documents are required for moving procedures" and submits the form.

[0407] Step 2:

[0408] The device sends the input to the LINE server. The message is received by the LINE server.

[0409] Step 3:

[0410] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[0411] Step 4:

[0412] The chatbot server uses a natural language processing engine to analyze the intent of messages. Simultaneously, it uses an emotion engine to analyze the user's emotions.

[0413] Step 5:

[0414] Based on the analysis results, the chatbot server recognizes the user's intention to initiate the moving process and their sense of urgency.

[0415] Step 6:

[0416] The chatbot server connects to the database and queries it to retrieve the necessary document information for the procedure. The database then returns the required document information.

[0417] Step 7:

[0418] The chatbot server generates a message informing the user of the necessary documents and sends it to the device via the LINE server, stating, "Thank you for your urgency. The following documents are required for the moving process: Resident registration certificate, utility bills, and identification."

[0419] Specific processing steps for inquiries regarding unclear points in billing statements

[0420] Step 1:

[0421] The user types, "This month's bill is high, please show me the details," and submits it.

[0422] Step 2:

[0423] The device sends the input to the LINE server. The message is received by the LINE server.

[0424] Step 3:

[0425] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[0426] Step 4:

[0427] The chatbot server uses a natural language processing engine to analyze intent and recognize that the user "wants to inquire about their billing details." At the same time, it uses an emotion engine to recognize the user's "confusion."

[0428] Step 5:

[0429] The chatbot server sends an SMS verification code to the user's registered phone number to verify their identity. It generates a message for the verification process and sends it to the device via the LINE server, stating, "We will now verify your identity. An SMS code has been sent to your registered phone number."

[0430] Step 6:

[0431] The user enters the code "123456" received via SMS into the LINE chat and sends it.

[0432] Step 7:

[0433] The device sends the input back to the LINE server. The message is received by the LINE server.

[0434] Step 8:

[0435] The LINE server forwards the entered SMS code to the chatbot server. The chatbot server verifies the code, and if it is correct, completes the identity verification process.

[0436] Step 9:

[0437] The chatbot server queries the database to retrieve the billing details. The database then returns the billing details information.

[0438] Step 10:

[0439] The chatbot server generates a message saying, "This month's billing details are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen," adds a clear explanation, and sends it to the user's terminal via the LINE server, saying, "We apologize for any confusion regarding the billing details. Here is a detailed breakdown."

[0440] As described above, the system of the present invention recognizes the user's intentions and emotions and provides an appropriate response.

[0441] (Example 2)

[0442] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0443] Conventional chatbot systems are limited to analyzing the intent behind user input, making it difficult to respond in a way that takes user emotions into account. As a result, responses to users are uniform, and communication is often not smooth. Furthermore, there are delays in providing information about specific processes and responding to users' anxieties and urgency, which leads to decreased user satisfaction. This invention aims to solve these problems.

[0444] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0445] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for recognizing the user's emotions using an emotion engine, means for determining and executing necessary processing based on the analyzed intent and emotions, and means for returning the processing results to the user. This enables an appropriate response that takes into account both the user's intent and emotions.

[0446] "Means for receiving user input" refers to the function of obtaining text messages and commands sent from the terminal used by the user.

[0447] A "natural language processing engine" is a software module that analyzes text entered by a user and understands its intent and meaning.

[0448] An "emotion engine" is a software module that identifies emotions and psychological states from user input and determines appropriate responses based on that information.

[0449] "Means for determining necessary processing based on analyzed intent and emotion" refers to a function that selects the next action or response based on the analysis results of the natural language processing engine and the emotion engine.

[0450] "Means of responding to the user with processing results" refers to a function that generates the determined response or action as a text message and sends it to the user.

[0451] "A means of obtaining and proposing information related to store visit reservations to the user" refers to a function that, when a user requests a store visit reservation, retrieves available dates and times and related information from a database and presents it to the user.

[0452] "A means of receiving a user's requested reservation date and time and saving the reservation information" refers to a function that receives the reservation date and time selected by the user and saves it in a database.

[0453] "Means of notifying users of confirmed reservation information" refers to a function that generates and sends a message to inform users of a confirmed reservation.

[0454] "A means of obtaining necessary document information for procedures and guiding users through it" refers to a function that retrieves the necessary documents and information for various procedures from a database and guides users through them appropriately.

[0455] "A means of determining processing priorities based on user emotions" refers to a function that determines the order and urgency of responses according to the user's psychological state, as analyzed by the emotion engine.

[0456] "A means of obtaining and introducing online procedural information to users" refers to a function that searches for and obtains information on procedures and services that users can perform online, and provides that information to users.

[0457] Modes for carrying out the invention

[0458] The system of the present invention is an advanced chatbot that combines intent analysis of input text and sentiment recognition to enable smooth communication with users. The specific configuration and operation of the system of the present invention are described below.

[0459] Overall system configuration

[0460] The system consists of the following elements:

[0461] 1. User terminal

[0462] These are devices such as smartphones and PCs, and users access them using the LINE app or a web browser.

[0463] 2. LINE Server

[0464] Its role is to receive user input and forward it to the chatbot server.

[0465] 3. Chatbot Server

[0466] This is the central part of the system that analyzes user input and generates responses.

[0467] It incorporates a natural language processing engine (e.g., Google Cloud Natural Language API) and an emotion engine (e.g., Microsoft® Azure® Text Analytics API) to analyze user intent and emotion.

[0468] Access and read / write data to a database (e.g., MySQL®) as needed.

[0469] 4. Database

[0470] It manages user information, reservation information, required documents for procedures, and billing information.

[0471] Program processing

[0472] 1. Receive user input

[0473] The user types "I want to make a reservation to visit the store" into the LINE app and sends it. The user's device sends this message to the LINE server.

[0474] 2. Perform natural language processing and sentiment recognition.

[0475] The server receives user input via the LINE server and forwards it to the chatbot server in JSON format. The chatbot server calls the Google Cloud Natural Language API to analyze the intent of the message. Simultaneously, it uses the Microsoft Azure Text Analytics API to recognize the user's sentiment.

[0476] 3. Determine the necessary processing based on the analysis results.

[0477] The chatbot server determines the next action to take based on the analysis results. For example, in the case of a store visit reservation, it generates a query to retrieve available dates and times from the database.

[0478] 4. Return the processing result to the user.

[0479] The chatbot server proposes available dates and times to the user. The message is generated in a specific format and sent to the user via the LINE server.

[0480] 5. Process the user's response.

[0481] The user selects their preferred date and time and replies. The chatbot server receives this selection and executes an SQL query to save it to the database.

[0482] 6. Send a reservation confirmation notice to the user.

[0483] The chatbot server generates a reservation confirmation message and sends it to the user: "Your reservation is confirmed. We look forward to seeing you on [Month] [Day] at [Time]."

[0484] Specific example

[0485] The following are examples of specific use cases.

[0486] 1. Make an appointment to visit the store.

[0487] If a user includes an anxious expression when making a reservation (for example, "I'm worried because it's my first time"), the emotion engine recognizes this anxiety. The chatbot server then offers detailed explanations and support options to alleviate the anxiety. For example, "Don't worry if it's your first time. We'll send you detailed instructions."

[0488] 2. Information on the documents required for the procedure

[0489] When a user requests procedural information and sends a message that includes an emotion such as "It's urgent," the emotion engine recognizes the "urgency." The chatbot server quickly provides the necessary document information and gives an appropriate response such as, "Thank you for your urgency. The required documents are as follows."

[0490] 3. Inquiry regarding unclear points in the billing statement.

[0491] If a user types "The explanation about the charges is unclear," the sentiment engine recognizes this as "confusion." The chatbot server then adds a more detailed and clearer explanation: "We apologize for the confusion regarding the charges. Here is a detailed breakdown."

[0492] Thus, the present invention realizes a chatbot system that provides appropriate responses according to the user's intentions and emotions by combining natural language processing and emotion recognition.

[0493] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0494] Step 1:

[0495] Receive user input

[0496] explanation:

[0497] The user types "I would like to make a reservation to visit the store" into the LINE app and sends it.

[0498] The device will send this message to the LINE server.

[0499] Specific actions:

[0500] The user types the message "I would like to make a reservation to visit" on the LINE app on their smartphone and presses the send button.

[0501] Input: User's text message

[0502] Output: Message forwarded to LINE server

[0503] Step 2:

[0504] Performing natural language processing and emotion recognition.

[0505] explanation:

[0506] The server forwards messages received from the LINE server to the chatbot server.

[0507] The chatbot server analyzes the intent of messages using a natural language processing engine (e.g., Google Cloud Natural Language API).

[0508] Use an emotion engine (e.g., Microsoft Azure Text Analytics API) to recognize the user's emotions.

[0509] Specific actions:

[0510] The server receives the user's message via the LINE server and forwards it to the chatbot server in JSON format.

[0511] The chatbot server sends a request to the Google Cloud Natural Language API to analyze the intent behind the "store visit reservation."

[0512] Simultaneously, a request is sent to the Microsoft Azure Text Analytics API to analyze the emotions (e.g., anxiety) contained in the message.

[0513] Input: Message forwarded from LINE server

[0514] Output: Analyzed intentions and emotions

[0515] Step 3:

[0516] Based on the analysis results, determine the necessary processing steps.

[0517] explanation:

[0518] Based on the analysis results, the chatbot server determines the next action to take.

[0519] For example, in the case of a store visit reservation, available dates and times are retrieved from a database (e.g., MySQL).

[0520] Specific actions:

[0521] The chatbot server generates database queries based on the analysis results.

[0522] The server queries the MySQL database to retrieve available dates and times for booking.

[0523] Input: Analyzed intentions and emotions

[0524] Output: Database query results including available dates and times for booking

[0525] Step 4:

[0526] Return the processing result to the user.

[0527] explanation:

[0528] The chatbot server then suggests available dates and times to the user.

[0529] The generated text message is sent to the LINE server as a suggestion.

[0530] Specific actions:

[0531] The chatbot server generates a suggestion message in a specific format and sends that message to the LINE server.

[0532] The LINE server forwards the message to the user's device.

[0533] Input: Database query results including available dates and times for booking

[0534] Output: Suggestion message sent to the user

[0535] Step 5:

[0536] Processing user responses

[0537] explanation:

[0538] The user selects their preferred date and time and replies.

[0539] The chatbot server receives the reply and saves the desired date and time to its database.

[0540] Specific actions:

[0541] The user sends a message saying, "I would like to meet on [Month] [Day] at [Time]."

[0542] The chatbot server receives this message and executes an SQL query to save the desired date and time to the database.

[0543] Input: User's reply message

[0544] Output: Reservation information stored in the database

[0545] Step 6:

[0546] Send a reservation confirmation notification to the user.

[0547] explanation:

[0548] The chatbot server generates and sends a message to the user notifying them that the reservation has been confirmed.

[0549] Specific actions:

[0550] The chatbot server generates the message, "Your reservation has been confirmed. We look forward to seeing you on [Month] [Day] at [Time]."

[0551] The message is sent to the user's device via the LINE server.

[0552] Input: Reservation information stored in the database

[0553] Output: Booking confirmation message sent to the user

[0554] (Application Example 2)

[0555] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0556] Traditional chatbot systems are limited to analyzing user intent and have the problem of not being able to provide appropriate responses that take into account user emotions. This degrades the user experience, and in particular, in ride reservation systems, support may be insufficient when passengers are feeling anxious or stressed. There is a need for a system that can smoothly respond to passengers when they need changes or urgent assistance.

[0557] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0558] In this invention, the server includes means for receiving user input, means for analyzing the intent and emotion of the input using a natural language processing engine and an emotion recognition system, means for determining and executing necessary processing based on the analyzed intent and emotion, and means for returning the processing results to the user. This enables appropriate responses that address not only the user's intent but also their emotions, allowing for quick and appropriate responses to requests such as ride reservations and route changes.

[0559] "User" refers to an individual or group that uses the system.

[0560] "Input" refers to the information that a user sends to the system.

[0561] A "natural language processing engine" is a computer program that analyzes user input to understand their intent.

[0562] An "emotion recognition system" is a computer program that analyzes emotions from user input.

[0563] "Intention" refers to the purpose or request that the user is trying to convey through their input.

[0564] "Emotions" refer to the psychological state that a user expresses through their input.

[0565] "Processing" refers to the actions the system takes based on the analyzed intentions and emotions.

[0566] "Response" refers to the answer that the system provides to the user.

[0567] "Ride reservation" refers to the act of a user reserving a vehicle for a specific date and time.

[0568] "Route change" refers to a request from a user to change their existing travel route.

[0569] A "support center" refers to an organization that provides assistance to users when they encounter problems or have questions.

[0570] "Dialogue" refers to the communication that takes place between a system and a user.

[0571] "Mobility experience" refers to the overall experience a user has when using an autonomous vehicle.

[0572] The system for implementing this invention is an advanced chatbot system aimed at improving the user's riding experience. This system consists of the following main components:

[0573] Overall system configuration

[0574] The system consists of the following four elements:

[0575] 1. User terminal

[0576] A user terminal refers to a mobile device such as a smartphone, which users use to access the system through applications. Users can make reservations and change routes through text input.

[0577] 2. Communication Server

[0578] This server is responsible for receiving user input in real time and forwarding it to the chatbot server. The communication server is crucial for ensuring stable network communication.

[0579] 3. Chatbot Server

[0580] The chatbot server is the heart of the system, handling all data processing and response generation. It includes the following software components:

[0581] Natural language processing engine (e.g., TextBlob): Analyzes user input text and understands intent.

[0582] Emotion recognition systems (e.g., emotion analysis using TextBlob): Analyze emotions from user input and adjust responses accordingly.

[0583] 4. Database Server

[0584] The database server manages the following information:

[0585] User Information

[0586] Reservation Information

[0587] Route information

[0588] Past conversation history

[0589] Explanation of the process

[0590] The system processes the data as follows:

[0591] Receiving user input

[0592] Text input is sent from the user's terminal and reaches the chatbot server via the communication server.

[0593] Natural language processing and sentiment analysis

[0594] The chatbot server uses a natural language processing engine to analyze the user's intent and an emotion recognition system to evaluate their emotions. For example, if a user types "I want to make a reservation," the intent is analyzed as "reservation" and the emotion as "positive."

[0595] Decision and execution of processing

[0596] Based on the analyzed intent and emotions, the chatbot server determines the appropriate action. For example, if a reservation request is made, it retrieves available dates and times from the database and generates suggestions.

[0597] Response to the user

[0598] The system returns the processing results to the user. When suggesting available dates and times, if the user expresses concerns, the system will respond in an empathetic manner, such as, "It seems you have concerns about your preferred date and time. Please feel free to contact us."

[0599] Examples of specific user interactions

[0600] The following is an example of a specific interaction between the user and the system:

[0601] User input: "I would like to make a reservation."

[0602] System response: "The following time slots are available for booking: 2023-11-01 10:00, 2023-11-01 14:00, 2023-11-01 18:00"

[0603] The system constantly analyzes user emotions and generates appropriate responses, allowing users to use the system with peace of mind.

[0604] Software and hardware used

[0605] Software: TextBlob (natural language processing and sentiment analysis), requests module (HTTP requests)

[0606] Hardware: Smartphones (user terminals), servers (communication servers and chatbot servers)

[0607] These components allow users to book and use autonomous vehicles intuitively and with confidence.

[0608] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0609] Step 1:

[0610] Receiving user input

[0611] The user terminal receives input (text messages) from the user. This input includes requests such as reservations and route changes. The received input is sent to the communication server.

[0612] Input: The user enters a message such as "I would like to make a reservation."

[0613] Processing: The user terminal sends input to the communication server.

[0614] Output: Communication server receives input message

[0615] Step 2:

[0616] Intent analysis using natural language processing

[0617] The communication server forwards the received user input to the chatbot server. The chatbot server uses a natural language processing engine to analyze the intent of the user's input.

[0618] Input: Input message forwarded from the communication server

[0619] Processing: A natural language processing engine analyzes the intent of the input (e.g., "request for reservation").

[0620] Output: Analyzed intent (e.g., "reservation")

[0621] Step 3:

[0622] Emotion analysis using an emotion recognition system

[0623] The chatbot server uses an emotion recognition system to analyze the emotions from the user's input.

[0624] Input: Input message forwarded from the communication server

[0625] Processing: The emotion recognition system analyzes the emotion of the input (e.g., "positive").

[0626] Output: Analyzed emotion (e.g., "positive")

[0627] Step 4:

[0628] Decision and execution of processing

[0629] The chatbot server determines the necessary actions based on the analyzed intent and emotion. For example, in the case of a reservation request, it retrieves available dates and times from the database.

[0630] Input: Analyzed intent and emotion (e.g., "reservation", "positive")

[0631] Process: Retrieve available date and time information from the database.

[0632] Output: Acquired reservation date and time information (e.g., "2023-11-01 10:00")

[0633] Step 5:

[0634] Generating responses to users

[0635] The chatbot server generates a response to the user based on the processing results. This response should include consideration for the user's emotions.

[0636] Input: Acquired reservation date and time information and analyzed sentiment (e.g., "2023-11-01 10:00", "Positive")

[0637] Process: Generate a response (Example: "The available time slots are as follows: 2023-11-01 10:00")

[0638] Output: Reply message to the user (Example: "The available time slots are as follows: 2023-11-01 10:00")

[0639] Step 6:

[0640] Sending a reply to the user

[0641] The chatbot server sends the generated response message to the user's terminal via the communication server. The user can then review the response and take the next action.

[0642] Input: Generated response message (Example: "The following time slots are available for booking: 2023-11-01 10:00")

[0643] Processing: Send a reply message to the user terminal via the communication server.

[0644] Output: Reply message displayed on the user's terminal

[0645] The above outlines the specific processing flow of the chatbot in the autonomous vehicle reservation system. By performing appropriate data processing and calculations at each step, it is possible to provide users with quick and appropriate responses.

[0646] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0647] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0648] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0649] [Second Embodiment]

[0650] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0651] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0652] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0653] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0654] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0655] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0656] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0657] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0658] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0659] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0660] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0661] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0662] The system of the present invention is an advanced chatbot for streamlining customer service and can perform various practical procedures based on user input. The system of the present invention is realized through a series of processes that receive user input, analyze it, and generate an appropriate response.

[0663] Overall system configuration

[0664] The system is broadly composed of the following elements:

[0665] 1. User terminal

[0666] This refers to a device that users access using the LINE app or a web browser.

[0667] 2. LINE Server

[0668] Its role is to receive user input and forward it to the chatbot server.

[0669] 3. Chatbot Server

[0670] It is the central part of the system that analyzes user input and generates appropriate responses.

[0671] It has a built-in natural language processing engine that analyzes the user's intent.

[0672] Access, read from, and write to the database as needed.

[0673] 4. Database

[0674] This is a supplementary system for managing user information, reservation information, and necessary document information for procedures.

[0675] Processing flow

[0676] 1. Receive user input

[0677] For example, a user might type "I want to make a reservation to visit the store" in the chat screen of the LINE app.

[0678] The device sends this input to the LINE server.

[0679] 2. Analyze user intent using a natural language processing engine.

[0680] The LINE server receives the input and forwards it to the chatbot server.

[0681] The chatbot server uses a natural language processing engine to analyze the intent of the input and recognizes that the user "wants to make a reservation to visit the store."

[0682] 3. Executing processing in response to user requests

[0683] For store visit reservations, the chatbot server retrieves available dates and times from the database.

[0684] The database returns available dates and times, and the chatbot server suggests possible dates and times to the user.

[0685] 4. Process the user's response.

[0686] The user selects their preferred date and time and replies in the chat, "Please make it June 12th at 10:00."

[0687] The chatbot server then analyzes this again, saves the desired date and time to the database, and confirms the reservation.

[0688] 5. Send a confirmation notice to the user.

[0689] The chatbot server generates the message "Your reservation has been confirmed. Please come to the store on June 12th at 10:00" and sends it to the user's device via the LINE server.

[0690] Specific example

[0691] The following are some specific use cases.

[0692] 1. Information on the documents required for the procedure

[0693] The user's terminal sends the message, "Please tell me what documents are required for the relocation procedure."

[0694] The chatbot server uses a natural language processing engine to analyze intent.

[0695] The system queries a database for information on "documents required for moving procedures" and retrieves the results.

[0696] The chatbot server generates a response message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the user's terminal.

[0697] 2. Introduction to online procedures

[0698] A message is sent to the user's terminal saying, "Please tell me what procedures I can do online."

[0699] The chatbot server receives this input and retrieves the appropriate online procedure information from its database.

[0700] The response message reads: "The following procedures can be done online: change of address, review contract details, and review billing statements. You can find more details at this link [link]."

[0701] 3. Inquiry regarding unclear points in the billing statement.

[0702] The user enters, "My bill this month is high, please show me the details."

[0703] The chatbot server requests identity verification and sends an SMS authentication code to the user's device.

[0704] The user enters an SMS code into the chat, and the chatbot server verifies the code.

[0705] The chatbot server retrieves the billing details from the database and sends a response message saying, "Your billing details for this month are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen."

[0706] In this way, the present invention realizes a system that provides a quick and effective response to the diverse demands of users.

[0707] The following describes the processing flow.

[0708] Step 1:

[0709] The user types "I want to make a reservation to visit" in the LINE app's chat screen. The user then sends this message.

[0710] Step 2:

[0711] The device sends user input to the LINE server. The message is received by the LINE server.

[0712] Step 3:

[0713] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[0714] Step 4:

[0715] The chatbot server uses a natural language processing engine to analyze the intent of incoming messages. The natural language processing engine identifies the intent as "I would like to make a reservation to visit the store."

[0716] Step 5:

[0717] The chatbot server queries the database for available dates and times and retrieves them. The database then sends the available dates and times back to the chatbot server.

[0718] Step 6:

[0719] The chatbot server generates a message for the user saying, "Please tell us your preferred date and time. Suggestions: June 12th, 10:00 AM, June 12th, 2:00 PM," and sends it to the user's device via the LINE server.

[0720] Step 7:

[0721] The user enters their desired date and time, such as "June 12th, 10:00 AM, please," on their device and sends it to the LINE server.

[0722] Step 8:

[0723] The LINE server receives input from the user and forwards it to the chatbot server. The chatbot server receives the message for the requested date and time.

[0724] Step 9:

[0725] The chatbot server saves the date and time of the received message to the database and updates the reservation information. The database stores the reservation information.

[0726] Step 10:

[0727] The chatbot server generates a reservation confirmation message saying, "Your reservation has been confirmed. Please come to the store on June 12th at 10:00," and sends it to the user's device via the LINE server.

[0728] The following are example steps for obtaining the necessary documents and for inquiring about any unclear points regarding the fee breakdown.

[0729] Instructions on the documents required for the procedure

[0730] Step 1:

[0731] The user uses their device to type "Please tell me what documents are required for moving procedures" and then sends the message.

[0732] Step 2:

[0733] The device sends input to the LINE server. The message is received by the LINE server.

[0734] Step 3:

[0735] The LINE server forwards the incoming message to the chatbot server. The chatbot server receives the message.

[0736] Step 4:

[0737] The chatbot server uses a natural language processing engine to analyze the intent of the input. The natural language processing engine identifies the intent as "I want to know what documents are needed for moving procedures."

[0738] Step 5:

[0739] The chatbot server queries the database for relevant document information and retrieves it. The database then sends the relevant information back to the chatbot server.

[0740] Step 6:

[0741] Based on the information it has acquired, the chatbot server generates a message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the device via the LINE server.

[0742] Inquiry regarding unclear points in the billing statement

[0743] Step 1:

[0744] The user types, "My bill this month is high, please show me the details," and sends it via the LINE app.

[0745] Step 2:

[0746] The device sends the input to the LINE server. The message is received by the LINE server.

[0747] Step 3:

[0748] The LINE server forwards the incoming message to the chatbot server. The chatbot server receives the message.

[0749] Step 4:

[0750] The chatbot server uses a natural language processing engine to analyze the user's intent and recognizes that they "want to inquire about their billing details."

[0751] Step 5:

[0752] The chatbot server sends an SMS verification code to the user's registered phone number to verify their identity and generates a message for the verification process. It then sends the message, "We are verifying your identity. An SMS code has been sent to your registered phone number," to the user's device via the LINE server.

[0753] Step 6:

[0754] The user enters the code "123456" received via SMS into the LINE chat and sends it.

[0755] Step 7:

[0756] The device sends the input back to the LINE server. The message is received by the LINE server.

[0757] Step 8:

[0758] The LINE server forwards the entered SMS code to the chatbot server. The chatbot server verifies the code, and if it is correct, completes the identity verification process.

[0759] Step 9:

[0760] The chatbot server queries the database to retrieve the billing details. The database then sends the billing details information back to the chatbot server.

[0761] Step 10:

[0762] The chatbot server generates the message "This month's charges are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage fee 1000 yen" and sends it to the user's terminal via the LINE server.

[0763] (Example 1)

[0764] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0765] In today's business environment, providing prompt and effective customer service is essential. However, many companies still rely on manual processes and inefficient systems, leading to wasted resources and decreased customer satisfaction. Furthermore, while rapid and accurate information provision is crucial, particularly for appointment scheduling and procedural guidance, integrated and automated systems to achieve this are lacking. Additionally, efficient online identity verification processes are inadequate. These challenges need to be addressed.

[0766] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0767] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for determining necessary processing based on the analyzed intent and executing that processing, and means for returning the processing results to the user. This makes it possible to automatically analyze user input and generate an appropriate response.

[0768] Furthermore, the server includes means for acquiring information related to store visit reservations and proposing them to the user, means for receiving the user's preferred reservation date and time and saving the reservation information, and means for confirming and notifying the reservation in a conversational format via a chat server. This enables efficient store visit reservations and their confirmation and notification.

[0769] Furthermore, the server includes means for obtaining necessary document information for procedures and guiding users through them, means for obtaining online procedure information and presenting it to users, and means for receiving user authentication information and verifying their identity. This automates the procedure guidance and identity verification processes, significantly improving user convenience.

[0770] "Means for receiving user input" refers to the interface or protocol that allows a server to receive text data or voice data entered by a user through a terminal.

[0771] A "natural language processing engine" refers to algorithms and technologies that analyze user input and understand its intent and content, and examples include machine learning models and rule-based processing systems.

[0772] "Means for determining necessary processing based on analyzed intent and executing that processing" refers to software and hardware that determine and execute specific actions according to the user's intent analyzed by a natural language processing engine.

[0773] "Means of returning processing results to the user" refers to methods or devices for notifying the user of the results of processing performed by the server, such as a message generation engine or a communication interface.

[0774] "Means of obtaining and proposing information related to store visit reservations to users" refers to systems and methods for obtaining information related to store visit reservations from databases or APIs and presenting that information to users.

[0775] "A means of receiving a user's requested reservation date and time and saving the reservation information" refers to an interface or protocol for receiving a reservation date and time specified by a user and saving it to a database.

[0776] "A means of confirming and notifying reservations interactively via a chat server" refers to chat software and related infrastructure that allows users and servers to exchange information interactively, confirm the final reservation, and notify the user of the result.

[0777] "Means of obtaining and guiding users to the documents required for a procedure" refers to applications or systems that obtain information about the documents required for a procedure from databases or other sources and guide users to that information.

[0778] "Means of obtaining and introducing online procedural information to users" refers to methods and systems for obtaining information on services and procedures offered online and introducing them to users.

[0779] "Means of receiving user authentication information and verifying identity" refers to systems and methods for receiving authentication information provided by users (e.g., passwords or SMS codes) and using that information to verify the user's identity.

[0780] The system of the present invention is an advanced chatbot for streamlining customer service and can perform various practical procedures based on user input. This system is realized through a series of processes that receive user input, analyze it, and generate an appropriate response. A specific embodiment of this system is described below.

[0781] System Configuration

[0782] This system is broadly composed of the following elements.

[0783] 1. User terminal

[0784] This is a device that users access using the LINE app or a web browser. It has the function of receiving user input and sending it to the server.

[0785] 2. LINE Server

[0786] Its role is to receive user input and forward it to the chatbot server. It also handles protocol conversion and temporary storage of input data.

[0787] 3. Chatbot Server

[0788] This is the core of the system, analyzing user input and generating appropriate responses. Specifically, it incorporates a natural language processing engine to analyze user intent. It utilizes APIs such as the Google NLP API as its natural language processing engine. It also accesses databases as needed to read and write information.

[0789] 4. Database

[0790] This is an auxiliary system for managing user information, reservation information, and necessary document information for procedures. For example, it uses a PostgreSQL database.

[0791] Program processing

[0792] 1. Receive user input

[0793] For example, a user might type "I want to make a reservation to visit the store" in the chat screen of the LINE app.

[0794] The device sends this input to the LINE server.

[0795] The LINE server forwards the received data to the chatbot server.

[0796] 2. Analyze user intent using a natural language processing engine.

[0797] The chatbot server uses a natural language processing engine to analyze the intent of the input and recognizes that the user "wants to make a reservation to visit the store."

[0798] 3. Executing processing in response to user requests

[0799] For store visit reservations, the chatbot server retrieves available dates and times from the database. For example, it might send an SQL query to a PostgreSQL database to retrieve the data.

[0800] The database returns available dates and times, and the chatbot server suggests possible dates and times to the user.

[0801] 4. Process the user's response.

[0802] The user selects their preferred date and time and replies via chat, for example, "June 12th at 10:00 AM, please."

[0803] The chatbot server then analyzes this again, saves the desired date and time to the database, and confirms the reservation.

[0804] 5. Send a confirmation notice to the user.

[0805] The chatbot server generates the message "Your reservation has been confirmed. Please come to the store on June 12th at 10:00" and sends it to the user's device via the LINE server.

[0806] Specific example

[0807] The following are some specific use cases.

[0808] 1. Information on the documents required for the procedure

[0809] The user's terminal sends the message, "Please tell me what documents are required for the relocation procedure."

[0810] The chatbot server uses a natural language processing engine to analyze intent.

[0811] The system queries a database for information on "documents required for moving procedures" and retrieves the results.

[0812] The chatbot server generates a response message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the user's terminal.

[0813] 2. Introduction to online procedures

[0814] A message is sent to the user's terminal saying, "Please tell me what procedures I can do online."

[0815] The chatbot server receives this input and retrieves the appropriate online procedure information from its database.

[0816] The response message reads: "The following procedures can be done online: change of address, review contract details, and review billing statements. You can find more details at this link [link]."

[0817] 3. Inquiry regarding unclear points in the billing statement.

[0818] The user enters, "My bill this month is high, please show me the details."

[0819] The chatbot server requests identity verification and sends an SMS authentication code to the user's device.

[0820] The user enters an SMS code into the chat, and the chatbot server verifies the code.

[0821] The chatbot server retrieves the billing details from the database and sends a response message saying, "Your billing details for this month are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen."

[0822] Example of a prompt

[0823] 1. If the user wishes to make an appointment:

[0824] User: I want to make an appointment to visit the store.

[0825] System: Your appointment request has been received. Available dates and times are as follows: June 10th, 2:00 PM and June 12th, 10:00 AM. Please select your preferred date and time.

[0826] User: Please make it June 12th at 10:00.

[0827] System: Your reservation has been confirmed. Please come to the store on June 12th at 10:00.

[0828] 2. When the user asks about the documents required for the procedure:

[0829] User: What documents are required for moving procedures?

[0830] System: The following documents are required for the moving process: Resident registration certificate, utility bills, and identification.

[0831] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0832] Step 1:

[0833] Receive user input

[0834] The user types "I would like to make a reservation to visit the store" in the chat screen of the LINE app.

[0835] Input: Text data sent by the user via the LINE app.

[0836] The device sends input data to the LINE server via the LINE API. The data is transmitted encrypted.

[0837] The LINE server receives this input data and forwards it to the chatbot server using a specific protocol.

[0838] Output: User input data transferred to the chatbot server.

[0839] Step 2:

[0840] Analyze user intent using a natural language processing engine.

[0841] The chatbot server analyzes the received data.

[0842] Input: User input data transferred to the chatbot server.

[0843] The chatbot server calls a natural language processing engine such as the Google NLP API to analyze the input. Specifically, it extracts the intention "to make a store visit reservation" from the input "I want to make a store visit reservation."

[0844] Output: The analyzed user intent and related information are generated in JSON format.

[0845] Step 3:

[0846] Execute processing in response to user requests.

[0847] The chatbot server processes user requests based on the analysis results.

[0848] Input: Analyzed user intent and related information.

[0849] For example, in the case of a store visit reservation, the chatbot server connects to the database and retrieves information on available dates and times.

[0850] The chatbot server generates SQL queries and sends them to a database (e.g., PostgreSQL) to retrieve data.

[0851] The database searches for available date and time slots and sends the results back to the chatbot server.

[0852] Output: Retrieved reservation availability information.

[0853] Step 4:

[0854] Suggest possible dates and times to the user.

[0855] The chatbot server generates a message to notify the user of the available reservation date and time information it has retrieved.

[0856] Input: Retrieved information on available reservation dates and times.

[0857] The chatbot server generates a message and sends it to the LINE server.

[0858] The LINE server delivers notification messages to the user's device.

[0859] Output: The message reads: "The following dates and times are available for booking: June 10th at 2:00 PM, June 12th at 10:00 AM. Please choose your preferred date and time."

[0860] Step 5:

[0861] Processing user responses

[0862] The user selects their preferred date and time and replies, "Please make it June 12th at 10:00."

[0863] Input: The user's preferred date and time.

[0864] The user's terminal sends this response back to the LINE server.

[0865] The LINE server forwards the received response to the chatbot server.

[0866] The chatbot server then uses its natural language processing engine again to analyze the response and extract the user's preferred date and time.

[0867] Input: A response message containing the user's preferred date and time.

[0868] The chatbot server generates an SQL query to save the desired date and time to the database and sends it to the database.

[0869] The database stores the reservation information and sends the results back to the chatbot server.

[0870] Output: The desired date and time have been saved in the database.

[0871] Step 6:

[0872] Send a confirmation notice to the user.

[0873] The chatbot server generates the message, "Your reservation has been confirmed. Please come to the store on June 12th at 10:00."

[0874] Input: Desired date and time information stored in the database.

[0875] The chatbot server sends that message to the LINE server.

[0876] The LINE server delivers the message to the user's device.

[0877] The user receives a confirmation message.

[0878] Output: Message: "Your reservation has been confirmed. Please come to the store on June 12th at 10:00."

[0879] (Application Example 1)

[0880] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0881] To improve the work efficiency of factory workers, there is a need for a system that can provide work instructions and solve problems in real time. In particular, intuitive voice-based operation and rapid information delivery are required. However, conventional systems have difficulty analyzing voice input and making real-time decisions, which hinders efforts to improve work efficiency.

[0882] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0883] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for determining necessary processing based on the analyzed intent and executing that processing, means for returning the processing results to the user, means for analyzing voice input and converting the voice to text, and means for retrieving information from a database based on the analyzed intent and providing work instructions. This makes it possible for factory workers to obtain accurate instructions and information in real time by using voice input.

[0884] "Means for receiving user input" refers to devices or software used to incorporate information and commands provided by the user into the system.

[0885] A "natural language processing engine" is an algorithm and program that analyzes input text to understand its meaning and intent.

[0886] "Means for determining necessary processing based on analyzed intent" refers to the process by which a natural language processing engine selects the next action to take based on the results of its analysis.

[0887] "The means to carry out that process" refers to the system or device used to actually put the decided action into action.

[0888] "Means for returning processing results to the user" refer to communication means or display devices for providing feedback to the user regarding the results and information of the actions performed.

[0889] "Means for analyzing voice input and converting speech to text" refers to software or hardware that uses speech recognition technology to convert a user's voice into text information.

[0890] "Methods for retrieving information from a database" refers to the procedures for searching for and retrieving the required information from a database that stores the necessary data.

[0891] "Means for providing work instructions" refers to a mechanism for communicating specific work procedures and instructions to the user based on analysis and acquired information.

[0892] "Means of acquiring and proposing information related to store visit reservations to users" refers to the process of acquiring information such as available reservation dates, times, and locations, and presenting that information to the user.

[0893] "Means for saving reservation information" refers to a data management function that records reservation data selected by the user and allows it to be retrieved or updated as needed.

[0894] "Means of providing relevant business processes" refers to a system that provides the instructions and procedures necessary to perform specific tasks or procedures within a factory.

[0895] "Means of obtaining necessary document information for a procedure and informing the user" refers to a method of obtaining the documents and information necessary to perform a specific procedure and notifying the user of them.

[0896] "Means of obtaining and introducing online procedural information to users" refers to a function that collects information on procedures that can be performed via the internet and introduces it to users.

[0897] The present invention provides an advanced chatbot system for improving the work efficiency of factory workers, capable of providing practical work instructions and problem-solving based on user voice input. The overall configuration of this system is as follows.

[0898] Overall system configuration

[0899] The system of the present invention consists of the following elements.

[0900] 1. User terminal

[0901] These are devices accessed by workers using smart glasses, head-mounted displays, or PC microphones.

[0902] 2. Server

[0903] It is a central processing unit for performing speech recognition and natural language processing.

[0904] It has a built-in natural language processing engine (e.g., Google Cloud NLP) that converts voice input into text and analyzes the user's intent.

[0905] 3. Database

[0906] This system manages work instructions, assembly procedures, and information related to work procedures.

[0907] Processing flow

[0908] 1. Receiving voice input

[0909] The user terminal receives voice input spoken by the worker into smart glasses or a head-mounted display (e.g., "Tell me the assembly procedure for the next product").

[0910] 2. Speech Recognition and Text Conversion

[0911] The device sends the received audio to the server, where it uses speech recognition technology (e.g., the SpeechRecognition library) to convert the audio into text.

[0912] 3. Intent Analysis and Data Acquisition

[0913] The server analyzes the converted text using a natural language processing engine and queries the database for work instructions and necessary information.

[0914] 4. Providing work instructions

[0915] The server organizes the information retrieved from the database and provides work instructions to the user in text or voice.

[0916] For example, when providing instructions for assembling the following product, instructions such as "Step 1: Attach part A. Step 2: Tighten the screws..." are provided.

[0917] Hardware and software to use

[0918] 1. Hardware

[0919] Smart glasses, head-mounted displays, PC microphones, and other devices that allow users to access information.

[0920] Server (Central processing unit for performing advanced processing)

[0921] 2. Software

[0922] Speech recognition libraries (e.g., SpeechRecognition)

[0923] Natural language processing engine (e.g., Google Cloud NLP)

[0924] Database management system (management of work instructions and procedure information)

[0925] Specific example

[0926] In the factory, a new worker uses smart glasses to give voice instructions such as, "Tell me the assembly procedure for the next product." The server analyzes this voice, retrieves the procedure information from the database, displays it on the worker's screen, and also provides voice guidance.

[0927] Example of a prompt:

[0928] "Please tell me the assembly instructions for the following product."

[0929] This system allows factory workers to receive accurate instructions and information in real time by using voice input, thereby improving operational efficiency.

[0930] This invention is a system that highly analyzes voice input to improve work efficiency and provides necessary information and instructions in real time.

[0931] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0932] Step 1:

[0933] Users input voice commands through smart glasses or head-mounted displays.

[0934] As a concrete example, the user might say, "Please tell me the assembly instructions for the following product."

[0935] The input is the user's voice, and the output is audio data.

[0936] Step 2:

[0937] The device that receives the audio data sends that audio data to the server.

[0938] Specifically, the device sends audio data to the server via the network.

[0939] The input is audio data, and the output is data sent to the server.

[0940] Step 3:

[0941] The server converts the received audio data into text using speech recognition software (e.g., the SpeechRecognition library).

[0942] Specifically, the server starts the speech recognition engine and converts the speech data into text data.

[0943] The input is audio data, and the output is converted text data.

[0944] Step 4:

[0945] The server analyzes the converted text data using a natural language processing engine (e.g., Google Cloud NLP) to understand the user's intent.

[0946] In terms of specific operations, the server invokes a natural language processing engine to analyze the text data and extract the intended meaning.

[0947] The input is text data, and the output is parsed intent data.

[0948] Step 5:

[0949] Based on the analyzed intent, the server retrieves relevant information from the database.

[0950] In terms of specific actions, the server queries the database to retrieve the necessary work procedure information.

[0951] The input is intent data, and the output is work instruction information retrieved from the database.

[0952] Step 6:

[0953] The server organizes the acquired work instruction information and generates text or voice messages to respond to the user.

[0954] In terms of specific operations, the server formats the information into a format that is easy to transmit to the user and generates a message.

[0955] The input is work instruction information retrieved from the database, and the output is the generated message.

[0956] Step 7:

[0957] The server sends the generated message to the terminal, and the terminal notifies the user.

[0958] In terms of specific operations, the server sends a message to the terminal over the network, and the terminal displays instructions to the user via voice and text.

[0959] The input is the generated message, and the output is the notification to the user.

[0960] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0961] The system of the present invention is an advanced chatbot that combines intent analysis of input text and emotion recognition to enable smooth communication with users. The system of the present invention has the function of recognizing the user's emotions and responding appropriately according to those emotions.

[0962] Overall system configuration

[0963] The system consists of the following elements:

[0964] 1. User terminal

[0965] This refers to a device that users access using the LINE app or a web browser.

[0966] 2. LINE Server

[0967] Its role is to receive user input and forward it to the chatbot server.

[0968] 3. Chatbot Server

[0969] This is the central part of the system that analyzes user input and generates responses.

[0970] It incorporates a natural language processing engine and an emotion engine to analyze the user's intentions and emotions.

[0971] Access the database and perform read and write operations as needed.

[0972] 4. Database

[0973] It manages user information, reservation information, required documents for procedures, and billing information.

[0974] Processing flow

[0975] 1. Receive user input

[0976] The user types "I would like to make a reservation to visit the store" into the LINE app and sends it.

[0977] 2. Perform natural language processing and sentiment recognition.

[0978] The LINE server forwards the input to the chatbot server.

[0979] The chatbot server uses a natural language processing engine to analyze the intent of the input.

[0980] At the same time, an emotion engine is used to recognize the user's emotions.

[0981] 3. Determine the necessary processing based on the analysis results.

[0982] The chatbot server determines the necessary actions based on the analyzed intent and emotions.

[0983] For example, in the case of a store visit reservation, available dates and times are retrieved from the database.

[0984] 4. Return the processing result to the user.

[0985] The system then suggests available dates and times to the user.

[0986] 5. Process the user's response.

[0987] The user selects their preferred date and time and replies.

[0988] The chatbot server receives the requested date and time and saves it to the database. The reservation is then confirmed.

[0989] 6. Send a reservation confirmation notice to the user.

[0990] Generate a confirmation message and send it to the user.

[0991] Specific example

[0992] The following are examples of specific use cases.

[0993] 1. Make an appointment to visit the store.

[0994] If a user includes anxious expressions when making a reservation (for example, "I'm worried because it's my first time"), the emotion engine recognizes this as "anxiety."

[0995] The chatbot server offers detailed explanations and support options to alleviate anxiety. "Don't worry if this is your first time visiting. We'll send you detailed instructions."

[0996] 2. Information on the documents required for the procedure

[0997] When a user requests procedural information, if a message containing an emotion such as "It's urgent" is sent, the emotion engine recognizes the "urgency."

[0998] The chatbot server quickly provides the necessary document information and responds appropriately with phrases like, "Thank you for your urgency. The required documents are as follows."

[0999] 3. Inquiry regarding unclear points in the billing statement.

[1000] If a user enters "The explanation about the fees is unclear," the sentiment engine recognizes this as "confusion."

[1001] The chatbot server adds a detailed and easy-to-understand explanation: "We apologize for any confusion regarding the billing details. Here is a detailed breakdown."

[1002] Thus, the present invention realizes a chatbot system that provides appropriate responses according to the user's intentions and emotions by combining natural language processing and emotion recognition.

[1003] The following describes the processing flow.

[1004] Specific processing steps for making a store visit reservation

[1005] Step 1:

[1006] The user types "I want to make a reservation to visit the store" in the LINE app's chat screen and sends it.

[1007] Step 2:

[1008] The device sends the user's message to the LINE server. The message is received by the LINE server.

[1009] Step 3:

[1010] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[1011] Step 4:

[1012] The chatbot server uses a natural language processing engine to analyze the intent of messages. Simultaneously, it uses an emotion engine to analyze the user's emotions.

[1013] Step 5:

[1014] Based on the analysis results, the chatbot server recognizes the user's intention to make an appointment and any feelings of anxiety.

[1015] Step 6:

[1016] The chatbot server connects to the database and queries it to retrieve available dates and times for booking. The database then returns the available dates and times.

[1017] Step 7:

[1018] The chatbot server generates a message suggesting possible dates and times to the user, saying, "Please let us know your preferred date and time. Suggestions: June 12th, 10:00 AM, June 12th, 2:00 PM. We also offer support to ensure a smooth experience even for first-time visitors." This message is then sent to the user's device via the LINE server.

[1019] Step 8:

[1020] The user replies, "Please make it June 12th at 10:00."

[1021] Step 9:

[1022] The device sends the user's reply to the LINE server. The LINE server receives the message and forwards it to the chatbot server.

[1023] Step 10:

[1024] The chatbot server saves the date and time of the received message to its database and updates the reservation information.

[1025] Step 11:

[1026] The database stores the reservation information. The chatbot server generates a reservation confirmation message and sends it to the user's device via the LINE server, stating, "Your reservation has been confirmed. Please come to our store on June 12th at 10:00. Please feel free to contact us if you have any questions."

[1027] Specific processing steps for the documents required for the procedure

[1028] Step 1:

[1029] The user types "Please tell me what documents are required for moving procedures" and submits the form.

[1030] Step 2:

[1031] The device sends the input to the LINE server. The message is received by the LINE server.

[1032] Step 3:

[1033] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[1034] Step 4:

[1035] The chatbot server uses a natural language processing engine to analyze the intent of messages. Simultaneously, it uses an emotion engine to analyze the user's emotions.

[1036] Step 5:

[1037] Based on the analysis results, the chatbot server recognizes the user's intention to initiate the moving process and their sense of urgency.

[1038] Step 6:

[1039] The chatbot server connects to the database and queries it to retrieve the necessary document information for the procedure. The database then returns the required document information.

[1040] Step 7:

[1041] The chatbot server generates a message informing the user of the necessary documents and sends it to the device via the LINE server, stating, "Thank you for your urgency. The following documents are required for the moving process: Resident registration certificate, utility bills, and identification."

[1042] Specific processing steps for inquiries regarding unclear points in billing statements

[1043] Step 1:

[1044] The user types, "This month's bill is high, please show me the details," and submits it.

[1045] Step 2:

[1046] The device sends the input to the LINE server. The message is received by the LINE server.

[1047] Step 3:

[1048] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[1049] Step 4:

[1050] The chatbot server uses a natural language processing engine to analyze intent and recognize that the user "wants to inquire about their billing details." At the same time, it uses an emotion engine to recognize the user's "confusion."

[1051] Step 5:

[1052] The chatbot server sends an SMS verification code to the user's registered phone number to verify their identity. It generates a message for the verification process and sends it to the device via the LINE server, stating, "We will now verify your identity. An SMS code has been sent to your registered phone number."

[1053] Step 6:

[1054] The user enters the code "123456" received via SMS into the LINE chat and sends it.

[1055] Step 7:

[1056] The device sends the input back to the LINE server. The message is received by the LINE server.

[1057] Step 8:

[1058] The LINE server forwards the entered SMS code to the chatbot server. The chatbot server verifies the code, and if it is correct, completes the identity verification process.

[1059] Step 9:

[1060] The chatbot server queries the database to retrieve the billing details. The database then returns the billing details information.

[1061] Step 10:

[1062] The chatbot server generates a message saying, "This month's billing details are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen," adds a clear explanation, and sends it to the user's terminal via the LINE server, saying, "We apologize for any confusion regarding the billing details. Here is a detailed breakdown."

[1063] As described above, the system of the present invention recognizes the user's intentions and emotions and provides an appropriate response.

[1064] (Example 2)

[1065] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1066] Conventional chatbot systems are limited to analyzing the intent behind user input, making it difficult to respond in a way that takes user emotions into account. As a result, responses to users are uniform, and communication is often not smooth. Furthermore, there are delays in providing information about specific processes and responding to users' anxieties and urgency, which leads to decreased user satisfaction. This invention aims to solve these problems.

[1067] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1068] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for recognizing the user's emotions using an emotion engine, means for determining and executing necessary processing based on the analyzed intent and emotions, and means for returning the processing results to the user. This enables an appropriate response that takes into account both the user's intent and emotions.

[1069] "Means for receiving user input" refers to the function of obtaining text messages and commands sent from the terminal used by the user.

[1070] A "natural language processing engine" is a software module that analyzes text entered by a user and understands its intent and meaning.

[1071] An "emotion engine" is a software module that identifies emotions and psychological states from user input and determines appropriate responses based on that information.

[1072] "Means for determining necessary processing based on analyzed intent and emotion" refers to a function that selects the next action or response based on the analysis results of the natural language processing engine and the emotion engine.

[1073] "Means of responding to the user with processing results" refers to a function that generates the determined response or action as a text message and sends it to the user.

[1074] "A means of obtaining and proposing information related to store visit reservations to the user" refers to a function that, when a user requests a store visit reservation, retrieves available dates and times and related information from a database and presents it to the user.

[1075] "A means of receiving a user's requested reservation date and time and saving the reservation information" refers to a function that receives the reservation date and time selected by the user and saves it in a database.

[1076] "Means of notifying users of confirmed reservation information" refers to a function that generates and sends a message to inform users of a confirmed reservation.

[1077] "A means of obtaining necessary document information for procedures and guiding users through it" refers to a function that retrieves the necessary documents and information for various procedures from a database and guides users through them appropriately.

[1078] "A means of determining processing priorities based on user emotions" refers to a function that determines the order and urgency of responses according to the user's psychological state, as analyzed by the emotion engine.

[1079] "A means of obtaining and introducing online procedural information to users" refers to a function that searches for and obtains information on procedures and services that users can perform online, and provides that information to users.

[1080] Modes for carrying out the invention

[1081] The system of the present invention is an advanced chatbot that combines intent analysis of input text and sentiment recognition to enable smooth communication with users. The specific configuration and operation of the system of the present invention are described below.

[1082] Overall system configuration

[1083] The system consists of the following elements:

[1084] 1. User terminal

[1085] These are devices such as smartphones and PCs, and users access them using the LINE app or a web browser.

[1086] 2. LINE Server

[1087] Its role is to receive user input and forward it to the chatbot server.

[1088] 3. Chatbot Server

[1089] This is the central part of the system that analyzes user input and generates responses.

[1090] It incorporates a natural language processing engine (e.g., Google Cloud Natural Language API) and an emotion engine (e.g., Microsoft Azure Text Analytics API) to analyze user intent and sentiment.

[1091] Access and read / write data to a database (e.g., MySQL) as needed.

[1092] 4. Database

[1093] It manages user information, reservation information, required documents for procedures, and billing information.

[1094] Program processing

[1095] 1. Receive user input

[1096] The user types "I want to make a reservation to visit the store" into the LINE app and sends it. The user's device sends this message to the LINE server.

[1097] 2. Perform natural language processing and sentiment recognition.

[1098] The server receives user input via the LINE server and forwards it to the chatbot server in JSON format. The chatbot server calls the Google Cloud Natural Language API to analyze the intent of the message. Simultaneously, it uses the Microsoft Azure Text Analytics API to recognize the user's sentiment.

[1099] 3. Determine the necessary processing based on the analysis results.

[1100] The chatbot server determines the next action to take based on the analysis results. For example, in the case of a store visit reservation, it generates a query to retrieve available dates and times from the database.

[1101] 4. Return the processing result to the user.

[1102] The chatbot server proposes available dates and times to the user. The message is generated in a specific format and sent to the user via the LINE server.

[1103] 5. Process the user's response.

[1104] The user selects their preferred date and time and replies. The chatbot server receives this selection and executes an SQL query to save it to the database.

[1105] 6. Send a reservation confirmation notice to the user.

[1106] The chatbot server generates a reservation confirmation message and sends it to the user: "Your reservation is confirmed. We look forward to seeing you on [Month] [Day] at [Time]."

[1107] Specific example

[1108] The following are examples of specific use cases.

[1109] 1. Make an appointment to visit the store.

[1110] If a user includes an anxious expression when making a reservation (for example, "I'm worried because it's my first time"), the emotion engine recognizes this anxiety. The chatbot server then offers detailed explanations and support options to alleviate the anxiety. For example, "Don't worry if it's your first time. We'll send you detailed instructions."

[1111] 2. Information on the documents required for the procedure

[1112] When a user requests procedural information and sends a message that includes an emotion such as "It's urgent," the emotion engine recognizes the "urgency." The chatbot server quickly provides the necessary document information and gives an appropriate response such as, "Thank you for your urgency. The required documents are as follows."

[1113] 3. Inquiry regarding unclear points in the billing statement.

[1114] If a user types "The explanation about the charges is unclear," the sentiment engine recognizes this as "confusion." The chatbot server then adds a more detailed and clearer explanation: "We apologize for the confusion regarding the charges. Here is a detailed breakdown."

[1115] Thus, the present invention realizes a chatbot system that provides appropriate responses according to the user's intentions and emotions by combining natural language processing and emotion recognition.

[1116] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1117] Step 1:

[1118] Receive user input

[1119] explanation:

[1120] The user types "I would like to make a reservation to visit the store" into the LINE app and sends it.

[1121] The device will send this message to the LINE server.

[1122] Specific actions:

[1123] The user types the message "I would like to make a reservation to visit" on the LINE app on their smartphone and presses the send button.

[1124] Input: User's text message

[1125] Output: Message forwarded to LINE server

[1126] Step 2:

[1127] Performing natural language processing and emotion recognition.

[1128] explanation:

[1129] The server forwards messages received from the LINE server to the chatbot server.

[1130] The chatbot server analyzes the intent of messages using a natural language processing engine (e.g., Google Cloud Natural Language API).

[1131] Use an emotion engine (e.g., Microsoft Azure Text Analytics API) to recognize the user's emotions.

[1132] Specific actions:

[1133] The server receives the user's message via the LINE server and forwards it to the chatbot server in JSON format.

[1134] The chatbot server sends a request to the Google Cloud Natural Language API to analyze the intent behind the "store visit reservation."

[1135] Simultaneously, a request is sent to the Microsoft Azure Text Analytics API to analyze the emotions (e.g., anxiety) contained in the message.

[1136] Input: Message forwarded from LINE server

[1137] Output: Analyzed intentions and emotions

[1138] Step 3:

[1139] Based on the analysis results, determine the necessary processing steps.

[1140] explanation:

[1141] Based on the analysis results, the chatbot server determines the next action to take.

[1142] For example, in the case of a store visit reservation, available dates and times are retrieved from a database (e.g., MySQL).

[1143] Specific actions:

[1144] The chatbot server generates database queries based on the analysis results.

[1145] The server queries the MySQL database to retrieve available dates and times for booking.

[1146] Input: Analyzed intentions and emotions

[1147] Output: Database query results including available dates and times for booking

[1148] Step 4:

[1149] Return the processing result to the user.

[1150] explanation:

[1151] The chatbot server then suggests available dates and times to the user.

[1152] The generated text message is sent to the LINE server as a suggestion.

[1153] Specific actions:

[1154] The chatbot server generates a suggestion message in a specific format and sends that message to the LINE server.

[1155] The LINE server forwards the message to the user's device.

[1156] Input: Database query results including available dates and times for booking

[1157] Output: Suggestion message sent to the user

[1158] Step 5:

[1159] Processing user responses

[1160] explanation:

[1161] The user selects their preferred date and time and replies.

[1162] The chatbot server receives the reply and saves the desired date and time to its database.

[1163] Specific actions:

[1164] The user sends a message saying, "I would like to meet on [Month] [Day] at [Time]."

[1165] The chatbot server receives this message and executes an SQL query to save the desired date and time to the database.

[1166] Input: User's reply message

[1167] Output: Reservation information stored in the database

[1168] Step 6:

[1169] Send a reservation confirmation notification to the user.

[1170] explanation:

[1171] The chatbot server generates and sends a message to the user notifying them that the reservation has been confirmed.

[1172] Specific actions:

[1173] The chatbot server generates the message, "Your reservation has been confirmed. We look forward to seeing you on [Month] [Day] at [Time]."

[1174] The message is sent to the user's device via the LINE server.

[1175] Input: Reservation information stored in the database

[1176] Output: Booking confirmation message sent to the user

[1177] (Application Example 2)

[1178] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1179] Traditional chatbot systems are limited to analyzing user intent and have the problem of not being able to provide appropriate responses that take into account user emotions. This degrades the user experience, and in particular, in ride reservation systems, support may be insufficient when passengers are feeling anxious or stressed. There is a need for a system that can smoothly respond to passengers when they need changes or urgent assistance.

[1180] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1181] In this invention, the server includes means for receiving user input, means for analyzing the intent and emotion of the input using a natural language processing engine and an emotion recognition system, means for determining and executing necessary processing based on the analyzed intent and emotion, and means for returning the processing results to the user. This enables appropriate responses that address not only the user's intent but also their emotions, allowing for quick and appropriate responses to requests such as ride reservations and route changes.

[1182] "User" refers to an individual or group that uses the system.

[1183] "Input" refers to the information that a user sends to the system.

[1184] A "natural language processing engine" is a computer program that analyzes user input to understand their intent.

[1185] An "emotion recognition system" is a computer program that analyzes emotions from user input.

[1186] "Intention" refers to the purpose or request that the user is trying to convey through their input.

[1187] "Emotions" refer to the psychological state that a user expresses through their input.

[1188] "Processing" refers to the actions the system takes based on the analyzed intentions and emotions.

[1189] "Response" refers to the answer that the system provides to the user.

[1190] "Ride reservation" refers to the act of a user reserving a vehicle for a specific date and time.

[1191] "Route change" refers to a request from a user to change their existing travel route.

[1192] A "support center" refers to an organization that provides assistance to users when they encounter problems or have questions.

[1193] "Dialogue" refers to the communication that takes place between a system and a user.

[1194] "Mobility experience" refers to the overall experience a user has when using an autonomous vehicle.

[1195] The system for implementing this invention is an advanced chatbot system aimed at improving the user's riding experience. This system consists of the following main components:

[1196] Overall system configuration

[1197] The system consists of the following four elements:

[1198] 1. User terminal

[1199] A user terminal refers to a mobile device such as a smartphone, which users use to access the system through applications. Users can make reservations and change routes through text input.

[1200] 2. Communication Server

[1201] This server is responsible for receiving user input in real time and forwarding it to the chatbot server. The communication server is crucial for ensuring stable network communication.

[1202] 3. Chatbot Server

[1203] The chatbot server is the heart of the system, handling all data processing and response generation. It includes the following software components:

[1204] Natural language processing engine (e.g., TextBlob): Analyzes user input text and understands intent.

[1205] Emotion recognition systems (e.g., emotion analysis using TextBlob): Analyze emotions from user input and adjust responses accordingly.

[1206] 4. Database Server

[1207] The database server manages the following information:

[1208] User Information

[1209] Reservation Information

[1210] Route information

[1211] Past conversation history

[1212] Explanation of the process

[1213] The system processes the data as follows:

[1214] Receiving user input

[1215] Text input is sent from the user's terminal and reaches the chatbot server via the communication server.

[1216] Natural language processing and sentiment analysis

[1217] The chatbot server uses a natural language processing engine to analyze the user's intent and an emotion recognition system to evaluate their emotions. For example, if a user types "I want to make a reservation," the intent is analyzed as "reservation" and the emotion as "positive."

[1218] Decision and execution of processing

[1219] Based on the analyzed intent and emotions, the chatbot server determines the appropriate action. For example, if a reservation request is made, it retrieves available dates and times from the database and generates suggestions.

[1220] Response to the user

[1221] The system returns the processing results to the user. When suggesting available dates and times, if the user expresses concerns, it will respond in an empathetic manner, such as, "It seems you have concerns about your preferred date and time. Please feel free to contact us."

[1222] Examples of specific user interactions

[1223] The following is an example of a specific interaction between the user and the system:

[1224] User input: "I would like to make a reservation."

[1225] System response: "The following time slots are available for booking: 2023-11-01 10:00, 2023-11-01 14:00, 2023-11-01 18:00"

[1226] The system constantly analyzes user emotions and generates appropriate responses, allowing users to use the system with peace of mind.

[1227] Software and hardware used

[1228] Software: TextBlob (natural language processing and sentiment analysis), requests module (HTTP requests)

[1229] Hardware: Smartphones (user terminals), servers (communication servers and chatbot servers)

[1230] These components allow users to book and use autonomous vehicles intuitively and with confidence.

[1231] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1232] Step 1:

[1233] Receiving user input

[1234] The user terminal receives input (text messages) from the user. This input includes requests such as reservations and route changes. The received input is sent to the communication server.

[1235] Input: The user enters a message such as "I would like to make a reservation."

[1236] Processing: The user terminal sends input to the communication server.

[1237] Output: Communication server receives input message

[1238] Step 2:

[1239] Intent analysis using natural language processing

[1240] The communication server forwards the received user input to the chatbot server. The chatbot server uses a natural language processing engine to analyze the intent of the user's input.

[1241] Input: Input message forwarded from the communication server

[1242] Processing: A natural language processing engine analyzes the intent of the input (e.g., "request for reservation").

[1243] Output: Analyzed intent (e.g., "reservation")

[1244] Step 3:

[1245] Emotion analysis using an emotion recognition system

[1246] The chatbot server uses an emotion recognition system to analyze the emotions from the user's input.

[1247] Input: Input message forwarded from the communication server

[1248] Processing: The emotion recognition system analyzes the emotion of the input (e.g., "positive").

[1249] Output: Analyzed emotion (e.g., "positive")

[1250] Step 4:

[1251] Decision and execution of processing

[1252] The chatbot server determines the necessary actions based on the analyzed intent and emotion. For example, in the case of a reservation request, it retrieves available dates and times from the database.

[1253] Input: Analyzed intentions and emotions (e.g., "reservation", "positive")

[1254] Process: Retrieve available date and time information from the database.

[1255] Output: Acquired reservation date and time information (e.g., "2023-11-01 10:00")

[1256] Step 5:

[1257] Generating responses to users

[1258] The chatbot server generates a response to the user based on the processing results. This response should include consideration for the user's emotions.

[1259] Input: Acquired reservation date and time information and analyzed sentiment (e.g., "2023-11-01 10:00", "Positive")

[1260] Process: Generate a response (Example: "The available time slots are as follows: 2023-11-01 10:00")

[1261] Output: Reply message to the user (Example: "The available time slots are as follows: 2023-11-01 10:00")

[1262] Step 6:

[1263] Sending a reply to the user

[1264] The chatbot server sends the generated response message to the user's terminal via the communication server. The user can then review the response and take the next action.

[1265] Input: Generated response message (Example: "The following time slots are available for booking: 2023-11-01 10:00")

[1266] Processing: Send a reply message to the user terminal via the communication server.

[1267] Output: Reply message displayed on the user's terminal

[1268] The above outlines the specific processing flow of the chatbot in the autonomous vehicle reservation system. By performing appropriate data processing and calculations at each step, it is possible to provide users with quick and appropriate responses.

[1269] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1270] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1271] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1272] [Third Embodiment]

[1273] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1274] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1275] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1276] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1277] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1278] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1279] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1280] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1281] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1282] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1283] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1284] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1285] The system of the present invention is an advanced chatbot for streamlining customer service and can perform various practical procedures based on user input. The system of the present invention is realized through a series of processes that receive user input, analyze it, and generate an appropriate response.

[1286] Overall system configuration

[1287] The system is broadly composed of the following elements:

[1288] 1. User terminal

[1289] This refers to a device that users access using the LINE app or a web browser.

[1290] 2. LINE Server

[1291] Its role is to receive user input and forward it to the chatbot server.

[1292] 3. Chatbot Server

[1293] It is the central part of the system that analyzes user input and generates appropriate responses.

[1294] It has a built-in natural language processing engine that analyzes the user's intent.

[1295] Access, read from, and write to the database as needed.

[1296] 4. Database

[1297] This is a supplementary system for managing user information, reservation information, and necessary document information for procedures.

[1298] Processing flow

[1299] 1. Receive user input

[1300] For example, a user might type "I want to make a reservation to visit the store" in the chat screen of the LINE app.

[1301] The device sends this input to the LINE server.

[1302] 2. Analyze user intent using a natural language processing engine.

[1303] The LINE server receives the input and forwards it to the chatbot server.

[1304] The chatbot server uses a natural language processing engine to analyze the intent of the input and recognizes that the user "wants to make a reservation to visit the store."

[1305] 3. Executing processing in response to user requests

[1306] For store visit reservations, the chatbot server retrieves available dates and times from the database.

[1307] The database returns available dates and times, and the chatbot server suggests possible dates and times to the user.

[1308] 4. Process the user's response.

[1309] The user selects their preferred date and time and replies in the chat, "Please make it June 12th at 10:00."

[1310] The chatbot server then analyzes this again, saves the desired date and time to the database, and confirms the reservation.

[1311] 5. Send a confirmation notice to the user.

[1312] The chatbot server generates the message "Your reservation has been confirmed. Please come to the store on June 12th at 10:00" and sends it to the user's device via the LINE server.

[1313] Specific example

[1314] The following are some specific use cases.

[1315] 1. Information on the documents required for the procedure

[1316] The user's terminal sends the message, "Please tell me what documents are required for the relocation procedure."

[1317] The chatbot server uses a natural language processing engine to analyze intent.

[1318] The system queries a database for information on "documents required for moving procedures" and retrieves the results.

[1319] The chatbot server generates a response message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the user's terminal.

[1320] 2. Introduction to online procedures

[1321] A message is sent to the user's terminal saying, "Please tell me what procedures I can do online."

[1322] The chatbot server receives this input and retrieves the appropriate online procedure information from its database.

[1323] The response message reads: "The following procedures can be done online: change of address, review contract details, and review billing statements. You can find more details at this link [link]."

[1324] 3. Inquiry regarding unclear points in the billing statement.

[1325] The user enters, "My bill this month is high, please show me the details."

[1326] The chatbot server requests identity verification and sends an SMS authentication code to the user's device.

[1327] The user enters an SMS code into the chat, and the chatbot server verifies the code.

[1328] The chatbot server retrieves the billing details from the database and sends a response message saying, "Your billing details for this month are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen."

[1329] In this way, the present invention realizes a system that provides a quick and effective response to the diverse demands of users.

[1330] The following describes the processing flow.

[1331] Step 1:

[1332] The user types "I want to make a reservation to visit" in the LINE app's chat screen. The user then sends this message.

[1333] Step 2:

[1334] The device sends user input to the LINE server. The message is received by the LINE server.

[1335] Step 3:

[1336] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[1337] Step 4:

[1338] The chatbot server uses a natural language processing engine to analyze the intent of incoming messages. The natural language processing engine identifies the intent as "I would like to make a reservation to visit the store."

[1339] Step 5:

[1340] The chatbot server queries the database for available dates and times and retrieves them. The database then sends the available dates and times back to the chatbot server.

[1341] Step 6:

[1342] The chatbot server generates a message for the user saying, "Please tell us your preferred date and time. Suggestions: June 12th, 10:00 AM, June 12th, 2:00 PM," and sends it to the user's device via the LINE server.

[1343] Step 7:

[1344] The user enters their desired date and time, such as "June 12th, 10:00 AM, please," on their device and sends it to the LINE server.

[1345] Step 8:

[1346] The LINE server receives input from the user and forwards it to the chatbot server. The chatbot server receives the message for the requested date and time.

[1347] Step 9:

[1348] The chatbot server saves the date and time of the received message to the database and updates the reservation information. The database stores the reservation information.

[1349] Step 10:

[1350] The chatbot server generates a reservation confirmation message saying, "Your reservation has been confirmed. Please come to the store on June 12th at 10:00," and sends it to the user's device via the LINE server.

[1351] The following are example steps for obtaining the necessary documents and for inquiring about any unclear points regarding the fee breakdown.

[1352] Instructions on the documents required for the procedure

[1353] Step 1:

[1354] The user uses their device to type "Please tell me what documents are required for moving procedures" and then sends the message.

[1355] Step 2:

[1356] The device sends input to the LINE server. The message is received by the LINE server.

[1357] Step 3:

[1358] The LINE server forwards the incoming message to the chatbot server. The chatbot server receives the message.

[1359] Step 4:

[1360] The chatbot server uses a natural language processing engine to analyze the intent of the input. The natural language processing engine identifies the intent as "I want to know what documents are needed for moving procedures."

[1361] Step 5:

[1362] The chatbot server queries the database for relevant document information and retrieves it. The database then sends the relevant information back to the chatbot server.

[1363] Step 6:

[1364] Based on the information it has acquired, the chatbot server generates a message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the device via the LINE server.

[1365] Inquiry regarding unclear points in the billing statement

[1366] Step 1:

[1367] The user types, "My bill this month is high, please show me the details," and sends it via the LINE app.

[1368] Step 2:

[1369] The device sends the input to the LINE server. The message is received by the LINE server.

[1370] Step 3:

[1371] The LINE server forwards the incoming message to the chatbot server. The chatbot server receives the message.

[1372] Step 4:

[1373] The chatbot server uses a natural language processing engine to analyze the user's intent and recognizes that they "want to inquire about their billing details."

[1374] Step 5:

[1375] The chatbot server sends an SMS verification code to the user's registered phone number to verify their identity and generates a message for the verification process. It then sends the message, "We are verifying your identity. An SMS code has been sent to your registered phone number," to the user's device via the LINE server.

[1376] Step 6:

[1377] The user enters the code "123456" received via SMS into the LINE chat and sends it.

[1378] Step 7:

[1379] The device sends the input back to the LINE server. The message is received by the LINE server.

[1380] Step 8:

[1381] The LINE server forwards the entered SMS code to the chatbot server. The chatbot server verifies the code, and if it is correct, completes the identity verification process.

[1382] Step 9:

[1383] The chatbot server queries the database to retrieve the billing details. The database then sends the billing details information back to the chatbot server.

[1384] Step 10:

[1385] The chatbot server generates the message "This month's charges are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage fee 1000 yen" and sends it to the user's terminal via the LINE server.

[1386] (Example 1)

[1387] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1388] In today's business environment, providing prompt and effective customer service is essential. However, many companies still rely on manual processes and inefficient systems, leading to wasted resources and decreased customer satisfaction. Furthermore, while rapid and accurate information provision is crucial, particularly for appointment scheduling and procedural guidance, integrated and automated systems to achieve this are lacking. Additionally, efficient online identity verification processes are inadequate. These challenges need to be addressed.

[1389] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1390] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for determining necessary processing based on the analyzed intent and executing that processing, and means for returning the processing results to the user. This makes it possible to automatically analyze user input and generate an appropriate response.

[1391] Furthermore, the server includes means for acquiring information related to store visit reservations and proposing them to the user, means for receiving the user's preferred reservation date and time and saving the reservation information, and means for confirming and notifying the reservation in a conversational format via a chat server. This enables efficient store visit reservations and their confirmation and notification.

[1392] Furthermore, the server includes means for obtaining necessary document information for procedures and guiding users through them, means for obtaining online procedure information and presenting it to users, and means for receiving user authentication information and verifying their identity. This automates the procedure guidance and identity verification processes, significantly improving user convenience.

[1393] "Means for receiving user input" refers to the interface or protocol that allows a server to receive text data or voice data entered by a user through a terminal.

[1394] A "natural language processing engine" refers to algorithms and technologies that analyze user input and understand its intent and content, and examples include machine learning models and rule-based processing systems.

[1395] "Means for determining necessary processing based on analyzed intent and executing that processing" refers to software and hardware that determine and execute specific actions according to the user's intent analyzed by a natural language processing engine.

[1396] "Means of returning processing results to the user" refers to methods or devices for notifying the user of the results of processing performed by the server, such as a message generation engine or a communication interface.

[1397] "Means of obtaining and proposing information related to store visit reservations to users" refers to systems and methods for obtaining information related to store visit reservations from databases or APIs and presenting that information to users.

[1398] "A means of receiving a user's requested reservation date and time and saving the reservation information" refers to an interface or protocol for receiving a reservation date and time specified by a user and saving it to a database.

[1399] "A means of confirming and notifying reservations interactively via a chat server" refers to chat software and related infrastructure that allows users and servers to exchange information interactively, confirm the final reservation, and notify the user of the result.

[1400] "Means of obtaining and guiding users to the documents required for a procedure" refers to applications or systems that obtain information about the documents required for a procedure from databases or other sources and guide users to that information.

[1401] "Means of obtaining and introducing online procedural information to users" refers to methods and systems for obtaining information on services and procedures offered online and introducing them to users.

[1402] "Means of receiving user authentication information and verifying identity" refers to systems and methods for receiving authentication information provided by users (e.g., passwords or SMS codes) and using that information to verify the user's identity.

[1403] The system of the present invention is an advanced chatbot for streamlining customer service and can perform various practical procedures based on user input. This system is realized through a series of processes that receive user input, analyze it, and generate an appropriate response. A specific embodiment of this system is described below.

[1404] System Configuration

[1405] This system is broadly composed of the following elements.

[1406] 1. User terminal

[1407] This is a device that users access using the LINE app or a web browser. It has the function of receiving user input and sending it to the server.

[1408] 2. LINE Server

[1409] Its role is to receive user input and forward it to the chatbot server. It also handles protocol conversion and temporary storage of input data.

[1410] 3. Chatbot Server

[1411] This is the core of the system, analyzing user input and generating appropriate responses. Specifically, it incorporates a natural language processing engine to analyze user intent. It utilizes APIs such as the Google NLP API as its natural language processing engine. It also accesses databases as needed to read and write information.

[1412] 4. Database

[1413] This is an auxiliary system for managing user information, reservation information, and necessary document information for procedures. For example, it uses a PostgreSQL database.

[1414] Program processing

[1415] 1. Receive user input

[1416] For example, a user might type "I want to make a reservation to visit the store" in the chat screen of the LINE app.

[1417] The device sends this input to the LINE server.

[1418] The LINE server forwards the received data to the chatbot server.

[1419] 2. Analyze user intent using a natural language processing engine.

[1420] The chatbot server uses a natural language processing engine to analyze the intent of the input and recognizes that the user "wants to make a reservation to visit the store."

[1421] 3. Executing processing in response to user requests

[1422] For store visit reservations, the chatbot server retrieves available dates and times from the database. For example, it might send an SQL query to a PostgreSQL database to retrieve the data.

[1423] The database returns available dates and times, and the chatbot server suggests possible dates and times to the user.

[1424] 4. Process the user's response.

[1425] The user selects their preferred date and time and replies via chat, for example, "June 12th at 10:00 AM, please."

[1426] The chatbot server then analyzes this again, saves the desired date and time to the database, and confirms the reservation.

[1427] 5. Send a confirmation notice to the user.

[1428] The chatbot server generates the message "Your reservation has been confirmed. Please come to the store on June 12th at 10:00" and sends it to the user's device via the LINE server.

[1429] Specific example

[1430] The following are some specific use cases.

[1431] 1. Information on the documents required for the procedure

[1432] The user's terminal sends the message, "Please tell me what documents are required for the relocation procedure."

[1433] The chatbot server uses a natural language processing engine to analyze intent.

[1434] The system queries a database for information on "documents required for moving procedures" and retrieves the results.

[1435] The chatbot server generates a response message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the user's terminal.

[1436] 2. Introduction to online procedures

[1437] A message is sent to the user's terminal saying, "Please tell me what procedures I can do online."

[1438] The chatbot server receives this input and retrieves the appropriate online procedure information from its database.

[1439] The response message reads: "The following procedures can be done online: change of address, review contract details, and review billing statements. You can find more details at this link [link]."

[1440] 3. Inquiry regarding unclear points in the billing statement.

[1441] The user enters, "My bill this month is high, please show me the details."

[1442] The chatbot server requests identity verification and sends an SMS authentication code to the user's device.

[1443] The user enters an SMS code into the chat, and the chatbot server verifies the code.

[1444] The chatbot server retrieves the billing details from the database and sends a response message saying, "Your billing details for this month are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen."

[1445] Example of a prompt

[1446] 1. If the user wishes to make an appointment:

[1447] User: I want to make an appointment to visit the store.

[1448] System: Your appointment request has been received. Available dates and times are as follows: June 10th, 2:00 PM and June 12th, 10:00 AM. Please select your preferred date and time.

[1449] User: Please make it June 12th at 10:00.

[1450] System: Your reservation has been confirmed. Please come to the store on June 12th at 10:00.

[1451] 2. When the user asks about the documents required for the procedure:

[1452] User: What documents are required for moving procedures?

[1453] System: The following documents are required for the moving process: Resident registration certificate, utility bills, and identification.

[1454] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1455] Step 1:

[1456] Receive user input

[1457] The user types "I would like to make a reservation to visit the store" in the chat screen of the LINE app.

[1458] Input: Text data sent by the user via the LINE app.

[1459] The device sends input data to the LINE server via the LINE API. The data is transmitted encrypted.

[1460] The LINE server receives this input data and forwards it to the chatbot server using a specific protocol.

[1461] Output: User input data transferred to the chatbot server.

[1462] Step 2:

[1463] Analyze user intent using a natural language processing engine.

[1464] The chatbot server analyzes the received data.

[1465] Input: User input data transferred to the chatbot server.

[1466] The chatbot server calls a natural language processing engine such as the Google NLP API to analyze the input. Specifically, it extracts the intention "to make a store visit reservation" from the input "I want to make a store visit reservation."

[1467] Output: The analyzed user intent and related information are generated in JSON format.

[1468] Step 3:

[1469] Execute processing in response to user requests.

[1470] The chatbot server processes user requests based on the analysis results.

[1471] Input: Analyzed user intent and related information.

[1472] For example, in the case of a store visit reservation, the chatbot server connects to the database and retrieves information on available dates and times.

[1473] The chatbot server generates SQL queries and sends them to a database (e.g., PostgreSQL) to retrieve data.

[1474] The database searches for available date and time slots and sends the results back to the chatbot server.

[1475] Output: Retrieved reservation availability information.

[1476] Step 4:

[1477] Suggest possible dates and times to the user.

[1478] The chatbot server generates a message to notify the user of the available reservation date and time information it has retrieved.

[1479] Input: Retrieved information on available reservation dates and times.

[1480] The chatbot server generates a message and sends it to the LINE server.

[1481] The LINE server delivers notification messages to the user's device.

[1482] Output: The message reads: "The following dates and times are available for booking: June 10th at 2:00 PM, June 12th at 10:00 AM. Please choose your preferred date and time."

[1483] Step 5:

[1484] Processing user responses

[1485] The user selects their preferred date and time and replies, "Please make it June 12th at 10:00."

[1486] Input: The user's preferred date and time.

[1487] The user's terminal sends this response back to the LINE server.

[1488] The LINE server forwards the received response to the chatbot server.

[1489] The chatbot server then uses its natural language processing engine again to analyze the response and extract the user's preferred date and time.

[1490] Input: A response message containing the user's preferred date and time.

[1491] The chatbot server generates an SQL query to save the desired date and time to the database and sends it to the database.

[1492] The database stores the reservation information and sends the results back to the chatbot server.

[1493] Output: The desired date and time have been saved in the database.

[1494] Step 6:

[1495] Send a confirmation notice to the user.

[1496] The chatbot server generates the message, "Your reservation has been confirmed. Please come to the store on June 12th at 10:00."

[1497] Input: Desired date and time information stored in the database.

[1498] The chatbot server sends that message to the LINE server.

[1499] The LINE server delivers the message to the user's device.

[1500] The user receives a confirmation message.

[1501] Output: Message: "Your reservation has been confirmed. Please come to the store on June 12th at 10:00."

[1502] (Application Example 1)

[1503] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1504] To improve the work efficiency of factory workers, there is a need for a system that can provide work instructions and solve problems in real time. In particular, intuitive voice-based operation and rapid information delivery are required. However, conventional systems have difficulty analyzing voice input and making real-time decisions, which hinders efforts to improve work efficiency.

[1505] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1506] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for determining necessary processing based on the analyzed intent and executing that processing, means for returning the processing results to the user, means for analyzing voice input and converting the voice to text, and means for retrieving information from a database based on the analyzed intent and providing work instructions. This makes it possible for factory workers to obtain accurate instructions and information in real time by using voice input.

[1507] "Means for receiving user input" refers to devices or software used to incorporate information and commands provided by the user into the system.

[1508] A "natural language processing engine" is an algorithm and program that analyzes input text to understand its meaning and intent.

[1509] "Means for determining necessary processing based on analyzed intent" refers to the process by which a natural language processing engine selects the next action to take based on the results of its analysis.

[1510] "The means to carry out that process" refers to the system or device used to actually put the decided action into action.

[1511] "Means for returning processing results to the user" refer to communication means or display devices for providing feedback to the user regarding the results and information of the actions performed.

[1512] "Means for analyzing voice input and converting speech to text" refers to software or hardware that uses speech recognition technology to convert a user's voice into text information.

[1513] "Methods for retrieving information from a database" refers to the procedures for searching for and retrieving the required information from a database that stores the necessary data.

[1514] "Means for providing work instructions" refers to a mechanism for communicating specific work procedures and instructions to the user based on analysis and acquired information.

[1515] "Means of acquiring and proposing information related to store visit reservations to users" refers to the process of acquiring information such as available reservation dates, times, and locations, and presenting that information to the user.

[1516] "Means for saving reservation information" refers to a data management function that records reservation data selected by the user and allows it to be retrieved or updated as needed.

[1517] "Means of providing relevant business processes" refers to a system that provides the instructions and procedures necessary to perform specific tasks or procedures within a factory.

[1518] "Means of obtaining necessary document information for a procedure and informing the user" refers to a method of obtaining the documents and information necessary to perform a specific procedure and notifying the user of them.

[1519] "Means of obtaining and introducing online procedural information to users" refers to a function that collects information on procedures that can be performed via the internet and introduces it to users.

[1520] The present invention provides an advanced chatbot system for improving the work efficiency of factory workers, capable of providing practical work instructions and problem-solving based on user voice input. The overall configuration of this system is as follows.

[1521] Overall system configuration

[1522] The system of the present invention consists of the following elements.

[1523] 1. User terminal

[1524] These are devices accessed by workers using smart glasses, head-mounted displays, or PC microphones.

[1525] 2. Server

[1526] It is a central processing unit for performing speech recognition and natural language processing.

[1527] It has a built-in natural language processing engine (e.g., Google Cloud NLP) that converts voice input into text and analyzes the user's intent.

[1528] 3. Database

[1529] This system manages work instructions, assembly procedures, and information related to work procedures.

[1530] Processing flow

[1531] 1. Receiving voice input

[1532] The user terminal receives voice input spoken by the worker into smart glasses or a head-mounted display (e.g., "Tell me the assembly procedure for the next product").

[1533] 2. Speech Recognition and Text Conversion

[1534] The device sends the received audio to the server, where it uses speech recognition technology (e.g., the SpeechRecognition library) to convert the audio into text.

[1535] 3. Intent Analysis and Data Acquisition

[1536] The server analyzes the converted text using a natural language processing engine and queries the database for work instructions and necessary information.

[1537] 4. Providing work instructions

[1538] The server organizes the information retrieved from the database and provides work instructions to the user in text or voice.

[1539] For example, when providing instructions for assembling the following product, instructions such as "Step 1: Attach part A. Step 2: Tighten the screws..." are provided.

[1540] Hardware and software to use

[1541] 1. Hardware

[1542] Smart glasses, head-mounted displays, PC microphones, and other devices that allow users to access information.

[1543] Server (Central processing unit for performing advanced processing)

[1544] 2. Software

[1545] Speech recognition libraries (e.g., SpeechRecognition)

[1546] Natural language processing engine (e.g., Google Cloud NLP)

[1547] Database management system (management of work instructions and procedure information)

[1548] Specific example

[1549] In the factory, a new worker uses smart glasses to give voice instructions such as, "Tell me the assembly procedure for the next product." The server analyzes this voice, retrieves the procedure information from the database, displays it on the worker's screen, and also provides voice guidance.

[1550] Example of a prompt:

[1551] "Please tell me the assembly instructions for the following product."

[1552] This system allows factory workers to receive accurate instructions and information in real time by using voice input, thereby improving operational efficiency.

[1553] This invention is a system that highly analyzes voice input to improve work efficiency and provides necessary information and instructions in real time.

[1554] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1555] Step 1:

[1556] Users input voice commands through smart glasses or head-mounted displays.

[1557] As a concrete example, the user might say, "Please tell me the assembly instructions for the following product."

[1558] The input is the user's voice, and the output is audio data.

[1559] Step 2:

[1560] The device that receives the audio data sends that audio data to the server.

[1561] Specifically, the device sends audio data to the server via the network.

[1562] The input is audio data, and the output is data sent to the server.

[1563] Step 3:

[1564] The server converts the received audio data into text using speech recognition software (e.g., the SpeechRecognition library).

[1565] Specifically, the server starts the speech recognition engine and converts the speech data into text data.

[1566] The input is audio data, and the output is converted text data.

[1567] Step 4:

[1568] The server analyzes the converted text data using a natural language processing engine (e.g., Google Cloud NLP) to understand the user's intent.

[1569] In terms of specific operations, the server invokes a natural language processing engine to analyze the text data and extract the intended meaning.

[1570] The input is text data, and the output is parsed intent data.

[1571] Step 5:

[1572] Based on the analyzed intent, the server retrieves relevant information from the database.

[1573] In terms of specific actions, the server queries the database to retrieve the necessary work procedure information.

[1574] The input is intent data, and the output is work instruction information retrieved from the database.

[1575] Step 6:

[1576] The server organizes the acquired work instruction information and generates text or voice messages to respond to the user.

[1577] In terms of specific operations, the server formats the information into a format that is easy to transmit to the user and generates a message.

[1578] The input is work instruction information retrieved from the database, and the output is the generated message.

[1579] Step 7:

[1580] The server sends the generated message to the terminal, and the terminal notifies the user.

[1581] In terms of specific operations, the server sends a message to the terminal over the network, and the terminal displays instructions to the user via voice and text.

[1582] The input is the generated message, and the output is the notification to the user.

[1583] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1584] The system of the present invention is an advanced chatbot that combines intent analysis of input text and emotion recognition to enable smooth communication with users. The system of the present invention has the function of recognizing the user's emotions and responding appropriately according to those emotions.

[1585] Overall system configuration

[1586] The system consists of the following elements:

[1587] 1. User terminal

[1588] This refers to a device that users access using the LINE app or a web browser.

[1589] 2. LINE Server

[1590] Its role is to receive user input and forward it to the chatbot server.

[1591] 3. Chatbot Server

[1592] This is the central part of the system that analyzes user input and generates responses.

[1593] It incorporates a natural language processing engine and an emotion engine to analyze the user's intentions and emotions.

[1594] Access the database and perform read and write operations as needed.

[1595] 4. Database

[1596] It manages user information, reservation information, required documents for procedures, and billing information.

[1597] Processing flow

[1598] 1. Receive user input

[1599] The user types "I would like to make a reservation to visit the store" into the LINE app and sends it.

[1600] 2. Perform natural language processing and sentiment recognition.

[1601] The LINE server forwards the input to the chatbot server.

[1602] The chatbot server uses a natural language processing engine to analyze the intent of the input.

[1603] At the same time, an emotion engine is used to recognize the user's emotions.

[1604] 3. Determine the necessary processing based on the analysis results.

[1605] The chatbot server determines the necessary actions based on the analyzed intent and emotions.

[1606] For example, in the case of a store visit reservation, available dates and times are retrieved from the database.

[1607] 4. Return the processing result to the user.

[1608] The system then suggests available dates and times to the user.

[1609] 5. Process the user's response.

[1610] The user selects their preferred date and time and replies.

[1611] The chatbot server receives the requested date and time and saves it to the database. The reservation is then confirmed.

[1612] 6. Send a reservation confirmation notice to the user.

[1613] Generate a confirmation message and send it to the user.

[1614] Specific example

[1615] The following are examples of specific use cases.

[1616] 1. Make an appointment to visit the store.

[1617] If a user includes anxious expressions when making a reservation (for example, "I'm worried because it's my first time"), the emotion engine recognizes this as "anxiety."

[1618] The chatbot server offers detailed explanations and support options to alleviate anxiety. "Don't worry if this is your first time visiting. We'll send you detailed instructions."

[1619] 2. Information on the documents required for the procedure

[1620] When a user requests procedural information, if a message containing an emotion such as "It's urgent" is sent, the emotion engine recognizes the "urgency."

[1621] The chatbot server quickly provides the necessary document information and responds appropriately with phrases like, "Thank you for your urgency. The required documents are as follows."

[1622] 3. Inquiry regarding unclear points in the billing statement.

[1623] If a user enters "The explanation about the fees is unclear," the sentiment engine recognizes this as "confusion."

[1624] The chatbot server adds a detailed and easy-to-understand explanation: "We apologize for any confusion regarding the billing details. Here is a detailed breakdown."

[1625] Thus, the present invention realizes a chatbot system that provides appropriate responses according to the user's intentions and emotions by combining natural language processing and emotion recognition.

[1626] The following describes the processing flow.

[1627] Specific processing steps for making a store visit reservation

[1628] Step 1:

[1629] The user types "I want to make a reservation to visit the store" in the LINE app's chat screen and sends it.

[1630] Step 2:

[1631] The device sends the user's message to the LINE server. The message is received by the LINE server.

[1632] Step 3:

[1633] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[1634] Step 4:

[1635] The chatbot server uses a natural language processing engine to analyze the intent of messages. Simultaneously, it uses an emotion engine to analyze the user's emotions.

[1636] Step 5:

[1637] Based on the analysis results, the chatbot server recognizes the user's intention to make an appointment and any feelings of anxiety.

[1638] Step 6:

[1639] The chatbot server connects to the database and queries it to retrieve available dates and times for booking. The database then returns the available dates and times.

[1640] Step 7:

[1641] The chatbot server generates a message suggesting possible dates and times to the user, saying, "Please let us know your preferred date and time. Suggestions: June 12th, 10:00 AM, June 12th, 2:00 PM. We also offer support to ensure a smooth experience even for first-time visitors." This message is then sent to the user's device via the LINE server.

[1642] Step 8:

[1643] The user replies, "Please make it June 12th at 10:00."

[1644] Step 9:

[1645] The device sends the user's reply to the LINE server. The LINE server receives the message and forwards it to the chatbot server.

[1646] Step 10:

[1647] The chatbot server saves the date and time of the received message to the database and updates the reservation information.

[1648] Step 11:

[1649] The database stores the reservation information. The chatbot server generates a reservation confirmation message and sends it to the user's device via the LINE server, stating, "Your reservation has been confirmed. Please come to our store on June 12th at 10:00. Please feel free to contact us if you have any questions."

[1650] Specific processing steps for documents required for the procedure

[1651] Step 1:

[1652] The user types "Please tell me what documents are required for moving procedures" and submits the form.

[1653] Step 2:

[1654] The device sends the input to the LINE server. The message is received by the LINE server.

[1655] Step 3:

[1656] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[1657] Step 4:

[1658] The chatbot server uses a natural language processing engine to analyze the intent of messages. Simultaneously, it uses an emotion engine to analyze the user's emotions.

[1659] Step 5:

[1660] Based on the analysis results, the chatbot server recognizes the user's intention to initiate the moving process and their sense of urgency.

[1661] Step 6:

[1662] The chatbot server connects to the database and queries it to retrieve the necessary document information for the procedure. The database then returns the required document information.

[1663] Step 7:

[1664] The chatbot server generates a message informing the user of the necessary documents and sends it to the device via the LINE server, stating, "Thank you for your urgency. The following documents are required for the moving process: Resident registration certificate, utility bills, and identification."

[1665] Specific processing steps for inquiries regarding unclear points in billing statements

[1666] Step 1:

[1667] The user types, "This month's bill is high, please show me the details," and submits it.

[1668] Step 2:

[1669] The device sends the input to the LINE server. The message is received by the LINE server.

[1670] Step 3:

[1671] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[1672] Step 4:

[1673] The chatbot server uses a natural language processing engine to analyze intent and recognize that the user "wants to inquire about their billing details." At the same time, it uses an emotion engine to recognize the user's "confusion."

[1674] Step 5:

[1675] The chatbot server sends an SMS verification code to the user's registered phone number to verify their identity. It generates a message for the verification process and sends it to the device via the LINE server, stating, "We will now verify your identity. An SMS code has been sent to your registered phone number."

[1676] Step 6:

[1677] The user enters the code "123456" received via SMS into the LINE chat and sends it.

[1678] Step 7:

[1679] The device sends the input back to the LINE server. The message is received by the LINE server.

[1680] Step 8:

[1681] The LINE server forwards the entered SMS code to the chatbot server. The chatbot server verifies the code, and if it is correct, completes the identity verification process.

[1682] Step 9:

[1683] The chatbot server queries the database to retrieve the billing details. The database then returns the billing details information.

[1684] Step 10:

[1685] The chatbot server generates a message saying, "This month's billing details are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen," adds a clear explanation, and sends it to the user's terminal via the LINE server, saying, "We apologize for any confusion regarding the billing details. Here is a detailed breakdown."

[1686] As described above, the system of the present invention recognizes the user's intentions and emotions and provides an appropriate response.

[1687] (Example 2)

[1688] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1689] Conventional chatbot systems are limited to analyzing the intent behind user input, making it difficult to respond in a way that takes user emotions into account. As a result, responses to users are uniform, and communication is often not smooth. Furthermore, there are delays in providing information about specific processes and responding to users' anxieties and urgency, which leads to decreased user satisfaction. This invention aims to solve these problems.

[1690] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1691] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for recognizing the user's emotions using an emotion engine, means for determining and executing necessary processing based on the analyzed intent and emotions, and means for returning the processing results to the user. This enables an appropriate response that takes into account both the user's intent and emotions.

[1692] "Means for receiving user input" refers to the function of obtaining text messages and commands sent from the terminal used by the user.

[1693] A "natural language processing engine" is a software module that analyzes text entered by a user and understands its intent and meaning.

[1694] An "emotion engine" is a software module that identifies emotions and psychological states from user input and determines appropriate responses based on that information.

[1695] "Means for determining necessary processing based on analyzed intent and emotion" refers to a function that selects the next action or response based on the analysis results of the natural language processing engine and the emotion engine.

[1696] "Means of responding to the user with processing results" refers to a function that generates the determined response or action as a text message and sends it to the user.

[1697] "A means of obtaining and proposing information related to store visit reservations to the user" refers to a function that, when a user requests a store visit reservation, retrieves available dates and times and related information from a database and presents it to the user.

[1698] "A means of receiving a user's requested reservation date and time and saving the reservation information" refers to a function that receives the reservation date and time selected by the user and saves it in a database.

[1699] "Means of notifying users of confirmed reservation information" refers to a function that generates and sends a message to inform users of a confirmed reservation.

[1700] "A means of obtaining necessary document information for procedures and guiding users through it" refers to a function that retrieves the necessary documents and information for various procedures from a database and guides users through them appropriately.

[1701] "A means of determining processing priorities based on user emotions" refers to a function that determines the order and urgency of responses according to the user's psychological state, as analyzed by the emotion engine.

[1702] "A means of obtaining and introducing online procedural information to users" refers to a function that searches for and obtains information on procedures and services that users can perform online, and provides that information to users.

[1703] Modes for carrying out the invention

[1704] The system of the present invention is an advanced chatbot that combines intent analysis of input text and sentiment recognition to enable smooth communication with users. The specific configuration and operation of the system of the present invention are described below.

[1705] Overall system configuration

[1706] The system consists of the following elements:

[1707] 1. User terminal

[1708] These are devices such as smartphones and PCs, and users access them using the LINE app or a web browser.

[1709] 2. LINE Server

[1710] Its role is to receive user input and forward it to the chatbot server.

[1711] 3. Chatbot Server

[1712] This is the central part of the system that analyzes user input and generates responses.

[1713] It incorporates a natural language processing engine (e.g., Google Cloud Natural Language API) and an emotion engine (e.g., Microsoft Azure Text Analytics API) to analyze user intent and sentiment.

[1714] Access and read / write data to a database (e.g., MySQL) as needed.

[1715] 4. Database

[1716] It manages user information, reservation information, required documents for procedures, and billing information.

[1717] Program processing

[1718] 1. Receive user input

[1719] The user types "I want to make a reservation to visit the store" into the LINE app and sends it. The user's device sends this message to the LINE server.

[1720] 2. Perform natural language processing and sentiment recognition.

[1721] The server receives user input via the LINE server and forwards it to the chatbot server in JSON format. The chatbot server calls the Google Cloud Natural Language API to analyze the intent of the message. Simultaneously, it uses the Microsoft Azure Text Analytics API to recognize the user's sentiment.

[1722] 3. Determine the necessary processing based on the analysis results.

[1723] The chatbot server determines the next action to take based on the analysis results. For example, in the case of a store visit reservation, it generates a query to retrieve available dates and times from the database.

[1724] 4. Return the processing result to the user.

[1725] The chatbot server proposes available dates and times to the user. The message is generated in a specific format and sent to the user via the LINE server.

[1726] 5. Process the user's response.

[1727] The user selects their preferred date and time and replies. The chatbot server receives this selection and executes an SQL query to save it to the database.

[1728] 6. Send a reservation confirmation notice to the user.

[1729] The chatbot server generates a reservation confirmation message and sends it to the user: "Your reservation is confirmed. We look forward to seeing you on [Month] [Day] at [Time]."

[1730] Specific example

[1731] The following are examples of specific use cases.

[1732] 1. Make an appointment to visit the store.

[1733] If a user includes an anxious expression when making a reservation (for example, "I'm worried because it's my first time"), the emotion engine recognizes this anxiety. The chatbot server then offers detailed explanations and support options to alleviate the anxiety. For example, "Don't worry if it's your first time. We'll send you detailed instructions."

[1734] 2. Information on the documents required for the procedure

[1735] When a user requests procedural information and sends a message that includes an emotion such as "It's urgent," the emotion engine recognizes the "urgency." The chatbot server quickly provides the necessary document information and gives an appropriate response such as, "Thank you for your urgency. The required documents are as follows."

[1736] 3. Inquiry regarding unclear points in the billing statement.

[1737] If a user types "The explanation about the charges is unclear," the sentiment engine recognizes this as "confusion." The chatbot server then adds a more detailed and clearer explanation: "We apologize for the confusion regarding the charges. Here is a detailed breakdown."

[1738] Thus, the present invention realizes a chatbot system that provides appropriate responses according to the user's intentions and emotions by combining natural language processing and emotion recognition.

[1739] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1740] Step 1:

[1741] Receive user input

[1742] explanation:

[1743] The user types "I would like to make a reservation to visit the store" into the LINE app and sends it.

[1744] The device will send this message to the LINE server.

[1745] Specific actions:

[1746] The user types the message "I would like to make a reservation to visit" on the LINE app on their smartphone and presses the send button.

[1747] Input: User's text message

[1748] Output: Message forwarded to LINE server

[1749] Step 2:

[1750] Performing natural language processing and emotion recognition.

[1751] explanation:

[1752] The server forwards messages received from the LINE server to the chatbot server.

[1753] The chatbot server analyzes the intent of messages using a natural language processing engine (e.g., Google Cloud Natural Language API).

[1754] Use an emotion engine (e.g., Microsoft Azure Text Analytics API) to recognize the user's emotions.

[1755] Specific actions:

[1756] The server receives the user's message via the LINE server and forwards it to the chatbot server in JSON format.

[1757] The chatbot server sends a request to the Google Cloud Natural Language API to analyze the intent behind the "store visit reservation."

[1758] Simultaneously, a request is sent to the Microsoft Azure Text Analytics API to analyze the emotions (e.g., anxiety) contained in the message.

[1759] Input: Message forwarded from LINE server

[1760] Output: Analyzed intentions and emotions

[1761] Step 3:

[1762] Based on the analysis results, determine the necessary processing steps.

[1763] explanation:

[1764] Based on the analysis results, the chatbot server determines the next action to take.

[1765] For example, in the case of a store visit reservation, available dates and times are retrieved from a database (e.g., MySQL).

[1766] Specific actions:

[1767] The chatbot server generates database queries based on the analysis results.

[1768] The server queries the MySQL database to retrieve available dates and times for booking.

[1769] Input: Analyzed intentions and emotions

[1770] Output: Database query results including available dates and times for booking

[1771] Step 4:

[1772] Return the processing result to the user.

[1773] explanation:

[1774] The chatbot server then suggests available dates and times to the user.

[1775] The generated text message is sent to the LINE server as a suggestion.

[1776] Specific actions:

[1777] The chatbot server generates a suggestion message in a specific format and sends that message to the LINE server.

[1778] The LINE server forwards the message to the user's device.

[1779] Input: Database query results including available dates and times for booking

[1780] Output: Suggestion message sent to the user

[1781] Step 5:

[1782] Processing user responses

[1783] explanation:

[1784] The user selects their preferred date and time and replies.

[1785] The chatbot server receives the reply and saves the desired date and time to its database.

[1786] Specific actions:

[1787] The user sends a message saying, "I would like to meet on [Month] [Day] at [Time]."

[1788] The chatbot server receives this message and executes an SQL query to save the desired date and time to the database.

[1789] Input: User's reply message

[1790] Output: Reservation information stored in the database

[1791] Step 6:

[1792] Send a reservation confirmation notification to the user.

[1793] explanation:

[1794] The chatbot server generates and sends a message to the user notifying them that the reservation has been confirmed.

[1795] Specific actions:

[1796] The chatbot server generates the message, "Your reservation has been confirmed. We look forward to seeing you on [Month] [Day] at [Time]."

[1797] The message is sent to the user's device via the LINE server.

[1798] Input: Reservation information stored in the database

[1799] Output: Booking confirmation message sent to the user

[1800] (Application Example 2)

[1801] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1802] Traditional chatbot systems are limited to analyzing user intent and have the problem of not being able to provide appropriate responses that take into account user emotions. This degrades the user experience, and in particular, in ride reservation systems, support may be insufficient when passengers are feeling anxious or stressed. There is a need for a system that can smoothly respond to passengers when they need changes or urgent assistance.

[1803] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1804] In this invention, the server includes means for receiving user input, means for analyzing the intent and emotion of the input using a natural language processing engine and an emotion recognition system, means for determining and executing necessary processing based on the analyzed intent and emotion, and means for returning the processing results to the user. This enables appropriate responses that address not only the user's intent but also their emotions, allowing for quick and appropriate responses to requests such as ride reservations and route changes.

[1805] "User" refers to an individual or group that uses the system.

[1806] "Input" refers to the information that a user sends to the system.

[1807] A "natural language processing engine" is a computer program that analyzes user input to understand their intent.

[1808] An "emotion recognition system" is a computer program that analyzes emotions from user input.

[1809] "Intention" refers to the purpose or request that the user is trying to convey through their input.

[1810] "Emotions" refer to the psychological state that a user expresses through their input.

[1811] "Processing" refers to the actions the system takes based on the analyzed intentions and emotions.

[1812] "Response" refers to the answer that the system provides to the user.

[1813] "Ride reservation" refers to the act of a user reserving a vehicle for a specific date and time.

[1814] "Route change" refers to a request from a user to change their existing travel route.

[1815] A "support center" refers to an organization that provides assistance to users when they encounter problems or have questions.

[1816] "Dialogue" refers to the communication that takes place between a system and a user.

[1817] "Mobility experience" refers to the overall experience a user has when using an autonomous vehicle.

[1818] The system for implementing this invention is an advanced chatbot system aimed at improving the user's riding experience. This system consists of the following main components:

[1819] Overall system configuration

[1820] The system consists of the following four elements:

[1821] 1. User terminal

[1822] A user terminal refers to a mobile device such as a smartphone, which users use to access the system through applications. Users can make reservations and change routes through text input.

[1823] 2. Communication Server

[1824] This server is responsible for receiving user input in real time and forwarding it to the chatbot server. The communication server is crucial for ensuring stable network communication.

[1825] 3. Chatbot Server

[1826] The chatbot server is the heart of the system, handling all data processing and response generation. It includes the following software components:

[1827] Natural language processing engine (e.g., TextBlob): Analyzes user input text and understands intent.

[1828] Emotion recognition systems (e.g., emotion analysis using TextBlob): Analyze emotions from user input and adjust responses accordingly.

[1829] 4. Database Server

[1830] The database server manages the following information:

[1831] User Information

[1832] Reservation Information

[1833] Route information

[1834] Past conversation history

[1835] Explanation of the process

[1836] The system processes the data as follows:

[1837] Receiving user input

[1838] Text input is sent from the user's terminal and reaches the chatbot server via the communication server.

[1839] Natural language processing and sentiment analysis

[1840] The chatbot server uses a natural language processing engine to analyze the user's intent and an emotion recognition system to evaluate their emotions. For example, if a user types "I want to make a reservation," the intent is analyzed as "reservation" and the emotion as "positive."

[1841] Decision and execution of processing

[1842] Based on the analyzed intent and emotions, the chatbot server determines the appropriate action. For example, if a reservation request is made, it retrieves available dates and times from the database and generates suggestions.

[1843] Response to the user

[1844] The system returns the processing results to the user. When suggesting available dates and times, if the user expresses concerns, it will respond in an empathetic manner, such as, "It seems you have concerns about your preferred date and time. Please feel free to contact us."

[1845] Examples of specific user interactions

[1846] The following is an example of a specific interaction between the user and the system:

[1847] User input: "I would like to make a reservation."

[1848] System response: "The following time slots are available for booking: 2023-11-01 10:00, 2023-11-01 14:00, 2023-11-01 18:00"

[1849] The system constantly analyzes user emotions and generates appropriate responses, allowing users to use the system with peace of mind.

[1850] Software and hardware used

[1851] Software: TextBlob (natural language processing and sentiment analysis), requests module (HTTP requests)

[1852] Hardware: Smartphones (user terminals), servers (communication servers and chatbot servers)

[1853] These components allow users to book and use autonomous vehicles intuitively and with confidence.

[1854] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1855] Step 1:

[1856] Receiving user input

[1857] The user terminal receives input (text messages) from the user. This input includes requests such as reservations and route changes. The received input is sent to the communication server.

[1858] Input: The user enters a message such as "I would like to make a reservation."

[1859] Processing: The user terminal sends input to the communication server.

[1860] Output: Communication server receives input message

[1861] Step 2:

[1862] Intent analysis using natural language processing

[1863] The communication server forwards the received user input to the chatbot server. The chatbot server uses a natural language processing engine to analyze the intent of the user's input.

[1864] Input: Input message forwarded from the communication server

[1865] Processing: A natural language processing engine analyzes the intent of the input (e.g., "request for reservation").

[1866] Output: Analyzed intent (e.g., "reservation")

[1867] Step 3:

[1868] Emotion analysis using an emotion recognition system

[1869] The chatbot server uses an emotion recognition system to analyze the emotions from the user's input.

[1870] Input: Input message forwarded from the communication server

[1871] Processing: The emotion recognition system analyzes the emotion of the input (e.g., "positive").

[1872] Output: Analyzed emotion (e.g., "positive")

[1873] Step 4:

[1874] Decision and execution of processing

[1875] The chatbot server determines the necessary actions based on the analyzed intent and emotion. For example, in the case of a reservation request, it retrieves available dates and times from the database.

[1876] Input: Analyzed intentions and emotions (e.g., "reservation", "positive")

[1877] Process: Retrieve available date and time information from the database.

[1878] Output: Acquired reservation date and time information (e.g., "2023-11-01 10:00")

[1879] Step 5:

[1880] Generating responses to users

[1881] The chatbot server generates a response to the user based on the processing results. This response should include consideration for the user's emotions.

[1882] Input: Acquired reservation date and time information and analyzed sentiment (e.g., "2023-11-01 10:00", "Positive")

[1883] Process: Generate a response (Example: "The available time slots are as follows: 2023-11-01 10:00")

[1884] Output: Reply message to the user (Example: "The available time slots are as follows: 2023-11-01 10:00")

[1885] Step 6:

[1886] Sending a reply to the user

[1887] The chatbot server sends the generated response message to the user's terminal via the communication server. The user can then review the response and take the next action.

[1888] Input: Generated response message (Example: "The following time slots are available for booking: 2023-11-01 10:00")

[1889] Processing: Send a reply message to the user terminal via the communication server.

[1890] Output: Reply message displayed on the user's terminal

[1891] The above outlines the specific processing flow of the chatbot in the autonomous vehicle reservation system. By performing appropriate data processing and calculations at each step, it is possible to provide users with quick and appropriate responses.

[1892] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1893] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1894] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1895] [Fourth Embodiment]

[1896] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1897] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1898] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1899] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1900] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1901] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1902] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1903] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1904] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1905] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1906] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1907] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1908] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1909] The system of the present invention is an advanced chatbot for streamlining customer service and can perform various practical procedures based on user input. The system of the present invention is realized through a series of processes that receive user input, analyze it, and generate an appropriate response.

[1910] Overall system configuration

[1911] The system is broadly composed of the following elements:

[1912] 1. User terminal

[1913] This refers to a device that users access using the LINE app or a web browser.

[1914] 2. LINE Server

[1915] Its role is to receive user input and forward it to the chatbot server.

[1916] 3. Chatbot Server

[1917] It is the central part of the system that analyzes user input and generates appropriate responses.

[1918] It has a built-in natural language processing engine that analyzes the user's intent.

[1919] Access, read from, and write to the database as needed.

[1920] 4. Database

[1921] This is a supplementary system for managing user information, reservation information, and necessary document information for procedures.

[1922] Processing flow

[1923] 1. Receive user input

[1924] For example, a user might type "I want to make a reservation to visit the store" in the chat screen of the LINE app.

[1925] The device sends this input to the LINE server.

[1926] 2. Analyze user intent using a natural language processing engine.

[1927] The LINE server receives the input and forwards it to the chatbot server.

[1928] The chatbot server uses a natural language processing engine to analyze the intent of the input and recognizes that the user "wants to make a reservation to visit the store."

[1929] 3. Executing processing in response to user requests

[1930] For store visit reservations, the chatbot server retrieves available dates and times from the database.

[1931] The database returns available dates and times, and the chatbot server suggests possible dates and times to the user.

[1932] 4. Process the user's response.

[1933] The user selects their preferred date and time and replies in the chat, "Please make it June 12th at 10:00."

[1934] The chatbot server then analyzes this again, saves the desired date and time to the database, and confirms the reservation.

[1935] 5. Send a confirmation notice to the user.

[1936] The chatbot server generates the message "Your reservation has been confirmed. Please come to the store on June 12th at 10:00" and sends it to the user's device via the LINE server.

[1937] Specific example

[1938] The following are some specific use cases.

[1939] 1. Information on the documents required for the procedure

[1940] The user's terminal sends the message, "Please tell me what documents are required for the moving procedure."

[1941] The chatbot server uses a natural language processing engine to analyze intent.

[1942] The system queries a database for information on "documents required for moving procedures" and retrieves the results.

[1943] The chatbot server generates a response message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the user's terminal.

[1944] 2. Introduction to online procedures

[1945] A message is sent to the user's terminal saying, "Please tell me what procedures I can do online."

[1946] The chatbot server receives this input and retrieves the appropriate online procedure information from its database.

[1947] The response message reads: "The following procedures can be done online: change of address, review contract details, and review billing statements. You can find more details at this link [link]."

[1948] 3. Inquiry regarding unclear points in the billing statement.

[1949] The user enters, "My bill this month is high, please show me the details."

[1950] The chatbot server requests identity verification and sends an SMS authentication code to the user's device.

[1951] The user enters an SMS code into the chat, and the chatbot server verifies the code.

[1952] The chatbot server retrieves the billing details from the database and sends a response message saying, "Your billing details for this month are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen."

[1953] In this way, the present invention realizes a system that provides a quick and effective response to the diverse demands of users.

[1954] The following describes the processing flow.

[1955] Step 1:

[1956] The user types "I want to make a reservation to visit" in the LINE app's chat screen. The user then sends this message.

[1957] Step 2:

[1958] The device sends user input to the LINE server. The message is received by the LINE server.

[1959] Step 3:

[1960] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[1961] Step 4:

[1962] The chatbot server uses a natural language processing engine to analyze the intent of incoming messages. The natural language processing engine identifies the intent as "I would like to make a reservation to visit the store."

[1963] Step 5:

[1964] The chatbot server queries the database for available dates and times and retrieves them. The database then sends the available dates and times back to the chatbot server.

[1965] Step 6:

[1966] The chatbot server generates a message for the user saying, "Please tell us your preferred date and time. Suggestions: June 12th, 10:00 AM, June 12th, 2:00 PM," and sends it to the user's device via the LINE server.

[1967] Step 7:

[1968] The user enters their desired date and time, such as "June 12th, 10:00 AM, please," on their device and sends it to the LINE server.

[1969] Step 8:

[1970] The LINE server receives input from the user and forwards it to the chatbot server. The chatbot server receives the message for the requested date and time.

[1971] Step 9:

[1972] The chatbot server saves the date and time of the received message to the database and updates the reservation information. The database stores the reservation information.

[1973] Step 10:

[1974] The chatbot server generates a reservation confirmation message saying, "Your reservation has been confirmed. Please come to the store on June 12th at 10:00," and sends it to the user's device via the LINE server.

[1975] The following are example steps for obtaining the necessary documents and for inquiring about any unclear points regarding the fee breakdown.

[1976] Instructions on the documents required for the procedure

[1977] Step 1:

[1978] The user uses their device to type "Please tell me what documents are required for moving procedures" and then sends the message.

[1979] Step 2:

[1980] The device sends input to the LINE server. The message is received by the LINE server.

[1981] Step 3:

[1982] The LINE server forwards the incoming message to the chatbot server. The chatbot server receives the message.

[1983] Step 4:

[1984] The chatbot server uses a natural language processing engine to analyze the intent of the input. The natural language processing engine identifies the intent as "I want to know what documents are needed for moving procedures."

[1985] Step 5:

[1986] The chatbot server queries the database for relevant document information and retrieves it. The database then sends the relevant information back to the chatbot server.

[1987] Step 6:

[1988] Based on the information it has acquired, the chatbot server generates a message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the device via the LINE server.

[1989] Inquiry regarding unclear points in the billing statement

[1990] Step 1:

[1991] The user types, "My bill this month is high, please show me the details," and sends it via the LINE app.

[1992] Step 2:

[1993] The device sends the input to the LINE server. The message is received by the LINE server.

[1994] Step 3:

[1995] The LINE server forwards the incoming message to the chatbot server. The chatbot server receives the message.

[1996] Step 4:

[1997] The chatbot server uses a natural language processing engine to analyze the user's intent and recognizes that they "want to inquire about their billing details."

[1998] Step 5:

[1999] The chatbot server sends an SMS verification code to the user's registered phone number to verify their identity and generates a message for the verification process. It then sends the message, "We are verifying your identity. An SMS code has been sent to your registered phone number," to the user's device via the LINE server.

[2000] Step 6:

[2001] The user enters the code "123456" received via SMS into the LINE chat and sends it.

[2002] Step 7:

[2003] The device sends the input back to the LINE server. The message is received by the LINE server.

[2004] Step 8:

[2005] The LINE server forwards the entered SMS code to the chatbot server. The chatbot server verifies the code, and if it is correct, completes the identity verification process.

[2006] Step 9:

[2007] The chatbot server queries the database to retrieve the billing details. The database then sends the billing details information back to the chatbot server.

[2008] Step 10:

[2009] The chatbot server generates the message "This month's charges are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage fee 1000 yen" and sends it to the user's terminal via the LINE server.

[2010] (Example 1)

[2011] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2012] In today's business environment, providing prompt and effective customer service is essential. However, many companies still rely on manual processes and inefficient systems, leading to wasted resources and decreased customer satisfaction. Furthermore, while rapid and accurate information provision is crucial, particularly for appointment scheduling and procedural guidance, integrated and automated systems to achieve this are lacking. Additionally, efficient online identity verification processes are inadequate. These challenges need to be addressed.

[2013] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[2014] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for determining necessary processing based on the analyzed intent and executing that processing, and means for returning the processing results to the user. This makes it possible to automatically analyze user input and generate an appropriate response.

[2015] Furthermore, the server includes means for acquiring information related to store visit reservations and proposing them to the user, means for receiving the user's preferred reservation date and time and saving the reservation information, and means for confirming and notifying the reservation in a conversational format via a chat server. This enables efficient store visit reservations and their confirmation and notification.

[2016] Furthermore, the server includes means for obtaining necessary document information for procedures and guiding users through them, means for obtaining online procedure information and presenting it to users, and means for receiving user authentication information and verifying their identity. This automates the procedure guidance and identity verification processes, significantly improving user convenience.

[2017] "Means for receiving user input" refers to the interface or protocol that allows a server to receive text data or voice data entered by a user through a terminal.

[2018] A "natural language processing engine" refers to algorithms and technologies that analyze user input and understand its intent and content, and examples include machine learning models and rule-based processing systems.

[2019] "Means for determining necessary processing based on analyzed intent and executing that processing" refers to software and hardware that determine and execute specific actions according to the user's intent analyzed by a natural language processing engine.

[2020] "Means of returning processing results to the user" refers to methods or devices for notifying the user of the results of processing performed by the server, such as a message generation engine or a communication interface.

[2021] "Means of obtaining and proposing information related to store visit reservations to users" refers to systems and methods for obtaining information related to store visit reservations from databases or APIs and presenting that information to users.

[2022] "Means for receiving reservation date and time requests from users and saving reservation information" refers to interfaces or protocols for receiving reservation date and time specified by users and saving them to a database.

[2023] "A means of confirming and notifying reservations interactively via a chat server" refers to chat software and related infrastructure that allows users and servers to exchange information interactively, confirm the final reservation, and notify the user of the result.

[2024] "Means of obtaining and guiding users to the documents required for a procedure" refers to applications or systems that obtain information about the documents required for a procedure from databases or other sources and guide users to that information.

[2025] "Means of obtaining and introducing online procedural information to users" refers to methods and systems for obtaining information on services and procedures offered online and introducing them to users.

[2026] "Means of receiving user authentication information and verifying identity" refers to systems and methods for receiving authentication information provided by users (e.g., passwords or SMS codes) and using that information to verify the user's identity.

[2027] The system of the present invention is an advanced chatbot for streamlining customer service and can perform various practical procedures based on user input. This system is realized through a series of processes that receive user input, analyze it, and generate an appropriate response. A specific embodiment of this system is described below.

[2028] System Configuration

[2029] This system is broadly composed of the following elements.

[2030] 1. User terminal

[2031] This is a device that users access using the LINE app or a web browser. It has the function of receiving user input and sending it to the server.

[2032] 2. LINE Server

[2033] Its role is to receive user input and forward it to the chatbot server. It also handles protocol conversion and temporary storage of input data.

[2034] 3. Chatbot Server

[2035] This is the core of the system, analyzing user input and generating appropriate responses. Specifically, it incorporates a natural language processing engine to analyze user intent. It utilizes APIs such as the Google NLP API as its natural language processing engine. It also accesses databases as needed to read and write information.

[2036] 4. Database

[2037] This is an auxiliary system for managing user information, reservation information, and necessary document information for procedures. For example, it uses a PostgreSQL database.

[2038] Program processing

[2039] 1. Receive user input

[2040] For example, a user might type "I want to make a reservation to visit the store" in the chat screen of the LINE app.

[2041] The device sends this input to the LINE server.

[2042] The LINE server forwards the received data to the chatbot server.

[2043] 2. Analyze user intent using a natural language processing engine.

[2044] The chatbot server uses a natural language processing engine to analyze the intent of the input and recognizes that the user "wants to make a reservation to visit the store."

[2045] 3. Executing processing in response to user requests

[2046] For store visit reservations, the chatbot server retrieves available dates and times from the database. For example, it might send an SQL query to a PostgreSQL database to retrieve the data.

[2047] The database returns available dates and times, and the chatbot server suggests possible dates and times to the user.

[2048] 4. Process the user's response.

[2049] The user selects their preferred date and time and replies via chat, for example, "June 12th at 10:00 AM, please."

[2050] The chatbot server then analyzes this again, saves the desired date and time to the database, and confirms the reservation.

[2051] 5. Send a confirmation notice to the user.

[2052] The chatbot server generates the message "Your reservation has been confirmed. Please come to the store on June 12th at 10:00" and sends it to the user's device via the LINE server.

[2053] Specific example

[2054] The following are some specific use cases.

[2055] 1. Information on the documents required for the procedure

[2056] The user's terminal sends the message, "Please tell me what documents are required for the moving procedure."

[2057] The chatbot server uses a natural language processing engine to analyze intent.

[2058] The system queries a database for information on "documents required for moving procedures" and retrieves the results.

[2059] The chatbot server generates a response message saying, "The following documents are required for moving procedures: resident registration certificate, utility bills, and identification," and sends it to the user's terminal.

[2060] 2. Introduction to online procedures

[2061] A message is sent to the user's terminal saying, "Please tell me what procedures I can do online."

[2062] The chatbot server receives this input and retrieves the appropriate online procedure information from its database.

[2063] The response message reads: "The following procedures can be done online: change of address, review contract details, and review billing statements. You can find more details at this link [link]."

[2064] 3. Inquiry regarding unclear points in the billing statement.

[2065] The user enters, "My bill this month is high, please show me the details."

[2066] The chatbot server requests identity verification and sends an SMS authentication code to the user's device.

[2067] The user enters an SMS code into the chat, and the chatbot server verifies the code.

[2068] The chatbot server retrieves the billing details from the database and sends a response message saying, "Your billing details for this month are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen."

[2069] Example of a prompt

[2070] 1. If the user wishes to make an appointment:

[2071] User: I want to make an appointment to visit the store.

[2072] System: Your appointment request has been received. Available dates and times are as follows: June 10th, 2:00 PM and June 12th, 10:00 AM. Please select your preferred date and time.

[2073] User: Please make it June 12th at 10:00.

[2074] System: Your reservation has been confirmed. Please come to the store on June 12th at 10:00.

[2075] 2. When the user asks about the documents required for the procedure:

[2076] User: What documents are required for moving procedures?

[2077] System: The following documents are required for the moving process: Certificate of residence, utility bills, and identification.

[2078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[2079] Step 1:

[2080] Receive user input

[2081] The user types "I would like to make a reservation to visit the store" in the chat screen of the LINE app.

[2082] Input: Text data sent by the user via the LINE app.

[2083] The device sends input data to the LINE server via the LINE API. The data is transmitted encrypted.

[2084] The LINE server receives this input data and forwards it to the chatbot server using a specific protocol.

[2085] Output: User input data transferred to the chatbot server.

[2086] Step 2:

[2087] Analyze user intent using a natural language processing engine.

[2088] The chatbot server analyzes the received data.

[2089] Input: User input data transferred to the chatbot server.

[2090] The chatbot server calls a natural language processing engine such as the Google NLP API to analyze the input. Specifically, it extracts the intention "to make a store visit reservation" from the input "I want to make a store visit reservation."

[2091] Output: The analyzed user intent and related information are generated in JSON format.

[2092] Step 3:

[2093] Execute processing in response to user requests.

[2094] The chatbot server processes user requests based on the analysis results.

[2095] Input: Analyzed user intent and related information.

[2096] For example, in the case of a store visit reservation, the chatbot server connects to the database and retrieves information on available dates and times.

[2097] The chatbot server generates SQL queries and sends them to a database (e.g., PostgreSQL) to retrieve data.

[2098] The database searches for available date and time slots and sends the results back to the chatbot server.

[2099] Output: Retrieved reservation availability information.

[2100] Step 4:

[2101] Suggest possible dates and times to the user.

[2102] The chatbot server generates a message to notify the user of the available reservation date and time information it has retrieved.

[2103] Input: Retrieved information on available reservation dates and times.

[2104] The chatbot server generates a message and sends it to the LINE server.

[2105] The LINE server delivers notification messages to the user's device.

[2106] Output: The message reads: "The following dates and times are available for booking: June 10th at 2:00 PM, June 12th at 10:00 AM. Please choose your preferred date and time."

[2107] Step 5:

[2108] Processing user responses

[2109] The user selects their preferred date and time and replies, "Please make it June 12th at 10:00."

[2110] Input: The user's preferred date and time.

[2111] The user's terminal sends this response back to the LINE server.

[2112] The LINE server forwards the received response to the chatbot server.

[2113] The chatbot server then uses its natural language processing engine again to analyze the response and extract the user's preferred date and time.

[2114] Input: A response message containing the user's preferred date and time.

[2115] The chatbot server generates an SQL query to save the desired date and time to the database and sends it to the database.

[2116] The database stores the reservation information and sends the results back to the chatbot server.

[2117] Output: The desired date and time have been saved in the database.

[2118] Step 6:

[2119] Send a confirmation notice to the user.

[2120] The chatbot server generates the message, "Your reservation has been confirmed. Please come to the store on June 12th at 10:00."

[2121] Input: Desired date and time information stored in the database.

[2122] The chatbot server sends that message to the LINE server.

[2123] The LINE server delivers the message to the user's device.

[2124] The user receives a confirmation message.

[2125] Output: Message: "Your reservation has been confirmed. Please come to the store on June 12th at 10:00."

[2126] (Application Example 1)

[2127] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2128] To improve the work efficiency of factory workers, there is a need for a system that can provide work instructions and solve problems in real time. In particular, intuitive voice-based operation and rapid information delivery are required. However, conventional systems have difficulty analyzing voice input and making real-time decisions, which hinders efforts to improve work efficiency.

[2129] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[2130] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for determining necessary processing based on the analyzed intent and executing that processing, means for returning the processing results to the user, means for analyzing voice input and converting the voice to text, and means for retrieving information from a database based on the analyzed intent and providing work instructions. This makes it possible for factory workers to obtain accurate instructions and information in real time by using voice input.

[2131] "Means for receiving user input" refers to devices or software used to incorporate information and commands provided by the user into the system.

[2132] A "natural language processing engine" is an algorithm and program that analyzes input text to understand its meaning and intent.

[2133] "Means for determining necessary processing based on analyzed intent" refers to the process by which a natural language processing engine selects the next action to take based on the results of its analysis.

[2134] "The means to carry out that process" refers to the system or device used to actually put the decided action into action.

[2135] "Means for returning processing results to the user" refer to communication means or display devices for providing feedback to the user regarding the results and information of the actions performed.

[2136] "Means for analyzing voice input and converting speech to text" refers to software or hardware that uses speech recognition technology to convert a user's voice into text information.

[2137] "Methods for retrieving information from a database" refers to the procedures for searching for and retrieving the required information from a database that stores the necessary data.

[2138] "Means for providing work instructions" refers to a mechanism for communicating specific work procedures and instructions to the user based on analysis and acquired information.

[2139] "Means of acquiring and proposing information related to store visit reservations to users" refers to the process of acquiring information such as available reservation dates, times, and locations, and presenting that information to the user.

[2140] "Means for saving reservation information" refers to a data management function that records reservation data selected by the user and allows it to be retrieved or updated as needed.

[2141] "Means of providing relevant business processes" refers to a system that provides the instructions and procedures necessary to perform specific tasks or procedures within a factory.

[2142] "Means of obtaining necessary document information for a procedure and informing the user" refers to a method of obtaining the documents and information necessary to perform a specific procedure and notifying the user of them.

[2143] "Means of obtaining and introducing online procedural information to users" refers to a function that collects information on procedures that can be performed via the internet and introduces it to users.

[2144] The present invention provides an advanced chatbot system for improving the work efficiency of factory workers, capable of providing practical work instructions and problem-solving based on user voice input. The overall configuration of this system is as follows.

[2145] Overall system configuration

[2146] The system of the present invention consists of the following elements.

[2147] 1. User terminal

[2148] These are devices accessed by workers using smart glasses, head-mounted displays, or PC microphones.

[2149] 2. Server

[2150] It is a central processing unit for performing speech recognition and natural language processing.

[2151] It has a built-in natural language processing engine (e.g., Google Cloud NLP) that converts voice input into text and analyzes the user's intent.

[2152] 3. Database

[2153] This system manages work instructions, assembly procedures, and information related to work procedures.

[2154] Processing flow

[2155] 1. Receiving voice input

[2156] The user terminal receives voice input spoken by the worker into smart glasses or a head-mounted display (e.g., "Tell me the assembly procedure for the next product").

[2157] 2. Speech Recognition and Text Conversion

[2158] The device sends the received audio to the server, where it uses speech recognition technology (e.g., the SpeechRecognition library) to convert the audio into text.

[2159] 3. Intent Analysis and Data Acquisition

[2160] The server analyzes the converted text using a natural language processing engine and queries the database for work instructions and necessary information.

[2161] 4. Providing work instructions

[2162] The server organizes the information retrieved from the database and provides work instructions to the user in text or voice.

[2163] For example, when providing instructions for assembling the following product, instructions such as "Step 1: Attach part A. Step 2: Tighten the screws..." are provided.

[2164] Hardware and software to use

[2165] 1. Hardware

[2166] Smart glasses, head-mounted displays, PC microphones, and other devices that allow users to access information.

[2167] Server (Central processing unit for performing advanced processing)

[2168] 2. Software

[2169] Speech recognition libraries (e.g., SpeechRecognition)

[2170] Natural language processing engine (e.g., Google Cloud NLP)

[2171] Database management system (management of work instructions and procedure information)

[2172] Specific example

[2173] In the factory, a new worker uses smart glasses to give voice instructions such as, "Tell me the assembly procedure for the next product." The server analyzes this voice, retrieves the procedure information from the database, displays it on the worker's screen, and also provides voice guidance.

[2174] Example of a prompt:

[2175] "Please tell me the assembly instructions for the following product."

[2176] This system allows factory workers to receive accurate instructions and information in real time by using voice input, thereby improving operational efficiency.

[2177] This invention is a system that highly analyzes voice input to improve work efficiency and provides necessary information and instructions in real time.

[2178] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[2179] Step 1:

[2180] Users input voice commands through smart glasses or head-mounted displays.

[2181] As a concrete example, the user might say, "Please tell me the assembly instructions for the following product."

[2182] The input is the user's voice, and the output is audio data.

[2183] Step 2:

[2184] The device that receives the audio data sends that audio data to the server.

[2185] Specifically, the device sends audio data to the server via the network.

[2186] The input is audio data, and the output is data sent to the server.

[2187] Step 3:

[2188] The server converts the received audio data into text using speech recognition software (e.g., the SpeechRecognition library).

[2189] Specifically, the server starts the speech recognition engine and converts the speech data into text data.

[2190] The input is audio data, and the output is converted text data.

[2191] Step 4:

[2192] The server analyzes the converted text data using a natural language processing engine (e.g., Google Cloud NLP) to understand the user's intent.

[2193] In terms of specific operations, the server invokes a natural language processing engine to analyze the text data and extract the intended meaning.

[2194] The input is text data, and the output is parsed intent data.

[2195] Step 5:

[2196] Based on the analyzed intent, the server retrieves relevant information from the database.

[2197] In terms of specific actions, the server queries the database to retrieve the necessary work procedure information.

[2198] The input is intent data, and the output is work instruction information retrieved from the database.

[2199] Step 6:

[2200] The server organizes the acquired work instruction information and generates text or voice messages to respond to the user.

[2201] In terms of specific operations, the server formats the information into a format that is easy to transmit to the user and generates a message.

[2202] The input is work instruction information retrieved from the database, and the output is the generated message.

[2203] Step 7:

[2204] The server sends the generated message to the terminal, and the terminal notifies the user.

[2205] In terms of specific operations, the server sends a message to the terminal over the network, and the terminal displays instructions to the user via voice and text.

[2206] The input is the generated message, and the output is the notification to the user.

[2207] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[2208] The system of the present invention is an advanced chatbot that combines intent analysis of input text and emotion recognition to enable smooth communication with users. The system of the present invention has the function of recognizing the user's emotions and responding appropriately according to those emotions.

[2209] Overall system configuration

[2210] The system consists of the following elements:

[2211] 1. User terminal

[2212] This refers to a device that users access using the LINE app or a web browser.

[2213] 2. LINE Server

[2214] Its role is to receive user input and forward it to the chatbot server.

[2215] 3. Chatbot Server

[2216] This is the central part of the system that analyzes user input and generates responses.

[2217] It incorporates a natural language processing engine and an emotion engine to analyze the user's intentions and emotions.

[2218] Access the database and perform read and write operations as needed.

[2219] 4. Database

[2220] It manages user information, reservation information, required documents for procedures, and billing information.

[2221] Processing flow

[2222] 1. Receive user input

[2223] The user types "I would like to make a reservation to visit the store" into the LINE app and sends it.

[2224] 2. Perform natural language processing and sentiment recognition.

[2225] The LINE server forwards the input to the chatbot server.

[2226] The chatbot server uses a natural language processing engine to analyze the intent of the input.

[2227] At the same time, an emotion engine is used to recognize the user's emotions.

[2228] 3. Determine the necessary processing based on the analysis results.

[2229] The chatbot server determines the necessary actions based on the analyzed intent and emotions.

[2230] For example, in the case of a store visit reservation, available dates and times are retrieved from the database.

[2231] 4. Return the processing result to the user.

[2232] The system then suggests available dates and times to the user.

[2233] 5. Process the user's response.

[2234] The user selects their preferred date and time and replies.

[2235] The chatbot server receives the requested date and time and saves it to the database. The reservation is then confirmed.

[2236] 6. Send a reservation confirmation notice to the user.

[2237] Generate a confirmation message and send it to the user.

[2238] Specific example

[2239] The following are examples of specific use cases.

[2240] 1. Make an appointment to visit the store.

[2241] If a user includes anxious expressions when making a reservation (for example, "I'm worried because it's my first time"), the emotion engine recognizes this as "anxiety."

[2242] The chatbot server offers detailed explanations and support options to alleviate anxiety. "Don't worry if this is your first time visiting. We'll send you detailed instructions."

[2243] 2. Information on the documents required for the procedure

[2244] When a user requests procedural information, if a message containing an emotion such as "It's urgent" is sent, the emotion engine recognizes the "urgency."

[2245] The chatbot server quickly provides the necessary document information and responds appropriately with phrases like, "Thank you for your urgency. The required documents are as follows."

[2246] 3. Inquiry regarding unclear points in the billing statement.

[2247] If a user enters "The explanation about the fees is unclear," the sentiment engine recognizes this as "confusion."

[2248] The chatbot server adds a detailed and easy-to-understand explanation: "We apologize for any confusion regarding the billing details. Here is a detailed breakdown."

[2249] Thus, the present invention realizes a chatbot system that provides appropriate responses according to the user's intentions and emotions by combining natural language processing and emotion recognition.

[2250] The following describes the processing flow.

[2251] Specific processing steps for making a store visit reservation

[2252] Step 1:

[2253] The user types "I want to make a reservation to visit the store" in the LINE app's chat screen and sends it.

[2254] Step 2:

[2255] The device sends the user's message to the LINE server. The message is received by the LINE server.

[2256] Step 3:

[2257] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[2258] Step 4:

[2259] The chatbot server uses a natural language processing engine to analyze the intent of messages. Simultaneously, it uses an emotion engine to analyze the user's emotions.

[2260] Step 5:

[2261] Based on the analysis results, the chatbot server recognizes the user's intention to make an appointment and any feelings of anxiety.

[2262] Step 6:

[2263] The chatbot server connects to the database and queries it to retrieve available dates and times for booking. The database then returns the available dates and times.

[2264] Step 7:

[2265] The chatbot server generates a message suggesting possible dates and times to the user, saying, "Please let us know your preferred date and time. Suggestions: June 12th, 10:00 AM, June 12th, 2:00 PM. We also offer support to ensure a smooth experience even for first-time visitors." This message is then sent to the user's device via the LINE server.

[2266] Step 8:

[2267] The user replies, "Please make it June 12th at 10:00."

[2268] Step 9:

[2269] The device sends the user's reply to the LINE server. The LINE server receives the message and forwards it to the chatbot server.

[2270] Step 10:

[2271] The chatbot server saves the date and time of the received message to the database and updates the reservation information.

[2272] Step 11:

[2273] The database stores the reservation information. The chatbot server generates a reservation confirmation message and sends it to the user's device via the LINE server, stating, "Your reservation has been confirmed. Please come to our store on June 12th at 10:00. Please feel free to contact us if you have any questions."

[2274] Specific processing steps for documents required for the procedure

[2275] Step 1:

[2276] The user types "Please tell me what documents are required for moving procedures" and submits the form.

[2277] Step 2:

[2278] The device sends the input to the LINE server. The message is received by the LINE server.

[2279] Step 3:

[2280] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[2281] Step 4:

[2282] The chatbot server uses a natural language processing engine to analyze the intent of messages. Simultaneously, it uses an emotion engine to analyze the user's emotions.

[2283] Step 5:

[2284] Based on the analysis results, the chatbot server recognizes the user's intention to initiate the moving process and their sense of urgency.

[2285] Step 6:

[2286] The chatbot server connects to the database and queries it to retrieve the necessary document information for the procedure. The database then returns the required document information.

[2287] Step 7:

[2288] The chatbot server generates a message informing the user of the necessary documents and sends it to the device via the LINE server, stating, "Thank you for your urgency. The following documents are required for the moving process: Resident registration certificate, utility bills, and identification."

[2289] Specific processing steps for inquiries regarding unclear points in billing statements

[2290] Step 1:

[2291] The user types, "This month's bill is high, please show me the details," and submits it.

[2292] Step 2:

[2293] The device sends the input to the LINE server. The message is received by the LINE server.

[2294] Step 3:

[2295] The LINE server forwards the received message to the chatbot server. The chatbot server receives the message.

[2296] Step 4:

[2297] The chatbot server uses a natural language processing engine to analyze intent and recognize that the user "wants to inquire about their billing details." At the same time, it uses an emotion engine to recognize the user's "confusion."

[2298] Step 5:

[2299] The chatbot server sends an SMS verification code to the user's registered phone number to verify their identity. It generates a message for the verification process and sends it to the device via the LINE server, stating, "We will now verify your identity. An SMS code has been sent to your registered phone number."

[2300] Step 6:

[2301] The user enters the code "123456" received via SMS into the LINE chat and sends it.

[2302] Step 7:

[2303] The device sends the input back to the LINE server. The message is received by the LINE server.

[2304] Step 8:

[2305] The LINE server forwards the entered SMS code to the chatbot server. The chatbot server verifies the code, and if it is correct, completes the identity verification process.

[2306] Step 9:

[2307] The chatbot server queries the database to retrieve the billing details. The database then returns the billing details information.

[2308] Step 10:

[2309] The chatbot server generates a message saying, "This month's billing details are as follows: Basic fee 5000 yen, call charges 1500 yen, data usage 1000 yen," adds a clear explanation, and sends it to the user's terminal via the LINE server, saying, "We apologize for any confusion regarding the billing details. Here is a detailed breakdown."

[2310] As described above, the system of the present invention recognizes the user's intentions and emotions and provides an appropriate response.

[2311] (Example 2)

[2312] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2313] Conventional chatbot systems are limited to analyzing the intent behind user input, making it difficult to respond in a way that takes user emotions into account. As a result, responses to users are uniform, and communication is often not smooth. Furthermore, there are delays in providing information about specific processes and responding to users' anxieties and urgency, which leads to decreased user satisfaction. This invention aims to solve these problems.

[2314] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[2315] In this invention, the server includes means for receiving user input, means for analyzing the intent of the input using a natural language processing engine, means for recognizing the user's emotions using an emotion engine, means for determining and executing necessary processing based on the analyzed intent and emotions, and means for returning the processing results to the user. This enables an appropriate response that takes into account both the user's intent and emotions.

[2316] "Means for receiving user input" refers to the function of obtaining text messages and commands sent from the terminal used by the user.

[2317] A "natural language processing engine" is a software module that analyzes text entered by a user and understands its intent and meaning.

[2318] An "emotion engine" is a software module that identifies emotions and psychological states from user input and determines appropriate responses based on that information.

[2319] "Means for determining necessary processing based on analyzed intent and emotion" refers to a function that selects the next action or response based on the analysis results of the natural language processing engine and the emotion engine.

[2320] "Means of responding to the user with processing results" refers to a function that generates the determined response or action as a text message and sends it to the user.

[2321] "A means of obtaining and proposing information related to store visit reservations to the user" refers to a function that, when a user requests a store visit reservation, retrieves available dates and times and related information from a database and presents it to the user.

[2322] "A means of receiving a user's requested reservation date and time and saving the reservation information" refers to a function that receives the reservation date and time selected by the user and saves it in a database.

[2323] "Means of notifying users of confirmed reservation information" refers to a function that generates and sends a message to inform users of a confirmed reservation.

[2324] "A means of obtaining necessary document information for procedures and guiding users through it" refers to a function that retrieves the necessary documents and information for various procedures from a database and guides users through them appropriately.

[2325] "A means of determining processing priorities based on user emotions" refers to a function that determines the order and urgency of responses according to the user's psychological state, as analyzed by the emotion engine.

[2326] "A means of obtaining and introducing online procedural information to users" refers to a function that searches for and obtains information on procedures and services that users can perform online, and provides that information to users.

[2327] Modes for carrying out the invention

[2328] The system of the present invention is an advanced chatbot that combines intent analysis of input text and sentiment recognition to enable smooth communication with users. The specific configuration and operation of the system of the present invention are described below.

[2329] Overall system configuration

[2330] The system consists of the following elements:

[2331] 1. User terminal

[2332] These are devices such as smartphones and PCs, and users access them using the LINE app or a web browser.

[2333] 2. LINE Server

[2334] Its role is to receive user input and forward it to the chatbot server.

[2335] 3. Chatbot Server

[2336] This is the central part of the system that analyzes user input and generates responses.

[2337] It incorporates a natural language processing engine (e.g., Google Cloud Natural Language API) and an emotion engine (e.g., Microsoft Azure Text Analytics API) to analyze user intent and sentiment.

[2338] Access and read / write data to a database (e.g., MySQL) as needed.

[2339] 4. Database

[2340] It manages user information, reservation information, required documents for procedures, and billing information.

[2341] Program processing

[2342] 1. Receive user input

[2343] The user types "I want to make a reservation to visit the store" into the LINE app and sends it. The user's device sends this message to the LINE server.

[2344] 2. Perform natural language processing and sentiment recognition.

[2345] The server receives user input via the LINE server and forwards it to the chatbot server in JSON format. The chatbot server calls the Google Cloud Natural Language API to analyze the intent of the message. Simultaneously, it uses the Microsoft Azure Text Analytics API to recognize the user's sentiment.

[2346] 3. Determine the necessary processing based on the analysis results.

[2347] The chatbot server determines the next action to take based on the analysis results. For example, in the case of a store visit reservation, it generates a query to retrieve available dates and times from the database.

[2348] 4. Return the processing result to the user.

[2349] The chatbot server proposes available dates and times to the user. The message is generated in a specific format and sent to the user via the LINE server.

[2350] 5. Process the user's response.

[2351] The user selects their preferred date and time and replies. The chatbot server receives this selection and executes an SQL query to save it to the database.

[2352] 6. Send a reservation confirmation notice to the user.

[2353] The chatbot server generates a reservation confirmation message and sends it to the user: "Your reservation is confirmed. We look forward to seeing you on [Month] [Day] at [Time]."

[2354] Specific example

[2355] The following are examples of specific use cases.

[2356] 1. Make an appointment to visit the store.

[2357] If a user includes an anxious expression when making a reservation (for example, "I'm worried because it's my first time"), the emotion engine recognizes this anxiety. The chatbot server then offers detailed explanations and support options to alleviate the anxiety. For example, "Don't worry if it's your first time. We'll send you detailed instructions."

[2358] 2. Information on the documents required for the procedure

[2359] When a user requests procedural information and sends a message that includes an emotion such as "It's urgent," the emotion engine recognizes the "urgency." The chatbot server quickly provides the necessary document information and gives an appropriate response such as, "Thank you for your urgency. The required documents are as follows."

[2360] 3. Inquiry regarding unclear points in the billing statement.

[2361] If a user types "The explanation about the charges is unclear," the sentiment engine recognizes this as "confusion." The chatbot server then adds a more detailed and clearer explanation: "We apologize for the confusion regarding the charges. Here is a detailed breakdown."

[2362] Thus, the present invention realizes a chatbot system that provides appropriate responses according to the user's intentions and emotions by combining natural language processing and emotion recognition.

[2363] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2364] Step 1:

[2365] Receive user input

[2366] explanation:

[2367] The user types "I would like to make a reservation to visit the store" into the LINE app and sends it.

[2368] The device will send this message to the LINE server.

[2369] Specific actions:

[2370] The user types the message "I would like to make a reservation to visit" on the LINE app on their smartphone and presses the send button.

[2371] Input: User's text message

[2372] Output: Message forwarded to LINE server

[2373] Step 2:

[2374] Performing natural language processing and emotion recognition.

[2375] explanation:

[2376] The server forwards messages received from the LINE server to the chatbot server.

[2377] The chatbot server analyzes the intent of messages using a natural language processing engine (e.g., Google Cloud Natural Language API).

[2378] Use an emotion engine (e.g., Microsoft Azure Text Analytics API) to recognize the user's emotions.

[2379] Specific actions:

[2380] The server receives the user's message via the LINE server and forwards it to the chatbot server in JSON format.

[2381] The chatbot server sends a request to the Google Cloud Natural Language API to analyze the intent behind the "store visit reservation."

[2382] Simultaneously, a request is sent to the Microsoft Azure Text Analytics API to analyze the emotions (e.g., anxiety) contained in the message.

[2383] Input: Message forwarded from LINE server

[2384] Output: Analyzed intentions and emotions

[2385] Step 3:

[2386] Based on the analysis results, determine the necessary processing steps.

[2387] explanation:

[2388] Based on the analysis results, the chatbot server determines the next action to take.

[2389] For example, in the case of a store visit reservation, available dates and times are retrieved from a database (e.g., MySQL).

[2390] Specific actions:

[2391] The chatbot server generates database queries based on the analysis results.

[2392] The server queries the MySQL database to retrieve available dates and times for booking.

[2393] Input: Analyzed intentions and emotions

[2394] Output: Database query results including available dates and times for booking

[2395] Step 4:

[2396] Return the processing result to the user.

[2397] explanation:

[2398] The chatbot server then suggests available dates and times to the user.

[2399] The generated text message is sent to the LINE server as a suggestion.

[2400] Specific actions:

[2401] The chatbot server generates a suggestion message in a specific format and sends that message to the LINE server.

[2402] The LINE server forwards the message to the user's device.

[2403] Input: Database query results including available dates and times for booking

[2404] Output: Suggestion message sent to the user

[2405] Step 5:

[2406] Processing user responses

[2407] explanation:

[2408] The user selects their preferred date and time and replies.

[2409] The chatbot server receives the reply and saves the desired date and time to its database.

[2410] Specific actions:

[2411] The user sends a message saying, "I would like to meet on [Month] [Day] at [Time]."

[2412] The chatbot server receives this message and executes an SQL query to save the desired date and time to the database.

[2413] Input: User's reply message

[2414] Output: Reservation information stored in the database

[2415] Step 6:

[2416] Send a reservation confirmation notification to the user.

[2417] explanation:

[2418] The chatbot server generates and sends a message to the user notifying them that the reservation has been confirmed.

[2419] Specific actions:

[2420] The chatbot server generates the message, "Your reservation has been confirmed. We look forward to seeing you on [Month] [Day] at [Time]."

[2421] The message is sent to the user's device via the LINE server.

[2422] Input: Reservation information stored in the database

[2423] Output: Booking confirmation message sent to the user

[2424] (Application Example 2)

[2425] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2426] Traditional chatbot systems are limited to analyzing user intent and have the problem of not being able to provide appropriate responses that take into account user emotions. This degrades the user experience, and in particular, in ride reservation systems, support may be insufficient when passengers are feeling anxious or stressed. There is a need for a system that can smoothly respond to passengers when they need changes or urgent assistance.

[2427] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2428] In this invention, the server includes means for receiving user input, means for analyzing the intent and emotion of the input using a natural language processing engine and an emotion recognition system, means for determining and executing necessary processing based on the analyzed intent and emotion, and means for returning the processing results to the user. This enables appropriate responses that address not only the user's intent but also their emotions, allowing for quick and appropriate responses to requests such as ride reservations and route changes.

[2429] "User" refers to an individual or group that uses the system.

[2430] "Input" refers to the information that a user sends to the system.

[2431] A "natural language processing engine" is a computer program that analyzes user input to understand their intent.

[2432] An "emotion recognition system" is a computer program that analyzes emotions from user input.

[2433] "Intention" refers to the purpose or request that the user is trying to convey through their input.

[2434] "Emotions" refer to the psychological state that a user expresses through their input.

[2435] "Processing" refers to the actions the system takes based on the analyzed intentions and emotions.

[2436] "Response" refers to the answer that the system provides to the user.

[2437] "Ride reservation" refers to the act of a user reserving a vehicle for a specific date and time.

[2438] "Route change" refers to a request from a user to change their existing travel route.

[2439] A "support center" refers to an organization that provides assistance to users when they encounter problems or have questions.

[2440] "Dialogue" refers to the communication that takes place between a system and a user.

[2441] "Mobility experience" refers to the overall experience a user has when using an autonomous vehicle.

[2442] The system for implementing this invention is an advanced chatbot system aimed at improving the user's riding experience. This system consists of the following main components:

[2443] Overall system configuration

[2444] The system consists of the following four elements:

[2445] 1. User terminal

[2446] A user terminal refers to a mobile device such as a smartphone, which users use to access the system through applications. Users can make reservations and change routes through text input.

[2447] 2. Communication Server

[2448] This server is responsible for receiving user input in real time and forwarding it to the chatbot server. The communication server is crucial for ensuring stable network communication.

[2449] 3. Chatbot Server

[2450] The chatbot server is the heart of the system, handling all data processing and response generation. It includes the following software components:

[2451] Natural language processing engine (e.g., TextBlob): Analyzes user input text and understands intent.

[2452] Emotion recognition systems (e.g., emotion analysis using TextBlob): Analyze emotions from user input and adjust responses accordingly.

[2453] 4. Database Server

[2454] The database server manages the following information:

[2455] User Information

[2456] Reservation Information

[2457] Route information

[2458] Past conversation history

[2459] Explanation of the process

[2460] The system processes the data as follows:

[2461] Receiving user input

[2462] Text input is sent from the user's terminal and reaches the chatbot server via the communication server.

[2463] Natural language processing and sentiment analysis

[2464] The chatbot server uses a natural language processing engine to analyze the user's intent and an emotion recognition system to evaluate their emotions. For example, if a user types "I want to make a reservation," the intent is analyzed as "reservation" and the emotion as "positive."

[2465] Decision and execution of processing

[2466] Based on the analyzed intent and emotions, the chatbot server determines the appropriate action. For example, if a reservation request is made, it retrieves available dates and times from the database and generates suggestions.

[2467] Response to the user

[2468] The system returns the processing results to the user. When suggesting available dates and times, if the user expresses concerns, it will respond in an empathetic manner, such as, "It seems you have concerns about your preferred date and time. Please feel free to contact us."

[2469] Examples of specific user interactions

[2470] The following is an example of a specific interaction between the user and the system:

[2471] User input: "I would like to make a reservation."

[2472] ...

Claims

1. A means for receiving user input, A means for analyzing the intent of the input using a natural language processing engine, A means for determining the necessary processing based on the analyzed intent and for executing that processing, A means of returning the processing result to the user, A system that includes this.

2. A means of obtaining information related to store visit reservations and proposing it to users, A means of receiving the user's requested reservation date and time and saving the reservation information, The system according to claim 1, further comprising:

3. A means of obtaining the necessary document information for the procedure and guiding the user, A means of obtaining and presenting online procedural information to users, The system according to claim 1, further comprising:

4. Means for verifying the user's identity, After successful identity verification, the means to obtain and provide the user's billing details information are as follows: The system according to claim 1, further comprising:

5. The system according to claim 4, which includes a means of verifying identity by SMS authentication.

Images