System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting authentication information for a user to access; means for verifying the authentication information and starting a conversation session with a user who has been successfully authenticated; means including artificial intelligence for receiving consultation contents from a user, analyzing the consultation contents, and proposing an appropriate procedure window; means for generating a necessary procedure form based on the procedure window proposed by the artificial intelligence and providing the form to the user; and means for receiving a procedure content input by the user and storing the procedure content in a database.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Procedures at city hall usually require users to visit multiple counters, which is time-consuming and troublesome for users. This problem is particularly serious for busy users who cannot find the time on weekdays, and smooth and efficient procedures are required. Furthermore, for users who live far away, physically accessing city hall itself is a major burden. Furthermore, it is difficult to determine which counter to go to, and visiting the wrong counter can lead to repeated visits and delays in procedures. There is a need to solve these issues and make city hall procedures easy for anyone to carry out. [Means for solving the problem]

[0005] To address the above-mentioned problems, the present invention provides a system that allows users to complete city hall procedures online using a virtual space. The system includes: a means for inputting authentication information for a user's access; a means for verifying the authentication information and initiating an interactive session for a successfully authenticated user; a means including an artificial intelligence (AI) for receiving a user's inquiry, analyzing the inquiry, and suggesting an appropriate procedure window; a means for generating a required procedure form based on the procedure window suggested by the AI ​​and providing it to the user; and a means for receiving the procedure details entered by the user and storing the procedure details in a database. The system also includes a means for sending a confirmation message after the user completes the procedure, and a means for receiving feedback from the user and storing the feedback in a database. The system further includes a means for the AI ​​to use natural language processing technology when analyzing the user's inquiry, and a means for providing the form generated based on the suggested procedure window with the user's past procedure data pre-populated. In this way, users can efficiently complete city hall procedures through a virtual space without visiting multiple windows.

[0006] "User" refers to an individual or corporation that uses this system to carry out city hall procedures.

[0007] "Authentication Information" refers to information used to verify a user's identity, such as a user ID and password.

[0008] An "interactive session" refers to a process in which a user and a system exchange information in a chat format.

[0009] "Consultation content" refers to text information including the procedures the user wants to carry out or the details of the inquiry.

[0010] "Artificial intelligence (AI)" refers to machine learning models and algorithms that analyze the consultation content entered by users and suggest appropriate procedures.

[0011] "Procedure counter" refers to a department or person in charge related to a specific city hall business.

[0012] "Procedure Form" means an electronic form for users to enter required information.

[0013] "Database" refers to a data storage system that stores user authentication information, procedure details, feedback, etc.

[0014] "Confirmation message" refers to a notification message that informs the user that the procedure has been completed.

[0015] "Feedback" refers to information that records opinions and improvements provided by users after a transaction.

[0016] "Natural language processing technology" refers to the technology used by artificial intelligence to analyze text entered by users.

[0017] "Past procedure data" refers to the details and history of procedures previously performed by the user. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This system provides technology that uses virtual space to complete city hall procedures online. The specific processing content of this system's program is explained below.

[0040] 1. Providing an authentication screen and user authentication

[0041] server

[0042] The server loads the necessary databases and AI models at system startup, initializes the system, verifies authentication information, and starts an interactive session for users who are successfully authenticated.

[0043] Terminal

[0044] When a user accesses the device, a login screen is displayed and they are prompted to enter their authentication information (user ID and password). If authentication is successful, an interactive session begins and a chat screen is displayed where they can enter their consultation details.

[0045] User

[0046] The user enters authentication information on the login screen, and once authentication is successful, enters the content of the consultation on the chat screen.

[0047] 2. Analysis of consultation content and presentation of contact points

[0048] server

[0049] The server receives the inquiry details entered by the user and analyzes them using artificial intelligence. Specifically, it uses natural language processing technology to extract keywords and identify the appropriate procedure counter. The artificial intelligence retrieves related procedure information from a database and suggests the most suitable counter for the user.

[0050] Terminal

[0051] The terminal displays the message sent from the server suggesting a contact point on the chat screen and guides the user on the next step.

[0052] User

[0053] The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[0054] 3. Provide and complete the procedure form

[0055] server

[0056] The server generates the necessary procedure form based on the proposed procedure window, and may pre-fill parts of the form using past procedure data. The generated form is then sent to the terminal.

[0057] Terminal

[0058] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[0059] User

[0060] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[0061] 4. Data storage and notification of completion of procedure

[0062] server

[0063] The server receives the procedure details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the completion of the procedure and sends it to the user. After the procedure is completed, it also sends a message to request feedback from the user.

[0064] Terminal

[0065] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[0066] User

[0067] Users will receive a confirmation message that the process has been completed, and if necessary, they can fill out a feedback form and submit their comments or suggestions for improvement.

[0068] Specific examples

[0069] As an example, let's take the case of changing an address. When a user logs in and types "I want to change my address" on the chat screen, the server analyzes the inquiry and suggests the office where the address change can be carried out. When the user follows the suggestions, enters their new address in the address change form and submits it, the server saves the data in its database and notifies the user that the procedure is complete. Through this series of procedures, the user can efficiently change their address without having to physically visit city hall.

[0070] The above is an embodiment of the present invention. This system allows users to efficiently complete city hall procedures from home, which is expected to significantly reduce time and effort.

[0071] The processing flow will be explained below.

[0072] Program processing steps

[0073] Authentication and Chat Start

[0074] Step 1:

[0075] The server establishes a database connection when the system starts up and loads the necessary tables and AI models into memory.

[0076] Step 2:

[0077] When a user accesses the device, a login screen is displayed and the user is prompted to enter their user ID and password.

[0078] Step 3:

[0079] The user enters the user ID and password on the login screen.

[0080] Step 4:

[0081] The server verifies the entered authentication information and creates a session if the authentication is successful.

[0082] Step 5:

[0083] If the authentication is successful, the terminal displays a chat screen and prompts the user to enter the content of the consultation.

[0084] Receiving and analyzing consultation content

[0085] Step 6:

[0086] The user inputs the content of the consultation on the chat screen and sends it.

[0087] Step 7:

[0088] The server analyzes the inquiry received from the user using natural language processing technology. Specifically, it extracts keywords from the input text and identifies the type of procedure.

[0089] Proposal of contact information and generation of procedure forms

[0090] Step 8:

[0091] The server identifies an appropriate procedure window based on the analyzed consultation content, generates a proposal message, and transmits it to the terminal.

[0092] Step 9:

[0093] The terminal displays the proposal message sent from the server on the chat screen.

[0094] Step 10:

[0095] The user checks the procedure window suggested by the server and selects the suggested procedure.

[0096] Step 11:

[0097] The server generates the appropriate procedure form based on the selected procedure, pre-populating the form with previous procedure data, if available.

[0098] Fill in the form and complete the procedure

[0099] Step 12:

[0100] The terminal displays the procedure form sent from the server and prompts the user to enter the necessary information.

[0101] Step 13:

[0102] The user enters the necessary information into the procedure form, checks the contents on the confirmation screen, and then presses the submit button.

[0103] Step 14:

[0104] The server receives the submitted procedure details and stores them in a database.

[0105] Step 15:

[0106] The server generates and sends a confirmation message to the user indicating that the transaction has been completed.

[0107] Collecting feedback

[0108] Step 16:

[0109] The terminal displays a feedback input screen along with a confirmation message that the procedure has been completed.

[0110] Step 17:

[0111] Users can submit their opinions and suggestions for improvement in a feedback form.

[0112] Step 18:

[0113] The server receives the feedback from the user and stores it in a database.

[0114] These are the specific processing steps of the virtual city hall online service program, which allows users to efficiently carry out city hall procedures from home.

[0115] Example 1

[0116] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0117] When completing procedures at city hall, users must physically visit the city hall, which is a time-consuming and labor-intensive process. Furthermore, procedures require users to manually fill out numerous documents, which increases the risk of errors. Another problem is that insufficient information is provided about procedures, which means users take a long time to identify the appropriate office and procedures.

[0118] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0119] In this invention, the server includes means for providing an interface for users to input authentication information for access, means for verifying the authentication information and starting an interactive session for users who are successfully authenticated, means for receiving the content of the consultation from the user, analyzing the content of the consultation, and using artificial intelligence including natural language processing technology to suggest an appropriate procedure window, means for generating the necessary procedure form based on the procedure window suggested by the artificial intelligence and providing the form with past procedure data pre-entered to the user, means for receiving the procedure content entered by the user and storing the procedure content in a database, and means for receiving feedback from the user. This enables users to efficiently complete procedures online without having to physically visit the city hall.

[0120] "Authentication information" refers to information such as user ID and password that a user enters to access a system.

[0121] An "interactive session" is a real-time communication process between a successfully authenticated user and a system.

[0122] "Consultation content" refers to questions or requests regarding procedures that users input into the system.

[0123] "Natural language processing technology" is a technology for analyzing text entered by a user and understanding its meaning.

[0124] "Artificial intelligence" refers to an automated intelligent system that analyzes the inquiry content entered by the user and suggests the appropriate procedure window.

[0125] A "procedure counter" is an interface on the system that provides the forms and information necessary to complete a specific government procedure.

[0126] A "procedure form" is an electronic format into which a user enters the information necessary to carry out a procedure.

[0127] A "database" is a system for storing and managing information such as procedure details and user feedback.

[0128] "Feedback" refers to opinions and suggestions for improvement that users provide to the system after completing a procedure.

[0129] This system provides technology that allows city hall procedures to be completed online using a virtual space. The detailed implementation method of this system is described below.

[0130] System Overview

[0131] The system operates primarily between three parties: the server, the terminal, and the user. It allows users to complete city hall procedures online through a series of processes, including initialization, user authentication, analysis of the consultation content, provision of a procedure form, data storage, and notification of procedure completion.

[0132] Hardware and software used

[0133] 1. Server

[0134] Hardware: A server with a powerful processor, ample memory, and SSD storage.

[0135] Software: Operating systems (e.g., Linux), database management systems (e.g., MySQL), artificial intelligence (AI) models (e.g., BERT or GPT), authentication services.

[0136] 2. Terminal

[0137] Hardware: Computer or smartphone.

[0138] Software: Web browser (e.g., GOOGLE CHROME®, Mozilla Firefox), chat interface.

[0139] System processing flow

[0140] 1. Providing an authentication screen and user authentication

[0141] Server: When the system starts up, the server loads the necessary database and AI model, initializes the system, verifies authentication information (user ID and password), and starts an interactive session for users who are successfully authenticated.

[0142] Terminal: When a user accesses the terminal, a login screen is displayed and they are prompted to enter their authentication information. If authentication is successful, an interactive session begins and a chat screen is displayed where they can enter their consultation details.

[0143] User: The user enters their authentication information on the login screen, and once authentication is successful, they enter the details of their inquiry on the chat screen.

[0144] 2. Analysis of consultation content and presentation of contact points

[0145] Server: The server receives the inquiry details entered by the user, extracts keywords using natural language processing technology, identifies the appropriate procedure counter, retrieves related procedure information from the database, and suggests the most suitable procedure counter to the user.

[0146] Terminal: The terminal displays the contact suggestion message sent from the server on the chat screen and guides the user on the next step.

[0147] User: The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[0148] 3. Provide and complete the procedure form

[0149] Server: The server generates the necessary transaction form based on the proposed transaction window. It may pre-fill parts of the form using past transaction data. The generated form is sent to the terminal.

[0150] Terminal: The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[0151] User: The user enters the required information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[0152] 4. Data storage and notification of completion of procedure

[0153] Server: The server receives the transaction details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the transaction completion and sends it to the user. After the transaction is completed, it also sends a message to request feedback from the user.

[0154] Terminal: The terminal displays a confirmation message for the user that the procedure has been completed and a feedback input screen.

[0155] User: The user receives a confirmation message that the process has been completed, and optionally fills in and submits a feedback form with their thoughts and suggestions for improvement.

[0156] Specific examples

[0157] Address change procedures

[0158] Consider the case where a user logs in and types "I want to change my address" on the chat screen. In this case, the server analyzes the consultation content, extracts the related keyword "address change," and identifies the appropriate contact point. The suggested contact point information is displayed on the terminal. When the user follows the suggestions, enters their new address in the address change form, and submits it, the server saves this information in the database and notifies them that the procedure is complete. This allows the user to change their address online without having to physically visit city hall.

[0159] Prompt Sentence Examples

[0160] Text format

[0161] When a user logs in and types "I want to change my address" on the chat screen, please tell us in detail how the system will guide them through the procedure and support them until it is completed.

[0162] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently carry out city hall procedures from home, and is expected to save time and effort.

[0163] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0164] Step 1: Initialize the system

[0165] server

[0166] When the system starts up, the server loads the necessary databases and AI models and initializes the system. Specifically, the server loads the necessary data and models from the file system or external storage and expands them into memory. It also reads and configures system configuration information and prepares authentication services and security protocols. This initialization process makes it possible to connect to various databases and issue security tokens.

[0167] Input: Database connection information, AI model file

[0168] Output: Initialization status of various services, system configuration complete

[0169] Step 2: Providing an authentication screen and user authentication

[0170] Terminal

[0171] When a user accesses the terminal, a login screen is displayed. This screen is designed using HTML and CSS and includes input fields for the user ID and password, as well as a login button.

[0172] User

[0173] The user enters authentication information on the login screen and presses the login button, which sends the entered user ID and password to the server.

[0174] server

[0175] The server receives the authentication information, compares it with the user information in its database, and if the comparison is successful, generates a session token and sends it to the device.

[0176] Input: User ID, Password

[0177] Output: Authentication result (success or failure), session token

[0178] Step 3: Analyze the consultation content and provide a contact point

[0179] Terminal

[0180] The device receives the session token sent from the server and displays a chat screen, which includes a text box for accepting user input and a send button.

[0181] User

[0182] The user inputs the content of the inquiry (for example, "I want to change my address") on the chat screen and presses the send button.

[0183] server

[0184] The server receives messages from users and analyzes their inquiries using natural language processing technology. Specifically, it uses an NLP model to extract keywords from the input text and retrieves related procedural information from a database. It then identifies the most appropriate procedure window and suggests it to the user.

[0185] Input: User message

[0186] Output: Proposal of procedure window

[0187] Step 4: Provide and complete the procedure form

[0188] server

[0189] The server generates the necessary procedure form based on the identified procedure window. This form is dynamically generated using an HTML template. It may also refer to past procedure data and pre-fill parts of the form. The generated form is sent to the terminal.

[0190] Terminal

[0191] The terminal displays the procedure form sent from the server, which contains fields for the user to fill in and prompts them for information using UI elements (text boxes, radio buttons, etc.).

[0192] User

[0193] The user enters the necessary information into the form, checks the details on the confirmation screen, and then presses the submit button. The entered information is sent from the terminal to the server.

[0194] Input: Procedure window information

[0195] Output: Procedure form

[0196] Step 5: Data storage and notification of completion

[0197] server

[0198] The server receives the transaction details submitted by the user and stores them in a database. If the transaction is successfully completed, it generates a confirmation message for the user and sends it to them. After the transaction is completed, it also sends a message to request feedback from the user.

[0199] Terminal

[0200] The terminal displays a confirmation message for the completion of the procedure and a feedback input screen to the user. The feedback input screen has a text area where the user can enter their opinions and suggestions for improvement.

[0201] User

[0202] The user receives a confirmation message that the procedure has been completed, and if necessary, enters opinions or suggestions for improvement in the feedback form and presses the submit button. The entered feedback information is sent to the server.

[0203] Input: Procedure details, user feedback

[0204] Output: Procedural information stored in the database, feedback confirmation message

[0205] These are the specific processing steps of the system program. This series of processes allows users to efficiently complete procedures online without having to physically visit city hall.

[0206] (Application example 1)

[0207] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0208] Today's consumers often encounter complex procedures and guidance when using virtual shopping malls. While they are expected to offer the same convenience and efficiency as offline shopping malls, traditional systems lack sufficient ways for users to quickly and efficiently complete procedures and guidance within a digital environment. Furthermore, as the number of inquiries and procedures from users increases, response times may be delayed, resulting in a decline in customer satisfaction.

[0209] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0210] In this invention, the server includes means for inputting authentication information for a user to access the virtual shopping mall, means for verifying the authentication information and starting an interactive session with a successfully authenticated user, means including artificial intelligence for receiving the content of a consultation from the user, analyzing the content of the consultation, and suggesting an appropriate procedure window, means for generating a required procedure form based on the procedure window suggested by the artificial intelligence and providing it to the user, means for receiving the procedure content input by the user and storing the procedure content in a database, and means for using an interactive agent in a virtual space to provide the user with procedure guidance and form input support, thereby enabling the user to quickly and efficiently complete procedures and guidance within the virtual shopping mall.

[0211] "User" refers to an individual or organization that uses the system to carry out procedures.

[0212] "Authentication information" refers to information such as user ID and password required when a user accesses a system.

[0213] An "interactive session" refers to a process in which a system and a user communicate with each other to proceed through a procedure.

[0214] "Consultation content" refers to the content of questions or requests entered by users into the system.

[0215] "Procedure counter" refers to the department or service where users should go to carry out specific procedures.

[0216] "Artificial intelligence" refers to technology that analyzes the content of users' inquiries within the system and suggests the most appropriate procedures.

[0217] "Natural language processing technology" refers to technology that enables computers to understand and analyze human language.

[0218] "Procedure form" refers to a format or page where a user can enter information necessary to proceed with a procedure.

[0219] "Database" refers to a storage device for centrally managing data such as user procedural information and feedback within the system.

[0220] An "interactive agent" refers to an automatic response system that assists users with procedures by interacting with them in a virtual space.

[0221] "Virtual space" refers to an imaginary space created using computer technology, and refers to a place where online activities take place.

[0222] The system allows users to quickly and efficiently complete complex procedures and directions within a virtual shopping mall.

[0223] Hardware and software used

[0224] 1. Server: This is the core part of the system. Hosting and database management are performed using AWS (registered trademark) (Amazon Web Services).

[0225] 2. Terminal: A smartphone, smart glasses, or head-mounted display is used as the interface for users to access the system.

[0226] 3. Natural language processing technology: The content of the user's inquiry is analyzed using Google (registered trademark) Cloud Natural Language API.

[0227] 4. Database: Amazon RDS (Relational Database Service) is used to store procedural data and feedback.

[0228] 5. Chatbot: Use Dialogflow to build a chat interface for the information center.

[0229] System details

[0230] 1. Providing an authentication screen and user authentication

[0231] When the system starts up, the server loads the necessary database and AI model and initializes the system. It verifies authentication information and starts an interactive session for users who are successfully authenticated. When a user accesses the device, it displays a login screen and prompts them to enter their authentication information. The user enters their authentication information on the login screen, and if authentication is successful, a chat screen is displayed.

[0232] 2. Analysis of consultation content and provision of guidance

[0233] When a user enters their inquiry into the chat, the server receives the content and analyzes keywords using the Google Cloud Natural Language API. The server identifies the appropriate counter or store where to go to complete the procedure and presents that information to the user. The device then displays the guidance message sent from the server on the chat screen.

[0234] 3. Provide and complete the procedure form

[0235] When a specific procedure is required, the server generates a procedure form and can pre-fill some parts based on past data. The generated form is sent to the terminal and displayed to the user. The user enters the necessary information into the form, checks the contents on the confirmation screen, and then presses the submit button.

[0236] 4. Data storage and notification of completion of procedure

[0237] The server receives the procedure details submitted by the user and stores them in Amazon RDS. After saving, it generates a confirmation message confirming the completion of the procedure and sends it to the user. After the procedure is complete, it also sends a message requesting feedback from the user. The terminal displays a confirmation message confirming the completion of the procedure and a feedback input screen to the user. The user receives the confirmation message confirming the completion of the procedure and, if necessary, enters opinions or suggestions for improvement in the feedback form and submits it.

[0238] Specific examples

[0239] When a user enters "I would like to process a return" into the virtual shopping mall's information center, the server analyzes the inquiry and provides instructions on how to process the return. When the user enters the necessary information into the appropriate form and submits it, the data is saved by the server and the user is notified that the process has been completed.

[0240] Prompt Sentence Examples

[0241] User: "I want to process a return"

[0242] Server: "To begin your return, please enter the required information in the form below."

[0243] [Present Form]

[0244] User: [fill out form]

[0245] Server: "Your transaction is complete. Thank you for using our service."

[0246] This system allows users to carry out procedures efficiently and smoothly within a virtual space.

[0247] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0248] Step 1:

[0249] When a user accesses the terminal, a login screen is displayed and the user is prompted to enter authentication information (user ID and password). The entered authentication information is then sent to the server. The input data is the user ID and password character string.

[0250] Step 2:

[0251] The server compares the received authentication information with existing information in the database. If the authentication information matches, it determines that authentication is successful and sends a message to the terminal to start an interactive session. The data calculation is to confirm that the entered authentication information matches the information in the database. If authentication is successful, the output is a message to start an interactive session.

[0252] Step 3:

[0253] The terminal receives a message of successful authentication and displays a chat screen for the user. The user inputs the content of the consultation on the chat screen. The input data is the user's free-form text message.

[0254] Step 4:

[0255] The server receives the inquiry content from the user and extracts and analyzes keywords using the Google Cloud Natural Language API. Based on the extracted keywords, it identifies the appropriate procedure counter and generates a guidance message. Data calculation involves keyword extraction using natural language processing technology and matching of procedure information. The output is a guidance message regarding the appropriate procedure counter.

[0256] Step 5:

[0257] The terminal displays the guidance message sent from the server on the chat screen and guides the user on the next step. The message displayed is a guidance message generated by the server.

[0258] Step 6:

[0259] The server generates the necessary procedure form and automatically fills in some of it based on past procedure data. The generated form is then sent to the terminal. Data calculation involves referencing past data and automatically generating the form. The output is a procedure form containing the necessary items that the user must enter.

[0260] Step 7:

[0261] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information. The user enters the necessary information into the form and presses the submit button. The input data is the information entered into the form.

[0262] Step 8:

[0263] The server receives the procedure details submitted by the user and stores them in a database. After the data has been saved, it generates a confirmation message for the completion of the procedure and sends it to the terminal. The data operation is to store the received data in the database. The output is a confirmation message for the completion of the procedure.

[0264] Step 9:

[0265] The terminal displays the confirmation message sent from the server indicating completion of the procedure and the feedback input screen to the user. The user receives the confirmation message, enters their opinions and suggestions for improvement in the feedback form, and presses the send button. The input data is the information entered in the feedback form.

[0266] Step 10:

[0267] The server receives feedback from users and stores it in a database. This makes it possible to use it to improve the operation of the system. Data calculation is the storage of feedback data in the database. The output is confirmation that the data has been saved.

[0268] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0269] This system combines technology that uses virtual space to complete city hall procedures online with an emotion engine that analyzes users' emotions, providing a more sophisticated and user-friendly experience. The specific processing content of this system's program is explained below.

[0270] 1. Providing an authentication screen and user authentication

[0271] server

[0272] The server establishes a database connection at system startup, loads the necessary tables, AI models, and emotion engines into memory, verifies authentication information, and starts an interactive session for users who are successfully authenticated.

[0273] Terminal

[0274] When a user accesses the device, a login screen is displayed and the user is prompted to enter authentication information (user ID and password). If authentication is successful, an interactive session begins and a chat screen is displayed.

[0275] User

[0276] The user enters authentication information on the login screen, and once authentication is successful, enters the content of the consultation on the chat screen.

[0277] 2. Analysis of consultation content and presentation of contact points

[0278] server

[0279] The server receives the consultation details entered by the user and analyzes them using artificial intelligence. Specifically, it uses natural language processing technology to extract keywords and identify the type of procedure. It also uses an emotion engine to analyze the consultation details and identify the user's emotions. Based on the results of this analysis, the server generates suggested messages for the procedure counter that take the user's emotions into consideration.

[0280] Terminal

[0281] The terminal displays the message sent from the server suggesting a contact point on the chat screen and guides the user on the next step.

[0282] User

[0283] The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[0284] 3. Provide and complete the procedure form

[0285] server

[0286] The server generates the necessary procedure form based on the proposed procedure window, possibly pre-filling parts of the form using past procedure data, and sends the generated form to the terminal.

[0287] Terminal

[0288] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[0289] User

[0290] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[0291] 4. Data storage and notification of completion of procedure

[0292] server

[0293] The server receives the procedure details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the completion of the procedure and sends it to the user. After the procedure is completed, it also sends a message to request feedback from the user.

[0294] Terminal

[0295] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[0296] User

[0297] Users will receive a confirmation message that the process has been completed, and if necessary, they can fill out a feedback form and submit their comments or suggestions for improvement.

[0298] Example: Address change procedure

[0299] As an example, let's consider the case of changing one's address. When a user logs in and enters "I want to change my address" on the chat screen, the server analyzes the inquiry using natural language processing technology. In parallel, the emotion engine analyzes the user's emotions from the text and determines that the user is under stress. In this case, the server suggests the appropriate office to change their address, along with an encouraging message such as "We will respond promptly. We will help you with your address change." When the user follows the suggestions and enters their new address into the address change form and submits it, the server saves the data in its database and notifies them that the procedure is complete. At this time, the user's emotional data is also saved in their profile. Through this series of procedures, users can change their address efficiently and safely without having to physically visit city hall.

[0300] The above is an embodiment of the present invention. This system not only allows users to efficiently carry out city hall procedures from home, but also enables responses that take into consideration the user's feelings, creating a more user-friendly environment.

[0301] The processing flow will be explained below.

[0302] Program processing steps

[0303] Authentication and Chat Start

[0304] Step 1:

[0305] The server establishes a database connection when the system starts up and loads the necessary tables, AI models, and emotion engines into memory.

[0306] Step 2:

[0307] When a user accesses the device, a login screen is displayed and the user is prompted to enter their user ID and password.

[0308] Step 3:

[0309] The user enters the user ID and password on the login screen.

[0310] Step 4:

[0311] The server verifies the entered authentication information and creates a session if the authentication is successful.

[0312] Step 5:

[0313] If the authentication is successful, the terminal displays a chat screen and prompts the user to enter the content of the consultation.

[0314] Receiving and analyzing consultation content

[0315] Step 6:

[0316] The user inputs the content of the consultation on the chat screen and sends it.

[0317] Step 7:

[0318] The server analyzes the inquiry received from the user using natural language processing technology. Specifically, it extracts keywords from the input text and identifies the type of procedure.

[0319] Step 8:

[0320] The server uses an emotion engine to analyze the user's emotions from the consultation text. Specifically, it identifies the user's emotional state (e.g., stress, relief, satisfaction) from the tone and expression of the text.

[0321] Proposal of contact information and generation of procedure forms

[0322] Step 9:

[0323] The server then identifies the appropriate procedure window based on the analysis results, generates a suggested message that takes into account the user's emotions, and sends it to the device. For example, if a user is feeling stressed, it will add a message of encouragement or reassurance.

[0324] Step 10:

[0325] The terminal displays the proposed message sent from the server on the chat screen and guides the user on the next step.

[0326] Step 11:

[0327] The user checks the procedure window suggested by the server and selects the suggested procedure.

[0328] Step 12:

[0329] The server generates the appropriate procedure form based on the selected procedure, pre-populates the form with previous procedure data, if available, and presents the form in an emotionally sensitive and user-friendly interface.

[0330] Fill in the form and complete the procedure

[0331] Step 13:

[0332] The terminal displays the procedure form sent from the server and prompts the user to enter the necessary information.

[0333] Step 14:

[0334] The user enters the necessary information into the procedure form, checks the contents on the confirmation screen, and then presses the submit button.

[0335] Step 15:

[0336] The server receives the submitted procedure details and stores them in a database.

[0337] Step 16:

[0338] The server generates and sends a confirmation message to the user indicating that the procedure has been completed, and after the procedure is completed, it uses the emotion engine to analyze the user's emotional state again.

[0339] Collecting feedback

[0340] Step 17:

[0341] The terminal displays a feedback input screen along with a confirmation message that the procedure has been completed.

[0342] Step 18:

[0343] Users can submit their opinions and suggestions for improvement in a feedback form.

[0344] Step 19:

[0345] The server receives user feedback, stores it in a database along with emotional data, and identifies areas for improvement based on the feedback, which will be reflected in the next update.

[0346] These are the specific processing steps of the virtual city hall online service that combines the emotion engine, allowing users to carry out city hall procedures efficiently and safely from home.

[0347] Example 2

[0348] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0349] There is a demand for a user-friendly environment that allows users to efficiently complete procedures online without having to physically visit a government office, and that also provides responses that take users' emotions into consideration. However, conventional systems have difficulty taking users' emotional states into account, and have been unable to reduce their stress and anxiety. Therefore, the objective of this invention is to provide a system that analyzes users' emotions and provides appropriate procedural guidance based on this analysis.

[0350] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting authentication information, means for verifying the authentication information and starting an interactive session with a user who has been successfully authenticated, means including artificial intelligence for receiving the consultation content, analyzing the consultation content, and suggesting an appropriate procedure window, means for generating and providing the necessary procedure form based on the procedure window suggested by the artificial intelligence, means for receiving the procedure content and saving the procedure content in data storage, means for performing emotion analysis and generating a suggested message based on the user's emotional state, and means for recording the user's emotional data and reflecting it in the user's profile. This allows the user to efficiently complete procedures online without having to physically visit a government office, and further allows the user to proceed with the procedures while reducing stress and anxiety by receiving guidance that takes into account the user's emotional state.

[0351] "Authentication information" means data provided by a user to identify himself or herself when accessing a system, including, for example, a user ID and password.

[0352] An "interactive session" is an interactive communication initiated by the system with a successfully authenticated user, and includes the ability for the user to input procedural consultations and requests into the system.

[0353] "Consultation content" refers to the content of questions or requests regarding specific procedures that users input into the system.

[0354] A "procedure window" is the optimal window suggested by the system for handling a specific type of procedure, and includes information such as which procedural documents are required.

[0355] "Artificial intelligence" is a technology used to analyze the content of users' inquiries and suggest appropriate procedure points, and has functions including natural language processing and emotion analysis.

[0356] A "procedure form" is a digital document that allows a user to enter information necessary to perform a specific procedure.

[0357] "Data storage" means a database or storage device that securely stores user transaction details and other related information.

[0358] "Emotion analysis" is a technology that detects the emotional state from the text entered by the user and identifies emotional states such as stress or relief.

[0359] A "suggested message" is a message that includes appropriate guidance or words of encouragement for the user based on the results of sentiment analysis.

[0360] A "profile" is an individual record that includes information such as a user's emotional data and past procedure history.

[0361] This invention is a system that combines technology that uses virtual space to complete city hall procedures online with an emotion engine to provide users with an advanced, user-friendly experience. Specific embodiments of this system are described below.

[0362] This system consists of three components: a server, a terminal, and a user. The server uses an EC2 instance on AWS, and the database uses AWS RDS. The artificial intelligence models used are TENSORFLOW (registered trademark) and GPT-4 (registered trademark), and the sentiment analysis engine uses Hugging Face's Transformers. The terminal uses a web browser, and the front end provides a web application using React.

[0363] First, when the system starts, the server establishes a connection to the database and loads the necessary tables, AI models, and emotion engines into memory. When a user accesses the system, the terminal displays a login screen prompting the user to enter their user ID and password. The authentication information entered by the user is verified by the server, and if authentication is successful, an interactive session begins.

[0364] When a user inputs the details of their consultation, the details are sent from the device to the server. The server then uses natural language processing technology to analyze the details and identify the appropriate procedure. At the same time, it also analyzes the user's emotional state using an emotion analysis engine and generates a suggested procedure window message based on this.

[0365] The server sends the generated proposal message to the terminal, which then displays the message on the chat screen to guide the user on the next step. When the user follows the proposed procedure window and fills in the necessary procedure form, the server references past procedure data and pre-fills the necessary parts. This procedure form is displayed on the terminal, and the user enters the necessary information and submits the form.

[0366] The server saves the submitted procedure details in a database, generates a confirmation message for the user to confirm the completion of the procedure, and sends it to the user. After the procedure is completed, the server also sends a message to request feedback from the user, and saves the received feedback in the database.

[0367] As a specific example, let's consider the case of changing an address. When a user types "I want to change my address" on the chat screen, the server analyzes the content of the inquiry using natural language processing technology, while an emotion engine simultaneously analyzes the user's emotions from the text. If the server determines that the user is under stress, it suggests a contact point for changing the address, along with an encouraging message such as "We will respond promptly. We will help you with your address change." When the user enters their new address into the address change form and submits it as suggested, the server saves the data in its database and notifies the user that the procedure is complete. Through this series of procedures, the user can change their address efficiently and safely without having to physically visit city hall.

[0368] Below is an example of a prompt sentence.

[0369] "I want to change my address, but I've been under a lot of stress lately. How do I go about the process?"

[0370] In response to this prompt, the generative AI model uses natural language processing techniques to provide procedural and emotionally appropriate responses.

[0371] In this way, the system provides efficient and user-friendly online procedures while taking into consideration the user's feelings.

[0372] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0373] Step 1:

[0374] The server connects to the database when the system starts up and loads the necessary tables, AI models, and emotion engines into memory.

[0375] Input: Start the server

[0376] Output: Establishing a database connection and loading required resources

[0377] Specifically, a database connection is established using Python's sqlalchemy, and a model using TensorFlow and an emotion engine using Hugging Face's Transformers library are loaded into memory.

[0378] Step 2:

[0379] When a user accesses the system, the terminal displays a login screen and prompts the user to enter their user ID and password.

[0380] Input: User access

[0381] Output: Login screen displayed

[0382] Specifically, the front end using React displays a login screen and waits for user input.

[0383] Step 3:

[0384] The user enters the user ID and password on the login screen.

[0385] Input: Authentication information (user ID and password)

[0386] Output: Sending authentication information

[0387] Specifically, the user enters authentication information into the login form and presses the submit button.

[0388] Step 4:

[0389] The server receives the authentication information sent from the terminal and authenticates it by comparing it with information in a database. If authentication is successful, an interactive session begins.

[0390] Input: Credentials

[0391] Output: Interactive session started upon successful authentication

[0392] Specifically, a JWT token is used to verify authentication information, and an interactive session begins when a valid token is generated.

[0393] Step 5:

[0394] The terminal confirms that the authentication was successful and displays the chat screen.

[0395] Input: Authentication success notification

[0396] Output: Chat screen display

[0397] Specifically, the React component generates and displays the chat screen.

[0398] Step 6:

[0399] The user inputs the details of the procedure to be discussed on the chat screen.

[0400] Input: Consultation details

[0401] Output: Sending consultation details

[0402] Specifically, the user enters the content of the consultation in the chat box and presses the send button.

[0403] Step 7:

[0404] The server receives the consultation details sent from the device, extracts keywords using natural language processing technology, identifies the type of procedure, and analyzes the user's emotions using an emotion engine.

[0405] Input: Consultation details

[0406] Output: Procedure type and sentiment analysis results

[0407] Specifically, keywords are extracted using Python's NLTK library, and an emotion analysis model determines the user's emotional state.

[0408] Step 8:

[0409] The server generates a message suggesting a procedure window that takes into consideration the user's emotional state based on the analysis results.

[0410] Input: Keyword extraction results, sentiment analysis results

[0411] Output: Proposal message generated

[0412] Specifically, code that dynamically generates a template message is executed.

[0413] Step 9:

[0414] The terminal displays the proposal message sent from the server on the chat screen and guides the user on the next step.

[0415] Input: Proposal message

[0416] Output: Display on the chat screen

[0417] Specifically, the React component displays a suggestion message.

[0418] Step 10:

[0419] The user fills in the necessary procedure forms according to the suggested procedure window.

[0420] Input: Proposal message

[0421] Output: Procedure form input

[0422] Specifically, the user follows the suggestions, enters the necessary information into the procedure form, and presses the submit button.

[0423] Step 11:

[0424] The server stores the procedure details sent by the user in a data storage, generates a confirmation message for completing the procedure, and sends it to the user.

[0425] Input: Procedure details

[0426] Output: Save to data storage, generate confirmation message

[0427] Specifically, it runs SQL queries to validate and save data and generates a confirmation message.

[0428] Step 12:

[0429] The terminal receives a confirmation message that the transaction has been completed and displays it to the user, as well as a form to collect feedback.

[0430] Input:Confirmation message

[0431] Output: Display confirmation message, display feedback form

[0432] Specifically, the React component displays a confirmation message and a feedback form.

[0433] Step 13:

[0434] The user checks the confirmation message indicating that the procedure has been completed, and if necessary, enters opinions or suggestions for improvement in the feedback form and submits it.

[0435] Input: Feedback

[0436] Output: Send feedback

[0437] Specifically, the user enters their opinion in the feedback form and presses the send button.

[0438] Step 14:

[0439] The server receives the feedback sent by the user and stores it in a data storage.

[0440] Input: Feedback

[0441] Output: Save Feedback

[0442] Specifically, it executes SQL queries to store the feedback data.

[0443] (Application example 2)

[0444] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0445] Conventional online processing systems simply focus on users completing the necessary procedures, and rarely take into account the user's emotional state. This makes it difficult to reduce the stress and anxiety users experience while completing the process. Furthermore, online shopping systems often lack recommendations for products and services that are tailored to the user's emotions, potentially resulting in a decrease in satisfaction. This invention solves these problems and provides high-quality services that take into account the user's emotions.

[0446] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input authentication information for access, means for verifying the authentication information and starting an interactive session with a user who has been successfully authenticated, means including an emotion analysis engine for analyzing the emotional state of the user, and means for recommending appropriate products and services based on the emotion analysis results. This makes it possible to provide procedures and services that correspond to the emotional state of the user.

[0447] "Authentication information" refers to the information that a user enters to access a system, and is usually a user ID and password.

[0448] An "interactive session" is a two-way communication exchange initiated with a successfully authenticated user.

[0449] "Artificial intelligence" is a system that uses advanced computing technology to understand and analyze the content of users' inquiries and take appropriate steps and make suggestions.

[0450] A "processing counter" is a designated location or portal where users can carry out the procedures they need.

[0451] A "transaction form" means an electronic form into which a user enters necessary information and contains fields necessary to complete a particular transaction.

[0452] A "database" is a system for managing stored information and data and making it accessible when needed.

[0453] An "emotion analysis engine" is a technology that analyzes and identifies a user's emotional state from their text or voice.

[0454] "Product and service recommendation" is the act of suggesting appropriate products and services to users based on their analyzed emotional state.

[0455] "Feedback" refers to information such as opinions, impressions, and areas for improvement from users, which can be used to improve and optimize the system.

[0456] "Natural language processing technology" is a computational technology that has the ability to understand, analyze, and generate human language.

[0457] The system embodying this invention allows users to use an online virtual store and perform product recommendations and procedures based on emotion analysis. This system operates in cooperation with three entities: a server, a terminal, and a user.

[0458] 1. Authentication and starting an interactive session

[0459] At system startup, the server first establishes a database connection and loads the necessary tables, artificial intelligence models, and sentiment analysis engine into memory, thereby verifying authentication information and preparing to start an interactive session with the user if authentication is successful.

[0460] When a user accesses the device, a login screen is displayed and they are prompted to enter their authentication information. If the authentication is successful, an interactive session begins and a chat box is displayed.

[0461] Users enter their authentication information on the login screen, and once authentication is successful, they can enter their consultation details through the chat box.

[0462] 2. Analysis of consultation content and proposal of contact points

[0463] When the server receives the user's inquiry, it analyzes the content using natural language processing technology. It then uses an emotion analysis engine to identify the user's emotional state. Based on the analysis results, the server generates a procedure window or product recommendation message that takes the user's emotional state into consideration.

[0464] The terminal displays messages sent from the server, such as counter suggestions and product recommendations, in a chat box and guides the user on the next step.

[0465] The user checks the counters and products suggested by the server and follows the instructions to proceed to the next step.

[0466] 3. Provide and complete the procedure form

[0467] The server generates the necessary procedure form based on the proposed procedure window. It can pre-fill parts of the form using past procedure data. The generated form is sent to the terminal.

[0468] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[0469] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[0470] 4. Data storage and notification of completion of procedure

[0471] The server receives the procedure submitted by the user and stores it in a database. After storing it, it generates a confirmation message of the procedure completion and sends it to the user. Furthermore, after the procedure is completed, it also sends a message to receive feedback from the user.

[0472] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[0473] Users will receive a confirmation message that the process has been completed, and if necessary, can fill out and submit a feedback form with their opinions or suggestions for improvement.

[0474] Hardware and software used

[0475] This system uses the following hardware and software:

[0476] Hardware: Smartphone (iOS or ANDROID (registered trademark))

[0477] Software: Python, OpenAI (registered trademark) API, natural language processing library, sentiment analysis engine

[0478] Examples of concrete examples and prompts

[0479] As a concrete example, consider the case where a user inputs "I want new running shoes." In this case, the server performs natural language processing and sentiment analysis, and if it derives, for example, "I'm tired," it will recommend comfortable running shoes.

[0480] Example prompt sentence:

[0481] "I want new running shoes. I've been working out too hard lately and I feel tired."

[0482] In this way, it is possible to recommend products and services that suit the user's emotional state, providing a high-quality online shopping experience.

[0483] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0484] Step 1:

[0485] The user launches the app on their smartphone and enters their user ID and password on the login screen.

[0486] Input: User ID and password

[0487] The server checks the received authentication information against its database and, if authentication is successful, begins an interactive session.

[0488] Output: A command to display a message indicating successful authentication and a chat box.

[0489] Step 2:

[0490] Based on the authentication success message, the terminal displays a chat box to the user.

[0491] Input: Authentication success message

[0492] The user enters the question (e.g., "I want new running shoes") into the chat box.

[0493] Output: Text data of the consultation content

[0494] Step 3:

[0495] The server receives the consultation content sent by the user and analyzes the content using natural language processing technology.

[0496] Input: Text data of consultation content

[0497] The server uses a generative AI model to extract keywords and process the data to identify the type of procedure.

[0498] Output: Extracted keywords and procedure types

[0499] Step 4:

[0500] The server uses an emotion analysis engine to analyze the user's emotional state based on the consultation content.

[0501] Input: Text data of consultation content

[0502] The server uses an emotion analysis engine to perform data calculations to identify emotional states.

[0503] Output: Parsed emotional state (e.g. "Tired")

[0504] Step 5:

[0505] Based on the analysis results, the server generates messages suggesting products and procedures that take into account the emotional state.

[0506] Input: extracted keywords, procedure type, and emotional state

[0507] The server uses a generative AI model to generate messages to recommend optimal products and procedures to users.

[0508] Output: Proposal message

[0509] Step 6:

[0510] The terminal displays the suggestion message sent from the server in the chat box.

[0511] Input: Proposal message

[0512] The user reviews the proposed procedures and products and then begins filling out the necessary forms.

[0513] Output: Display of suggestion message

[0514] Step 7:

[0515] The server receives the procedure form, which the user fills out according to the suggestions.

[0516] Input: Text data of procedure details

[0517] The server references past procedure data and pre-fills and processes data in the form.

[0518] Output: Completed procedure form

[0519] Step 8:

[0520] The server stores the completed procedure details in a database.

[0521] Input: Completed procedure form

[0522] When the server has finished saving, it generates a procedure completion message and sends it to the terminal.

[0523] Output: Procedure complete message

[0524] Step 9:

[0525] The terminal displays a confirmation message to the user that the procedure has been completed.

[0526] Input: Procedure completion message

[0527] The user confirms the completion of the procedure and enters feedback if necessary.

[0528] Output: A confirmation message for completing the procedure and a feedback input screen

[0529] Step 10:

[0530] The server receives feedback from users and stores it in a database.

[0531] Input: Text data of feedback content

[0532] The server analyzes the stored feedback and performs data calculations to help improve the service.

[0533] Output: Stored feedback data

[0534] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0535] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0536] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0537] [Second embodiment]

[0538] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0539] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0540] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0541] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0542] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0543] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0544] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0545] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0546] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0547] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0548] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0549] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0550] This system provides technology that uses virtual space to complete city hall procedures online. The specific processing content of this system's program is explained below.

[0551] 1. Providing an authentication screen and user authentication

[0552] server

[0553] The server loads the necessary databases and AI models at system startup, initializes the system, verifies authentication information, and starts an interactive session for users who are successfully authenticated.

[0554] Terminal

[0555] When a user accesses the device, a login screen is displayed and they are prompted to enter their authentication information (user ID and password). If authentication is successful, an interactive session begins and a chat screen is displayed where they can enter their consultation details.

[0556] User

[0557] The user enters authentication information on the login screen, and once authentication is successful, enters the content of the consultation on the chat screen.

[0558] 2. Analysis of consultation content and presentation of contact points

[0559] server

[0560] The server receives the inquiry details entered by the user and analyzes them using artificial intelligence. Specifically, it uses natural language processing technology to extract keywords and identify the appropriate procedure counter. The artificial intelligence retrieves related procedure information from a database and suggests the most suitable counter for the user.

[0561] Terminal

[0562] The terminal displays the message sent from the server suggesting a contact point on the chat screen and guides the user on the next step.

[0563] User

[0564] The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[0565] 3. Provide and complete the procedure form

[0566] server

[0567] The server generates the necessary procedure form based on the proposed procedure window, and may pre-fill parts of the form using past procedure data. The generated form is then sent to the terminal.

[0568] Terminal

[0569] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[0570] User

[0571] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[0572] 4. Data storage and notification of completion of procedure

[0573] server

[0574] The server receives the procedure details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the completion of the procedure and sends it to the user. After the procedure is completed, it also sends a message to request feedback from the user.

[0575] Terminal

[0576] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[0577] User

[0578] Users will receive a confirmation message that the process has been completed, and if necessary, they can fill out a feedback form and submit their comments or suggestions for improvement.

[0579] Specific examples

[0580] As an example, let's take the case of changing an address. When a user logs in and types "I want to change my address" on the chat screen, the server analyzes the inquiry and suggests the office where the address change can be carried out. When the user follows the suggestions, enters their new address in the address change form and submits it, the server saves the data in its database and notifies the user that the procedure is complete. Through this series of procedures, the user can efficiently change their address without having to physically visit city hall.

[0581] The above is an embodiment of the present invention. This system allows users to efficiently complete city hall procedures from home, which is expected to significantly reduce time and effort.

[0582] The processing flow will be explained below.

[0583] Program processing steps

[0584] Authentication and Chat Start

[0585] Step 1:

[0586] The server establishes a database connection when the system starts up and loads the necessary tables and AI models into memory.

[0587] Step 2:

[0588] When a user accesses the device, a login screen is displayed and the user is prompted to enter their user ID and password.

[0589] Step 3:

[0590] The user enters the user ID and password on the login screen.

[0591] Step 4:

[0592] The server verifies the entered authentication information and creates a session if the authentication is successful.

[0593] Step 5:

[0594] If the authentication is successful, the terminal displays a chat screen and prompts the user to enter the content of the consultation.

[0595] Receiving and analyzing consultation content

[0596] Step 6:

[0597] The user inputs the content of the consultation on the chat screen and sends it.

[0598] Step 7:

[0599] The server analyzes the inquiry received from the user using natural language processing technology. Specifically, it extracts keywords from the input text and identifies the type of procedure.

[0600] Proposal of contact information and generation of procedure forms

[0601] Step 8:

[0602] The server identifies an appropriate procedure window based on the analyzed consultation content, generates a proposal message, and transmits it to the terminal.

[0603] Step 9:

[0604] The terminal displays the proposal message sent from the server on the chat screen.

[0605] Step 10:

[0606] The user checks the procedure window suggested by the server and selects the suggested procedure.

[0607] Step 11:

[0608] The server generates the appropriate procedure form based on the selected procedure, pre-populating the form with previous procedure data, if available.

[0609] Fill in the form and complete the procedure

[0610] Step 12:

[0611] The terminal displays the procedure form sent from the server and prompts the user to enter the necessary information.

[0612] Step 13:

[0613] The user enters the necessary information into the procedure form, checks the contents on the confirmation screen, and then presses the submit button.

[0614] Step 14:

[0615] The server receives the submitted procedure details and stores them in a database.

[0616] Step 15:

[0617] The server generates and sends a confirmation message to the user indicating that the transaction has been completed.

[0618] Collecting feedback

[0619] Step 16:

[0620] The terminal displays a feedback input screen along with a confirmation message that the procedure has been completed.

[0621] Step 17:

[0622] Users can submit their opinions and suggestions for improvement in a feedback form.

[0623] Step 18:

[0624] The server receives the feedback from the user and stores it in a database.

[0625] These are the specific processing steps of the virtual city hall online service program, which allows users to efficiently carry out city hall procedures from home.

[0626] Example 1

[0627] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0628] When completing procedures at city hall, users must physically visit the city hall, which is a time-consuming and labor-intensive process. Furthermore, procedures require users to manually fill out numerous documents, which increases the risk of errors. Another problem is that insufficient information is provided about procedures, which means users take a long time to identify the appropriate office and procedures.

[0629] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0630] In this invention, the server includes means for providing an interface for users to input authentication information for access, means for verifying the authentication information and starting an interactive session for users who are successfully authenticated, means for receiving the content of the consultation from the user, analyzing the content of the consultation, and using artificial intelligence including natural language processing technology to suggest an appropriate procedure window, means for generating the necessary procedure form based on the procedure window suggested by the artificial intelligence and providing the form with past procedure data pre-entered to the user, means for receiving the procedure content entered by the user and storing the procedure content in a database, and means for receiving feedback from the user. This enables users to efficiently complete procedures online without having to physically visit the city hall.

[0631] "Authentication information" refers to information such as user ID and password that a user enters to access a system.

[0632] An "interactive session" is a real-time communication process between a successfully authenticated user and a system.

[0633] "Consultation content" refers to questions or requests regarding procedures that users input into the system.

[0634] "Natural language processing technology" is a technology for analyzing text entered by a user and understanding its meaning.

[0635] "Artificial intelligence" refers to an automated intelligent system that analyzes the inquiry content entered by the user and suggests the appropriate procedure window.

[0636] A "procedure counter" is an interface on the system that provides the forms and information necessary to complete a specific government procedure.

[0637] A "procedure form" is an electronic format into which a user enters the information necessary to carry out a procedure.

[0638] A "database" is a system for storing and managing information such as procedure details and user feedback.

[0639] "Feedback" refers to opinions and suggestions for improvement that users provide to the system after completing a procedure.

[0640] This system provides technology that allows city hall procedures to be completed online using a virtual space. The detailed implementation method of this system is described below.

[0641] System Overview

[0642] The system operates primarily between three parties: the server, the terminal, and the user. It allows users to complete city hall procedures online through a series of processes, including initialization, user authentication, analysis of the consultation content, provision of a procedure form, data storage, and notification of procedure completion.

[0643] Hardware and software used

[0644] 1. Server

[0645] Hardware: A server with a powerful processor, ample memory, and SSD storage.

[0646] Software: Operating systems (e.g., Linux), database management systems (e.g., MySQL), artificial intelligence (AI) models (e.g., BERT or GPT), and authentication services.

[0647] 2. Terminal

[0648] Hardware: Computer or smartphone.

[0649] Software: Web browser (e.g. Google Chrome, Mozilla Firefox), chat interface.

[0650] System processing flow

[0651] 1. Providing an authentication screen and user authentication

[0652] Server: When the system starts up, the server loads the necessary database and AI model, initializes the system, verifies authentication information (user ID and password), and starts an interactive session for users who are successfully authenticated.

[0653] Terminal: When a user accesses the terminal, a login screen is displayed and they are prompted to enter their authentication information. If authentication is successful, an interactive session begins and a chat screen is displayed where they can enter their consultation details.

[0654] User: The user enters their authentication information on the login screen, and once authentication is successful, they enter the details of their inquiry on the chat screen.

[0655] 2. Analysis of consultation content and presentation of contact points

[0656] Server: The server receives the inquiry details entered by the user, extracts keywords using natural language processing technology, identifies the appropriate procedure counter, retrieves related procedure information from the database, and suggests the most suitable procedure counter to the user.

[0657] Terminal: The terminal displays the contact suggestion message sent from the server on the chat screen and guides the user on the next step.

[0658] User: The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[0659] 3. Provide and complete the procedure form

[0660] Server: The server generates the necessary transaction form based on the proposed transaction window. It may pre-fill parts of the form using past transaction data. The generated form is sent to the terminal.

[0661] Terminal: The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[0662] User: The user enters the required information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[0663] 4. Data storage and notification of completion of procedure

[0664] Server: The server receives the transaction details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the transaction completion and sends it to the user. After the transaction is completed, it also sends a message to request feedback from the user.

[0665] Terminal: The terminal displays a confirmation message for the user that the procedure has been completed and a feedback input screen.

[0666] User: The user receives a confirmation message that the process has been completed, and optionally fills in and submits a feedback form with their thoughts and suggestions for improvement.

[0667] Specific examples

[0668] Address change procedures

[0669] Consider the case where a user logs in and types "I want to change my address" on the chat screen. In this case, the server analyzes the consultation content, extracts the related keyword "address change," and identifies the appropriate contact point. The suggested contact point information is displayed on the terminal. When the user follows the suggestions, enters their new address in the address change form, and submits it, the server saves this information in the database and notifies them that the procedure is complete. This allows the user to change their address online without having to physically visit city hall.

[0670] Prompt Sentence Examples

[0671] Text format

[0672] When a user logs in and types "I want to change my address" on the chat screen, please tell us in detail how the system will guide them through the procedure and support them until it is completed.

[0673] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently carry out city hall procedures from home, and is expected to save time and effort.

[0674] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0675] Step 1: Initialize the system

[0676] server

[0677] When the system starts up, the server loads the necessary databases and AI models and initializes the system. Specifically, the server loads the necessary data and models from the file system or external storage and expands them into memory. It also reads and configures system configuration information and prepares authentication services and security protocols. This initialization process makes it possible to connect to various databases and issue security tokens.

[0678] Input: Database connection information, AI model file

[0679] Output: Initialization status of various services, system configuration complete

[0680] Step 2: Providing an authentication screen and user authentication

[0681] Terminal

[0682] When a user accesses the terminal, a login screen is displayed. This screen is designed using HTML and CSS and includes input fields for the user ID and password, as well as a login button.

[0683] User

[0684] The user enters authentication information on the login screen and presses the login button, which sends the entered user ID and password to the server.

[0685] server

[0686] The server receives the authentication information, compares it with the user information in its database, and if the comparison is successful, generates a session token and sends it to the device.

[0687] Input: User ID, Password

[0688] Output: Authentication result (success or failure), session token

[0689] Step 3: Analyze the consultation content and provide a contact point

[0690] Terminal

[0691] The device receives the session token sent from the server and displays a chat screen, which includes a text box for accepting user input and a send button.

[0692] User

[0693] The user inputs the content of the inquiry (for example, "I want to change my address") on the chat screen and presses the send button.

[0694] server

[0695] The server receives messages from users and analyzes their inquiries using natural language processing technology. Specifically, it uses an NLP model to extract keywords from the input text and retrieves related procedural information from a database. It then identifies the most appropriate procedure window and suggests it to the user.

[0696] Input: User message

[0697] Output: Proposal of procedure window

[0698] Step 4: Provide and complete the procedure form

[0699] server

[0700] The server generates the necessary procedure form based on the identified procedure window. This form is dynamically generated using an HTML template. It may also refer to past procedure data and pre-fill parts of the form. The generated form is sent to the terminal.

[0701] Terminal

[0702] The terminal displays the procedure form sent from the server, which contains fields for the user to fill in and prompts them for information using UI elements (text boxes, radio buttons, etc.).

[0703] User

[0704] The user enters the necessary information into the form, checks the details on the confirmation screen, and then presses the submit button. The entered information is sent from the terminal to the server.

[0705] Input: Procedure window information

[0706] Output: Procedure form

[0707] Step 5: Data storage and notification of completion

[0708] server

[0709] The server receives the transaction details submitted by the user and stores them in a database. If the transaction is successfully completed, it generates a confirmation message for the user and sends it to them. After the transaction is completed, it also sends a message to request feedback from the user.

[0710] Terminal

[0711] The terminal displays a confirmation message for the completion of the procedure and a feedback input screen to the user. The feedback input screen has a text area where the user can enter their opinions and suggestions for improvement.

[0712] User

[0713] The user receives a confirmation message that the procedure has been completed, and if necessary, enters opinions or suggestions for improvement in the feedback form and presses the submit button. The entered feedback information is sent to the server.

[0714] Input: Procedure details, user feedback

[0715] Output: Procedural information stored in the database, feedback confirmation message

[0716] These are the specific processing steps of the system program. This series of processes allows users to efficiently complete procedures online without having to physically visit city hall.

[0717] (Application example 1)

[0718] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0719] Today's consumers often encounter complex procedures and guidance when using virtual shopping malls. While they are expected to offer the same convenience and efficiency as offline shopping malls, traditional systems lack sufficient ways for users to quickly and efficiently complete procedures and guidance within a digital environment. Furthermore, as the number of inquiries and procedures from users increases, response times may be delayed, resulting in a decline in customer satisfaction.

[0720] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0721] In this invention, the server includes means for inputting authentication information for a user to access the virtual shopping mall, means for verifying the authentication information and starting an interactive session with a successfully authenticated user, means including artificial intelligence for receiving the content of a consultation from the user, analyzing the content of the consultation, and suggesting an appropriate procedure window, means for generating a required procedure form based on the procedure window suggested by the artificial intelligence and providing it to the user, means for receiving the procedure content input by the user and storing the procedure content in a database, and means for using an interactive agent in a virtual space to provide the user with procedure guidance and form input support, thereby enabling the user to quickly and efficiently complete procedures and guidance within the virtual shopping mall.

[0722] "User" refers to an individual or organization that uses the system to carry out procedures.

[0723] "Authentication information" refers to information such as user ID and password required when a user accesses a system.

[0724] An "interactive session" refers to a process in which a system and a user communicate with each other to proceed through a procedure.

[0725] "Consultation content" refers to the content of questions or requests entered by users into the system.

[0726] "Procedure counter" refers to the department or service where users should go to carry out specific procedures.

[0727] "Artificial intelligence" refers to technology that analyzes the content of users' inquiries within the system and suggests the most appropriate procedures.

[0728] "Natural language processing technology" refers to technology that enables computers to understand and analyze human language.

[0729] "Procedure form" refers to a format or page where a user can enter information necessary to proceed with a procedure.

[0730] "Database" refers to a storage device for centrally managing data such as user procedural information and feedback within the system.

[0731] An "interactive agent" refers to an automatic response system that assists users with procedures by interacting with them in a virtual space.

[0732] "Virtual space" refers to an imaginary space created using computer technology, and refers to a place where online activities take place.

[0733] The system allows users to quickly and efficiently complete complex procedures and directions within a virtual shopping mall.

[0734] Hardware and software used

[0735] 1. Server: This is the core part of the system. Hosting and database management are performed using AWS (Amazon Web Services).

[0736] 2. Terminal: A smartphone, smart glasses, or head-mounted display is used as the interface for users to access the system.

[0737] 3. Natural language processing technology: The content of the user's inquiry is analyzed using the Google Cloud Natural Language API.

[0738] 4. Database: Amazon RDS (Relational Database Service) is used to store procedural data and feedback.

[0739] 5. Chatbot: Use Dialogflow to build a chat interface for the information center.

[0740] System details

[0741] 1. Providing an authentication screen and user authentication

[0742] When the system starts up, the server loads the necessary database and AI model and initializes the system. It verifies authentication information and starts an interactive session for users who are successfully authenticated. When a user accesses the device, it displays a login screen and prompts them to enter their authentication information. The user enters their authentication information on the login screen, and if authentication is successful, a chat screen is displayed.

[0743] 2. Analysis of consultation content and provision of guidance

[0744] When a user enters their inquiry into the chat, the server receives the content and analyzes keywords using the Google Cloud Natural Language API. The server identifies the appropriate counter or store where to go to complete the procedure and presents that information to the user. The device then displays the guidance message sent from the server on the chat screen.

[0745] 3. Provide and complete the procedure form

[0746] When a specific procedure is required, the server generates a procedure form and can pre-fill some parts based on past data. The generated form is sent to the terminal and displayed to the user. The user enters the necessary information into the form, checks the contents on the confirmation screen, and then presses the submit button.

[0747] 4. Data storage and notification of completion of procedure

[0748] The server receives the procedure details submitted by the user and stores them in Amazon RDS. After saving, it generates a confirmation message confirming the completion of the procedure and sends it to the user. After the procedure is complete, it also sends a message requesting feedback from the user. The terminal displays a confirmation message confirming the completion of the procedure and a feedback input screen to the user. The user receives the confirmation message confirming the completion of the procedure and, if necessary, enters opinions or suggestions for improvement in the feedback form and submits it.

[0749] Specific examples

[0750] When a user enters "I would like to process a return" into the virtual shopping mall's information center, the server analyzes the inquiry and provides instructions on how to process the return. When the user enters the necessary information into the appropriate form and submits it, the data is saved by the server and the user is notified that the process has been completed.

[0751] Prompt Sentence Examples

[0752] User: "I want to process a return"

[0753] Server: "To begin your return, please enter the required information in the form below."

[0754] [Present Form]

[0755] User: [fill out form]

[0756] Server: "Your transaction is complete. Thank you for using our service."

[0757] This system allows users to carry out procedures efficiently and smoothly within a virtual space.

[0758] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0759] Step 1:

[0760] When a user accesses the terminal, a login screen is displayed and the user is prompted to enter authentication information (user ID and password). The entered authentication information is then sent to the server. The input data is the user ID and password character string.

[0761] Step 2:

[0762] The server compares the received authentication information with existing information in the database. If the authentication information matches, it determines that authentication is successful and sends a message to the terminal to start an interactive session. The data calculation is to confirm that the entered authentication information matches the information in the database. If authentication is successful, the output is a message to start an interactive session.

[0763] Step 3:

[0764] The terminal receives a message of successful authentication and displays a chat screen for the user. The user inputs the content of the consultation on the chat screen. The input data is the user's free-form text message.

[0765] Step 4:

[0766] The server receives the inquiry content from the user and extracts and analyzes keywords using the Google Cloud Natural Language API. Based on the extracted keywords, it identifies the appropriate procedure counter and generates a guidance message. Data calculation involves keyword extraction using natural language processing technology and matching of procedure information. The output is a guidance message regarding the appropriate procedure counter.

[0767] Step 5:

[0768] The terminal displays the guidance message sent from the server on the chat screen and guides the user on the next step. The message displayed is a guidance message generated by the server.

[0769] Step 6:

[0770] The server generates the necessary procedure form and automatically fills in some of it based on past procedure data. The generated form is then sent to the terminal. Data calculation involves referencing past data and automatically generating the form. The output is a procedure form containing the necessary items that the user must enter.

[0771] Step 7:

[0772] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information. The user enters the necessary information into the form and presses the submit button. The input data is the information entered into the form.

[0773] Step 8:

[0774] The server receives the procedure details submitted by the user and stores them in a database. After the data has been saved, it generates a confirmation message for the completion of the procedure and sends it to the terminal. The data operation is to store the received data in the database. The output is a confirmation message for the completion of the procedure.

[0775] Step 9:

[0776] The terminal displays the confirmation message sent from the server indicating completion of the procedure and the feedback input screen to the user. The user receives the confirmation message, enters their opinions and suggestions for improvement in the feedback form, and presses the send button. The input data is the information entered in the feedback form.

[0777] Step 10:

[0778] The server receives feedback from users and stores it in a database. This makes it possible to use it to improve the operation of the system. Data calculation is the storage of feedback data in the database. The output is confirmation that the data has been saved.

[0779] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0780] This system combines technology that uses virtual space to complete city hall procedures online with an emotion engine that analyzes users' emotions, providing a more sophisticated and user-friendly experience. The specific processing content of this system's program is explained below.

[0781] 1. Providing an authentication screen and user authentication

[0782] server

[0783] The server establishes a database connection at system startup, loads the necessary tables, AI models, and emotion engines into memory, verifies authentication information, and starts an interactive session for users who are successfully authenticated.

[0784] Terminal

[0785] When a user accesses the device, a login screen is displayed and the user is prompted to enter authentication information (user ID and password). If authentication is successful, an interactive session begins and a chat screen is displayed.

[0786] User

[0787] The user enters authentication information on the login screen, and once authentication is successful, enters the content of the consultation on the chat screen.

[0788] 2. Analysis of consultation content and presentation of contact points

[0789] server

[0790] The server receives the consultation details entered by the user and analyzes them using artificial intelligence. Specifically, it uses natural language processing technology to extract keywords and identify the type of procedure. It also uses an emotion engine to analyze the consultation details and identify the user's emotions. Based on the results of this analysis, the server generates suggested messages for the procedure counter that take the user's emotions into consideration.

[0791] Terminal

[0792] The terminal displays the message sent from the server suggesting a contact point on the chat screen and guides the user on the next step.

[0793] User

[0794] The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[0795] 3. Provide and complete the procedure form

[0796] server

[0797] The server generates the necessary procedure form based on the proposed procedure window, possibly pre-filling parts of the form using past procedure data, and sends the generated form to the terminal.

[0798] Terminal

[0799] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[0800] User

[0801] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[0802] 4. Data storage and notification of completion of procedure

[0803] server

[0804] The server receives the procedure details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the completion of the procedure and sends it to the user. After the procedure is completed, it also sends a message to request feedback from the user.

[0805] Terminal

[0806] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[0807] User

[0808] Users will receive a confirmation message that the process has been completed, and if necessary, they can fill out a feedback form and submit their comments or suggestions for improvement.

[0809] Example: Address change procedure

[0810] As an example, let's consider the case of changing one's address. When a user logs in and enters "I want to change my address" on the chat screen, the server analyzes the inquiry using natural language processing technology. In parallel, the emotion engine analyzes the user's emotions from the text and determines that the user is under stress. In this case, the server suggests the appropriate office to change their address, along with an encouraging message such as "We will respond promptly. We will help you with your address change." When the user follows the suggestions and enters their new address into the address change form and submits it, the server saves the data in its database and notifies them that the procedure is complete. At this time, the user's emotional data is also saved in their profile. Through this series of procedures, users can change their address efficiently and safely without having to physically visit city hall.

[0811] The above is an embodiment of the present invention. This system not only allows users to efficiently carry out city hall procedures from home, but also enables responses that take into consideration the user's feelings, creating a more user-friendly environment.

[0812] The processing flow will be explained below.

[0813] Program processing steps

[0814] Authentication and Chat Start

[0815] Step 1:

[0816] The server establishes a database connection when the system starts up and loads the necessary tables, AI models, and emotion engines into memory.

[0817] Step 2:

[0818] When a user accesses the device, a login screen is displayed and the user is prompted to enter their user ID and password.

[0819] Step 3:

[0820] The user enters the user ID and password on the login screen.

[0821] Step 4:

[0822] The server verifies the entered authentication information and creates a session if the authentication is successful.

[0823] Step 5:

[0824] If the authentication is successful, the terminal displays a chat screen and prompts the user to enter the content of the consultation.

[0825] Receiving and analyzing consultation content

[0826] Step 6:

[0827] The user inputs the content of the consultation on the chat screen and sends it.

[0828] Step 7:

[0829] The server analyzes the inquiry received from the user using natural language processing technology. Specifically, it extracts keywords from the input text and identifies the type of procedure.

[0830] Step 8:

[0831] The server uses an emotion engine to analyze the user's emotions from the consultation text. Specifically, it identifies the user's emotional state (e.g., stress, relief, satisfaction) from the tone and expression of the text.

[0832] Proposal of contact information and generation of procedure forms

[0833] Step 9:

[0834] The server then identifies the appropriate procedure window based on the analysis results, generates a suggested message that takes into account the user's emotions, and sends it to the device. For example, if a user is feeling stressed, it will add a message of encouragement or reassurance.

[0835] Step 10:

[0836] The terminal displays the proposed message sent from the server on the chat screen and guides the user on the next step.

[0837] Step 11:

[0838] The user checks the procedure window suggested by the server and selects the suggested procedure.

[0839] Step 12:

[0840] The server generates the appropriate procedure form based on the selected procedure, pre-populates the form with previous procedure data, if available, and presents the form in an emotionally sensitive and user-friendly interface.

[0841] Fill in the form and complete the procedure

[0842] Step 13:

[0843] The terminal displays the procedure form sent from the server and prompts the user to enter the necessary information.

[0844] Step 14:

[0845] The user enters the necessary information into the procedure form, checks the contents on the confirmation screen, and then presses the submit button.

[0846] Step 15:

[0847] The server receives the submitted procedure details and stores them in a database.

[0848] Step 16:

[0849] The server generates and sends a confirmation message to the user indicating that the procedure has been completed, and after the procedure is completed, it uses the emotion engine to analyze the user's emotional state again.

[0850] Collecting feedback

[0851] Step 17:

[0852] The terminal displays a feedback input screen along with a confirmation message that the procedure has been completed.

[0853] Step 18:

[0854] Users can submit their opinions and suggestions for improvement in a feedback form.

[0855] Step 19:

[0856] The server receives user feedback, stores it in a database along with emotional data, and identifies areas for improvement based on the feedback, which will be reflected in the next update.

[0857] These are the specific processing steps of the virtual city hall online service that combines the emotion engine, allowing users to carry out city hall procedures efficiently and safely from home.

[0858] Example 2

[0859] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0860] There is a demand for a user-friendly environment that allows users to efficiently complete procedures online without having to physically visit a government office, and that also provides responses that take users' emotions into consideration. However, conventional systems have difficulty taking users' emotional states into account, and have been unable to reduce their stress and anxiety. Therefore, the objective of this invention is to provide a system that analyzes users' emotions and provides appropriate procedural guidance based on this analysis.

[0861] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting authentication information, means for verifying the authentication information and starting an interactive session with a user who has been successfully authenticated, means including artificial intelligence for receiving the consultation content, analyzing the consultation content, and suggesting an appropriate procedure window, means for generating and providing the necessary procedure form based on the procedure window suggested by the artificial intelligence, means for receiving the procedure content and saving the procedure content in data storage, means for performing emotion analysis and generating a suggested message based on the user's emotional state, and means for recording the user's emotional data and reflecting it in the user's profile. This allows the user to efficiently complete procedures online without having to physically visit a government office, and further allows the user to proceed with the procedures while reducing stress and anxiety by receiving guidance that takes into account the user's emotional state.

[0862] "Authentication information" means data provided by a user to identify himself or herself when accessing a system, including, for example, a user ID and password.

[0863] An "interactive session" is an interactive communication initiated by the system with a successfully authenticated user, and includes the ability for the user to input procedural consultations and requests into the system.

[0864] "Consultation content" refers to the content of questions or requests regarding specific procedures that users input into the system.

[0865] A "procedure window" is the optimal window suggested by the system for handling a specific type of procedure, and includes information such as which procedural documents are required.

[0866] "Artificial intelligence" is a technology used to analyze the content of users' inquiries and suggest appropriate procedure points, and has functions including natural language processing and emotion analysis.

[0867] A "procedure form" is a digital document that allows a user to enter information necessary to perform a specific procedure.

[0868] "Data storage" means a database or storage device that securely stores user transaction details and other related information.

[0869] "Emotion analysis" is a technology that detects the emotional state from the text entered by the user and identifies emotional states such as stress or relief.

[0870] A "suggested message" is a message that includes appropriate guidance or words of encouragement for the user based on the results of sentiment analysis.

[0871] A "profile" is an individual record that includes information such as a user's emotional data and past procedure history.

[0872] This invention is a system that combines technology that uses virtual space to complete city hall procedures online with an emotion engine to provide users with an advanced, user-friendly experience. Specific embodiments of this system are described below.

[0873] This system consists of three components: a server, a terminal, and a user. The server uses an EC2 instance on AWS, and the database uses AWS RDS. The artificial intelligence model uses TensorFlow and GPT-4, and the sentiment analysis engine uses Hugging Face's Transformers. The terminal uses a web browser, and the front end provides a web application using React.

[0874] First, when the system starts, the server establishes a connection to the database and loads the necessary tables, AI models, and emotion engines into memory. When a user accesses the system, the terminal displays a login screen prompting the user to enter their user ID and password. The authentication information entered by the user is verified by the server, and if authentication is successful, an interactive session begins.

[0875] When a user inputs the details of their consultation, the details are sent from the device to the server. The server then uses natural language processing technology to analyze the details and identify the appropriate procedure. At the same time, it also analyzes the user's emotional state using an emotion analysis engine and generates a suggested procedure window message based on this.

[0876] The server sends the generated proposal message to the terminal, which then displays the message on the chat screen to guide the user on the next step. When the user follows the proposed procedure window and fills in the necessary procedure form, the server references past procedure data and pre-fills the necessary parts. This procedure form is displayed on the terminal, and the user enters the necessary information and submits the form.

[0877] The server saves the submitted procedure details in a database, generates a confirmation message for the user to confirm the completion of the procedure, and sends it to the user. After the procedure is completed, the server also sends a message to request feedback from the user, and saves the received feedback in the database.

[0878] As a specific example, let's consider the case of changing an address. When a user types "I want to change my address" on the chat screen, the server analyzes the content of the inquiry using natural language processing technology, while an emotion engine simultaneously analyzes the user's emotions from the text. If the server determines that the user is under stress, it suggests a contact point for changing the address, along with an encouraging message such as "We will respond promptly. We will help you with your address change." When the user enters their new address into the address change form and submits it as suggested, the server saves the data in its database and notifies the user that the procedure is complete. Through this series of procedures, the user can change their address efficiently and safely without having to physically visit city hall.

[0879] Below is an example of a prompt sentence.

[0880] "I want to change my address, but I've been under a lot of stress lately. How do I go about the process?"

[0881] In response to this prompt, the generative AI model uses natural language processing techniques to provide procedural and emotionally appropriate responses.

[0882] In this way, the system provides efficient and user-friendly online procedures while taking into consideration the user's feelings.

[0883] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0884] Step 1:

[0885] The server connects to the database when the system starts up and loads the necessary tables, AI models, and emotion engines into memory.

[0886] Input: Start the server

[0887] Output: Establishing a database connection and loading required resources

[0888] Specifically, a database connection is established using Python's sqlalchemy, and a model using TensorFlow and an emotion engine using Hugging Face's Transformers library are loaded into memory.

[0889] Step 2:

[0890] When a user accesses the system, the terminal displays a login screen and prompts the user to enter their user ID and password.

[0891] Input: User access

[0892] Output: Login screen displayed

[0893] Specifically, the front end using React displays a login screen and waits for user input.

[0894] Step 3:

[0895] The user enters the user ID and password on the login screen.

[0896] Input: Authentication information (user ID and password)

[0897] Output: Sending authentication information

[0898] Specifically, the user enters authentication information into the login form and presses the submit button.

[0899] Step 4:

[0900] The server receives the authentication information sent from the terminal and authenticates it by comparing it with information in a database. If authentication is successful, an interactive session begins.

[0901] Input: Credentials

[0902] Output: Interactive session started upon successful authentication

[0903] Specifically, a JWT token is used to verify authentication information, and an interactive session begins when a valid token is generated.

[0904] Step 5:

[0905] The terminal confirms that the authentication was successful and displays the chat screen.

[0906] Input: Authentication success notification

[0907] Output: Chat screen display

[0908] Specifically, the React component generates and displays the chat screen.

[0909] Step 6:

[0910] The user inputs the details of the procedure to be discussed on the chat screen.

[0911] Input: Consultation details

[0912] Output: Sending consultation details

[0913] Specifically, the user enters the content of the consultation in the chat box and presses the send button.

[0914] Step 7:

[0915] The server receives the consultation details sent from the device, extracts keywords using natural language processing technology, identifies the type of procedure, and analyzes the user's emotions using an emotion engine.

[0916] Input: Consultation details

[0917] Output: Procedure type and sentiment analysis results

[0918] Specifically, keywords are extracted using Python's NLTK library, and an emotion analysis model determines the user's emotional state.

[0919] Step 8:

[0920] The server generates a message suggesting a procedure window that takes into consideration the user's emotional state based on the analysis results.

[0921] Input: Keyword extraction results, sentiment analysis results

[0922] Output: Proposal message generated

[0923] Specifically, code that dynamically generates a template message is executed.

[0924] Step 9:

[0925] The terminal displays the proposal message sent from the server on the chat screen and guides the user on the next step.

[0926] Input: Proposal message

[0927] Output: Display on the chat screen

[0928] Specifically, the React component displays a suggestion message.

[0929] Step 10:

[0930] The user fills in the necessary procedure forms according to the suggested procedure window.

[0931] Input: Proposal message

[0932] Output: Procedure form input

[0933] Specifically, the user follows the suggestions, enters the necessary information into the procedure form, and presses the submit button.

[0934] Step 11:

[0935] The server stores the procedure details sent by the user in a data storage, generates a confirmation message for completing the procedure, and sends it to the user.

[0936] Input: Procedure details

[0937] Output: Save to data storage, generate confirmation message

[0938] Specifically, it runs SQL queries to validate and save data and generates a confirmation message.

[0939] Step 12:

[0940] The terminal receives a confirmation message that the transaction has been completed and displays it to the user, as well as a form to collect feedback.

[0941] Input:Confirmation message

[0942] Output: Display confirmation message, display feedback form

[0943] Specifically, the React component displays a confirmation message and a feedback form.

[0944] Step 13:

[0945] The user checks the confirmation message indicating that the procedure has been completed, and if necessary, enters opinions or suggestions for improvement in the feedback form and submits it.

[0946] Input: Feedback

[0947] Output: Send feedback

[0948] Specifically, the user enters their opinion in the feedback form and presses the send button.

[0949] Step 14:

[0950] The server receives the feedback sent by the user and stores it in a data storage.

[0951] Input: Feedback

[0952] Output: Save Feedback

[0953] Specifically, it executes SQL queries to store the feedback data.

[0954] (Application example 2)

[0955] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0956] Conventional online processing systems simply focus on users completing the necessary procedures, and rarely take into account the user's emotional state. This makes it difficult to reduce the stress and anxiety users experience while completing the process. Furthermore, online shopping systems often lack recommendations for products and services that are tailored to the user's emotions, potentially resulting in a decrease in satisfaction. This invention solves these problems and provides high-quality services that take into account the user's emotions.

[0957] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input authentication information for access, means for verifying the authentication information and starting an interactive session with a user who has been successfully authenticated, means including an emotion analysis engine for analyzing the emotional state of the user, and means for recommending appropriate products and services based on the emotion analysis results. This makes it possible to provide procedures and services that correspond to the emotional state of the user.

[0958] "Authentication information" refers to the information that a user enters to access a system, and is usually a user ID and password.

[0959] An "interactive session" is a two-way communication exchange initiated with a successfully authenticated user.

[0960] "Artificial intelligence" is a system that uses advanced computing technology to understand and analyze the content of users' inquiries and take appropriate steps and make suggestions.

[0961] A "processing counter" is a designated location or portal where users can carry out the procedures they need.

[0962] A "transaction form" means an electronic form into which a user enters necessary information and contains fields necessary to complete a particular transaction.

[0963] A "database" is a system for managing stored information and data and making it accessible when needed.

[0964] An "emotion analysis engine" is a technology that analyzes and identifies a user's emotional state from their text or voice.

[0965] "Product and service recommendation" is the act of suggesting appropriate products and services to users based on their analyzed emotional state.

[0966] "Feedback" refers to information such as opinions, impressions, and areas for improvement from users, which can be used to improve and optimize the system.

[0967] "Natural language processing technology" is a computational technology that has the ability to understand, analyze, and generate human language.

[0968] The system embodying this invention allows users to use an online virtual store and perform product recommendations and procedures based on emotion analysis. This system operates in cooperation with three entities: a server, a terminal, and a user.

[0969] 1. Authentication and starting an interactive session

[0970] At system startup, the server first establishes a database connection and loads the necessary tables, artificial intelligence models, and sentiment analysis engine into memory, thereby verifying authentication information and preparing to start an interactive session with the user if authentication is successful.

[0971] When a user accesses the device, a login screen is displayed and they are prompted to enter their authentication information. If the authentication is successful, an interactive session begins and a chat box is displayed.

[0972] Users enter their authentication information on the login screen, and once authentication is successful, they can enter their consultation details through the chat box.

[0973] 2. Analysis of consultation content and proposal of contact points

[0974] When the server receives the user's inquiry, it analyzes the content using natural language processing technology. It then uses an emotion analysis engine to identify the user's emotional state. Based on the analysis results, the server generates a procedure window or product recommendation message that takes the user's emotional state into consideration.

[0975] The terminal displays messages sent from the server, such as counter suggestions and product recommendations, in a chat box and guides the user on the next step.

[0976] The user checks the counters and products suggested by the server and follows the instructions to proceed to the next step.

[0977] 3. Provide and complete the procedure form

[0978] The server generates the necessary procedure form based on the proposed procedure window. It can pre-fill parts of the form using past procedure data. The generated form is sent to the terminal.

[0979] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[0980] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[0981] 4. Data storage and notification of completion of procedure

[0982] The server receives the procedure submitted by the user and stores it in a database. After storing it, it generates a confirmation message of the procedure completion and sends it to the user. Furthermore, after the procedure is completed, it also sends a message to receive feedback from the user.

[0983] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[0984] Users will receive a confirmation message that the process has been completed, and if necessary, can fill out and submit a feedback form with their opinions or suggestions for improvement.

[0985] Hardware and software used

[0986] This system uses the following hardware and software:

[0987] Hardware: Smartphone (iOS or Android)

[0988] Software: Python, OpenAI API, natural language processing library, sentiment analysis engine

[0989] Examples of concrete examples and prompts

[0990] As a concrete example, consider the case where a user inputs "I want new running shoes." In this case, the server performs natural language processing and sentiment analysis, and if it derives, for example, "I'm tired," it will recommend comfortable running shoes.

[0991] Example prompt sentence:

[0992] "I want new running shoes. I've been working out too hard lately and I feel tired."

[0993] In this way, it is possible to recommend products and services that suit the user's emotional state, providing a high-quality online shopping experience.

[0994] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0995] Step 1:

[0996] The user launches the app on their smartphone and enters their user ID and password on the login screen.

[0997] Input: User ID and password

[0998] The server checks the received authentication information against its database and, if authentication is successful, begins an interactive session.

[0999] Output: A command to display a message indicating successful authentication and a chat box.

[1000] Step 2:

[1001] Based on the authentication success message, the terminal displays a chat box to the user.

[1002] Input: Authentication success message

[1003] The user enters the question (e.g., "I want new running shoes") into the chat box.

[1004] Output: Text data of the consultation content

[1005] Step 3:

[1006] The server receives the consultation content sent by the user and analyzes the content using natural language processing technology.

[1007] Input: Text data of consultation content

[1008] The server uses a generative AI model to extract keywords and process the data to identify the type of procedure.

[1009] Output: Extracted keywords and procedure types

[1010] Step 4:

[1011] The server uses an emotion analysis engine to analyze the user's emotional state based on the consultation content.

[1012] Input: Text data of consultation content

[1013] The server uses an emotion analysis engine to perform data calculations to identify emotional states.

[1014] Output: Parsed emotional state (e.g. "Tired")

[1015] Step 5:

[1016] Based on the analysis results, the server generates messages suggesting products and procedures that take into account the emotional state.

[1017] Input: extracted keywords, procedure type, and emotional state

[1018] The server uses a generative AI model to generate messages to recommend optimal products and procedures to users.

[1019] Output: Proposal message

[1020] Step 6:

[1021] The terminal displays the suggestion message sent from the server in the chat box.

[1022] Input: Proposal message

[1023] The user reviews the proposed procedures and products and then begins filling out the necessary forms.

[1024] Output: Display of suggestion message

[1025] Step 7:

[1026] The server receives the procedure form, which the user fills out according to the suggestions.

[1027] Input: Text data of procedure details

[1028] The server references past procedure data and pre-fills and processes data in the form.

[1029] Output: Completed procedure form

[1030] Step 8:

[1031] The server stores the completed procedure details in a database.

[1032] Input: Completed procedure form

[1033] When the server has finished saving, it generates a procedure completion message and sends it to the terminal.

[1034] Output: Procedure complete message

[1035] Step 9:

[1036] The terminal displays a confirmation message to the user that the procedure has been completed.

[1037] Input: Procedure completion message

[1038] The user confirms the completion of the procedure and enters feedback if necessary.

[1039] Output: A confirmation message for completing the procedure and a feedback input screen

[1040] Step 10:

[1041] The server receives feedback from users and stores it in a database.

[1042] Input: Text data of feedback content

[1043] The server analyzes the stored feedback and performs data calculations to help improve the service.

[1044] Output: Stored feedback data

[1045] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1046] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1047] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1048] [Third embodiment]

[1049] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1050] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1051] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1052] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1053] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1054] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1055] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1056] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1057] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1058] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1059] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1060] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1061] This system provides technology that uses virtual space to complete city hall procedures online. The specific processing content of this system's program is explained below.

[1062] 1. Providing an authentication screen and user authentication

[1063] server

[1064] The server loads the necessary databases and AI models at system startup, initializes the system, verifies authentication information, and starts an interactive session for users who are successfully authenticated.

[1065] Terminal

[1066] When a user accesses the device, a login screen is displayed and they are prompted to enter their authentication information (user ID and password). If authentication is successful, an interactive session begins and a chat screen is displayed where they can enter their consultation details.

[1067] User

[1068] The user enters authentication information on the login screen, and once authentication is successful, enters the content of the consultation on the chat screen.

[1069] 2. Analysis of consultation content and presentation of contact points

[1070] server

[1071] The server receives the inquiry details entered by the user and analyzes them using artificial intelligence. Specifically, it uses natural language processing technology to extract keywords and identify the appropriate procedure counter. The artificial intelligence retrieves related procedure information from a database and suggests the most suitable counter for the user.

[1072] Terminal

[1073] The terminal displays the message sent from the server suggesting a contact point on the chat screen and guides the user on the next step.

[1074] User

[1075] The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[1076] 3. Provide and complete the procedure form

[1077] server

[1078] The server generates the necessary procedure form based on the proposed procedure window, and may pre-fill parts of the form using past procedure data. The generated form is then sent to the terminal.

[1079] Terminal

[1080] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[1081] User

[1082] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[1083] 4. Data storage and notification of completion of procedure

[1084] server

[1085] The server receives the procedure details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the completion of the procedure and sends it to the user. After the procedure is completed, it also sends a message to request feedback from the user.

[1086] Terminal

[1087] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[1088] User

[1089] Users will receive a confirmation message that the process has been completed, and if necessary, they can fill out a feedback form and submit their comments or suggestions for improvement.

[1090] Specific examples

[1091] As an example, let's take the case of changing an address. When a user logs in and types "I want to change my address" on the chat screen, the server analyzes the inquiry and suggests the office where the address change can be carried out. When the user follows the suggestions, enters their new address in the address change form and submits it, the server saves the data in its database and notifies the user that the procedure is complete. Through this series of procedures, the user can efficiently change their address without having to physically visit city hall.

[1092] The above is an embodiment of the present invention. This system allows users to efficiently complete city hall procedures from home, which is expected to significantly reduce time and effort.

[1093] The processing flow will be explained below.

[1094] Program processing steps

[1095] Authentication and Chat Start

[1096] Step 1:

[1097] The server establishes a database connection when the system starts up and loads the necessary tables and AI models into memory.

[1098] Step 2:

[1099] When a user accesses the device, a login screen is displayed and the user is prompted to enter their user ID and password.

[1100] Step 3:

[1101] The user enters the user ID and password on the login screen.

[1102] Step 4:

[1103] The server verifies the entered authentication information and creates a session if the authentication is successful.

[1104] Step 5:

[1105] If the authentication is successful, the terminal displays a chat screen and prompts the user to enter the content of the consultation.

[1106] Receiving and analyzing consultation content

[1107] Step 6:

[1108] The user inputs the content of the consultation on the chat screen and sends it.

[1109] Step 7:

[1110] The server analyzes the inquiry received from the user using natural language processing technology. Specifically, it extracts keywords from the input text and identifies the type of procedure.

[1111] Proposal of contact information and generation of procedure forms

[1112] Step 8:

[1113] The server identifies an appropriate procedure window based on the analyzed consultation content, generates a proposal message, and transmits it to the terminal.

[1114] Step 9:

[1115] The terminal displays the proposal message sent from the server on the chat screen.

[1116] Step 10:

[1117] The user checks the procedure window suggested by the server and selects the suggested procedure.

[1118] Step 11:

[1119] The server generates the appropriate procedure form based on the selected procedure, pre-populating the form with previous procedure data, if available.

[1120] Fill in the form and complete the procedure

[1121] Step 12:

[1122] The terminal displays the procedure form sent from the server and prompts the user to enter the necessary information.

[1123] Step 13:

[1124] The user enters the necessary information into the procedure form, checks the contents on the confirmation screen, and then presses the submit button.

[1125] Step 14:

[1126] The server receives the submitted procedure details and stores them in a database.

[1127] Step 15:

[1128] The server generates and sends a confirmation message to the user indicating that the transaction has been completed.

[1129] Collecting feedback

[1130] Step 16:

[1131] The terminal displays a feedback input screen along with a confirmation message that the procedure has been completed.

[1132] Step 17:

[1133] Users can submit their opinions and suggestions for improvement in a feedback form.

[1134] Step 18:

[1135] The server receives the feedback from the user and stores it in a database.

[1136] These are the specific processing steps of the virtual city hall online service program, which allows users to efficiently carry out city hall procedures from home.

[1137] Example 1

[1138] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1139] When completing procedures at city hall, users must physically visit the city hall, which is a time-consuming and labor-intensive process. Furthermore, procedures require users to manually fill out numerous documents, which increases the risk of errors. Another problem is that insufficient information is provided about procedures, which means users take a long time to identify the appropriate office and procedures.

[1140] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1141] In this invention, the server includes means for providing an interface for users to input authentication information for access, means for verifying the authentication information and starting an interactive session for users who are successfully authenticated, means for receiving the content of the consultation from the user, analyzing the content of the consultation, and using artificial intelligence including natural language processing technology to suggest an appropriate procedure window, means for generating the necessary procedure form based on the procedure window suggested by the artificial intelligence and providing the form with past procedure data pre-entered to the user, means for receiving the procedure content entered by the user and storing the procedure content in a database, and means for receiving feedback from the user. This enables users to efficiently complete procedures online without having to physically visit the city hall.

[1142] "Authentication information" refers to information such as user ID and password that a user enters to access a system.

[1143] An "interactive session" is a real-time communication process between a successfully authenticated user and a system.

[1144] "Consultation content" refers to questions or requests regarding procedures that users input into the system.

[1145] "Natural language processing technology" is a technology for analyzing text entered by a user and understanding its meaning.

[1146] "Artificial intelligence" refers to an automated intelligent system that analyzes the inquiry content entered by the user and suggests the appropriate procedure window.

[1147] A "procedure counter" is an interface on the system that provides the forms and information necessary to complete a specific government procedure.

[1148] A "procedure form" is an electronic format into which a user enters the information necessary to carry out a procedure.

[1149] A "database" is a system for storing and managing information such as procedure details and user feedback.

[1150] "Feedback" refers to opinions and suggestions for improvement that users provide to the system after completing a procedure.

[1151] This system provides technology that allows city hall procedures to be completed online using a virtual space. The detailed implementation method of this system is described below.

[1152] System Overview

[1153] The system operates primarily between three parties: the server, the terminal, and the user. It allows users to complete city hall procedures online through a series of processes, including initialization, user authentication, analysis of the consultation content, provision of a procedure form, data storage, and notification of procedure completion.

[1154] Hardware and software used

[1155] 1. Server

[1156] Hardware: A server with a powerful processor, ample memory, and SSD storage.

[1157] Software: Operating systems (e.g., Linux), database management systems (e.g., MySQL), artificial intelligence (AI) models (e.g., BERT or GPT), and authentication services.

[1158] 2. Terminal

[1159] Hardware: Computer or smartphone.

[1160] Software: Web browser (e.g. Google Chrome, Mozilla Firefox), chat interface.

[1161] System processing flow

[1162] 1. Providing an authentication screen and user authentication

[1163] Server: When the system starts up, the server loads the necessary database and AI model, initializes the system, verifies authentication information (user ID and password), and starts an interactive session for users who are successfully authenticated.

[1164] Terminal: When a user accesses the terminal, a login screen is displayed and they are prompted to enter their authentication information. If authentication is successful, an interactive session begins and a chat screen is displayed where they can enter their consultation details.

[1165] User: The user enters their authentication information on the login screen, and once authentication is successful, they enter the details of their inquiry on the chat screen.

[1166] 2. Analysis of consultation content and presentation of contact points

[1167] Server: The server receives the inquiry details entered by the user, extracts keywords using natural language processing technology, identifies the appropriate procedure counter, retrieves related procedure information from the database, and suggests the most suitable procedure counter to the user.

[1168] Terminal: The terminal displays the contact suggestion message sent from the server on the chat screen and guides the user on the next step.

[1169] User: The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[1170] 3. Provide and complete the procedure form

[1171] Server: The server generates the necessary transaction form based on the proposed transaction window. It may pre-fill parts of the form using past transaction data. The generated form is sent to the terminal.

[1172] Terminal: The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[1173] User: The user enters the required information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[1174] 4. Data storage and notification of completion of procedure

[1175] Server: The server receives the transaction details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the transaction completion and sends it to the user. After the transaction is completed, it also sends a message to request feedback from the user.

[1176] Terminal: The terminal displays a confirmation message for the user that the procedure has been completed and a feedback input screen.

[1177] User: The user receives a confirmation message that the process has been completed, and optionally fills in and submits a feedback form with their thoughts and suggestions for improvement.

[1178] Specific examples

[1179] Address change procedures

[1180] Consider the case where a user logs in and types "I want to change my address" on the chat screen. In this case, the server analyzes the consultation content, extracts the related keyword "address change," and identifies the appropriate contact point. The suggested contact point information is displayed on the terminal. When the user follows the suggestions, enters their new address in the address change form, and submits it, the server saves this information in the database and notifies them that the procedure is complete. This allows the user to change their address online without having to physically visit city hall.

[1181] Prompt Sentence Examples

[1182] Text format

[1183] When a user logs in and types "I want to change my address" on the chat screen, please tell us in detail how the system will guide them through the procedure and support them until it is completed.

[1184] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently carry out city hall procedures from home, and is expected to save time and effort.

[1185] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1186] Step 1: Initialize the system

[1187] server

[1188] When the system starts up, the server loads the necessary databases and AI models and initializes the system. Specifically, the server loads the necessary data and models from the file system or external storage and expands them into memory. It also reads and configures system configuration information and prepares authentication services and security protocols. This initialization process makes it possible to connect to various databases and issue security tokens.

[1189] Input: Database connection information, AI model file

[1190] Output: Initialization status of various services, system configuration complete

[1191] Step 2: Providing an authentication screen and user authentication

[1192] Terminal

[1193] When a user accesses the terminal, a login screen is displayed. This screen is designed using HTML and CSS and includes input fields for the user ID and password, as well as a login button.

[1194] User

[1195] The user enters authentication information on the login screen and presses the login button, which sends the entered user ID and password to the server.

[1196] server

[1197] The server receives the authentication information, compares it with the user information in its database, and if the comparison is successful, generates a session token and sends it to the device.

[1198] Input: User ID, Password

[1199] Output: Authentication result (success or failure), session token

[1200] Step 3: Analyze the consultation content and provide a contact point

[1201] Terminal

[1202] The device receives the session token sent from the server and displays a chat screen, which includes a text box for accepting user input and a send button.

[1203] User

[1204] The user inputs the content of the inquiry (for example, "I want to change my address") on the chat screen and presses the send button.

[1205] server

[1206] The server receives messages from users and analyzes their inquiries using natural language processing technology. Specifically, it uses an NLP model to extract keywords from the input text and retrieves related procedural information from a database. It then identifies the most appropriate procedure window and suggests it to the user.

[1207] Input: User message

[1208] Output: Proposal of procedure window

[1209] Step 4: Provide and complete the procedure form

[1210] server

[1211] The server generates the necessary procedure form based on the identified procedure window. This form is dynamically generated using an HTML template. It may also refer to past procedure data and pre-fill parts of the form. The generated form is sent to the terminal.

[1212] Terminal

[1213] The terminal displays the procedure form sent from the server, which contains fields for the user to fill in and prompts them for information using UI elements (text boxes, radio buttons, etc.).

[1214] User

[1215] The user enters the necessary information into the form, checks the details on the confirmation screen, and then presses the submit button. The entered information is sent from the terminal to the server.

[1216] Input: Procedure window information

[1217] Output: Procedure form

[1218] Step 5: Data storage and notification of completion

[1219] server

[1220] The server receives the transaction details submitted by the user and stores them in a database. If the transaction is successfully completed, it generates a confirmation message for the user and sends it to them. After the transaction is completed, it also sends a message to request feedback from the user.

[1221] Terminal

[1222] The terminal displays a confirmation message for the completion of the procedure and a feedback input screen to the user. The feedback input screen has a text area where the user can enter their opinions and suggestions for improvement.

[1223] User

[1224] The user receives a confirmation message that the procedure has been completed, and if necessary, enters opinions or suggestions for improvement in the feedback form and presses the submit button. The entered feedback information is sent to the server.

[1225] Input: Procedure details, user feedback

[1226] Output: Procedural information stored in the database, feedback confirmation message

[1227] These are the specific processing steps of the system program. This series of processes allows users to efficiently complete procedures online without having to physically visit city hall.

[1228] (Application example 1)

[1229] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1230] Today's consumers often encounter complex procedures and guidance when using virtual shopping malls. While they are expected to offer the same convenience and efficiency as offline shopping malls, traditional systems lack sufficient ways for users to quickly and efficiently complete procedures and guidance within a digital environment. Furthermore, as the number of inquiries and procedures from users increases, response times may be delayed, resulting in a decline in customer satisfaction.

[1231] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1232] In this invention, the server includes means for inputting authentication information for a user to access the virtual shopping mall, means for verifying the authentication information and starting an interactive session with a successfully authenticated user, means including artificial intelligence for receiving the content of a consultation from the user, analyzing the content of the consultation, and suggesting an appropriate procedure window, means for generating a required procedure form based on the procedure window suggested by the artificial intelligence and providing it to the user, means for receiving the procedure content input by the user and storing the procedure content in a database, and means for using an interactive agent in a virtual space to provide the user with procedure guidance and form input support, thereby enabling the user to quickly and efficiently complete procedures and guidance within the virtual shopping mall.

[1233] "User" refers to an individual or organization that uses the system to carry out procedures.

[1234] "Authentication information" refers to information such as user ID and password required when a user accesses a system.

[1235] An "interactive session" refers to a process in which a system and a user communicate with each other to proceed through a procedure.

[1236] "Consultation content" refers to the content of questions or requests entered by users into the system.

[1237] "Procedure counter" refers to the department or service where users should go to carry out specific procedures.

[1238] "Artificial intelligence" refers to technology that analyzes the content of users' inquiries within the system and suggests the most appropriate procedures.

[1239] "Natural language processing technology" refers to technology that enables computers to understand and analyze human language.

[1240] "Procedure form" refers to a format or page where a user can enter information necessary to proceed with a procedure.

[1241] "Database" refers to a storage device for centrally managing data such as user procedural information and feedback within the system.

[1242] An "interactive agent" refers to an automatic response system that assists users with procedures by interacting with them in a virtual space.

[1243] "Virtual space" refers to an imaginary space created using computer technology, and refers to a place where online activities take place.

[1244] The system allows users to quickly and efficiently complete complex procedures and directions within a virtual shopping mall.

[1245] Hardware and software used

[1246] 1. Server: This is the core part of the system. Hosting and database management are performed using AWS (Amazon Web Services).

[1247] 2. Terminal: A smartphone, smart glasses, or head-mounted display is used as the interface for users to access the system.

[1248] 3. Natural language processing technology: The content of the user's inquiry is analyzed using the Google Cloud Natural Language API.

[1249] 4. Database: Amazon RDS (Relational Database Service) is used to store procedural data and feedback.

[1250] 5. Chatbot: Use Dialogflow to build a chat interface for the information center.

[1251] System details

[1252] 1. Providing an authentication screen and user authentication

[1253] When the system starts up, the server loads the necessary database and AI model and initializes the system. It verifies authentication information and starts an interactive session for users who are successfully authenticated. When a user accesses the device, it displays a login screen and prompts them to enter their authentication information. The user enters their authentication information on the login screen, and if authentication is successful, a chat screen is displayed.

[1254] 2. Analysis of consultation content and provision of guidance

[1255] When a user enters their inquiry into the chat, the server receives the content and analyzes keywords using the Google Cloud Natural Language API. The server identifies the appropriate counter or store where to go to complete the procedure and presents that information to the user. The device then displays the guidance message sent from the server on the chat screen.

[1256] 3. Provide and complete the procedure form

[1257] When a specific procedure is required, the server generates a procedure form and can pre-fill some parts based on past data. The generated form is sent to the terminal and displayed to the user. The user enters the necessary information into the form, checks the contents on the confirmation screen, and then presses the submit button.

[1258] 4. Data storage and notification of completion of procedure

[1259] The server receives the procedure details submitted by the user and stores them in Amazon RDS. After saving, it generates a confirmation message confirming the completion of the procedure and sends it to the user. After the procedure is complete, it also sends a message requesting feedback from the user. The terminal displays a confirmation message confirming the completion of the procedure and a feedback input screen to the user. The user receives the confirmation message confirming the completion of the procedure and, if necessary, enters opinions or suggestions for improvement in the feedback form and submits it.

[1260] Specific examples

[1261] When a user enters "I would like to process a return" into the virtual shopping mall's information center, the server analyzes the inquiry and provides instructions on how to process the return. When the user enters the necessary information into the appropriate form and submits it, the data is saved by the server and the user is notified that the process has been completed.

[1262] Prompt Sentence Examples

[1263] User: "I want to process a return"

[1264] Server: "To begin your return, please enter the required information in the form below."

[1265] [Present Form]

[1266] User: [fill out form]

[1267] Server: "Your transaction is complete. Thank you for using our service."

[1268] This system allows users to carry out procedures efficiently and smoothly within a virtual space.

[1269] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1270] Step 1:

[1271] When a user accesses the terminal, a login screen is displayed and the user is prompted to enter authentication information (user ID and password). The entered authentication information is then sent to the server. The input data is the user ID and password character string.

[1272] Step 2:

[1273] The server compares the received authentication information with existing information in the database. If the authentication information matches, it determines that authentication is successful and sends a message to the terminal to start an interactive session. The data calculation is to confirm that the entered authentication information matches the information in the database. If authentication is successful, the output is a message to start an interactive session.

[1274] Step 3:

[1275] The terminal receives a message of successful authentication and displays a chat screen for the user. The user inputs the content of the consultation on the chat screen. The input data is the user's free-form text message.

[1276] Step 4:

[1277] The server receives the inquiry content from the user and extracts and analyzes keywords using the Google Cloud Natural Language API. Based on the extracted keywords, it identifies the appropriate procedure counter and generates a guidance message. Data calculation involves keyword extraction using natural language processing technology and matching of procedure information. The output is a guidance message regarding the appropriate procedure counter.

[1278] Step 5:

[1279] The terminal displays the guidance message sent from the server on the chat screen and guides the user on the next step. The message displayed is a guidance message generated by the server.

[1280] Step 6:

[1281] The server generates the necessary procedure form and automatically fills in some of it based on past procedure data. The generated form is then sent to the terminal. Data calculation involves referencing past data and automatically generating the form. The output is a procedure form containing the necessary items that the user must enter.

[1282] Step 7:

[1283] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information. The user enters the necessary information into the form and presses the submit button. The input data is the information entered into the form.

[1284] Step 8:

[1285] The server receives the procedure details submitted by the user and stores them in a database. After the data has been saved, it generates a confirmation message for the completion of the procedure and sends it to the terminal. The data operation is to store the received data in the database. The output is a confirmation message for the completion of the procedure.

[1286] Step 9:

[1287] The terminal displays the confirmation message sent from the server indicating completion of the procedure and the feedback input screen to the user. The user receives the confirmation message, enters their opinions and suggestions for improvement in the feedback form, and presses the send button. The input data is the information entered in the feedback form.

[1288] Step 10:

[1289] The server receives feedback from users and stores it in a database. This makes it possible to use it to improve the operation of the system. Data calculation is the storage of feedback data in the database. The output is confirmation that the data has been saved.

[1290] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1291] This system combines technology that uses virtual space to complete city hall procedures online with an emotion engine that analyzes users' emotions, providing a more sophisticated and user-friendly experience. The specific processing content of this system's program is explained below.

[1292] 1. Providing an authentication screen and user authentication

[1293] server

[1294] The server establishes a database connection at system startup, loads the necessary tables, AI models, and emotion engines into memory, verifies authentication information, and starts an interactive session for users who are successfully authenticated.

[1295] Terminal

[1296] When a user accesses the device, a login screen is displayed and the user is prompted to enter authentication information (user ID and password). If authentication is successful, an interactive session begins and a chat screen is displayed.

[1297] User

[1298] The user enters authentication information on the login screen, and once authentication is successful, enters the content of the consultation on the chat screen.

[1299] 2. Analysis of consultation content and presentation of contact points

[1300] server

[1301] The server receives the consultation details entered by the user and analyzes them using artificial intelligence. Specifically, it uses natural language processing technology to extract keywords and identify the type of procedure. It also uses an emotion engine to analyze the consultation details and identify the user's emotions. Based on the results of this analysis, the server generates suggested messages for the procedure counter that take the user's emotions into consideration.

[1302] Terminal

[1303] The terminal displays the message sent from the server suggesting a contact point on the chat screen and guides the user on the next step.

[1304] User

[1305] The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[1306] 3. Provide and complete the procedure form

[1307] server

[1308] The server generates the necessary procedure form based on the proposed procedure window, possibly pre-filling parts of the form using past procedure data, and sends the generated form to the terminal.

[1309] Terminal

[1310] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[1311] User

[1312] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[1313] 4. Data storage and notification of completion of procedure

[1314] server

[1315] The server receives the procedure details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the completion of the procedure and sends it to the user. After the procedure is completed, it also sends a message to request feedback from the user.

[1316] Terminal

[1317] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[1318] User

[1319] Users will receive a confirmation message that the process has been completed, and if necessary, they can fill out a feedback form and submit their comments or suggestions for improvement.

[1320] Example: Address change procedure

[1321] As an example, let's consider the case of changing one's address. When a user logs in and enters "I want to change my address" on the chat screen, the server analyzes the inquiry using natural language processing technology. In parallel, the emotion engine analyzes the user's emotions from the text and determines that the user is under stress. In this case, the server suggests the appropriate office to change their address, along with an encouraging message such as "We will respond promptly. We will help you with your address change." When the user follows the suggestions and enters their new address into the address change form and submits it, the server saves the data in its database and notifies them that the procedure is complete. At this time, the user's emotional data is also saved in their profile. Through this series of procedures, users can change their address efficiently and safely without having to physically visit city hall.

[1322] The above is an embodiment of the present invention. This system not only allows users to efficiently carry out city hall procedures from home, but also enables responses that take into consideration the user's feelings, creating a more user-friendly environment.

[1323] The processing flow will be explained below.

[1324] Program processing steps

[1325] Authentication and Chat Start

[1326] Step 1:

[1327] The server establishes a database connection when the system starts up and loads the necessary tables, AI models, and emotion engines into memory.

[1328] Step 2:

[1329] When a user accesses the device, a login screen is displayed and the user is prompted to enter their user ID and password.

[1330] Step 3:

[1331] The user enters the user ID and password on the login screen.

[1332] Step 4:

[1333] The server verifies the entered authentication information and creates a session if the authentication is successful.

[1334] Step 5:

[1335] If the authentication is successful, the terminal displays a chat screen and prompts the user to enter the content of the consultation.

[1336] Receiving and analyzing consultation content

[1337] Step 6:

[1338] The user inputs the content of the consultation on the chat screen and sends it.

[1339] Step 7:

[1340] The server analyzes the inquiry received from the user using natural language processing technology. Specifically, it extracts keywords from the input text and identifies the type of procedure.

[1341] Step 8:

[1342] The server uses an emotion engine to analyze the user's emotions from the consultation text. Specifically, it identifies the user's emotional state (e.g., stress, relief, satisfaction) from the tone and expression of the text.

[1343] Proposal of contact information and generation of procedure forms

[1344] Step 9:

[1345] The server then identifies the appropriate procedure window based on the analysis results, generates a suggested message that takes into account the user's emotions, and sends it to the device. For example, if a user is feeling stressed, it will add a message of encouragement or reassurance.

[1346] Step 10:

[1347] The terminal displays the proposed message sent from the server on the chat screen and guides the user on the next step.

[1348] Step 11:

[1349] The user checks the procedure window suggested by the server and selects the suggested procedure.

[1350] Step 12:

[1351] The server generates the appropriate procedure form based on the selected procedure, pre-populates the form with previous procedure data, if available, and presents the form in an emotionally sensitive and user-friendly interface.

[1352] Fill in the form and complete the procedure

[1353] Step 13:

[1354] The terminal displays the procedure form sent from the server and prompts the user to enter the necessary information.

[1355] Step 14:

[1356] The user enters the necessary information into the procedure form, checks the contents on the confirmation screen, and then presses the submit button.

[1357] Step 15:

[1358] The server receives the submitted procedure details and stores them in a database.

[1359] Step 16:

[1360] The server generates and sends a confirmation message to the user indicating that the procedure has been completed, and after the procedure is completed, it uses the emotion engine to analyze the user's emotional state again.

[1361] Collecting feedback

[1362] Step 17:

[1363] The terminal displays a feedback input screen along with a confirmation message that the procedure has been completed.

[1364] Step 18:

[1365] Users can submit their opinions and suggestions for improvement in a feedback form.

[1366] Step 19:

[1367] The server receives user feedback, stores it in a database along with emotional data, and identifies areas for improvement based on the feedback, which will be reflected in the next update.

[1368] These are the specific processing steps of the virtual city hall online service that combines the emotion engine, allowing users to carry out city hall procedures efficiently and safely from home.

[1369] Example 2

[1370] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1371] There is a demand for a user-friendly environment that allows users to efficiently complete procedures online without having to physically visit a government office, and that also provides responses that take users' emotions into consideration. However, conventional systems have difficulty taking users' emotional states into account, and have been unable to reduce their stress and anxiety. Therefore, the objective of this invention is to provide a system that analyzes users' emotions and provides appropriate procedural guidance based on this analysis.

[1372] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting authentication information, means for verifying the authentication information and starting an interactive session with a user who has been successfully authenticated, means including artificial intelligence for receiving the consultation content, analyzing the consultation content, and suggesting an appropriate procedure window, means for generating and providing the necessary procedure form based on the procedure window suggested by the artificial intelligence, means for receiving the procedure content and saving the procedure content in data storage, means for performing emotion analysis and generating a suggested message based on the user's emotional state, and means for recording the user's emotional data and reflecting it in the user's profile. This allows the user to efficiently complete procedures online without having to physically visit a government office, and further allows the user to proceed with the procedures while reducing stress and anxiety by receiving guidance that takes into account the user's emotional state.

[1373] "Authentication information" means data provided by a user to identify himself or herself when accessing a system, including, for example, a user ID and password.

[1374] An "interactive session" is an interactive communication initiated by the system with a successfully authenticated user, and includes the ability for the user to input procedural consultations and requests into the system.

[1375] "Consultation content" refers to the content of questions or requests regarding specific procedures that users input into the system.

[1376] A "procedure window" is the optimal window suggested by the system for handling a specific type of procedure, and includes information such as which procedural documents are required.

[1377] "Artificial intelligence" is a technology used to analyze the content of users' inquiries and suggest appropriate procedure points, and has functions including natural language processing and emotion analysis.

[1378] A "procedure form" is a digital document that allows a user to enter information necessary to perform a specific procedure.

[1379] "Data storage" means a database or storage device that securely stores user transaction details and other related information.

[1380] "Emotion analysis" is a technology that detects the emotional state from the text entered by the user and identifies emotional states such as stress or relief.

[1381] A "suggested message" is a message that includes appropriate guidance or words of encouragement for the user based on the results of sentiment analysis.

[1382] A "profile" is an individual record that includes information such as a user's emotional data and past procedure history.

[1383] This invention is a system that combines technology that uses virtual space to complete city hall procedures online with an emotion engine to provide users with an advanced, user-friendly experience. Specific embodiments of this system are described below.

[1384] This system consists of three components: a server, a terminal, and a user. The server uses an EC2 instance on AWS, and the database uses AWS RDS. The artificial intelligence model uses TensorFlow and GPT-4, and the sentiment analysis engine uses Hugging Face's Transformers. The terminal uses a web browser, and the front end provides a web application using React.

[1385] First, when the system starts, the server establishes a connection to the database and loads the necessary tables, AI models, and emotion engines into memory. When a user accesses the system, the terminal displays a login screen prompting the user to enter their user ID and password. The authentication information entered by the user is verified by the server, and if authentication is successful, an interactive session begins.

[1386] When a user inputs the details of their consultation, the details are sent from the device to the server. The server then uses natural language processing technology to analyze the details and identify the appropriate procedure. At the same time, it also analyzes the user's emotional state using an emotion analysis engine and generates a suggested procedure window message based on this.

[1387] The server sends the generated proposal message to the terminal, which then displays the message on the chat screen to guide the user on the next step. When the user follows the proposed procedure window and fills in the necessary procedure form, the server references past procedure data and pre-fills the necessary parts. This procedure form is displayed on the terminal, and the user enters the necessary information and submits the form.

[1388] The server saves the submitted procedure details in a database, generates a confirmation message for the user to confirm the completion of the procedure, and sends it to the user. After the procedure is completed, the server also sends a message to request feedback from the user, and saves the received feedback in the database.

[1389] As a specific example, let's consider the case of changing an address. When a user types "I want to change my address" on the chat screen, the server analyzes the content of the inquiry using natural language processing technology, while an emotion engine simultaneously analyzes the user's emotions from the text. If the server determines that the user is under stress, it suggests a contact point for changing the address, along with an encouraging message such as "We will respond promptly. We will help you with your address change." When the user enters their new address into the address change form and submits it as suggested, the server saves the data in its database and notifies the user that the procedure is complete. Through this series of procedures, the user can change their address efficiently and safely without having to physically visit city hall.

[1390] Below is an example of a prompt sentence.

[1391] "I want to change my address, but I've been under a lot of stress lately. How do I go about the process?"

[1392] In response to this prompt, the generative AI model uses natural language processing techniques to provide procedural and emotionally appropriate responses.

[1393] In this way, the system provides efficient and user-friendly online procedures while taking into consideration the user's feelings.

[1394] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1395] Step 1:

[1396] The server connects to the database when the system starts up and loads the necessary tables, AI models, and emotion engines into memory.

[1397] Input: Start the server

[1398] Output: Establishing a database connection and loading required resources

[1399] Specifically, a database connection is established using Python's sqlalchemy, and a model using TensorFlow and an emotion engine using Hugging Face's Transformers library are loaded into memory.

[1400] Step 2:

[1401] When a user accesses the system, the terminal displays a login screen and prompts the user to enter their user ID and password.

[1402] Input: User access

[1403] Output: Login screen displayed

[1404] Specifically, the front end using React displays a login screen and waits for user input.

[1405] Step 3:

[1406] The user enters the user ID and password on the login screen.

[1407] Input: Authentication information (user ID and password)

[1408] Output: Sending authentication information

[1409] Specifically, the user enters authentication information into the login form and presses the submit button.

[1410] Step 4:

[1411] The server receives the authentication information sent from the terminal and authenticates it by comparing it with information in a database. If authentication is successful, an interactive session begins.

[1412] Input: Credentials

[1413] Output: Interactive session started upon successful authentication

[1414] Specifically, a JWT token is used to verify authentication information, and an interactive session begins when a valid token is generated.

[1415] Step 5:

[1416] The terminal confirms that the authentication was successful and displays the chat screen.

[1417] Input: Authentication success notification

[1418] Output: Chat screen display

[1419] Specifically, the React component generates and displays the chat screen.

[1420] Step 6:

[1421] The user inputs the details of the procedure to be discussed on the chat screen.

[1422] Input: Consultation details

[1423] Output: Sending consultation details

[1424] Specifically, the user enters the content of the consultation in the chat box and presses the send button.

[1425] Step 7:

[1426] The server receives the consultation details sent from the device, extracts keywords using natural language processing technology, identifies the type of procedure, and analyzes the user's emotions using an emotion engine.

[1427] Input: Consultation details

[1428] Output: Procedure type and sentiment analysis results

[1429] Specifically, keywords are extracted using Python's NLTK library, and an emotion analysis model determines the user's emotional state.

[1430] Step 8:

[1431] The server generates a message suggesting a procedure window that takes into consideration the user's emotional state based on the analysis results.

[1432] Input: Keyword extraction results, sentiment analysis results

[1433] Output: Proposal message generated

[1434] Specifically, code that dynamically generates a template message is executed.

[1435] Step 9:

[1436] The terminal displays the proposal message sent from the server on the chat screen and guides the user on the next step.

[1437] Input: Proposal message

[1438] Output: Display on the chat screen

[1439] Specifically, the React component displays a suggestion message.

[1440] Step 10:

[1441] The user fills in the necessary procedure forms according to the suggested procedure window.

[1442] Input: Proposal message

[1443] Output: Procedure form input

[1444] Specifically, the user follows the suggestions, enters the necessary information into the procedure form, and presses the submit button.

[1445] Step 11:

[1446] The server stores the procedure details sent by the user in a data storage, generates a confirmation message for completing the procedure, and sends it to the user.

[1447] Input: Procedure details

[1448] Output: Save to data storage, generate confirmation message

[1449] Specifically, it runs SQL queries to validate and save data and generates a confirmation message.

[1450] Step 12:

[1451] The terminal receives a confirmation message that the transaction has been completed and displays it to the user, as well as a form to collect feedback.

[1452] Input:Confirmation message

[1453] Output: Display confirmation message, display feedback form

[1454] Specifically, the React component displays a confirmation message and a feedback form.

[1455] Step 13:

[1456] The user checks the confirmation message indicating that the procedure has been completed, and if necessary, enters opinions or suggestions for improvement in the feedback form and submits it.

[1457] Input: Feedback

[1458] Output: Send feedback

[1459] Specifically, the user enters their opinion in the feedback form and presses the send button.

[1460] Step 14:

[1461] The server receives the feedback sent by the user and stores it in a data storage.

[1462] Input: Feedback

[1463] Output: Save Feedback

[1464] Specifically, it executes SQL queries to store the feedback data.

[1465] (Application example 2)

[1466] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1467] Conventional online processing systems simply focus on users completing the necessary procedures, and rarely take into account the user's emotional state. This makes it difficult to reduce the stress and anxiety users experience while completing the process. Furthermore, online shopping systems often lack recommendations for products and services that are tailored to the user's emotions, potentially resulting in a decrease in satisfaction. This invention solves these problems and provides high-quality services that take into account the user's emotions.

[1468] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input authentication information for access, means for verifying the authentication information and starting an interactive session with a user who has been successfully authenticated, means including an emotion analysis engine for analyzing the emotional state of the user, and means for recommending appropriate products and services based on the emotion analysis results. This makes it possible to provide procedures and services that correspond to the emotional state of the user.

[1469] "Authentication information" refers to the information that a user enters to access a system, and is usually a user ID and password.

[1470] An "interactive session" is a two-way communication exchange initiated with a successfully authenticated user.

[1471] "Artificial intelligence" is a system that uses advanced computing technology to understand and analyze the content of users' inquiries and take appropriate steps and make suggestions.

[1472] A "processing counter" is a designated location or portal where users can carry out the procedures they need.

[1473] A "transaction form" means an electronic form into which a user enters necessary information and contains fields necessary to complete a particular transaction.

[1474] A "database" is a system for managing stored information and data and making it accessible when needed.

[1475] An "emotion analysis engine" is a technology that analyzes and identifies a user's emotional state from their text or voice.

[1476] "Product and service recommendation" is the act of suggesting appropriate products and services to users based on their analyzed emotional state.

[1477] "Feedback" refers to information such as opinions, impressions, and areas for improvement from users, which can be used to improve and optimize the system.

[1478] "Natural language processing technology" is a computational technology that has the ability to understand, analyze, and generate human language.

[1479] The system embodying this invention allows users to use an online virtual store and perform product recommendations and procedures based on emotion analysis. This system operates in cooperation with three entities: a server, a terminal, and a user.

[1480] 1. Authentication and starting an interactive session

[1481] At system startup, the server first establishes a database connection and loads the necessary tables, artificial intelligence models, and sentiment analysis engine into memory, thereby verifying authentication information and preparing to start an interactive session with the user if authentication is successful.

[1482] When a user accesses the device, a login screen is displayed and they are prompted to enter their authentication information. If the authentication is successful, an interactive session begins and a chat box is displayed.

[1483] Users enter their authentication information on the login screen, and once authentication is successful, they can enter their consultation details through the chat box.

[1484] 2. Analysis of consultation content and proposal of contact points

[1485] When the server receives the user's inquiry, it analyzes the content using natural language processing technology. It then uses an emotion analysis engine to identify the user's emotional state. Based on the analysis results, the server generates a procedure window or product recommendation message that takes the user's emotional state into consideration.

[1486] The terminal displays messages sent from the server, such as counter suggestions and product recommendations, in a chat box and guides the user on the next step.

[1487] The user checks the counters and products suggested by the server and follows the instructions to proceed to the next step.

[1488] 3. Provide and complete the procedure form

[1489] The server generates the necessary procedure form based on the proposed procedure window. It can pre-fill parts of the form using past procedure data. The generated form is sent to the terminal.

[1490] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[1491] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[1492] 4. Data storage and notification of completion of procedure

[1493] The server receives the procedure submitted by the user and stores it in a database. After storing it, it generates a confirmation message of the procedure completion and sends it to the user. Furthermore, after the procedure is completed, it also sends a message to receive feedback from the user.

[1494] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[1495] Users will receive a confirmation message that the process has been completed, and if necessary, can fill out and submit a feedback form with their opinions or suggestions for improvement.

[1496] Hardware and software used

[1497] This system uses the following hardware and software:

[1498] Hardware: Smartphone (iOS or Android)

[1499] Software: Python, OpenAI API, natural language processing library, sentiment analysis engine

[1500] Examples of concrete examples and prompts

[1501] As a concrete example, consider the case where a user inputs "I want new running shoes." In this case, the server performs natural language processing and sentiment analysis, and if it derives, for example, "I'm tired," it will recommend comfortable running shoes.

[1502] Example prompt sentence:

[1503] "I want new running shoes. I've been working out too hard lately and I feel tired."

[1504] In this way, it is possible to recommend products and services that suit the user's emotional state, providing a high-quality online shopping experience.

[1505] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1506] Step 1:

[1507] The user launches the app on their smartphone and enters their user ID and password on the login screen.

[1508] Input: User ID and password

[1509] The server checks the received authentication information against its database and, if authentication is successful, begins an interactive session.

[1510] Output: A command to display a message indicating successful authentication and a chat box.

[1511] Step 2:

[1512] Based on the authentication success message, the terminal displays a chat box to the user.

[1513] Input: Authentication success message

[1514] The user enters the question (e.g., "I want new running shoes") into the chat box.

[1515] Output: Text data of the consultation content

[1516] Step 3:

[1517] The server receives the consultation content sent by the user and analyzes the content using natural language processing technology.

[1518] Input: Text data of consultation content

[1519] The server uses a generative AI model to extract keywords and process the data to identify the type of procedure.

[1520] Output: Extracted keywords and procedure types

[1521] Step 4:

[1522] The server uses an emotion analysis engine to analyze the user's emotional state based on the consultation content.

[1523] Input: Text data of consultation content

[1524] The server uses an emotion analysis engine to perform data calculations to identify emotional states.

[1525] Output: Parsed emotional state (e.g. "Tired")

[1526] Step 5:

[1527] Based on the analysis results, the server generates messages suggesting products and procedures that take into account the emotional state.

[1528] Input: extracted keywords, procedure type, and emotional state

[1529] The server uses a generative AI model to generate messages to recommend optimal products and procedures to users.

[1530] Output: Proposal message

[1531] Step 6:

[1532] The terminal displays the suggestion message sent from the server in the chat box.

[1533] Input: Proposal message

[1534] The user reviews the proposed procedures and products and then begins filling out the necessary forms.

[1535] Output: Display of suggestion message

[1536] Step 7:

[1537] The server receives the procedure form, which the user fills out according to the suggestions.

[1538] Input: Text data of procedure details

[1539] The server references past procedure data and pre-fills and processes data in the form.

[1540] Output: Completed procedure form

[1541] Step 8:

[1542] The server stores the completed procedure details in a database.

[1543] Input: Completed procedure form

[1544] When the server has finished saving, it generates a procedure completion message and sends it to the terminal.

[1545] Output: Procedure complete message

[1546] Step 9:

[1547] The terminal displays a confirmation message to the user that the procedure has been completed.

[1548] Input: Procedure completion message

[1549] The user confirms the completion of the procedure and enters feedback if necessary.

[1550] Output: A confirmation message for completing the procedure and a feedback input screen

[1551] Step 10:

[1552] The server receives feedback from users and stores it in a database.

[1553] Input: Text data of feedback content

[1554] The server analyzes the stored feedback and performs data calculations to help improve the service.

[1555] Output: Stored feedback data

[1556] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1557] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1558] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1559] [Fourth embodiment]

[1560] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1561] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1562] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1563] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1564] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1565] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1566] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1567] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1568] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1569] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1570] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1571] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1572] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1573] This system provides technology that uses virtual space to complete city hall procedures online. The specific processing content of this system's program is explained below.

[1574] 1. Providing an authentication screen and user authentication

[1575] server

[1576] The server loads the necessary databases and AI models at system startup, initializes the system, verifies authentication information, and starts an interactive session for users who are successfully authenticated.

[1577] Terminal

[1578] When a user accesses the device, a login screen is displayed and they are prompted to enter their authentication information (user ID and password). If authentication is successful, an interactive session begins and a chat screen is displayed where they can enter their consultation details.

[1579] User

[1580] The user enters authentication information on the login screen, and once authentication is successful, enters the content of the consultation on the chat screen.

[1581] 2. Analysis of consultation content and presentation of contact points

[1582] server

[1583] The server receives the inquiry details entered by the user and analyzes them using artificial intelligence. Specifically, it uses natural language processing technology to extract keywords and identify the appropriate procedure counter. The artificial intelligence retrieves related procedure information from a database and suggests the most suitable counter for the user.

[1584] Terminal

[1585] The terminal displays the message sent from the server suggesting a contact point on the chat screen and guides the user on the next step.

[1586] User

[1587] The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[1588] 3. Provide and complete the procedure form

[1589] server

[1590] The server generates the necessary procedure form based on the proposed procedure window, and may pre-fill parts of the form using past procedure data. The generated form is then sent to the terminal.

[1591] Terminal

[1592] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[1593] User

[1594] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[1595] 4. Data storage and notification of completion of procedure

[1596] server

[1597] The server receives the procedure details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the completion of the procedure and sends it to the user. After the procedure is completed, it also sends a message to request feedback from the user.

[1598] Terminal

[1599] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[1600] User

[1601] Users will receive a confirmation message that the process has been completed, and if necessary, they can fill out a feedback form and submit their comments or suggestions for improvement.

[1602] Specific examples

[1603] As an example, let's take the case of changing an address. When a user logs in and types "I want to change my address" on the chat screen, the server analyzes the inquiry and suggests the office where the address change can be carried out. When the user follows the suggestions, enters their new address in the address change form and submits it, the server saves the data in its database and notifies the user that the procedure is complete. Through this series of procedures, the user can efficiently change their address without having to physically visit city hall.

[1604] The above is an embodiment of the present invention. This system allows users to efficiently complete city hall procedures from home, which is expected to significantly reduce time and effort.

[1605] The processing flow will be explained below.

[1606] Program processing steps

[1607] Authentication and Chat Start

[1608] Step 1:

[1609] The server establishes a database connection when the system starts up and loads the necessary tables and AI models into memory.

[1610] Step 2:

[1611] When a user accesses the device, a login screen is displayed and the user is prompted to enter their user ID and password.

[1612] Step 3:

[1613] The user enters the user ID and password on the login screen.

[1614] Step 4:

[1615] The server verifies the entered authentication information and creates a session if the authentication is successful.

[1616] Step 5:

[1617] If the authentication is successful, the terminal displays a chat screen and prompts the user to enter the content of the consultation.

[1618] Receiving and analyzing consultation content

[1619] Step 6:

[1620] The user inputs the content of the consultation on the chat screen and sends it.

[1621] Step 7:

[1622] The server analyzes the inquiry received from the user using natural language processing technology. Specifically, it extracts keywords from the input text and identifies the type of procedure.

[1623] Proposal of contact information and generation of procedure forms

[1624] Step 8:

[1625] The server identifies an appropriate procedure window based on the analyzed consultation content, generates a proposal message, and transmits it to the terminal.

[1626] Step 9:

[1627] The terminal displays the proposal message sent from the server on the chat screen.

[1628] Step 10:

[1629] The user checks the procedure window suggested by the server and selects the suggested procedure.

[1630] Step 11:

[1631] The server generates the appropriate procedure form based on the selected procedure, pre-populating the form with previous procedure data, if available.

[1632] Fill in the form and complete the procedure

[1633] Step 12:

[1634] The terminal displays the procedure form sent from the server and prompts the user to enter the necessary information.

[1635] Step 13:

[1636] The user enters the necessary information into the procedure form, checks the contents on the confirmation screen, and then presses the submit button.

[1637] Step 14:

[1638] The server receives the submitted procedure details and stores them in a database.

[1639] Step 15:

[1640] The server generates and sends a confirmation message to the user indicating that the transaction has been completed.

[1641] Collecting feedback

[1642] Step 16:

[1643] The terminal displays a feedback input screen along with a confirmation message that the procedure has been completed.

[1644] Step 17:

[1645] Users can submit their opinions and suggestions for improvement in a feedback form.

[1646] Step 18:

[1647] The server receives the feedback from the user and stores it in a database.

[1648] These are the specific processing steps of the virtual city hall online service program, which allows users to efficiently carry out city hall procedures from home.

[1649] Example 1

[1650] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1651] When completing procedures at city hall, users must physically visit the city hall, which is a time-consuming and labor-intensive process. Furthermore, procedures require users to manually fill out numerous documents, which increases the risk of errors. Another problem is that insufficient information is provided about procedures, which means users take a long time to identify the appropriate office and procedures.

[1652] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1653] In this invention, the server includes means for providing an interface for users to input authentication information for access, means for verifying the authentication information and starting an interactive session for users who are successfully authenticated, means for receiving the content of the consultation from the user, analyzing the content of the consultation, and using artificial intelligence including natural language processing technology to suggest an appropriate procedure window, means for generating the necessary procedure form based on the procedure window suggested by the artificial intelligence and providing the form with past procedure data pre-entered to the user, means for receiving the procedure content entered by the user and storing the procedure content in a database, and means for receiving feedback from the user. This enables users to efficiently complete procedures online without having to physically visit the city hall.

[1654] "Authentication information" refers to information such as user ID and password that a user enters to access a system.

[1655] An "interactive session" is a real-time communication process between a successfully authenticated user and a system.

[1656] "Consultation content" refers to questions or requests regarding procedures that users input into the system.

[1657] "Natural language processing technology" is a technology for analyzing text entered by a user and understanding its meaning.

[1658] "Artificial intelligence" refers to an automated intelligent system that analyzes the inquiry content entered by the user and suggests the appropriate procedure window.

[1659] A "procedure counter" is an interface on the system that provides the forms and information necessary to complete a specific government procedure.

[1660] A "procedure form" is an electronic format into which a user enters the information necessary to carry out a procedure.

[1661] A "database" is a system for storing and managing information such as procedure details and user feedback.

[1662] "Feedback" refers to opinions and suggestions for improvement that users provide to the system after completing a procedure.

[1663] This system provides technology that allows city hall procedures to be completed online using a virtual space. The detailed implementation method of this system is described below.

[1664] System Overview

[1665] The system operates primarily between three parties: the server, the terminal, and the user. It allows users to complete city hall procedures online through a series of processes, including initialization, user authentication, analysis of the consultation content, provision of a procedure form, data storage, and notification of procedure completion.

[1666] Hardware and software used

[1667] 1. Server

[1668] Hardware: A server with a powerful processor, ample memory, and SSD storage.

[1669] Software: Operating systems (e.g., Linux), database management systems (e.g., MySQL), artificial intelligence (AI) models (e.g., BERT or GPT), and authentication services.

[1670] 2. Terminal

[1671] Hardware: Computer or smartphone.

[1672] Software: Web browser (e.g. Google Chrome, Mozilla Firefox), chat interface.

[1673] System processing flow

[1674] 1. Providing an authentication screen and user authentication

[1675] Server: When the system starts up, the server loads the necessary database and AI model, initializes the system, verifies authentication information (user ID and password), and starts an interactive session for users who are successfully authenticated.

[1676] Terminal: When a user accesses the terminal, a login screen is displayed and they are prompted to enter their authentication information. If authentication is successful, an interactive session begins and a chat screen is displayed where they can enter their consultation details.

[1677] User: The user enters their authentication information on the login screen, and once authentication is successful, they enter the details of their inquiry on the chat screen.

[1678] 2. Analysis of consultation content and presentation of contact points

[1679] Server: The server receives the inquiry details entered by the user, extracts keywords using natural language processing technology, identifies the appropriate procedure counter, retrieves related procedure information from the database, and suggests the most suitable procedure counter to the user.

[1680] Terminal: The terminal displays the contact suggestion message sent from the server on the chat screen and guides the user on the next step.

[1681] User: The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[1682] 3. Provide and complete the procedure form

[1683] Server: The server generates the necessary transaction form based on the proposed transaction window. It may pre-fill parts of the form using past transaction data. The generated form is sent to the terminal.

[1684] Terminal: The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[1685] User: The user enters the required information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[1686] 4. Data storage and notification of completion of procedure

[1687] Server: The server receives the transaction details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the transaction completion and sends it to the user. After the transaction is completed, it also sends a message to request feedback from the user.

[1688] Terminal: The terminal displays a confirmation message for the user that the procedure has been completed and a feedback input screen.

[1689] User: The user receives a confirmation message that the process has been completed, and optionally fills in and submits a feedback form with their thoughts and suggestions for improvement.

[1690] Specific examples

[1691] Address change procedures

[1692] Consider the case where a user logs in and types "I want to change my address" on the chat screen. In this case, the server analyzes the consultation content, extracts the related keyword "address change," and identifies the appropriate contact point. The suggested contact point information is displayed on the terminal. When the user follows the suggestions, enters their new address in the address change form, and submits it, the server saves this information in the database and notifies them that the procedure is complete. This allows the user to change their address online without having to physically visit city hall.

[1693] Prompt Sentence Examples

[1694] Text format

[1695] When a user logs in and types "I want to change my address" on the chat screen, please tell us in detail how the system will guide them through the procedure and support them until it is completed.

[1696] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently carry out city hall procedures from home, and is expected to save time and effort.

[1697] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1698] Step 1: Initialize the system

[1699] server

[1700] When the system starts up, the server loads the necessary databases and AI models and initializes the system. Specifically, the server loads the necessary data and models from the file system or external storage and expands them into memory. It also reads and configures system configuration information and prepares authentication services and security protocols. This initialization process makes it possible to connect to various databases and issue security tokens.

[1701] Input: Database connection information, AI model file

[1702] Output: Initialization status of various services, system configuration complete

[1703] Step 2: Providing an authentication screen and user authentication

[1704] Terminal

[1705] When a user accesses the terminal, a login screen is displayed. This screen is designed using HTML and CSS and includes input fields for the user ID and password, as well as a login button.

[1706] User

[1707] The user enters authentication information on the login screen and presses the login button, which sends the entered user ID and password to the server.

[1708] server

[1709] The server receives the authentication information, compares it with the user information in its database, and if the comparison is successful, generates a session token and sends it to the device.

[1710] Input: User ID, Password

[1711] Output: Authentication result (success or failure), session token

[1712] Step 3: Analyze the consultation content and provide a contact point

[1713] Terminal

[1714] The device receives the session token sent from the server and displays a chat screen, which includes a text box for accepting user input and a send button.

[1715] User

[1716] The user inputs the content of the inquiry (for example, "I want to change my address") on the chat screen and presses the send button.

[1717] server

[1718] The server receives messages from users and analyzes their inquiries using natural language processing technology. Specifically, it uses an NLP model to extract keywords from the input text and retrieves related procedural information from a database. It then identifies the most appropriate procedure window and suggests it to the user.

[1719] Input: User message

[1720] Output: Proposal of procedure window

[1721] Step 4: Provide and complete the procedure form

[1722] server

[1723] The server generates the necessary procedure form based on the identified procedure window. This form is dynamically generated using an HTML template. It may also refer to past procedure data and pre-fill parts of the form. The generated form is sent to the terminal.

[1724] Terminal

[1725] The terminal displays the procedure form sent from the server, which contains fields for the user to fill in and prompts them for information using UI elements (text boxes, radio buttons, etc.).

[1726] User

[1727] The user enters the necessary information into the form, checks the details on the confirmation screen, and then presses the submit button. The entered information is sent from the terminal to the server.

[1728] Input: Procedure window information

[1729] Output: Procedure form

[1730] Step 5: Data storage and notification of completion

[1731] server

[1732] The server receives the transaction details submitted by the user and stores them in a database. If the transaction is successfully completed, it generates a confirmation message for the user and sends it to them. After the transaction is completed, it also sends a message to request feedback from the user.

[1733] Terminal

[1734] The terminal displays a confirmation message for the completion of the procedure and a feedback input screen to the user. The feedback input screen has a text area where the user can enter their opinions and suggestions for improvement.

[1735] User

[1736] The user receives a confirmation message that the procedure has been completed, and if necessary, enters opinions or suggestions for improvement in the feedback form and presses the submit button. The entered feedback information is sent to the server.

[1737] Input: Procedure details, user feedback

[1738] Output: Procedural information stored in the database, feedback confirmation message

[1739] These are the specific processing steps of the system program. This series of processes allows users to efficiently complete procedures online without having to physically visit city hall.

[1740] (Application example 1)

[1741] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1742] Today's consumers often encounter complex procedures and guidance when using virtual shopping malls. While they are expected to offer the same convenience and efficiency as offline shopping malls, traditional systems lack sufficient ways for users to quickly and efficiently complete procedures and guidance within a digital environment. Furthermore, as the number of inquiries and procedures from users increases, response times may be delayed, resulting in a decline in customer satisfaction.

[1743] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1744] In this invention, the server includes means for inputting authentication information for a user to access the virtual shopping mall, means for verifying the authentication information and starting an interactive session with a successfully authenticated user, means including artificial intelligence for receiving the content of a consultation from the user, analyzing the content of the consultation, and suggesting an appropriate procedure window, means for generating a required procedure form based on the procedure window suggested by the artificial intelligence and providing it to the user, means for receiving the procedure content input by the user and storing the procedure content in a database, and means for using an interactive agent in a virtual space to provide the user with procedure guidance and form input support, thereby enabling the user to quickly and efficiently complete procedures and guidance within the virtual shopping mall.

[1745] "User" refers to an individual or organization that uses the system to carry out procedures.

[1746] "Authentication information" refers to information such as user ID and password required when a user accesses a system.

[1747] An "interactive session" refers to a process in which a system and a user communicate with each other to proceed through a procedure.

[1748] "Consultation content" refers to the content of questions or requests entered by users into the system.

[1749] "Procedure counter" refers to the department or service where users should go to carry out specific procedures.

[1750] "Artificial intelligence" refers to technology that analyzes the content of users' inquiries within the system and suggests the most appropriate procedures.

[1751] "Natural language processing technology" refers to technology that enables computers to understand and analyze human language.

[1752] "Procedure form" refers to a format or page where a user can enter information necessary to proceed with a procedure.

[1753] "Database" refers to a storage device for centrally managing data such as user procedural information and feedback within the system.

[1754] An "interactive agent" refers to an automatic response system that assists users with procedures by interacting with them in a virtual space.

[1755] "Virtual space" refers to an imaginary space created using computer technology, and refers to a place where online activities take place.

[1756] The system allows users to quickly and efficiently complete complex procedures and directions within a virtual shopping mall.

[1757] Hardware and software used

[1758] 1. Server: This is the core part of the system. Hosting and database management are performed using AWS (Amazon Web Services).

[1759] 2. Terminal: A smartphone, smart glasses, or head-mounted display is used as the interface for users to access the system.

[1760] 3. Natural language processing technology: The content of the user's inquiry is analyzed using the Google Cloud Natural Language API.

[1761] 4. Database: Amazon RDS (Relational Database Service) is used to store procedural data and feedback.

[1762] 5. Chatbot: Use Dialogflow to build a chat interface for the information center.

[1763] System details

[1764] 1. Providing an authentication screen and user authentication

[1765] When the system starts up, the server loads the necessary database and AI model and initializes the system. It verifies authentication information and starts an interactive session for users who are successfully authenticated. When a user accesses the device, it displays a login screen and prompts them to enter their authentication information. The user enters their authentication information on the login screen, and if authentication is successful, a chat screen is displayed.

[1766] 2. Analysis of consultation content and provision of guidance

[1767] When a user enters their inquiry into the chat, the server receives the content and analyzes keywords using the Google Cloud Natural Language API. The server identifies the appropriate counter or store where to go to complete the procedure and presents that information to the user. The device then displays the guidance message sent from the server on the chat screen.

[1768] 3. Provide and complete the procedure form

[1769] When a specific procedure is required, the server generates a procedure form and can pre-fill some parts based on past data. The generated form is sent to the terminal and displayed to the user. The user enters the necessary information into the form, checks the contents on the confirmation screen, and then presses the submit button.

[1770] 4. Data storage and notification of completion of procedure

[1771] The server receives the procedure details submitted by the user and stores them in Amazon RDS. After saving, it generates a confirmation message confirming the completion of the procedure and sends it to the user. After the procedure is complete, it also sends a message requesting feedback from the user. The terminal displays a confirmation message confirming the completion of the procedure and a feedback input screen to the user. The user receives the confirmation message confirming the completion of the procedure and, if necessary, enters opinions or suggestions for improvement in the feedback form and submits it.

[1772] Specific examples

[1773] When a user enters "I would like to process a return" into the virtual shopping mall's information center, the server analyzes the inquiry and provides instructions on how to process the return. When the user enters the necessary information into the appropriate form and submits it, the data is saved by the server and the user is notified that the process has been completed.

[1774] Prompt Sentence Examples

[1775] User: "I want to process a return"

[1776] Server: "To begin your return, please enter the required information in the form below."

[1777] [Present Form]

[1778] User: [fill out form]

[1779] Server: "Your transaction is complete. Thank you for using our service."

[1780] This system allows users to carry out procedures efficiently and smoothly within a virtual space.

[1781] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1782] Step 1:

[1783] When a user accesses the terminal, a login screen is displayed and the user is prompted to enter authentication information (user ID and password). The entered authentication information is then sent to the server. The input data is the user ID and password character string.

[1784] Step 2:

[1785] The server compares the received authentication information with existing information in the database. If the authentication information matches, it determines that authentication is successful and sends a message to the terminal to start an interactive session. The data calculation is to confirm that the entered authentication information matches the information in the database. If authentication is successful, the output is a message to start an interactive session.

[1786] Step 3:

[1787] The terminal receives a message of successful authentication and displays a chat screen for the user. The user inputs the content of the consultation on the chat screen. The input data is the user's free-form text message.

[1788] Step 4:

[1789] The server receives the inquiry content from the user and extracts and analyzes keywords using the Google Cloud Natural Language API. Based on the extracted keywords, it identifies the appropriate procedure counter and generates a guidance message. Data calculation involves keyword extraction using natural language processing technology and matching of procedure information. The output is a guidance message regarding the appropriate procedure counter.

[1790] Step 5:

[1791] The terminal displays the guidance message sent from the server on the chat screen and guides the user on the next step. The message displayed is a guidance message generated by the server.

[1792] Step 6:

[1793] The server generates the necessary procedure form and automatically fills in some of it based on past procedure data. The generated form is then sent to the terminal. Data calculation involves referencing past data and automatically generating the form. The output is a procedure form containing the necessary items that the user must enter.

[1794] Step 7:

[1795] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information. The user enters the necessary information into the form and presses the submit button. The input data is the information entered into the form.

[1796] Step 8:

[1797] The server receives the procedure details submitted by the user and stores them in a database. After the data has been saved, it generates a confirmation message for the completion of the procedure and sends it to the terminal. The data operation is to store the received data in the database. The output is a confirmation message for the completion of the procedure.

[1798] Step 9:

[1799] The terminal displays the confirmation message sent from the server indicating completion of the procedure and the feedback input screen to the user. The user receives the confirmation message, enters their opinions and suggestions for improvement in the feedback form, and presses the send button. The input data is the information entered in the feedback form.

[1800] Step 10:

[1801] The server receives feedback from users and stores it in a database. This makes it possible to use it to improve the operation of the system. Data calculation is the storage of feedback data in the database. The output is confirmation that the data has been saved.

[1802] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1803] This system combines technology that uses virtual space to complete city hall procedures online with an emotion engine that analyzes users' emotions, providing a more sophisticated and user-friendly experience. The specific processing content of this system's program is explained below.

[1804] 1. Providing an authentication screen and user authentication

[1805] server

[1806] The server establishes a database connection at system startup, loads the necessary tables, AI models, and emotion engines into memory, verifies authentication information, and starts an interactive session for users who are successfully authenticated.

[1807] Terminal

[1808] When a user accesses the device, a login screen is displayed and the user is prompted to enter authentication information (user ID and password). If authentication is successful, an interactive session begins and a chat screen is displayed.

[1809] User

[1810] The user enters authentication information on the login screen, and once authentication is successful, enters the content of the consultation on the chat screen.

[1811] 2. Analysis of consultation content and presentation of contact points

[1812] server

[1813] The server receives the consultation details entered by the user and analyzes them using artificial intelligence. Specifically, it uses natural language processing technology to extract keywords and identify the type of procedure. It also uses an emotion engine to analyze the consultation details and identify the user's emotions. Based on the results of this analysis, the server generates suggested messages for the procedure counter that take the user's emotions into consideration.

[1814] Terminal

[1815] The terminal displays the message sent from the server suggesting a contact point on the chat screen and guides the user on the next step.

[1816] User

[1817] The user checks the procedure window suggested by the server and follows the instructions to proceed with the next procedure.

[1818] 3. Provide and complete the procedure form

[1819] server

[1820] The server generates the necessary procedure form based on the proposed procedure window, possibly pre-filling parts of the form using past procedure data, and sends the generated form to the terminal.

[1821] Terminal

[1822] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[1823] User

[1824] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[1825] 4. Data storage and notification of completion of procedure

[1826] server

[1827] The server receives the procedure details submitted by the user and stores them in a database. After storing them, it generates a confirmation message for the completion of the procedure and sends it to the user. After the procedure is completed, it also sends a message to request feedback from the user.

[1828] Terminal

[1829] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[1830] User

[1831] Users will receive a confirmation message that the process has been completed, and if necessary, they can fill out a feedback form and submit their comments or suggestions for improvement.

[1832] Example: Address change procedure

[1833] As an example, let's consider the case of changing one's address. When a user logs in and enters "I want to change my address" on the chat screen, the server analyzes the inquiry using natural language processing technology. In parallel, the emotion engine analyzes the user's emotions from the text and determines that the user is under stress. In this case, the server suggests the appropriate office to change their address, along with an encouraging message such as "We will respond promptly. We will help you with your address change." When the user follows the suggestions and enters their new address into the address change form and submits it, the server saves the data in its database and notifies them that the procedure is complete. At this time, the user's emotional data is also saved in their profile. Through this series of procedures, users can change their address efficiently and safely without having to physically visit city hall.

[1834] The above is an embodiment of the present invention. This system not only allows users to efficiently carry out city hall procedures from home, but also enables responses that take into consideration the user's feelings, creating a more user-friendly environment.

[1835] The processing flow will be explained below.

[1836] Program processing steps

[1837] Authentication and Chat Start

[1838] Step 1:

[1839] The server establishes a database connection when the system starts up and loads the necessary tables, AI models, and emotion engines into memory.

[1840] Step 2:

[1841] When a user accesses the device, a login screen is displayed and the user is prompted to enter their user ID and password.

[1842] Step 3:

[1843] The user enters the user ID and password on the login screen.

[1844] Step 4:

[1845] The server verifies the entered authentication information and creates a session if the authentication is successful.

[1846] Step 5:

[1847] If the authentication is successful, the terminal displays a chat screen and prompts the user to enter the content of the consultation.

[1848] Receiving and analyzing consultation content

[1849] Step 6:

[1850] The user inputs the content of the consultation on the chat screen and sends it.

[1851] Step 7:

[1852] The server analyzes the inquiry received from the user using natural language processing technology. Specifically, it extracts keywords from the input text and identifies the type of procedure.

[1853] Step 8:

[1854] The server uses an emotion engine to analyze the user's emotions from the consultation text. Specifically, it identifies the user's emotional state (e.g., stress, relief, satisfaction) from the tone and expression of the text.

[1855] Proposal of contact information and generation of procedure forms

[1856] Step 9:

[1857] The server then identifies the appropriate procedure window based on the analysis results, generates a suggested message that takes into account the user's emotions, and sends it to the device. For example, if a user is feeling stressed, it will add a message of encouragement or reassurance.

[1858] Step 10:

[1859] The terminal displays the proposed message sent from the server on the chat screen and guides the user on the next step.

[1860] Step 11:

[1861] The user checks the procedure window suggested by the server and selects the suggested procedure.

[1862] Step 12:

[1863] The server generates the appropriate procedure form based on the selected procedure, pre-populates the form with previous procedure data, if available, and presents the form in an emotionally sensitive and user-friendly interface.

[1864] Fill in the form and complete the procedure

[1865] Step 13:

[1866] The terminal displays the procedure form sent from the server and prompts the user to enter the necessary information.

[1867] Step 14:

[1868] The user enters the necessary information into the procedure form, checks the contents on the confirmation screen, and then presses the submit button.

[1869] Step 15:

[1870] The server receives the submitted procedure details and stores them in a database.

[1871] Step 16:

[1872] The server generates and sends a confirmation message to the user indicating that the procedure has been completed, and after the procedure is completed, it uses the emotion engine to analyze the user's emotional state again.

[1873] Collecting feedback

[1874] Step 17:

[1875] The terminal displays a feedback input screen along with a confirmation message that the procedure has been completed.

[1876] Step 18:

[1877] Users can submit their opinions and suggestions for improvement in a feedback form.

[1878] Step 19:

[1879] The server receives user feedback, stores it in a database along with emotional data, and identifies areas for improvement based on the feedback, which will be reflected in the next update.

[1880] These are the specific processing steps of the virtual city hall online service that combines the emotion engine, allowing users to carry out city hall procedures efficiently and safely from home.

[1881] Example 2

[1882] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1883] There is a demand for a user-friendly environment that allows users to efficiently complete procedures online without having to physically visit a government office, and that also provides responses that take users' emotions into consideration. However, conventional systems have difficulty taking users' emotional states into account, and have been unable to reduce their stress and anxiety. Therefore, the objective of this invention is to provide a system that analyzes users' emotions and provides appropriate procedural guidance based on this analysis.

[1884] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting authentication information, means for verifying the authentication information and starting an interactive session with a user who has been successfully authenticated, means including artificial intelligence for receiving the consultation content, analyzing the consultation content, and suggesting an appropriate procedure window, means for generating and providing the necessary procedure form based on the procedure window suggested by the artificial intelligence, means for receiving the procedure content and saving the procedure content in data storage, means for performing emotion analysis and generating a suggested message based on the user's emotional state, and means for recording the user's emotional data and reflecting it in the user's profile. This allows the user to efficiently complete procedures online without having to physically visit a government office, and further allows the user to proceed with the procedures while reducing stress and anxiety by receiving guidance that takes into account the user's emotional state.

[1885] "Authentication information" means data provided by a user to identify himself or herself when accessing a system, including, for example, a user ID and password.

[1886] An "interactive session" is an interactive communication initiated by the system with a successfully authenticated user, and includes the ability for the user to input procedural consultations and requests into the system.

[1887] "Consultation content" refers to the content of questions or requests regarding specific procedures that users input into the system.

[1888] A "procedure window" is the optimal window suggested by the system for handling a specific type of procedure, and includes information such as which procedural documents are required.

[1889] "Artificial intelligence" is a technology used to analyze the content of users' inquiries and suggest appropriate procedure points, and has functions including natural language processing and emotion analysis.

[1890] A "procedure form" is a digital document that allows a user to enter information necessary to perform a specific procedure.

[1891] "Data storage" means a database or storage device that securely stores user transaction details and other related information.

[1892] "Emotion analysis" is a technology that detects the emotional state from the text entered by the user and identifies emotional states such as stress or relief.

[1893] A "suggested message" is a message that includes appropriate guidance or words of encouragement for the user based on the results of sentiment analysis.

[1894] A "profile" is an individual record that includes information such as a user's emotional data and past procedure history.

[1895] This invention is a system that combines technology that uses virtual space to complete city hall procedures online with an emotion engine to provide users with an advanced, user-friendly experience. Specific embodiments of this system are described below.

[1896] This system consists of three components: a server, a terminal, and a user. The server uses an EC2 instance on AWS, and the database uses AWS RDS. The artificial intelligence model uses TensorFlow and GPT-4, and the sentiment analysis engine uses Hugging Face's Transformers. The terminal uses a web browser, and the front end provides a web application using React.

[1897] First, when the system starts, the server establishes a connection to the database and loads the necessary tables, AI models, and emotion engines into memory. When a user accesses the system, the terminal displays a login screen prompting the user to enter their user ID and password. The authentication information entered by the user is verified by the server, and if authentication is successful, an interactive session begins.

[1898] When a user inputs the details of their consultation, the details are sent from the device to the server. The server then uses natural language processing technology to analyze the details and identify the appropriate procedure. At the same time, it also analyzes the user's emotional state using an emotion analysis engine and generates a suggested procedure window message based on this.

[1899] The server sends the generated proposal message to the terminal, which then displays the message on the chat screen to guide the user on the next step. When the user follows the proposed procedure window and fills in the necessary procedure form, the server references past procedure data and pre-fills the necessary parts. This procedure form is displayed on the terminal, and the user enters the necessary information and submits the form.

[1900] The server saves the submitted procedure details in a database, generates a confirmation message for the user to confirm the completion of the procedure, and sends it to the user. After the procedure is completed, the server also sends a message to request feedback from the user, and saves the received feedback in the database.

[1901] As a specific example, let's consider the case of changing an address. When a user types "I want to change my address" on the chat screen, the server analyzes the content of the inquiry using natural language processing technology, while an emotion engine simultaneously analyzes the user's emotions from the text. If the server determines that the user is under stress, it suggests a contact point for changing the address, along with an encouraging message such as "We will respond promptly. We will help you with your address change." When the user enters their new address into the address change form and submits it as suggested, the server saves the data in its database and notifies the user that the procedure is complete. Through this series of procedures, the user can change their address efficiently and safely without having to physically visit city hall.

[1902] Below is an example of a prompt sentence.

[1903] "I want to change my address, but I've been under a lot of stress lately. How do I go about the process?"

[1904] In response to this prompt, the generative AI model uses natural language processing techniques to provide procedural and emotionally appropriate responses.

[1905] In this way, the system provides efficient and user-friendly online procedures while taking into consideration the user's feelings.

[1906] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1907] Step 1:

[1908] The server connects to the database when the system starts up and loads the necessary tables, AI models, and emotion engines into memory.

[1909] Input: Start the server

[1910] Output: Establishing a database connection and loading required resources

[1911] Specifically, a database connection is established using Python's sqlalchemy, and a model using TensorFlow and an emotion engine using Hugging Face's Transformers library are loaded into memory.

[1912] Step 2:

[1913] When a user accesses the system, the terminal displays a login screen and prompts the user to enter their user ID and password.

[1914] Input: User access

[1915] Output: Login screen displayed

[1916] Specifically, the front end using React displays a login screen and waits for user input.

[1917] Step 3:

[1918] The user enters the user ID and password on the login screen.

[1919] Input: Authentication information (user ID and password)

[1920] Output: Sending authentication information

[1921] Specifically, the user enters authentication information into the login form and presses the submit button.

[1922] Step 4:

[1923] The server receives the authentication information sent from the terminal and authenticates it by comparing it with information in a database. If authentication is successful, an interactive session begins.

[1924] Input: Credentials

[1925] Output: Interactive session started upon successful authentication

[1926] Specifically, a JWT token is used to verify authentication information, and an interactive session begins when a valid token is generated.

[1927] Step 5:

[1928] The terminal confirms that the authentication was successful and displays the chat screen.

[1929] Input: Authentication success notification

[1930] Output: Chat screen display

[1931] Specifically, the React component generates and displays the chat screen.

[1932] Step 6:

[1933] The user inputs the details of the procedure to be discussed on the chat screen.

[1934] Input: Consultation details

[1935] Output: Sending consultation details

[1936] Specifically, the user enters the content of the consultation in the chat box and presses the send button.

[1937] Step 7:

[1938] The server receives the consultation details sent from the device, extracts keywords using natural language processing technology, identifies the type of procedure, and analyzes the user's emotions using an emotion engine.

[1939] Input: Consultation details

[1940] Output: Procedure type and sentiment analysis results

[1941] Specifically, keywords are extracted using Python's NLTK library, and an emotion analysis model determines the user's emotional state.

[1942] Step 8:

[1943] The server generates a message suggesting a procedure window that takes into consideration the user's emotional state based on the analysis results.

[1944] Input: Keyword extraction results, sentiment analysis results

[1945] Output: Proposal message generated

[1946] Specifically, code that dynamically generates a template message is executed.

[1947] Step 9:

[1948] The terminal displays the proposal message sent from the server on the chat screen and guides the user on the next step.

[1949] Input: Proposal message

[1950] Output: Display on the chat screen

[1951] Specifically, the React component displays a suggestion message.

[1952] Step 10:

[1953] The user fills in the necessary procedure forms according to the suggested procedure window.

[1954] Input: Proposal message

[1955] Output: Procedure form input

[1956] Specifically, the user follows the suggestions, enters the necessary information into the procedure form, and presses the submit button.

[1957] Step 11:

[1958] The server stores the procedure details sent by the user in a data storage, generates a confirmation message for completing the procedure, and sends it to the user.

[1959] Input: Procedure details

[1960] Output: Save to data storage, generate confirmation message

[1961] Specifically, it runs SQL queries to validate and save data and generates a confirmation message.

[1962] Step 12:

[1963] The terminal receives a confirmation message that the transaction has been completed and displays it to the user, as well as a form to collect feedback.

[1964] Input:Confirmation message

[1965] Output: Display confirmation message, display feedback form

[1966] Specifically, the React component displays a confirmation message and a feedback form.

[1967] Step 13:

[1968] The user checks the confirmation message indicating that the procedure has been completed, and if necessary, enters opinions or suggestions for improvement in the feedback form and submits it.

[1969] Input: Feedback

[1970] Output: Send feedback

[1971] Specifically, the user enters their opinion in the feedback form and presses the send button.

[1972] Step 14:

[1973] The server receives the feedback sent by the user and stores it in a data storage.

[1974] Input: Feedback

[1975] Output: Save Feedback

[1976] Specifically, it executes SQL queries to store the feedback data.

[1977] (Application example 2)

[1978] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1979] Conventional online processing systems simply focus on users completing the necessary procedures, and rarely take into account the user's emotional state. This makes it difficult to reduce the stress and anxiety users experience while completing the process. Furthermore, online shopping systems often lack recommendations for products and services that are tailored to the user's emotions, potentially resulting in a decrease in satisfaction. This invention solves these problems and provides high-quality services that take into account the user's emotions.

[1980] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input authentication information for access, means for verifying the authentication information and starting an interactive session with a user who has been successfully authenticated, means including an emotion analysis engine for analyzing the emotional state of the user, and means for recommending appropriate products and services based on the emotion analysis results. This makes it possible to provide procedures and services that correspond to the emotional state of the user.

[1981] "Authentication information" refers to the information that a user enters to access a system, and is usually a user ID and password.

[1982] An "interactive session" is a two-way communication exchange initiated with a successfully authenticated user.

[1983] "Artificial intelligence" is a system that uses advanced computing technology to understand and analyze the content of users' inquiries and take appropriate steps and make suggestions.

[1984] A "processing counter" is a designated location or portal where users can carry out the procedures they need.

[1985] A "transaction form" means an electronic form into which a user enters necessary information and contains fields necessary to complete a particular transaction.

[1986] A "database" is a system for managing stored information and data and making it accessible when needed.

[1987] An "emotion analysis engine" is a technology that analyzes and identifies a user's emotional state from their text or voice.

[1988] "Product and service recommendation" is the act of suggesting appropriate products and services to users based on their analyzed emotional state.

[1989] "Feedback" refers to information such as opinions, impressions, and areas for improvement from users, which can be used to improve and optimize the system.

[1990] "Natural language processing technology" is a computational technology that has the ability to understand, analyze, and generate human language.

[1991] The system embodying this invention allows users to use an online virtual store and perform product recommendations and procedures based on emotion analysis. This system operates in cooperation with three entities: a server, a terminal, and a user.

[1992] 1. Authentication and starting an interactive session

[1993] At system startup, the server first establishes a database connection and loads the necessary tables, artificial intelligence models, and sentiment analysis engine into memory, thereby verifying authentication information and preparing to start an interactive session with the user if authentication is successful.

[1994] When a user accesses the device, a login screen is displayed and they are prompted to enter their authentication information. If the authentication is successful, an interactive session begins and a chat box is displayed.

[1995] Users enter their authentication information on the login screen, and once authentication is successful, they can enter their consultation details through the chat box.

[1996] 2. Analysis of consultation content and proposal of contact points

[1997] When the server receives the user's inquiry, it analyzes the content using natural language processing technology. It then uses an emotion analysis engine to identify the user's emotional state. Based on the analysis results, the server generates a procedure window or product recommendation message that takes the user's emotional state into consideration.

[1998] The terminal displays messages sent from the server, such as counter suggestions and product recommendations, in a chat box and guides the user on the next step.

[1999] The user checks the counters and products suggested by the server and follows the instructions to proceed to the next step.

[2000] 3. Provide and complete the procedure form

[2001] The server generates the necessary procedure form based on the proposed procedure window. It can pre-fill parts of the form using past procedure data. The generated form is sent to the terminal.

[2002] The terminal displays the procedure form sent from the server to the user and prompts them to enter the necessary information.

[2003] The user enters the necessary information into the displayed form, checks the contents on the confirmation screen, and then presses the submit button.

[2004] 4. Data storage and notification of completion of procedure

[2005] The server receives the procedure submitted by the user and stores it in a database. After storing it, it generates a confirmation message of the procedure completion and sends it to the user. Furthermore, after the procedure is completed, it also sends a message to receive feedback from the user.

[2006] The terminal displays a confirmation message for completing the procedure and a feedback input screen to the user.

[2007] Users will receive a confirmation message that the process has been completed, and if necessary, can fill out and submit a feedback form with their opinions or suggestions for improvement.

[2008] Hardware and software used

[2009] This system uses the following hardware and software:

[2010] Hardware: Smartphone (iOS or Android)

[2011] Software: Python, OpenAI API, natural language processing library, sentiment analysis engine

[2012] Examples of concrete examples and prompts

[2013] As a concrete example, consider the case where a user inputs "I want new running shoes." In this case, the server performs natural language processing and sentiment analysis, and if it derives, for example, "I'm tired," it will recommend comfortable running shoes.

[2014] Example prompt sentence:

[2015] "I want new running shoes. I've been working out too hard lately and I feel tired."

[2016] In this way, it is possible to recommend products and services that suit the user's emotional state, providing a high-quality online shopping experience.

[2017] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2018] Step 1:

[2019] The user launches the app on their smartphone and enters their user ID and password on the login screen.

[2020] Input: User ID and password

[2021] The server checks the received authentication information against its database and, if authentication is successful, begins an interactive session.

[2022] Output: A command to display a message indicating successful authentication and a chat box.

[2023] Step 2:

[2024] Based on the authentication success message, the terminal displays a chat box to the user.

[2025] Input: Authentication success message

[2026] The user enters the question (e.g., "I want new running shoes") into the chat box.

[2027] Output: Text data of the consultation content

[2028] Step 3:

[2029] The server receives the consultation content sent by the user and analyzes the content using natural language processing technology.

[2030] Input: Text data of consultation content

[2031] The server uses a generative AI model to extract keywords and process the data to identify the type of procedure.

[2032] Output: Extracted keywords and procedure types

[2033] Step 4:

[2034] The server uses an emotion analysis engine to analyze the user's emotional state based on the consultation content.

[2035] Input: Text data of consultation content

[2036] The server uses an emotion analysis engine to perform data calculations to identify emotional states.

[2037] Output: Parsed emotional state (e.g. "Tired")

[2038] Step 5:

[2039] Based on the analysis results, the server generates messages suggesting products and procedures that take into account the emotional state.

[2040] Input: extracted keywords, procedure type, and emotional state

[2041] The server uses a generative AI model to generate messages to recommend optimal products and procedures to users.

[2042] Output: Proposal message

[2043] Step 6:

[2044] The terminal displays the suggestion message sent from the server in the chat box.

[2045] Input: Proposal message

[2046] The user reviews the proposed procedures and products and then begins filling out the necessary forms.

[2047] Output: Display of suggestion message

[2048] Step 7:

[2049] The server receives the procedure form, which the user fills out according to the suggestions.

[2050] Input: Text data of procedure details

[2051] The server references past procedure data and pre-fills and processes data in the form.

[2052] Output: Completed procedure form

[2053] Step 8:

[2054] The server stores the completed procedure details in a database.

[2055] Input: Completed procedure form

[2056] When the server has finished saving, it generates a procedure completion message and sends it to the terminal.

[2057] Output: Procedure complete message

[2058] Step 9:

[2059] The terminal displays a confirmation message to the user that the procedure has been completed.

[2060] Input: Procedure completion message

[2061] The user confirms the completion of the procedure and enters feedback if necessary.

[2062] Output: A confirmation message for completing the procedure and a feedback input screen

[2063] Step 10:

[2064] The server receives feedback from users and stores it in a database.

[2065] Input: Text data of feedback content

[2066] The server analyzes the stored feedback and performs data calculations to help improve the service.

[2067] Output: Stored feedback data

[2068] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2069] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2070] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2071] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2072] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2073] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2074] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2075] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2076] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2077] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2078] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2079] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2080] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2081] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2082] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, ...

Claims

1. means for a user to enter access authentication information; means for verifying authentication information and initiating an interactive session for successfully authenticated users; A means including artificial intelligence for receiving the content of an inquiry from a user, analyzing the content of the inquiry, and suggesting an appropriate procedure window; A means for generating a necessary procedure form based on the procedure window suggested by the artificial intelligence and providing it to the user; A means for receiving the procedure details input by the user and storing the procedure details in a database; A system including:

2. means for sending a confirmation message after the user has completed the transaction; means for receiving feedback from users and storing the feedback in a database; The system of claim 1 further comprising:

3. A means for using natural language processing technology when the artificial intelligence analyzes the content of the user's consultation; A means for providing a form generated based on the proposed procedure window with the user's past procedure data pre-entered; The system of claim 1 , comprising:

Images