System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting a condition by a user; means for generating a virtual personality based on the condition; means for performing a chat between the virtual personality and the user; means for presenting a plurality of options during the chat and generating a feedback based on a selection of the user; and means for providing the feedback to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There are problems with online communication, with some not continuing well and others abandoning it midway. These problems affect a wide range of communication methods, from platonic friendships to coworkers and even business relationships. Furthermore, when interacting online, it is often difficult to find common topics or interests, and conversations often don't flow smoothly. There is a need for a system that removes these barriers and facilitates the steps to face-to-face meetings and interactions. [Means for solving the problem]

[0005] The present invention provides a system in which a user inputs target conditions and generates a virtual personality based on those conditions. The virtual personality chats with the user and presents multiple options during the chat. Feedback is generated and provided to the user based on the options selected by the user, improving online communication skills. The system also includes a means for recording the user's progress and utilizing this information in the next session, thereby providing continuous and effective support. Furthermore, the system includes a means for checking the format of the condition data entered by the user, preventing data entry errors and improving the reliability of the entire system.

[0006] "User" refers to an individual or corporation that uses this system.

[0007] "Conditions" refers to input data for generating a virtual personality, such as age, gender, place of origin, company, position, personality, family structure, address, and preferences.

[0008] "Virtual personality" refers to a simulated personality of a conversation partner that is generated based on conditions entered by the user.

[0009] "Chat" refers to a text-based interaction between a user and a virtual persona.

[0010] "Choices" refers to multiple response or action options presented to a user during a chat.

[0011] "Feedback" refers to suggestions and improvements generated based on the options selected by the user.

[0012] "Recording" refers to the act of saving data such as user chat sessions, choices, and feedback.

[0013] A "session" refers to a period of interaction during which a user initiates and terminates a chat with a virtual persona.

[0014] "Format check" refers to the process of checking whether the format and content of the condition data entered by the user are correct. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system that generates a virtual personality based on conditions input by a user and improves the user's communication skills through chat with that virtual personality.

[0037] System Overview

[0038] The system includes a means for a user to input conditions and generate a virtual personality based on the conditions, a means for chatting between the generated virtual personality and the user, a means for presenting multiple options during the chat, a means for generating feedback based on the user's selection, and a means for providing the feedback to the user. It also includes a means for recording the user's progress and utilizing the progress in the next session.

[0039] Explanation of program processing

[0040] 1. The user enters the conditions

[0041] User: Uses a device to enter the target conditions (age, gender, place of origin, company, position, personality, family structure, address, preferences, etc.).

[0042] Terminal: Checks the format of the entered data to make sure there are no errors. If there are no errors, the data is sent to the server.

[0043] 2. Send the data to the server

[0044] Terminal: Sends condition data serialized in JSON format to the server.

[0045] Server: Receives the request, parses the data, and obtains the user's criteria information.

[0046] 3. The server generates a virtual personality

[0047] Server: Based on the acquired condition information, a virtual personality is generated using a generative AI model (e.g., GPT-3, BERT). The generated virtual personality data is stored in storage and linked to the user's session information.

[0048] 4. Send virtual personality information to the user's device

[0049] Server: Sends the generated virtual personality data to the user's device as an HTTP response.

[0050] Terminal: Analyzes the virtual personality's profile information and displays it in the chat window.

[0051] 5. The user starts chatting with the virtual persona

[0052] User: Type a message in the chat window.

[0053] Terminal: Sends the entered message to the server.

[0054] Server: Analyzes the message and invokes the AI ​​model for the virtual personality to generate an appropriate response.

[0055] 6. The virtual personality responds and offers options

[0056] Server: Uses AI models to generate responses for the virtual personality, and also generates multiple options to present to the user.

[0057] Server: Sends the generated responses and options to the user's terminal.

[0058] Terminal: Displays the virtual personality's responses and options in the chat window.

[0059] 7. The user selects an option

[0060] User: Choose one of the options presented.

[0061] Terminal: Sends the selected option information to the server.

[0062] Server: Generates the next response or feedback based on the option information.

[0063] 8. Providing Feedback

[0064] Server: Generates feedback on the choices and provides suggestions and improvements to the user.

[0065] Server: Sends the generated feedback to the user's device.

[0066] Terminal: Show feedback in the chat window.

[0067] 9. Record and track user growth

[0068] Server: Records chat session data and stores it in a database for future sessions. This data is used to support the continued growth of our users.

[0069] Specific examples

[0070] Step 1: User enters criteria

[0071] User: Enter the target criteria as "male in his 20s, university student, tennis lover, living in Tokyo."

[0072] Terminal: Checks the format of the condition data, and if there are no errors, sends it to the server.

[0073] Step 2: Send the data to the server

[0074] Terminal: Converts the condition data into JSON format and sends it to the server.

[0075] Server: Receives data and extracts condition information.

[0076] Step 3: The server generates a virtual personality

[0077] Server: Generates a virtual personality using a generative AI model based on condition information.

[0078] Server: Stores the data of the generated virtual personality in storage and links it to session information.

[0079] Step 4: Send virtual personality information to the user's device

[0080] Server: Sends virtual personality data to the user's device.

[0081] Terminal: Display the virtual personality's profile in the chat window.

[0082] Step 5: The user starts chatting with the virtual persona

[0083] User: Type the message "Hi! You said you like tennis. How often do you play?" into the chat window.

[0084] Terminal: Sends a message to the server.

[0085] Server: Parses the message and invokes the AI ​​model to generate a response.

[0086] Step 6: The virtual persona responds and offers options

[0087] Server: The virtual personality responds with "Hello! I play two or three times a week. Who's Tanaka?" and generates multiple options.

[0088] Server: Sends the response and options to the user's terminal.

[0089] Terminal: Show responses and options in the chat window.

[0090] Step 7: User selects a choice

[0091] User: Select "Talk about another hobby" from the options.

[0092] Terminal: Sends the selection information to the server.

[0093] Server: Generates the next response or feedback.

[0094] Step 8: Provide feedback

[0095] Server: Generate feedback "When talking about different hobbies, it's important to explore common interests."

[0096] Server: Sends feedback to the user's device.

[0097] Terminal: Display feedback in the chat window.

[0098] Step 9: Record and track user growth

[0099] Server: Records chat session data and stores it in a database for future sessions.

[0100] In this way, users can comfortably practice their communication skills in a virtual environment and gain confidence in real-life face-to-face conversations.

[0101] The processing flow will be explained below.

[0102] Step 1:

[0103] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) into the condition input form on the device.

[0104] Step 2:

[0105] Terminal: Check the format of the entered data to ensure that the data format is correct and that required fields are not missing.

[0106] Step 3:

[0107] Terminal: Condition data that passes the format check is serialized into JSON format and sent to the server as an HTTP POST request.

[0108] Step 4:

[0109] Server: Analyzes the received request and extracts the condition data sent.

[0110] Step 5:

[0111] Server: Using the condition data, it sends a request to a generative AI model (e.g., GPT-3 or BERT) to generate a virtual personality.

[0112] Step 6:

[0113] Server: Stores the generated virtual personality data (such as name, age, personality, hobbies, etc.) in storage and links it to the user's session information.

[0114] Step 7:

[0115] Server: Sends the virtual personality data to the user's terminal as an HTTP response.

[0116] Step 8:

[0117] Terminal: Analyzes the received virtual personality data and displays profile information in the chat window.

[0118] Step 9:

[0119] User: Type a message in the chat window.

[0120] Step 10:

[0121] Terminal: Sends the entered message to the server.

[0122] Step 11:

[0123] Server: Analyzes the received message and generates an appropriate reply by calling an AI model for the virtual personality to respond.

[0124] Step 12:

[0125] Server: Generates multiple options to present to the user along with the responses of the generated virtual personality.

[0126] Step 13:

[0127] Server: Sends the virtual personality's responses and options to the user's terminal.

[0128] Step 14:

[0129] Terminal: Displays the virtual personality's replies and options in the chat window.

[0130] Step 15:

[0131] User: Select one of the options presented and send the selection to the terminal.

[0132] Step 16:

[0133] Terminal: Sends the selection information to the server.

[0134] Step 17:

[0135] Server: Generates the next response or feedback based on the selected information.

[0136] Step 18:

[0137] Server: Sends the generated feedback and the next response to the user's device.

[0138] Step 19:

[0139] Terminal: Show feedback and next response in the chat window.

[0140] Step 20:

[0141] Server: Records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions.

[0142] Example 1

[0143] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0144] Previous systems for improving communication skills had the problem of making it difficult for users to receive appropriate support. Specifically, it was difficult to generate a virtual personality based on the user's individual conditions, and there was a lack of means to continuously support the user's growth through dialogue with that virtual personality. Furthermore, there was a lack of technology for format checking of user input data and generating appropriate responses in real time, making it difficult to improve the user experience.

[0145] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0146] In this invention, the server includes: means for a user to input conditions; means for utilizing a generative AI model to generate a virtual personality based on the conditions; means for chatting between the virtual personality and the user; means for presenting multiple options during the chat and generating feedback based on the user's selection; means for providing the feedback to the user; means for displaying profile information of the virtual personality on a chat screen; and means for calling a generative AI model to analyze messages input by the user and generate appropriate responses. This allows users to continuously improve their communication skills through real-time interactions with virtual personalities based on their conditions.

[0147] "User" refers to a person who operates the system, inputs conditions, and interacts with a virtual personality.

[0148] "Conditions" refer to individual information such as age, sex, occupation, hobbies, and place of residence entered by the user.

[0149] A "virtual personality" refers to a virtual entity that can be used as a conversation partner and is generated based on the user's conditions using a generative AI model.

[0150] A "generative AI model" refers to an artificial intelligence model that generates a virtual personality using natural language processing based on conditions entered by the user.

[0151] "Chat" refers to a text-based interaction between a user and a virtual persona.

[0152] "Choices" refers to multiple response or reaction options that a user is presented with by a virtual persona during a chat.

[0153] "Feedback" refers to information such as suggestions and improvements provided based on the user's selections.

[0154] A "prompt" refers to an input instruction that causes a generative AI model to generate a specific response.

[0155] "Profile information" refers to information that a user can know, such as the virtual personality's personality, hobbies, and background.

[0156] "Session information" refers to the interaction history and associated data between a user and a virtual persona.

[0157] "Format check" refers to the process of checking whether the data entered by the user conforms to the specified format.

[0158] "HTTP response" refers to the data format returned from the server to the client (terminal).

[0159] This invention is a system that generates a virtual personality based on conditions input by a user and improves the user's communication skills through chatting with that virtual personality.

[0160] Hardware and Software Configuration

[0161] This system mainly consists of the following components:

[0162] 1. User's device: a device such as a personal computer, smartphone, or tablet.

[0163] 2. Server: Cloud server or local server.

[0164] 3. Generative AI models: Use natural language generation models such as GPT-3 and BERT.

[0165] Data processing and calculation

[0166] The system performs the following steps:

[0167] 1. The user enters the conditions

[0168] The user enters information such as age, gender, occupation, hobbies, and place of residence through the device interface. The device then performs a format check on the entered data to ensure there are no errors. If there are no errors, the data is serialized in JSON format and sent to the server.

[0169] 2. Acquire and analyze data

[0170] The server receives the condition data sent in JSON format, parses it, and extracts the user's condition information, which is then stored in temporary storage.

[0171] 3. Generate a virtual personality

[0172] The server sends a prompt to the generative AI model (e.g., GPT-3, BERT) based on the acquired condition information. The prompt includes the condition information entered by the user. For example, "Please generate a virtual personality based on the following conditions: age in his 20s, gender male, university student, tennis lover, residing in the Tokyo metropolitan area." The profile information of the generated virtual personality is saved in storage and linked to the user's session information.

[0173] 4. Send virtual personality information to the user device

[0174] The server sends the generated virtual personality data to the user's device as an HTTP response. The device analyzes this data and displays the virtual personality's profile (name, characteristics, hobbies, etc.) in a chat window.

[0175] 5. The user starts chatting with the virtual persona

[0176] The user types a message into the chat window, for example, "Hello! You said you like tennis. How often do you play?" The device then sends the typed message to the server.

[0177] 6. The virtual personality responds and offers options

[0178] The server analyzes the received message and calls the generative AI model again to generate a response from the virtual personality. For example, it generates a response like, "Hello! I play two or three times a week. How about you?" The generated response and multiple options (for example, "I play once a week," "Talk about another hobby," etc.) are sent to the user's device. The device displays the virtual personality's response and options in a chat window.

[0179] 7. The user selects an option

[0180] The user selects one of the options presented and confirms the choice by clicking or other operation. The device then sends information about the selected option to the server.

[0181] 8. Providing Feedback

[0182] The server generates feedback based on the user's choices, such as advice like, "When talking about different hobbies, it's important to find common interests." The generated feedback is sent to the user's device and displayed in the chat window.

[0183] 9. Record and track user growth

[0184] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for reference during the next session, which is used to support the user's continuous growth.

[0185] In this way, this system allows users to learn practical communication skills in a safe virtual environment and improve their ability to interact in the real world.

[0186] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0187] Step 1:

[0188] The user enters conditions (age, gender, occupation, hobbies, place of residence, etc.) using a device. The entered data is format-checked on the device to ensure there are no errors. If there are no errors, the data is serialized in JSON format.

[0189] Input: The user enters the conditions into the input form on the device.

[0190] Output: Serialized JSON formatted condition data

[0191] Step 2:

[0192] The device sends the condition data that has passed the format check to the server via an HTTP POST request. The server receives this request, parses the data, and extracts the user's condition information. The extracted data is saved in temporary storage.

[0193] Input: Condition data in serialized JSON format

[0194] Output: Parsed condition information data

[0195] Step 3:

[0196] The server then sends prompts to the generative AI model based on the extracted condition information. The prompts contain the user's condition information and prompt the model to generate speech content and a profile.

[0197] Input: Parsed condition information

[0198] Output: The prompt sent to the generative AI model

[0199] Step 4:

[0200] A generative AI model (e.g., GPT-3, BERT) generates a virtual personality profile based on the prompt text. This profile information is received on the server side, linked to the user's session information, and stored in storage.

[0201] Input: prompt statement

[0202] Output: Profile information of the generated virtual personality

[0203] Step 5:

[0204] The server sends the profile information of the virtual personality to the user's device as an HTTP response. The device receives this data, analyzes it, and displays the profile information (name, characteristics, hobbies, etc.) in the chat window.

[0205] Input: Profile information of the generated virtual personality

[0206] Output: Profile information displayed in the chat window

[0207] Step 6:

[0208] The user enters a message in the chat window. The device sends the message to the server, which receives the message and calls the generative AI model again to generate a response from the virtual personality.

[0209] Input: The message entered by the user

[0210] Output: A response message from the generative AI model

[0211] Step 7:

[0212] The server sends the generated response message and multiple options to the user's terminal, which receives it and displays the response message and options in a chat window.

[0213] Input: Response message from generative AI model, multiple choice

[0214] Output: Response message and options displayed in the chat window

[0215] Step 8:

[0216] The user selects one of the options presented and confirms the choice by clicking or other operation. The device then sends information about the selected option to the server.

[0217] Input: The choice selected by the user

[0218] Output: Choice information sent to the server

[0219] Step 9:

[0220] The server receives and analyzes the selection information and generates feedback, which is then sent back to the user's terminal and displayed in the terminal's chat window.

[0221] Input: Information about the option selected by the user

[0222] Output: Generated feedback

[0223] Step 10:

[0224] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for reference during the next session, to support the user's continuous growth.

[0225] Input: Chat session data

[0226] Output: Session information stored in the database

[0227] (Application example 1)

[0228] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0229] Security operators and guards are required to have advanced communication skills in order to respond quickly and appropriately to emergencies and daily security activities, but there is a lack of appropriate training methods and tools to efficiently improve these skills.

[0230] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0231] In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for generating a virtual personality based on a situation related to security activities, and means for communicating between the virtual personality and the user regarding an emergency. This enables security operators and security guards to efficiently improve the communication skills required for their work through virtual conversations based on actual situations.

[0232] The "means for user input" is a digital input interface that allows a user to provide specific requirements and situational information to the system.

[0233] The "means for generating a virtual personality based on the above conditions" is a technology that uses artificial intelligence to create a virtual conversation partner based on input condition information.

[0234] The "means for chatting between the virtual personality and the user" is a communication function that allows the generated virtual personality and the user to exchange text messages in real time.

[0235] "Means for presenting multiple options during chat and generating feedback based on the user's selection" is a mechanism that provides the user with several options during a conversation and generates an appropriate response or evaluation based on the selection.

[0236] The "means for providing the feedback to the user" is an output function for displaying or notifying the generated feedback or evaluation to the user.

[0237] The "means for generating a virtual personality based on a situation related to security activities" is a technology for generating a virtual personality based on a specific scenario related to security activities or emergency situations.

[0238] "Means for communication between the virtual personality and the user regarding an emergency" refers to a communication function for giving instructions and sharing information in an emergency through virtual dialogue.

[0239] This invention utilizes a virtual dialogue system to improve the effective communication skills of security operators and guards in emergency situations. The system is composed of a user, a terminal, and a server.

[0240] System Overview

[0241] The user uses a device to input specific conditions (for example, "Occupation: Security Operator," "Emergency: Fire," "Location: Office Building," "Instructions: Evacuation Instructions," etc.). The device checks the format of this condition data and then sends it to the server in JSON format. The server generates a virtual personality using a generative AI model (for example, GPT-2) based on the received condition information. The server then saves the virtual personality data and session information in storage and provides it to the user.

[0242] The user starts chatting with the virtual persona on the device. The message entered by the user is sent to the server, which uses an AI model to generate an appropriate response. The generated response and multiple options presented to the user are sent to the device and displayed in a chat window. The user selects one of the options, and feedback is generated and provided to the user based on that selection.

[0243] The system also includes a means to record the user's progress and use it for future sessions. This data supports the user's continuous improvement.

[0244] Hardware and software used

[0245] Hardware: Smartphone

[0246] Software: Python, generative AI model (GPT-2)

[0247] Explanation of data processing or data calculation

[0248] 1. Enter conditions and submit

[0249] The user inputs conditions such as "occupation: security operator," "emergency situation: fire," "location: office building," and "instructions: evacuation instructions."

[0250] The terminal checks the format of the condition data, and if there are no errors, sends it to the server in JSON format.

[0251] 2. Creation and provision of virtual personalities

[0252] The server generates a virtual personality using a generative AI model (GPT-2) based on the received condition information.

[0253] Data on the generated virtual personality is sent back to the user's terminal.

[0254] 3. Interacting with Virtual Personalities

[0255] The user inputs a message such as "Hello, there is an emergency. There is a fire. Please give us your instructions."

[0256] The server analyzes the received message using an AI model and generates an appropriate response.

[0257] The generated responses and options are displayed on the user's terminal.

[0258] 4. Providing Feedback

[0259] The server generates and provides appropriate feedback to the user based on the user's selection.

[0260] Provide specific advice and improvements to help users improve their skills.

[0261] Examples of concrete examples and prompts

[0262] Examples:

[0263] Enter the conditions: "Occupation: Security Operator", "Emergency: Fire", "Location: Office Building", "Instructions: Evacuation Instructions"

[0264] Initial message: "Hello, emergency. Fire. Your instructions needed."

[0265] Example prompt sentence:

[0266] Conditions: "Occupation: Security Operator", "Emergency: Fire", "Location: Office Building", "Instructions: Evacuation"

[0267] Start message: "Hello, emergency. Fire. Please respond."

[0268] In this way, users can safely practice their communication skills in a scenario that closely resembles real-world work.

[0269] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0270] Step 1:

[0271] The user inputs conditions. Using their smartphone, the user inputs conditions such as "Occupation: Security Operator," "Emergency: Fire," "Location: Office Building," and "Instructions: Evacuation Instructions." The device checks the format of the input data to ensure there are no errors. After checking, the device serializes the condition data into JSON format and sends it to the server.

[0272] Step 2:

[0273] The server receives the data. The server receives the condition data in JSON format sent from the device. The server parses the data, extracts the condition information, and prepares it as input data for use in the next processing step.

[0274] Step 3:

[0275] The server generates a virtual personality. Based on the extracted condition information, the server invokes a generative AI model (e.g., GPT-2) to generate the virtual personality. This generative model generates appropriate personality attributes based on the condition information and constructs data for the virtual personality. The data for the generated virtual personality is stored in the server's storage and linked to the user's session information.

[0276] Step 4:

[0277] The server sends information about the virtual personality to the user's device. The server then sends the generated virtual personality data to the user's device as an HTTP response. The device receives this data, analyzes it, and displays the virtual personality's profile information in the chat window.

[0278] Step 5:

[0279] The user starts chatting with the virtual persona. The user types a message into the chat window on the device: "Hello, there's an emergency. There's a fire. Please give us your instructions." The device sends the typed message to the server. The server receives the message, analyzes it, and calls the AI ​​model to generate an appropriate response from the virtual persona.

[0280] Step 6:

[0281] The server generates a response and presents the user with multiple options. The server uses an AI model to generate a response for the virtual personality, and then generates multiple options to present to the user. For example, a response such as "Please remain calm and head for the emergency exit" is generated. The generated response and options are sent to the user's device as an HTTP response. The device receives this and displays it in a chat window.

[0282] Step 7:

[0283] The user selects an option. The user selects one of the options presented and enters the selection information. The selection information is sent to the server via the device. The server again uses the AI ​​model to generate the next response or feedback based on the selected option information.

[0284] Step 8:

[0285] The server generates feedback and provides it to the user. The server generates feedback for the selected option and provides the user with suggestions and improvements. For example, the server might generate feedback such as "It is important to communicate instructions specifically and concisely." The generated feedback is sent to the user's device as an HTTP response and displayed in the chat window.

[0286] Step 9:

[0287] The server records the user's progress and uses it for the next session. The server records the chat session data and saves it in a database to use for the next session. This recorded data is used by the user to continuously improve their communication skills.

[0288] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0289] This system generates a virtual personality based on user input and improves the user's communication skills through chat with the virtual personality. It also incorporates an emotion engine that recognizes the user's emotions and provides feedback to improve the effectiveness of the chat.

[0290] System Overview

[0291] The system includes the following main means:

[0292] 1. A way for users to enter conditions

[0293] 2. Means for generating a virtual personality based on the above conditions

[0294] 3. Means for chatting between the virtual personality and the user

[0295] 4. A way to present multiple options during a chat and generate feedback based on the user's selection

[0296] 5. Means for providing said feedback to the user

[0297] 6. Means of recognizing user emotions using an emotion engine

[0298] 7. A way to track user progress and use it in the next session

[0299] 8. A means of formatting user-entered condition data

[0300] Explanation of program processing

[0301] 1. The user enters the conditions

[0302] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) on the device.

[0303] Terminal: Checks the format of the entered data and sends it to the server if there are no errors.

[0304] 2. Send the data to the server

[0305] Terminal: Serialize the condition data in JSON format and send it to the server as an HTTP POST request.

[0306] Server: Receives the request and parses the data to extract condition information.

[0307] 3. The server generates a virtual personality

[0308] Server: Sends a request to a generative AI model (e.g., GPT-3 or BERT) based on condition information to generate a virtual personality.

[0309] Server: Stores the virtual personality data (name, age, personality, hobbies, etc.) in storage and links it to the user's session information.

[0310] 4. Send virtual personality information to the user's device

[0311] Server: Sends the generated virtual personality data to the user's device as an HTTP response.

[0312] Terminal: Analyzes the virtual personality's profile information and displays it in the chat window.

[0313] 5. The user starts chatting with the virtual persona

[0314] User: Type a message in the chat window.

[0315] Terminal: Sends the entered message to the server.

[0316] Server: Analyzes the message and invokes the AI ​​model for the virtual personality to generate an appropriate response.

[0317] 6. The virtual personality responds and offers options

[0318] Server: Uses AI models to generate responses for the virtual personality, and also generates multiple options to present to the user.

[0319] Server: Sends the generated responses and options to the user's terminal.

[0320] Terminal: Displays the virtual personality's responses and options in the chat window.

[0321] 7. The user selects an option

[0322] User: Select one of the options presented and send the selection to the terminal.

[0323] Terminal: Sends the selection information to the server.

[0324] Server: Generates the next response or feedback based on the selected information.

[0325] 8. Providing Feedback

[0326] Server: Generates feedback on the choices and provides suggestions and improvements to the user.

[0327] Server: Sends the generated feedback to the user's device.

[0328] Terminal: Show feedback in the chat window.

[0329] 9. The Emotional Engine

[0330] Server: Analyzes the text entered by the user using the emotion engine and extracts emotion information.

[0331] Server: Adjusts the virtual personality's responses based on emotional information and generates dialogue that takes the user's emotional state into account.

[0332] 10. Record and track user growth

[0333] Server: Records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions. This data is used to support the user's ongoing growth.

[0334] Specific examples

[0335] Step 1: User enters criteria

[0336] User: Enters the following criteria into the input form on the device: "Male in his 20s, university student, tennis lover, living in Tokyo."

[0337] Terminal: Checks the format of the condition data, and if there are no errors, sends it to the server.

[0338] Step 2: Send the data to the server

[0339] Terminal: Serialize the condition data into JSON format and send it to the server as an HTTP POST request.

[0340] Server: Parse the data and extract the condition information.

[0341] Step 3: The server generates a virtual personality

[0342] Server: Based on the condition information, a virtual personality "Yamada-kun" is generated using a generative AI model.

[0343] Server: Stores the virtual personality data in storage and links it to session information.

[0344] Step 4: Send virtual personality information to the user's device

[0345] Server: Sends the generated virtual personality data to the user's device.

[0346] Terminal: Display the virtual personality's profile information in the chat window.

[0347] Step 5: The user starts chatting with the virtual persona

[0348] User: Type in the chat window, "Hi! You said you like tennis. How often do you play?"

[0349] Terminal: Sends the entered message to the server.

[0350] Server: Parses the message and invokes the AI ​​model to generate a response.

[0351] Step 6: The virtual persona responds and offers options

[0352] Server: Using an AI model, it generates the virtual personality's response, "Hello! I play two or three times a week. What about you?", while also generating options such as "Continue talking about tennis," "Go back to my introduction," or "Talk about another hobby."

[0353] Server: Sends the response and options to the user's terminal.

[0354] Terminal: Show responses and options in the chat window.

[0355] Step 7: User selects a choice

[0356] User: Select "Talk about another hobby" from the options.

[0357] Terminal: Sends the selection information to the server.

[0358] Server: Generates the next response or feedback.

[0359] Step 8: Provide feedback

[0360] Server: Generate feedback for the choice: "Talking about other hobbies is a good opportunity to learn new things about each other."

[0361] Server: Sends feedback to the user's device.

[0362] Terminal: Show feedback in the chat window.

[0363] Step 9: Emotional Engine in Action

[0364] Server: Using an emotion engine, the text entered by the user is analyzed and emotional information such as "Looks fun" or "Very interested" is extracted.

[0365] Server: Based on the emotional information, the virtual personality's responses are adjusted to generate more friendly dialogue.

[0366] Step 10: Record and track user growth

[0367] Server: Records chat session data and stores it in a database for future sessions.

[0368] In this way, users can safely hone their communication skills in a virtual environment, understand their own emotional state through the emotion engine, and gain confidence in real-life face-to-face conversations.

[0369] The processing flow will be explained below.

[0370] Step 1:

[0371] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) into the condition input form on the device.

[0372] Step 2:

[0373] Terminal: Checks the format of the entered data to ensure that the format and required fields are correct. If there are any errors, an error message is displayed prompting the user to make corrections.

[0374] Step 3:

[0375] Terminal: Condition data that passes the format check is serialized into JSON format and sent to the server as an HTTP POST request.

[0376] Step 4:

[0377] Server: Receives the HTTP POST request and parses the data to extract user criteria information.

[0378] Step 5:

[0379] Server: Based on the extracted condition information, a request is sent to a generative AI model (e.g., GPT-3 or BERT) to generate a virtual personality.

[0380] Step 6:

[0381] Server: The generated virtual personality data (name, age, personality, hobbies, etc.) is stored in a storage and linked to the user's session information.

[0382] Step 7:

[0383] Server: Converts the virtual personality data into JSON format and sends it to the user's device as an HTTP response.

[0384] Step 8:

[0385] Terminal: Analyzes the received virtual personality data and displays profile information in the chat window.

[0386] Step 9:

[0387] User: Type a message in the chat window.

[0388] Step 10:

[0389] Terminal: Sends the entered message to the server.

[0390] Step 11:

[0391] Server: Analyzes the received message and calls the AI ​​model for the virtual personality to generate an appropriate response.

[0392] Step 12:

[0393] Server: Generates multiple options along with the responses of the generated virtual personality.

[0394] Step 13:

[0395] Server: Sends the generated responses and options to the user's terminal.

[0396] Step 14:

[0397] Terminal: A chat window displays the virtual personality's responses and the options presented.

[0398] Step 15:

[0399] User: Select one of the displayed options and send the selected information to the terminal.

[0400] Step 16:

[0401] Terminal: Sends the selection information to the server.

[0402] Step 17:

[0403] Server: Analyzes the selection information and generates the next response or feedback based on it.

[0404] Step 18:

[0405] Server: Sends the generated feedback and the next response to the user's terminal.

[0406] Step 19:

[0407] Terminal: Show feedback and next response in the chat window.

[0408] Step 20:

[0409] Server: Using the emotion engine, analyzes the message entered by the user and extracts emotional information.

[0410] Step 21:

[0411] Server: Based on the emotional information, the server appropriately adjusts the responses of the virtual personality and generates a dialogue that takes into account the user's emotional state.

[0412] Step 22:

[0413] Server: Sends the adjusted response to the user's device, and displays emotional information as feedback.

[0414] Step 23:

[0415] Terminal: Responses containing emotional information are displayed in the chat window, allowing users to check their own emotional state.

[0416] Step 24:

[0417] Server: Records all data from the chat session (inputs, responses, options, feedback, emotional information, etc.) and stores it in a database for future sessions.

[0418] Example 2

[0419] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0420] In modern society, providing a training environment for users to improve their communication skills is an important issue. In particular, there is a need for a system that provides real-time feedback while taking into account the user's emotional state. There is also a need for a means to record the user's progress over the long term and provide dialogue based on that record.

[0421] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0422] In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for presenting multiple options during the chat and generating feedback based on the user's selection, means for providing the feedback to the user, means for analyzing the user's emotions, means for adjusting the virtual personality's responses based on the emotion information, means for generating appropriate responses and suggestions for the user's selections, means for recording chat session data and using it in the next session, and means for formatting the condition data entered by the user. This allows users to train their communication skills in a virtual environment with peace of mind, understand their own emotional state through the emotion engine, and gain confidence in actual conversations.

[0423] A "user" is an individual who utilizes the system to chat with a virtual personality and improve their communication skills.

[0424] "Conditions" refers to information such as age, gender, occupation, and hobbies entered by the user.

[0425] A "virtual personality" is a character that can be used as a conversation partner, generated by a generative AI model based on conditions entered by the user.

[0426] "Chat" refers to the exchange of messages between a user and a virtual persona.

[0427] "Choices" refers to multiple options presented to a user during a chat.

[0428] "Feedback" refers to suggestions and improvements generated based on user selections.

[0429] "Emotion" refers to the psychological state that the emotion engine analyzes and extracts from the message entered by the user.

[0430] An "emotion engine" is software or algorithms that analyze user-entered text and extract emotional information.

[0431] A "response" refers to a message that a virtual personality returns in response to a user's message.

[0432] A "session" refers to a series of chat exchanges between a user and a virtual persona.

[0433] "Format check" refers to the process of checking whether the condition data entered by the user is in the correct format.

[0434] A "generative AI model" refers to an artificial intelligence model that generates a virtual personality based on given conditions.

[0435] A "prompt sentence" refers to an input sentence that requests the generative AI model to generate a virtual personality based on certain conditions.

[0436] This system generates a virtual personality based on user input and improves the user's communication skills through chat with the virtual personality. It also incorporates an emotion engine that recognizes the user's emotions and provides feedback to improve the effectiveness of the chat.

[0437] The system includes the following main means:

[0438] 1. A way for users to enter conditions

[0439] 2. Means for generating a virtual personality based on the above conditions

[0440] 3. Means for chatting between the virtual personality and the user

[0441] 4. A way to present multiple options during a chat and generate feedback based on the user's selection

[0442] 5. Means for providing said feedback to the user

[0443] 6. Means of recognizing user emotions using an emotion engine

[0444] 7. A way to track user progress and use it in the next session

[0445] 8. A method for format checking user-entered condition data

[0446] The program for this system uses the following hardware and software:

[0447] Terminal: A device that accepts user input and sends data to a server. Examples include PCs, smartphones, and tablets.

[0448] Server: A central computer system that processes data, generates virtual personalities, manages chats, analyzes emotions, and provides feedback. Examples include virtual machines on a cloud service or dedicated servers.

[0449] Generative AI model: An artificial intelligence model that generates virtual personalities based on conditions. Examples include OpenAI's GPT-3 and BERT.

[0450] Emotion engine: A software component that analyzes text entered by a user and extracts emotional information. It is implemented using a natural language processing library or an AI engine.

[0451] The following example is a concrete example of the flow of operations.

[0452] Example 1: User enters criteria

[0453] The user enters the following criteria into the input form on the terminal: "Male in his 20s, university student, tennis lover, living in Tokyo."

[0454] The terminal checks the format of the input condition data, and if there are no errors, sends it to the server.

[0455] Example prompt sentence:

[0456] "Generate a hobby profile for a male in his 20s, a university student, who likes tennis and lives in Tokyo."

[0457] Example 2: Server generates virtual personality

[0458] The server generates a virtual personality "Yamada-kun" using a generative AI model based on the condition information and links it to the session information.

[0459] Example 3: Chatting with a virtual personality

[0460] The user types into the chat window, "Hi! You said you like tennis. How often do you play?"

[0461] The server uses an emotion engine to analyze the text entered by the user and extract emotional information such as "it looks fun" or "it looks very interesting."

[0462] The server uses an AI model to generate a response and then presents the user with multiple options.

[0463] In this way, this system, which operates in cooperation between users, devices, and servers, provides an environment in which users can improve their communication skills with peace of mind, and by providing appropriate feedback based on emotion analysis, users can proceed with conversations with confidence in real-life situations.

[0464] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0465] Step 1:

[0466] The user enters the conditions

[0467] The user inputs the target conditions (e.g., age, gender, occupation, hobbies, etc.) into the input form provided on the terminal. For example, the user inputs "male in his 20s, university student, tennis lover, living in Tokyo."

[0468] The terminal receives the input data (age, gender, occupation, hobbies, etc.) and performs a format check. If any errors are found, the terminal notifies the user of the information.

[0469] Input: Condition data entered by the user in the input form

[0470] Output: Correctly formatted condition data (after format check)

[0471] Step 2:

[0472] Sending data to the server

[0473] The terminal serializes the condition data that has passed the format check into JSON format.

[0474] The device sends the serialized JSON data to the server as an HTTP POST request using the endpoint " / create_personality".

[0475] The server receives the request and parses the data to extract the necessary condition information.

[0476] Input: JSON format of condition data sent from the terminal

[0477] Output: Server-side parsed condition data

[0478] Step 3:

[0479] The server generates a virtual personality

[0480] The server sends a request to a generative AI model (e.g., GPT-3 or BERT) based on the parsed condition information. Specifically, the prompt statement "Generate a hobby profile for a male in his 20s, a university student, who likes tennis and lives in Tokyo" is passed to the generative AI model.

[0481] The server retrieves the virtual personality data (such as name, age, personality, hobbies, etc.) returned by the generative AI model and stores it in internal storage (e.g., a NoSQL database). This data is associated with the user's session information.

[0482] Input: Parsed condition data

[0483] Output: Virtual personality data returned from the generative AI model

[0484] Step 4:

[0485] Send virtual personality information to the user's device

[0486] The server sends the generated virtual personality data to the user's device as an HTTP response. Specifically, the response is composed of the user's session ID and the data.

[0487] The device analyzes the received virtual personality data and displays it in the chat window. For example, it might display information like "Yamada, male in his 20s, university student, tennis lover, living in Tokyo."

[0488] Input: Virtual personality data from the server

[0489] Output: Virtual personality profile information displayed in the chat window

[0490] Step 5:

[0491] A user initiates a chat with a virtual persona

[0492] The user types a message into the chat window, for example, "Hi! I saw you said you like tennis. How often do you play?"

[0493] The terminal sends the entered message to the server by sending a POST request to the endpoint " / chat".

[0494] The server analyzes the received message and processes the text content using its internal natural language processing engine.

[0495] Input: A chat message from the user

[0496] Output: Analysis results from the natural language processing engine

[0497] Step 6:

[0498] The virtual personality responds and offers options

[0499] The server uses a natural language processing engine to analyze the user's message and generate a response from the virtual personality, such as "Hello! I play two to three times a week."

[0500] The server generates multiple options for the user (e.g., "Continue talking about tennis," "Return to self-introduction," "Talk about another hobby").

[0501] The server sends the generated responses and options to the user's terminal.

[0502] The terminal displays the virtual personality's responses and options in a chat window.

[0503] Input: User's chat message and choices

[0504] Output: Response message and options from the virtual personality

[0505] Step 7:

[0506] The user selects an option

[0507] The user selects one of the presented options and transmits the selected information to the terminal.

[0508] The device sends the selection information to the server. Specifically, it sends a request including the option ID to the endpoint " / select_option".

[0509] The server analyzes the received selection information and generates the following response or feedback:

[0510] Input: User selection information

[0511] Output: The next response or feedback

[0512] Step 8:

[0513] Providing feedback

[0514] The server generates feedback based on the user's selection, for example, "Talking about other hobbies is a good opportunity to discover new aspects of each other."

[0515] The server transmits the generated feedback to the user's terminal.

[0516] The device displays the feedback in a chat window.

[0517] Input: User selection information

[0518] Output: Feedback message

[0519] Step 9:

[0520] The Emotional Engine in Action

[0521] The server uses an emotion engine to analyze the text entered by the user, extracting emotion information such as "fun" or "interesting."

[0522] The server adjusts the responses of the virtual personality based on the emotional information and generates a dialogue that takes into account the user's emotional state.

[0523] Input: User's chat message

[0524] Output: Emotional information and tailored response message

[0525] Step 10:

[0526] Record and track user growth

[0527] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions. This data is used to support the user's ongoing growth.

[0528] Input: Chat session data

[0529] Output: Session data stored in the database

[0530] As described above, users, devices, and servers work together to operate the system efficiently, improving users' communication skills and providing appropriate feedback based on emotion analysis.

[0531] (Application example 2)

[0532] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0533] Efficient work instruction and worker safety training are important issues in modern manufacturing sites. In particular, quickly and reliably learning new procedures and technologies is essential for improving productivity. However, traditional methods require face-to-face instruction, and the quality of instruction depends on the skill and experience of the instructor, making it difficult to provide consistent instruction. It is also difficult to provide feedback that reflects the emotions and level of understanding of workers.

[0534] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for presenting multiple options during the chat and generating feedback based on the user's selection, means for providing the feedback to the user, means for extracting emotional information from data input by the user using an emotion engine, means for adjusting responses and feedback based on the user's emotional information, and means installed in industrial machinery for generating a training virtual personality and providing guidance to workers. This enables real-time guidance and feedback according to the emotional state of the worker.

[0535] The "means for the user to input conditions" is an interface device that allows the user to input conditions for generating a virtual personality.

[0536] The "means for generating a virtual personality" is a system that generates a virtual personality with specific attributes and characteristics based on conditions entered by the user.

[0537] The "means for chatting" is a communication interface for sending and receiving messages between the user and the virtual personality.

[0538] The "means for presenting options and generating feedback" is a function that presents multiple options to the user during a chat and provides feedback based on the selection.

[0539] A "means for providing feedback to a user" is an interface device for presenting the generated feedback to a user.

[0540] The "means for extracting emotional information from user input data using an emotion engine" is a function for analyzing data input by a user and identifying an emotional state from that data.

[0541] The "means for adjusting responses and feedback based on the user's emotional information" is a function that appropriately adjusts the responses and feedback of the virtual personality based on the emotional information extracted by the emotion engine.

[0542] "Means to be installed in industrial machines, generate educational virtual personalities, and provide guidance to workers" refers to a system that is installed in industrial machines and provides education and guidance to workers through virtual personalities.

[0543] The following describes an embodiment of the present invention. This system generates a virtual personality based on conditions entered by a user, and provides training to workers through the virtual personality installed on industrial machines. This will be described in detail below.

[0544] First, the user inputs conditions using an interface device, such as the worker's years of experience, new work procedures, language, etc. The input conditions are serialized by the terminal and sent to the server as an HTTP POST request.

[0545] Next, based on the condition data received by the server, a generative AI model is used to generate a virtual personality. This generative AI model uses natural language processing models such as GPT-3 and BERT. The generated virtual personality has attributes and characteristics that are suitable for the worker.

[0546] After the virtual personality is generated, the server installs the data into the industrial machine. The installed virtual personality then interacts with the worker through chat. During the chat, the worker is presented with multiple options, and feedback is generated based on the options. The feedback is then provided to the worker via their terminal.

[0547] This system incorporates an emotion engine that extracts emotional information from user input data. The server uses the emotion engine to analyze the user's emotional state and adjusts the virtual personality's responses and feedback based on the emotional information.

[0548] This process allows for real-time guidance and feedback that takes into account the worker's emotional state. User progress data is also recorded and used for the next session.

[0549] The hardware used includes industrial machines (e.g., KUKA, FANUC), tablets and smart glasses for workers, and software including Flask (a web application framework), generative AI models (e.g., GPT-3, BERT), and emotion recognition modules.

[0550] example

[0551] Consider a scenario where a worker is learning a new safety procedure. For example, the following conditions are entered:

[0552] Worker experience: Beginner

[0553] New work procedure: Machine maintenance

[0554] Language: Japanese

[0555] The server generates a virtual personality based on these conditions. The virtual personality is designed to provide simple and easy-to-understand instruction for beginners. During the dialogue, if the worker asks, "Which part of the machine should I check first?", the virtual personality generates an appropriate response and presents specific options. The prompt text in this case is as follows:

[0556] "Worker: What part of the machine should I check first?

[0557] Virtual personality: First, it is important to check the power source of the machine. Please choose the next step from the options below:

[0558] 1. Turn off the power to the power unit

[0559] 2. Check the oil in the power parts

[0560] 3. Check with other workers

[0561] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0562] Step 1:

[0563] The user inputs conditions such as the worker's years of experience, new work procedures, and language on the interface device. The terminal checks the format of this input data, and if there are no errors, it serializes it and sends it to the server. The input includes the worker's experience, procedures, and language, and the output generates condition data in JSON format.

[0564] Step 2:

[0565] The server receives the condition data. The server receives the condition data sent from the device, parses it, and extracts the condition information. The input is JSON-formatted condition data, and the parsed condition information is obtained as output.

[0566] Step 3:

[0567] The server generates a virtual personality. Based on the extracted condition information, the server sends a request to a generative AI model (e.g., GPT-3, BERT) to generate a virtual personality. The data of the generated virtual personality is stored on the server and linked to the user's session information. The condition information is input, and the data of the generated virtual personality is obtained as output.

[0568] Step 4:

[0569] The server installs the virtual personality information into the industrial machine. The server sends the generated virtual personality data to the industrial machine and installs it. The input is the virtual personality data, and the output is the virtual personality installed in the industrial machine.

[0570] Step 5:

[0571] A user starts a chat with a virtual persona. A worker starts a chat with a virtual persona using a tablet or smart glasses. The worker types a message in the chat window, and the device sends the message to the server. There is a user message as input, and a message sent to the server as output.

[0572] Step 6:

[0573] The server generates responses and options for the virtual personality. The server analyzes the received message and generates appropriate responses and options using a generative AI model. It also analyzes the user's emotional information using an emotion engine. The input is the user message, and the output is the virtual personality's responses, options, and emotional information.

[0574] Step 7:

[0575] The server sends responses, options, and feedback to the user. The server then sends the generated responses, options, and feedback based on the emotional information to the user's device. The inputs are the virtual personality's responses, options, and emotional information, and the output is the data sent to the device.

[0576] Step 8:

[0577] The user selects an option. They choose one from multiple options displayed in the chat window and send the selection information to the terminal. The option data is input, and the selection information is sent to the server as output.

[0578] Step 9:

[0579] The server generates the next response or feedback based on the selection. The server generates the next response or feedback based on the selection information and sends it to the device. The selection information is the input, and the next response or feedback is sent to the device as the output.

[0580] Step 10:

[0581] The emotion engine analyzes the user's input. The server uses the emotion engine to analyze the user's input data and extract emotional information. The user's input data is the input, and emotional information is obtained as the output.

[0582] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0583] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0584] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0585] [Second embodiment]

[0586] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0587] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0588] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0589] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0590] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0591] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0592] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0593] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0594] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0595] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0596] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0597] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0598] The present invention is a system that generates a virtual personality based on conditions input by a user and improves the user's communication skills through chat with that virtual personality.

[0599] System Overview

[0600] The system includes a means for a user to input conditions and generate a virtual personality based on the conditions, a means for chatting between the generated virtual personality and the user, a means for presenting multiple options during the chat, a means for generating feedback based on the user's selection, and a means for providing the feedback to the user. It also includes a means for recording the user's progress and utilizing the progress in the next session.

[0601] Explanation of program processing

[0602] 1. The user enters the conditions

[0603] User: Uses a device to enter the target conditions (age, gender, place of origin, company, position, personality, family structure, address, preferences, etc.).

[0604] Terminal: Checks the format of the entered data to make sure there are no errors. If there are no errors, the data is sent to the server.

[0605] 2. Send the data to the server

[0606] Terminal: Sends condition data serialized in JSON format to the server.

[0607] Server: Receives the request, parses the data, and obtains the user's criteria information.

[0608] 3. The server generates a virtual personality

[0609] Server: Based on the acquired condition information, a virtual personality is generated using a generative AI model (e.g., GPT-3, BERT). The generated virtual personality data is stored in storage and linked to the user's session information.

[0610] 4. Send virtual personality information to the user's device

[0611] Server: Sends the generated virtual personality data to the user's device as an HTTP response.

[0612] Terminal: Analyzes the virtual personality's profile information and displays it in the chat window.

[0613] 5. The user starts chatting with the virtual persona

[0614] User: Type a message in the chat window.

[0615] Terminal: Sends the entered message to the server.

[0616] Server: Analyzes the message and invokes the AI ​​model for the virtual personality to generate an appropriate response.

[0617] 6. The virtual personality responds and offers options

[0618] Server: Uses AI models to generate responses for the virtual personality, and also generates multiple options to present to the user.

[0619] Server: Sends the generated responses and options to the user's terminal.

[0620] Terminal: Displays the virtual personality's responses and options in the chat window.

[0621] 7. The user selects an option

[0622] User: Choose one of the options presented.

[0623] Terminal: Sends the selected option information to the server.

[0624] Server: Generates the next response or feedback based on the option information.

[0625] 8. Providing Feedback

[0626] Server: Generates feedback on the choices and provides suggestions and improvements to the user.

[0627] Server: Sends the generated feedback to the user's device.

[0628] Terminal: Show feedback in the chat window.

[0629] 9. Record and track user growth

[0630] Server: Records chat session data and stores it in a database for future sessions. This data is used to support the continued growth of our users.

[0631] Specific examples

[0632] Step 1: User enters criteria

[0633] User: Enter the target criteria as "male in his 20s, university student, tennis lover, living in Tokyo."

[0634] Terminal: Checks the format of the condition data, and if there are no errors, sends it to the server.

[0635] Step 2: Send the data to the server

[0636] Terminal: Converts the condition data into JSON format and sends it to the server.

[0637] Server: Receives data and extracts condition information.

[0638] Step 3: The server generates a virtual personality

[0639] Server: Generates a virtual personality using a generative AI model based on condition information.

[0640] Server: Stores the data of the generated virtual personality in storage and links it to session information.

[0641] Step 4: Send virtual personality information to the user's device

[0642] Server: Sends virtual personality data to the user's device.

[0643] Terminal: Display the virtual personality's profile in the chat window.

[0644] Step 5: The user starts chatting with the virtual persona

[0645] User: Type the message "Hi! You said you like tennis. How often do you play?" into the chat window.

[0646] Terminal: Sends a message to the server.

[0647] Server: Parses the message and invokes the AI ​​model to generate a response.

[0648] Step 6: The virtual persona responds and offers options

[0649] Server: The virtual personality responds with "Hello! I play two or three times a week. Who's Tanaka?" and generates multiple options.

[0650] Server: Sends the response and options to the user's terminal.

[0651] Terminal: Show responses and options in the chat window.

[0652] Step 7: User selects a choice

[0653] User: Select "Talk about another hobby" from the options.

[0654] Terminal: Sends the selection information to the server.

[0655] Server: Generates the next response or feedback.

[0656] Step 8: Provide feedback

[0657] Server: Generate feedback "When talking about different hobbies, it's important to explore common interests."

[0658] Server: Sends feedback to the user's device.

[0659] Terminal: Display feedback in the chat window.

[0660] Step 9: Record and track user growth

[0661] Server: Records chat session data and stores it in a database for future sessions.

[0662] In this way, users can comfortably practice their communication skills in a virtual environment and gain confidence in real-life face-to-face conversations.

[0663] The processing flow will be explained below.

[0664] Step 1:

[0665] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) into the condition input form on the device.

[0666] Step 2:

[0667] Terminal: Check the format of the entered data to ensure that the data format is correct and that required fields are not missing.

[0668] Step 3:

[0669] Terminal: Condition data that passes the format check is serialized into JSON format and sent to the server as an HTTP POST request.

[0670] Step 4:

[0671] Server: Analyzes the received request and extracts the condition data sent.

[0672] Step 5:

[0673] Server: Using the condition data, it sends a request to a generative AI model (e.g., GPT-3 or BERT) to generate a virtual personality.

[0674] Step 6:

[0675] Server: Stores the generated virtual personality data (such as name, age, personality, hobbies, etc.) in storage and links it to the user's session information.

[0676] Step 7:

[0677] Server: Sends the virtual personality data to the user's terminal as an HTTP response.

[0678] Step 8:

[0679] Terminal: Analyzes the received virtual personality data and displays profile information in the chat window.

[0680] Step 9:

[0681] User: Type a message in the chat window.

[0682] Step 10:

[0683] Terminal: Sends the entered message to the server.

[0684] Step 11:

[0685] Server: Analyzes the received message and generates an appropriate reply by calling an AI model for the virtual personality to respond.

[0686] Step 12:

[0687] Server: Generates multiple options to present to the user along with the responses of the generated virtual personality.

[0688] Step 13:

[0689] Server: Sends the virtual personality's responses and options to the user's terminal.

[0690] Step 14:

[0691] Terminal: Displays the virtual personality's replies and options in the chat window.

[0692] Step 15:

[0693] User: Select one of the options presented and send the selection to the terminal.

[0694] Step 16:

[0695] Terminal: Sends the selection information to the server.

[0696] Step 17:

[0697] Server: Generates the next response or feedback based on the selected information.

[0698] Step 18:

[0699] Server: Sends the generated feedback and the next response to the user's device.

[0700] Step 19:

[0701] Terminal: Show feedback and next response in the chat window.

[0702] Step 20:

[0703] Server: Records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions.

[0704] Example 1

[0705] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0706] Previous systems for improving communication skills had the problem of making it difficult for users to receive appropriate support. Specifically, it was difficult to generate a virtual personality based on the user's individual conditions, and there was a lack of means to continuously support the user's growth through dialogue with that virtual personality. Furthermore, there was a lack of technology for format checking of user input data and generating appropriate responses in real time, making it difficult to improve the user experience.

[0707] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0708] In this invention, the server includes: means for a user to input conditions; means for utilizing a generative AI model to generate a virtual personality based on the conditions; means for chatting between the virtual personality and the user; means for presenting multiple options during the chat and generating feedback based on the user's selection; means for providing the feedback to the user; means for displaying profile information of the virtual personality on a chat screen; and means for calling a generative AI model to analyze messages input by the user and generate appropriate responses. This allows users to continuously improve their communication skills through real-time interactions with virtual personalities based on their conditions.

[0709] "User" refers to a person who operates the system, inputs conditions, and interacts with a virtual personality.

[0710] "Conditions" refer to individual information such as age, sex, occupation, hobbies, and place of residence entered by the user.

[0711] A "virtual personality" refers to a virtual entity that can be used as a conversation partner and is generated based on the user's conditions using a generative AI model.

[0712] A "generative AI model" refers to an artificial intelligence model that generates a virtual personality using natural language processing based on conditions entered by the user.

[0713] "Chat" refers to a text-based interaction between a user and a virtual persona.

[0714] "Choices" refers to multiple response or reaction options that a user is presented with by a virtual persona during a chat.

[0715] "Feedback" refers to information such as suggestions and improvements provided based on the user's selections.

[0716] A "prompt" refers to an input instruction that causes a generative AI model to generate a specific response.

[0717] "Profile information" refers to information that a user can know, such as the virtual personality's personality, hobbies, and background.

[0718] "Session information" refers to the interaction history and associated data between a user and a virtual persona.

[0719] "Format check" refers to the process of checking whether the data entered by the user conforms to the specified format.

[0720] "HTTP response" refers to the data format returned from the server to the client (terminal).

[0721] This invention is a system that generates a virtual personality based on conditions input by a user and improves the user's communication skills through chatting with that virtual personality.

[0722] Hardware and Software Configuration

[0723] This system mainly consists of the following components:

[0724] 1. User's device: a device such as a personal computer, smartphone, or tablet.

[0725] 2. Server: Cloud server or local server.

[0726] 3. Generative AI models: Use natural language generation models such as GPT-3 and BERT.

[0727] Data processing and calculation

[0728] The system performs the following steps:

[0729] 1. The user enters the conditions

[0730] The user enters information such as age, gender, occupation, hobbies, and place of residence through the device interface. The device then performs a format check on the entered data to ensure there are no errors. If there are no errors, the data is serialized in JSON format and sent to the server.

[0731] 2. Acquire and analyze data

[0732] The server receives the condition data sent in JSON format, parses it, and extracts the user's condition information, which is then stored in temporary storage.

[0733] 3. Generate a virtual personality

[0734] The server sends a prompt to the generative AI model (e.g., GPT-3, BERT) based on the acquired condition information. The prompt includes the condition information entered by the user. For example, "Please generate a virtual personality based on the following conditions: age in his 20s, gender male, university student, tennis lover, residing in the Tokyo metropolitan area." The profile information of the generated virtual personality is saved in storage and linked to the user's session information.

[0735] 4. Send virtual personality information to the user device

[0736] The server sends the generated virtual personality data to the user's device as an HTTP response. The device analyzes this data and displays the virtual personality's profile (name, characteristics, hobbies, etc.) in a chat window.

[0737] 5. The user starts chatting with the virtual persona

[0738] The user types a message into the chat window, for example, "Hello! You said you like tennis. How often do you play?" The device then sends the typed message to the server.

[0739] 6. The virtual personality responds and offers options

[0740] The server analyzes the received message and calls the generative AI model again to generate a response from the virtual personality. For example, it generates a response like, "Hello! I play two or three times a week. How about you?" The generated response and multiple options (for example, "I play once a week," "Talk about another hobby," etc.) are sent to the user's device. The device displays the virtual personality's response and options in a chat window.

[0741] 7. The user selects an option

[0742] The user selects one of the options presented and confirms the choice by clicking or other operation. The device then sends information about the selected option to the server.

[0743] 8. Providing Feedback

[0744] The server generates feedback based on the user's choices, such as advice like, "When talking about different hobbies, it's important to find common interests." The generated feedback is sent to the user's device and displayed in the chat window.

[0745] 9. Record and track user growth

[0746] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for reference during the next session, which is used to support the user's continuous growth.

[0747] In this way, this system allows users to learn practical communication skills in a safe virtual environment and improve their ability to interact in the real world.

[0748] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0749] Step 1:

[0750] The user enters conditions (age, gender, occupation, hobbies, place of residence, etc.) using a device. The entered data is format-checked on the device to ensure there are no errors. If there are no errors, the data is serialized in JSON format.

[0751] Input: The user enters the conditions into the input form on the device.

[0752] Output: Serialized JSON formatted condition data

[0753] Step 2:

[0754] The device sends the condition data that has passed the format check to the server via an HTTP POST request. The server receives this request, parses the data, and extracts the user's condition information. The extracted data is saved in temporary storage.

[0755] Input: Condition data in serialized JSON format

[0756] Output: Parsed condition information data

[0757] Step 3:

[0758] The server then sends prompts to the generative AI model based on the extracted condition information. The prompts contain the user's condition information and prompt the model to generate speech content and a profile.

[0759] Input: Parsed condition information

[0760] Output: The prompt sent to the generative AI model

[0761] Step 4:

[0762] A generative AI model (e.g., GPT-3, BERT) generates a virtual personality profile based on the prompt text. This profile information is received on the server side, linked to the user's session information, and stored in storage.

[0763] Input: prompt statement

[0764] Output: Profile information of the generated virtual personality

[0765] Step 5:

[0766] The server sends the profile information of the virtual personality to the user's device as an HTTP response. The device receives this data, analyzes it, and displays the profile information (name, characteristics, hobbies, etc.) in the chat window.

[0767] Input: Profile information of the generated virtual personality

[0768] Output: Profile information displayed in the chat window

[0769] Step 6:

[0770] The user enters a message in the chat window. The device sends the message to the server, which receives the message and calls the generative AI model again to generate a response from the virtual personality.

[0771] Input: The message entered by the user

[0772] Output: A response message from the generative AI model

[0773] Step 7:

[0774] The server sends the generated response message and multiple options to the user's terminal, which receives it and displays the response message and options in a chat window.

[0775] Input: Response message from generative AI model, multiple choice

[0776] Output: Response message and options displayed in the chat window

[0777] Step 8:

[0778] The user selects one of the options presented and confirms the choice by clicking or other operation. The device then sends information about the selected option to the server.

[0779] Input: The choice selected by the user

[0780] Output: Choice information sent to the server

[0781] Step 9:

[0782] The server receives and analyzes the selection information and generates feedback, which is then sent back to the user's terminal and displayed in the terminal's chat window.

[0783] Input: Information about the option selected by the user

[0784] Output: Generated feedback

[0785] Step 10:

[0786] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for reference during the next session, to support the user's continuous growth.

[0787] Input: Chat session data

[0788] Output: Session information stored in the database

[0789] (Application example 1)

[0790] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0791] Security operators and guards are required to have advanced communication skills in order to respond quickly and appropriately to emergencies and daily security activities, but there is a lack of appropriate training methods and tools to efficiently improve these skills.

[0792] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0793] In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for generating a virtual personality based on a situation related to security activities, and means for communicating between the virtual personality and the user regarding an emergency. This enables security operators and security guards to efficiently improve the communication skills required for their work through virtual conversations based on actual situations.

[0794] The "means for user input" is a digital input interface that allows a user to provide specific requirements and situational information to the system.

[0795] The "means for generating a virtual personality based on the above conditions" is a technology that uses artificial intelligence to create a virtual conversation partner based on input condition information.

[0796] The "means for chatting between the virtual personality and the user" is a communication function that allows the generated virtual personality and the user to exchange text messages in real time.

[0797] "Means for presenting multiple options during chat and generating feedback based on the user's selection" is a mechanism that provides the user with several options during a conversation and generates an appropriate response or evaluation based on the selection.

[0798] The "means for providing the feedback to the user" is an output function for displaying or notifying the generated feedback or evaluation to the user.

[0799] The "means for generating a virtual personality based on a situation related to security activities" is a technology for generating a virtual personality based on a specific scenario related to security activities or emergency situations.

[0800] "Means for communication between the virtual personality and the user regarding an emergency" refers to a communication function for giving instructions and sharing information in an emergency through virtual dialogue.

[0801] This invention utilizes a virtual dialogue system to improve the effective communication skills of security operators and guards in emergency situations. The system is composed of a user, a terminal, and a server.

[0802] System Overview

[0803] The user uses a device to input specific conditions (for example, "Occupation: Security Operator," "Emergency: Fire," "Location: Office Building," "Instructions: Evacuation Instructions," etc.). The device checks the format of this condition data and then sends it to the server in JSON format. The server generates a virtual personality using a generative AI model (for example, GPT-2) based on the received condition information. The server then saves the virtual personality data and session information in storage and provides it to the user.

[0804] The user starts chatting with the virtual persona on the device. The message entered by the user is sent to the server, which uses an AI model to generate an appropriate response. The generated response and multiple options presented to the user are sent to the device and displayed in a chat window. The user selects one of the options, and feedback is generated and provided to the user based on that selection.

[0805] The system also includes a means to record the user's progress and use it for future sessions. This data supports the user's continuous improvement.

[0806] Hardware and software used

[0807] Hardware: Smartphone

[0808] Software: Python, generative AI model (GPT-2)

[0809] Explanation of data processing or data calculation

[0810] 1. Enter conditions and submit

[0811] The user inputs conditions such as "occupation: security operator," "emergency situation: fire," "location: office building," and "instructions: evacuation instructions."

[0812] The terminal checks the format of the condition data, and if there are no errors, sends it to the server in JSON format.

[0813] 2. Creation and provision of virtual personalities

[0814] The server generates a virtual personality using a generative AI model (GPT-2) based on the received condition information.

[0815] Data on the generated virtual personality is sent back to the user's terminal.

[0816] 3. Interacting with Virtual Personalities

[0817] The user inputs a message such as "Hello, there is an emergency. There is a fire. Please give us your instructions."

[0818] The server analyzes the received message using an AI model and generates an appropriate response.

[0819] The generated responses and options are displayed on the user's terminal.

[0820] 4. Providing Feedback

[0821] The server generates and provides appropriate feedback to the user based on the user's selection.

[0822] Provide specific advice and improvements to help users improve their skills.

[0823] Examples of concrete examples and prompts

[0824] Examples:

[0825] Enter the conditions: "Occupation: Security Operator", "Emergency: Fire", "Location: Office Building", "Instructions: Evacuation Instructions"

[0826] Initial message: "Hello, emergency. Fire. Your instructions needed."

[0827] Example prompt sentence:

[0828] Conditions: "Occupation: Security Operator", "Emergency: Fire", "Location: Office Building", "Instructions: Evacuation"

[0829] Dialogue start message: "Hello, emergency. Fire. Your instructions needed."

[0830] In this way, users can safely practice their communication skills in a scenario that closely resembles real-world work.

[0831] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0832] Step 1:

[0833] The user inputs conditions. Using their smartphone, the user inputs conditions such as "Occupation: Security Operator," "Emergency: Fire," "Location: Office Building," and "Instructions: Evacuation Instructions." The device checks the format of the input data to ensure there are no errors. After checking, the device serializes the condition data into JSON format and sends it to the server.

[0834] Step 2:

[0835] The server receives the data. The server receives the condition data in JSON format sent from the device. The server parses the data, extracts the condition information, and prepares it as input data for use in the next processing step.

[0836] Step 3:

[0837] The server generates a virtual personality. Based on the extracted condition information, the server invokes a generative AI model (e.g., GPT-2) to generate the virtual personality. This generative model generates appropriate personality attributes based on the condition information and constructs data for the virtual personality. The data for the generated virtual personality is stored in the server's storage and linked to the user's session information.

[0838] Step 4:

[0839] The server sends information about the virtual personality to the user's device. The server then sends the generated virtual personality data to the user's device as an HTTP response. The device receives this data, analyzes it, and displays the virtual personality's profile information in the chat window.

[0840] Step 5:

[0841] The user starts chatting with the virtual persona. The user types a message into the chat window on the device: "Hello, there's an emergency. There's a fire. Please give us your instructions." The device sends the typed message to the server. The server receives the message, analyzes it, and calls the AI ​​model to generate an appropriate response from the virtual persona.

[0842] Step 6:

[0843] The server generates a response and presents the user with multiple options. The server uses an AI model to generate a response for the virtual personality, and then generates multiple options to present to the user. For example, a response such as "Please remain calm and head for the emergency exit" is generated. The generated response and options are sent to the user's device as an HTTP response. The device receives this and displays it in a chat window.

[0844] Step 7:

[0845] The user selects an option. The user selects one of the options presented and enters the selection information. The selection information is sent to the server via the device. The server again uses the AI ​​model to generate the next response or feedback based on the selected option information.

[0846] Step 8:

[0847] The server generates feedback and provides it to the user. The server generates feedback for the selected option and provides the user with suggestions and improvements. For example, the server might generate feedback such as "It is important to communicate instructions specifically and concisely." The generated feedback is sent to the user's device as an HTTP response and displayed in the chat window.

[0848] Step 9:

[0849] The server records the user's progress and uses it for the next session. The server records the chat session data and saves it in a database to use for the next session. This recorded data is used by the user to continuously improve their communication skills.

[0850] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0851] This system generates a virtual personality based on user input and improves the user's communication skills through chat with the virtual personality. It also incorporates an emotion engine that recognizes the user's emotions and provides feedback to improve the effectiveness of the chat.

[0852] System Overview

[0853] The system includes the following main means:

[0854] 1. A way for users to enter conditions

[0855] 2. Means for generating a virtual personality based on the above conditions

[0856] 3. Means for chatting between the virtual personality and the user

[0857] 4. A way to present multiple options during a chat and generate feedback based on the user's selection

[0858] 5. Means for providing said feedback to the user

[0859] 6. Means of recognizing user emotions using an emotion engine

[0860] 7. A way to track user progress and use it in the next session

[0861] 8. A means of formatting user-entered condition data

[0862] Explanation of program processing

[0863] 1. The user enters the conditions

[0864] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) on the device.

[0865] Terminal: Checks the format of the entered data and sends it to the server if there are no errors.

[0866] 2. Send the data to the server

[0867] Terminal: Serialize the condition data in JSON format and send it to the server as an HTTP POST request.

[0868] Server: Receives the request and parses the data to extract condition information.

[0869] 3. The server generates a virtual personality

[0870] Server: Sends a request to a generative AI model (e.g., GPT-3 or BERT) based on condition information to generate a virtual personality.

[0871] Server: Stores the virtual personality data (name, age, personality, hobbies, etc.) in storage and links it to the user's session information.

[0872] 4. Send virtual personality information to the user's device

[0873] Server: Sends the generated virtual personality data to the user's device as an HTTP response.

[0874] Terminal: Analyzes the virtual personality's profile information and displays it in the chat window.

[0875] 5. The user starts chatting with the virtual persona

[0876] User: Type a message in the chat window.

[0877] Terminal: Sends the entered message to the server.

[0878] Server: Analyzes the message and invokes the AI ​​model for the virtual personality to generate an appropriate response.

[0879] 6. The virtual personality responds and offers options

[0880] Server: Uses AI models to generate responses for the virtual personality, and also generates multiple options to present to the user.

[0881] Server: Sends the generated responses and options to the user's terminal.

[0882] Terminal: Displays the virtual personality's responses and options in the chat window.

[0883] 7. The user selects an option

[0884] User: Select one of the options presented and send the selection to the terminal.

[0885] Terminal: Sends the selection information to the server.

[0886] Server: Generates the next response or feedback based on the selected information.

[0887] 8. Providing Feedback

[0888] Server: Generates feedback on the choices and provides suggestions and improvements to the user.

[0889] Server: Sends the generated feedback to the user's device.

[0890] Terminal: Show feedback in the chat window.

[0891] 9. The Emotional Engine

[0892] Server: Analyzes the text entered by the user using the emotion engine and extracts emotion information.

[0893] Server: Adjusts the virtual personality's responses based on emotional information and generates dialogue that takes the user's emotional state into account.

[0894] 10. Record and track user growth

[0895] Server: Records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions. This data is used to support the user's ongoing growth.

[0896] Specific examples

[0897] Step 1: User enters criteria

[0898] User: Enters the following criteria into the input form on the device: "Male in his 20s, university student, tennis lover, living in Tokyo."

[0899] Terminal: Checks the format of the condition data, and if there are no errors, sends it to the server.

[0900] Step 2: Send the data to the server

[0901] Terminal: Serialize the condition data into JSON format and send it to the server as an HTTP POST request.

[0902] Server: Parse the data and extract the condition information.

[0903] Step 3: The server generates a virtual personality

[0904] Server: Based on the condition information, a virtual personality "Yamada-kun" is generated using a generative AI model.

[0905] Server: Stores the virtual personality data in storage and links it to session information.

[0906] Step 4: Send virtual personality information to the user's device

[0907] Server: Sends the generated virtual personality data to the user's device.

[0908] Terminal: Display the virtual personality's profile information in the chat window.

[0909] Step 5: The user starts chatting with the virtual persona

[0910] User: Type in the chat window, "Hi! You said you like tennis. How often do you play?"

[0911] Terminal: Sends the entered message to the server.

[0912] Server: Parses the message and invokes the AI ​​model to generate a response.

[0913] Step 6: The virtual persona responds and offers options

[0914] Server: Using an AI model, it generates the virtual personality's response, "Hello! I play two or three times a week. What about you?", while also generating options such as "Continue talking about tennis," "Go back to my introduction," or "Talk about another hobby."

[0915] Server: Sends the response and options to the user's terminal.

[0916] Terminal: Show responses and options in the chat window.

[0917] Step 7: User selects a choice

[0918] User: Select "Talk about another hobby" from the options.

[0919] Terminal: Sends the selection information to the server.

[0920] Server: Generates the next response or feedback.

[0921] Step 8: Provide feedback

[0922] Server: Generate feedback for the choice: "Talking about other hobbies is a good opportunity to learn new things about each other."

[0923] Server: Sends feedback to the user's device.

[0924] Terminal: Show feedback in the chat window.

[0925] Step 9: Emotional Engine in Action

[0926] Server: Using an emotion engine, the text entered by the user is analyzed and emotional information such as "Looks fun" or "Very interested" is extracted.

[0927] Server: Based on the emotional information, the virtual personality's responses are adjusted to generate more friendly dialogue.

[0928] Step 10: Record and track user growth

[0929] Server: Records chat session data and stores it in a database for future sessions.

[0930] In this way, users can safely hone their communication skills in a virtual environment, understand their own emotional state through the emotion engine, and gain confidence in real-life face-to-face conversations.

[0931] The processing flow will be explained below.

[0932] Step 1:

[0933] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) into the condition input form on the device.

[0934] Step 2:

[0935] Terminal: Checks the format of the entered data to ensure that the format and required fields are correct. If there are any errors, an error message is displayed prompting the user to make corrections.

[0936] Step 3:

[0937] Terminal: Condition data that passes the format check is serialized into JSON format and sent to the server as an HTTP POST request.

[0938] Step 4:

[0939] Server: Receives the HTTP POST request and parses the data to extract user criteria information.

[0940] Step 5:

[0941] Server: Based on the extracted condition information, a request is sent to a generative AI model (e.g., GPT-3 or BERT) to generate a virtual personality.

[0942] Step 6:

[0943] Server: The generated virtual personality data (name, age, personality, hobbies, etc.) is stored in a storage and linked to the user's session information.

[0944] Step 7:

[0945] Server: Converts the virtual personality data into JSON format and sends it to the user's device as an HTTP response.

[0946] Step 8:

[0947] Terminal: Analyzes the received virtual personality data and displays profile information in the chat window.

[0948] Step 9:

[0949] User: Type a message in the chat window.

[0950] Step 10:

[0951] Terminal: Sends the entered message to the server.

[0952] Step 11:

[0953] Server: Analyzes the received message and calls the AI ​​model for the virtual personality to generate an appropriate response.

[0954] Step 12:

[0955] Server: Generates multiple options along with the responses of the generated virtual personality.

[0956] Step 13:

[0957] Server: Sends the generated responses and options to the user's terminal.

[0958] Step 14:

[0959] Terminal: A chat window displays the virtual personality's responses and the options presented.

[0960] Step 15:

[0961] User: Select one of the displayed options and send the selected information to the terminal.

[0962] Step 16:

[0963] Terminal: Sends the selection information to the server.

[0964] Step 17:

[0965] Server: Analyzes the selection information and generates the next response or feedback based on it.

[0966] Step 18:

[0967] Server: Sends the generated feedback and the next response to the user's terminal.

[0968] Step 19:

[0969] Terminal: Show feedback and next response in the chat window.

[0970] Step 20:

[0971] Server: Using the emotion engine, analyzes the message entered by the user and extracts emotional information.

[0972] Step 21:

[0973] Server: Based on the emotional information, the server appropriately adjusts the responses of the virtual personality and generates a dialogue that takes into account the user's emotional state.

[0974] Step 22:

[0975] Server: Sends the adjusted response to the user's device, and displays emotional information as feedback.

[0976] Step 23:

[0977] Terminal: Responses containing emotional information are displayed in the chat window, allowing users to check their own emotional state.

[0978] Step 24:

[0979] Server: Records all data from the chat session (inputs, responses, options, feedback, emotional information, etc.) and stores it in a database for future sessions.

[0980] Example 2

[0981] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0982] In modern society, providing a training environment for users to improve their communication skills is an important issue. In particular, there is a need for a system that provides real-time feedback while taking into account the user's emotional state. There is also a need for a means to record the user's progress over the long term and provide dialogue based on that record.

[0983] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0984] In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for presenting multiple options during the chat and generating feedback based on the user's selection, means for providing the feedback to the user, means for analyzing the user's emotions, means for adjusting the virtual personality's responses based on the emotion information, means for generating appropriate responses and suggestions for the user's selections, means for recording chat session data and using it in the next session, and means for formatting the condition data entered by the user. This allows users to train their communication skills in a virtual environment with peace of mind, understand their own emotional state through the emotion engine, and gain confidence in actual conversations.

[0985] A "user" is an individual who utilizes the system to chat with a virtual personality and improve their communication skills.

[0986] "Conditions" refers to information such as age, gender, occupation, and hobbies entered by the user.

[0987] A "virtual personality" is a character that can be used as a conversation partner, generated by a generative AI model based on conditions entered by the user.

[0988] "Chat" refers to the exchange of messages between a user and a virtual persona.

[0989] "Choices" refers to multiple options presented to a user during a chat.

[0990] "Feedback" refers to suggestions and improvements generated based on user selections.

[0991] "Emotion" refers to the psychological state that the emotion engine analyzes and extracts from the message entered by the user.

[0992] An "emotion engine" is software or algorithms that analyze user-entered text and extract emotional information.

[0993] A "response" refers to a message that a virtual personality returns in response to a user's message.

[0994] A "session" refers to a series of chat exchanges between a user and a virtual persona.

[0995] "Format check" refers to the process of checking whether the condition data entered by the user is in the correct format.

[0996] A "generative AI model" refers to an artificial intelligence model that generates a virtual personality based on given conditions.

[0997] A "prompt sentence" refers to an input sentence that requests the generative AI model to generate a virtual personality based on certain conditions.

[0998] This system generates a virtual personality based on user input and improves the user's communication skills through chat with the virtual personality. It also incorporates an emotion engine that recognizes the user's emotions and provides feedback to improve the effectiveness of the chat.

[0999] The system includes the following main means:

[1000] 1. A way for users to enter conditions

[1001] 2. Means for generating a virtual personality based on the above conditions

[1002] 3. Means for chatting between the virtual personality and the user

[1003] 4. A way to present multiple options during a chat and generate feedback based on the user's selection

[1004] 5. Means for providing said feedback to the user

[1005] 6. Means of recognizing user emotions using an emotion engine

[1006] 7. A way to track user progress and use it in the next session

[1007] 8. A method for format checking user-entered condition data

[1008] The program for this system uses the following hardware and software:

[1009] Terminal: A device that accepts user input and sends data to a server. Examples include PCs, smartphones, and tablets.

[1010] Server: A central computer system that processes data, generates virtual personalities, manages chats, analyzes emotions, and provides feedback. Examples include virtual machines on a cloud service or dedicated servers.

[1011] Generative AI model: An artificial intelligence model that generates virtual personalities based on conditions. Examples include OpenAI's GPT-3 and BERT.

[1012] Emotion engine: A software component that analyzes text entered by a user and extracts emotional information. It is implemented using a natural language processing library or an AI engine.

[1013] The following example is a concrete example of the flow of operations.

[1014] Example 1: User enters criteria

[1015] The user enters the following criteria into the input form on the terminal: "Male in his 20s, university student, tennis lover, living in Tokyo."

[1016] The terminal checks the format of the input condition data, and if there are no errors, sends it to the server.

[1017] Example prompt sentence:

[1018] "Generate a hobby profile for a male in his 20s, a university student, who likes tennis and lives in Tokyo."

[1019] Example 2: Server generates virtual personality

[1020] The server generates a virtual personality "Yamada-kun" using a generative AI model based on the condition information and links it to the session information.

[1021] Example 3: Chatting with a virtual personality

[1022] The user types into the chat window, "Hi! You said you like tennis. How often do you play?"

[1023] The server uses an emotion engine to analyze the text entered by the user and extract emotional information such as "it looks fun" or "it looks very interesting."

[1024] The server uses an AI model to generate a response and then presents the user with multiple options.

[1025] In this way, this system, which operates in cooperation between users, devices, and servers, provides an environment in which users can improve their communication skills with peace of mind, and by providing appropriate feedback based on emotion analysis, users can proceed with conversations with confidence in real-life situations.

[1026] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1027] Step 1:

[1028] The user enters the conditions

[1029] The user inputs the target conditions (e.g., age, gender, occupation, hobbies, etc.) into the input form provided on the terminal. For example, the user inputs "male in his 20s, university student, tennis lover, living in Tokyo."

[1030] The terminal receives the input data (age, gender, occupation, hobbies, etc.) and performs a format check. If any errors are found, the terminal notifies the user of the information.

[1031] Input: Condition data entered by the user in the input form

[1032] Output: Correctly formatted condition data (after format check)

[1033] Step 2:

[1034] Sending data to the server

[1035] The terminal serializes the condition data that has passed the format check into JSON format.

[1036] The device sends the serialized JSON data to the server as an HTTP POST request using the endpoint " / create_personality".

[1037] The server receives the request and parses the data to extract the necessary condition information.

[1038] Input: JSON format of condition data sent from the terminal

[1039] Output: Server-side parsed condition data

[1040] Step 3:

[1041] The server generates a virtual personality

[1042] The server sends a request to a generative AI model (e.g., GPT-3 or BERT) based on the parsed condition information. Specifically, the prompt statement "Generate a hobby profile for a male in his 20s, a university student, who likes tennis and lives in Tokyo" is passed to the generative AI model.

[1043] The server retrieves the virtual personality data (such as name, age, personality, hobbies, etc.) returned by the generative AI model and stores it in internal storage (e.g., a NoSQL database). This data is associated with the user's session information.

[1044] Input: Parsed condition data

[1045] Output: Virtual personality data returned from the generative AI model

[1046] Step 4:

[1047] Send virtual personality information to the user's device

[1048] The server sends the generated virtual personality data to the user's device as an HTTP response. Specifically, the response is composed of the user's session ID and the data.

[1049] The device analyzes the received virtual personality data and displays it in the chat window. For example, it might display information like "Yamada, male in his 20s, university student, tennis lover, living in Tokyo."

[1050] Input: Virtual personality data from the server

[1051] Output: Virtual personality profile information displayed in the chat window

[1052] Step 5:

[1053] A user initiates a chat with a virtual persona

[1054] The user types a message into the chat window, for example, "Hi! I saw you said you like tennis. How often do you play?"

[1055] The terminal sends the entered message to the server by sending a POST request to the endpoint " / chat".

[1056] The server analyzes the received message and processes the text content using its internal natural language processing engine.

[1057] Input: A chat message from the user

[1058] Output: Analysis results from the natural language processing engine

[1059] Step 6:

[1060] The virtual personality responds and offers options

[1061] The server uses a natural language processing engine to analyze the user's message and generate a response from the virtual personality, such as "Hello! I play two to three times a week."

[1062] The server generates multiple options for the user (e.g., "Continue talking about tennis," "Return to self-introduction," "Talk about another hobby").

[1063] The server sends the generated responses and options to the user's terminal.

[1064] The terminal displays the virtual personality's responses and options in a chat window.

[1065] Input: User's chat message and choices

[1066] Output: Response message and options from the virtual personality

[1067] Step 7:

[1068] The user selects an option

[1069] The user selects one of the presented options and transmits the selected information to the terminal.

[1070] The device sends the selection information to the server. Specifically, it sends a request including the option ID to the endpoint " / select_option".

[1071] The server analyzes the received selection information and generates the following response or feedback:

[1072] Input: User selection information

[1073] Output: The next response or feedback

[1074] Step 8:

[1075] Providing feedback

[1076] The server generates feedback based on the user's selection, for example, "Talking about other hobbies is a good opportunity to discover new aspects of each other."

[1077] The server transmits the generated feedback to the user's terminal.

[1078] The device displays the feedback in a chat window.

[1079] Input: User selection information

[1080] Output: Feedback message

[1081] Step 9:

[1082] The Emotional Engine in Action

[1083] The server uses an emotion engine to analyze the text entered by the user, extracting emotion information such as "fun" or "interesting."

[1084] The server adjusts the responses of the virtual personality based on the emotional information and generates a dialogue that takes into account the user's emotional state.

[1085] Input: User's chat message

[1086] Output: Emotional information and tailored response message

[1087] Step 10:

[1088] Record and track user growth

[1089] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions. This data is used to support the user's ongoing growth.

[1090] Input: Chat session data

[1091] Output: Session data stored in the database

[1092] As described above, users, devices, and servers work together to operate the system efficiently, improving users' communication skills and providing appropriate feedback based on emotion analysis.

[1093] (Application example 2)

[1094] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1095] Efficient work instruction and worker safety training are important issues in modern manufacturing sites. In particular, quickly and reliably learning new procedures and technologies is essential for improving productivity. However, traditional methods require face-to-face instruction, and the quality of instruction depends on the skill and experience of the instructor, making it difficult to provide consistent instruction. It is also difficult to provide feedback that reflects the emotions and level of understanding of workers.

[1096] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for presenting multiple options during the chat and generating feedback based on the user's selection, means for providing the feedback to the user, means for extracting emotional information from data input by the user using an emotion engine, means for adjusting responses and feedback based on the user's emotional information, and means installed in industrial machinery for generating a training virtual personality and providing guidance to workers. This enables real-time guidance and feedback according to the emotional state of the worker.

[1097] The "means for the user to input conditions" is an interface device that allows the user to input conditions for generating a virtual personality.

[1098] The "means for generating a virtual personality" is a system that generates a virtual personality with specific attributes and characteristics based on conditions entered by the user.

[1099] The "means for chatting" is a communication interface for sending and receiving messages between the user and the virtual personality.

[1100] The "means for presenting options and generating feedback" is a function that presents multiple options to the user during a chat and provides feedback based on the selection.

[1101] A "means for providing feedback to a user" is an interface device for presenting the generated feedback to a user.

[1102] The "means for extracting emotional information from user input data using an emotion engine" is a function for analyzing data input by a user and identifying an emotional state from that data.

[1103] The "means for adjusting responses and feedback based on the user's emotional information" is a function that appropriately adjusts the responses and feedback of the virtual personality based on the emotional information extracted by the emotion engine.

[1104] "Means to be installed in industrial machines, generate educational virtual personalities, and provide guidance to workers" refers to a system that is installed in industrial machines and provides education and guidance to workers through virtual personalities.

[1105] The following describes an embodiment of the present invention. This system generates a virtual personality based on conditions entered by a user, and provides training to workers through the virtual personality installed on industrial machines. This will be described in detail below.

[1106] First, the user inputs conditions using an interface device, such as the worker's years of experience, new work procedures, language, etc. The input conditions are serialized by the terminal and sent to the server as an HTTP POST request.

[1107] Next, based on the condition data received by the server, a generative AI model is used to generate a virtual personality. This generative AI model uses natural language processing models such as GPT-3 and BERT. The generated virtual personality has attributes and characteristics that are suitable for the worker.

[1108] After the virtual personality is generated, the server installs the data into the industrial machine. The installed virtual personality then interacts with the worker through chat. During the chat, the worker is presented with multiple options, and feedback is generated based on the options. The feedback is then provided to the worker via their terminal.

[1109] This system incorporates an emotion engine that extracts emotional information from user input data. The server uses the emotion engine to analyze the user's emotional state and adjusts the virtual personality's responses and feedback based on the emotional information.

[1110] This process allows for real-time guidance and feedback that takes into account the worker's emotional state. User progress data is also recorded and used for the next session.

[1111] The hardware used includes industrial machines (e.g., KUKA, FANUC), tablets and smart glasses for workers, and software including Flask (a web application framework), generative AI models (e.g., GPT-3, BERT), and emotion recognition modules.

[1112] example

[1113] Consider a scenario where a worker is learning a new safety procedure. For example, the following conditions are entered:

[1114] Worker experience: Beginner

[1115] New work procedure: Machine maintenance

[1116] Language: Japanese

[1117] The server generates a virtual personality based on these conditions. The virtual personality is designed to provide simple and easy-to-understand instruction for beginners. During the dialogue, if the worker asks, "Which part of the machine should I check first?", the virtual personality generates an appropriate response and presents specific options. The prompt text in this case is as follows:

[1118] "Worker: What part of the machine should I check first?

[1119] Virtual personality: First, it is important to check the power source of the machine. Please choose the next step from the options below:

[1120] 1. Turn off the power to the power unit

[1121] 2. Check the oil in the power parts

[1122] 3. Check with other workers

[1123] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1124] Step 1:

[1125] The user inputs conditions such as the worker's years of experience, new work procedures, and language on the interface device. The terminal checks the format of this input data, and if there are no errors, it serializes it and sends it to the server. The input includes the worker's experience, procedures, and language, and the output generates condition data in JSON format.

[1126] Step 2:

[1127] The server receives the condition data. The server receives the condition data sent from the device, parses it, and extracts the condition information. The input is JSON-formatted condition data, and the parsed condition information is obtained as output.

[1128] Step 3:

[1129] The server generates a virtual personality. Based on the extracted condition information, the server sends a request to a generative AI model (e.g., GPT-3, BERT) to generate a virtual personality. The data of the generated virtual personality is stored on the server and linked to the user's session information. The condition information is input, and the data of the generated virtual personality is obtained as output.

[1130] Step 4:

[1131] The server installs the virtual personality information into the industrial machine. The server sends the generated virtual personality data to the industrial machine and installs it. The input is the virtual personality data, and the output is the virtual personality installed in the industrial machine.

[1132] Step 5:

[1133] A user starts a chat with a virtual persona. A worker starts a chat with a virtual persona using a tablet or smart glasses. The worker types a message in the chat window, and the device sends the message to the server. There is a user message as input, and a message sent to the server as output.

[1134] Step 6:

[1135] The server generates responses and options for the virtual personality. The server analyzes the received message and generates appropriate responses and options using a generative AI model. It also analyzes the user's emotional information using an emotion engine. The input is the user message, and the output is the virtual personality's responses, options, and emotional information.

[1136] Step 7:

[1137] The server sends responses, options, and feedback to the user. The server then sends the generated responses, options, and feedback based on the emotional information to the user's device. The inputs are the virtual personality's responses, options, and emotional information, and the output is the data sent to the device.

[1138] Step 8:

[1139] The user selects an option. They choose one from multiple options displayed in the chat window and send the selection information to the terminal. The option data is input, and the selection information is sent to the server as output.

[1140] Step 9:

[1141] The server generates the next response or feedback based on the selection. The server generates the next response or feedback based on the selection information and sends it to the device. The selection information is the input, and the next response or feedback is sent to the device as the output.

[1142] Step 10:

[1143] The emotion engine analyzes the user's input. The server uses the emotion engine to analyze the user's input data and extract emotional information. The user's input data is the input, and emotional information is obtained as the output.

[1144] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1145] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1146] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1147] [Third embodiment]

[1148] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1149] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1150] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1151] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1152] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1153] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1154] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1155] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1156] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1157] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1158] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1159] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1160] The present invention is a system that generates a virtual personality based on conditions input by a user and improves the user's communication skills through chat with that virtual personality.

[1161] System Overview

[1162] The system includes a means for a user to input conditions and generate a virtual personality based on the conditions, a means for chatting between the generated virtual personality and the user, a means for presenting multiple options during the chat, a means for generating feedback based on the user's selection, and a means for providing the feedback to the user. It also includes a means for recording the user's progress and utilizing the progress in the next session.

[1163] Explanation of program processing

[1164] 1. The user enters the conditions

[1165] User: Uses a device to enter the target conditions (age, gender, place of origin, company, position, personality, family structure, address, preferences, etc.).

[1166] Terminal: Checks the format of the entered data to make sure there are no errors. If there are no errors, the data is sent to the server.

[1167] 2. Send the data to the server

[1168] Terminal: Sends condition data serialized in JSON format to the server.

[1169] Server: Receives the request, parses the data, and obtains the user's criteria information.

[1170] 3. The server generates a virtual personality

[1171] Server: Based on the acquired condition information, a virtual personality is generated using a generative AI model (e.g., GPT-3, BERT). The generated virtual personality data is stored in storage and linked to the user's session information.

[1172] 4. Send virtual personality information to the user's device

[1173] Server: Sends the generated virtual personality data to the user's device as an HTTP response.

[1174] Terminal: Analyzes the virtual personality's profile information and displays it in the chat window.

[1175] 5. The user starts chatting with the virtual persona

[1176] User: Type a message in the chat window.

[1177] Terminal: Sends the entered message to the server.

[1178] Server: Analyzes the message and invokes the AI ​​model for the virtual personality to generate an appropriate response.

[1179] 6. The virtual personality responds and offers options

[1180] Server: Uses AI models to generate responses for the virtual personality, and also generates multiple options to present to the user.

[1181] Server: Sends the generated responses and options to the user's terminal.

[1182] Terminal: Displays the virtual personality's responses and options in the chat window.

[1183] 7. The user selects an option

[1184] User: Choose one of the options presented.

[1185] Terminal: Sends the selected option information to the server.

[1186] Server: Generates the next response or feedback based on the option information.

[1187] 8. Providing Feedback

[1188] Server: Generates feedback on the choices and provides suggestions and improvements to the user.

[1189] Server: Sends the generated feedback to the user's device.

[1190] Terminal: Show feedback in the chat window.

[1191] 9. Record and track user growth

[1192] Server: Records chat session data and stores it in a database for future sessions. This data is used to support the continued growth of our users.

[1193] Specific examples

[1194] Step 1: User enters criteria

[1195] User: Enter the target criteria as "male in his 20s, university student, tennis lover, living in Tokyo."

[1196] Terminal: Checks the format of the condition data, and if there are no errors, sends it to the server.

[1197] Step 2: Send the data to the server

[1198] Terminal: Converts the condition data into JSON format and sends it to the server.

[1199] Server: Receives data and extracts condition information.

[1200] Step 3: The server generates a virtual personality

[1201] Server: Generates a virtual personality using a generative AI model based on condition information.

[1202] Server: Stores the data of the generated virtual personality in storage and links it to session information.

[1203] Step 4: Send virtual personality information to the user's device

[1204] Server: Sends virtual personality data to the user's device.

[1205] Terminal: Display the virtual personality's profile in the chat window.

[1206] Step 5: The user starts chatting with the virtual persona

[1207] User: Type the message "Hi! You said you like tennis. How often do you play?" into the chat window.

[1208] Terminal: Sends a message to the server.

[1209] Server: Parses the message and invokes the AI ​​model to generate a response.

[1210] Step 6: The virtual persona responds and offers options

[1211] Server: The virtual personality responds with "Hello! I play two or three times a week. Who's Tanaka?" and generates multiple options.

[1212] Server: Sends the response and options to the user's terminal.

[1213] Terminal: Show responses and options in the chat window.

[1214] Step 7: User selects a choice

[1215] User: Select "Talk about another hobby" from the options.

[1216] Terminal: Sends the selection information to the server.

[1217] Server: Generates the next response or feedback.

[1218] Step 8: Provide feedback

[1219] Server: Generate feedback "When talking about different hobbies, it's important to explore common interests."

[1220] Server: Sends feedback to the user's device.

[1221] Terminal: Display feedback in the chat window.

[1222] Step 9: Record and track user growth

[1223] Server: Records chat session data and stores it in a database for future sessions.

[1224] In this way, users can comfortably practice their communication skills in a virtual environment and gain confidence in real-life face-to-face conversations.

[1225] The processing flow will be explained below.

[1226] Step 1:

[1227] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) into the condition input form on the device.

[1228] Step 2:

[1229] Terminal: Check the format of the entered data to ensure that the data format is correct and that required fields are not missing.

[1230] Step 3:

[1231] Terminal: Condition data that passes the format check is serialized into JSON format and sent to the server as an HTTP POST request.

[1232] Step 4:

[1233] Server: Analyzes the received request and extracts the condition data sent.

[1234] Step 5:

[1235] Server: Using the condition data, it sends a request to a generative AI model (e.g., GPT-3 or BERT) to generate a virtual personality.

[1236] Step 6:

[1237] Server: Stores the generated virtual personality data (such as name, age, personality, hobbies, etc.) in storage and links it to the user's session information.

[1238] Step 7:

[1239] Server: Sends the virtual personality data to the user's terminal as an HTTP response.

[1240] Step 8:

[1241] Terminal: Analyzes the received virtual personality data and displays profile information in the chat window.

[1242] Step 9:

[1243] User: Type a message in the chat window.

[1244] Step 10:

[1245] Terminal: Sends the entered message to the server.

[1246] Step 11:

[1247] Server: Analyzes the received message and generates an appropriate reply by invoking an AI model for the virtual personality to respond.

[1248] Step 12:

[1249] Server: Generates multiple options to present to the user along with the responses of the generated virtual personality.

[1250] Step 13:

[1251] Server: Sends the virtual personality's responses and options to the user's terminal.

[1252] Step 14:

[1253] Terminal: Displays the virtual personality's replies and options in the chat window.

[1254] Step 15:

[1255] User: Select one of the options presented and send the selection to the terminal.

[1256] Step 16:

[1257] Terminal: Sends the selection information to the server.

[1258] Step 17:

[1259] Server: Generates the next response or feedback based on the selected information.

[1260] Step 18:

[1261] Server: Sends the generated feedback and the next response to the user's device.

[1262] Step 19:

[1263] Terminal: Show feedback and next response in the chat window.

[1264] Step 20:

[1265] Server: Records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions.

[1266] Example 1

[1267] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1268] Previous systems for improving communication skills had the problem of making it difficult for users to receive appropriate support. Specifically, it was difficult to generate a virtual personality based on the user's individual conditions, and there was a lack of means to continuously support the user's growth through dialogue with that virtual personality. Furthermore, there was a lack of technology for format checking of user input data and generating appropriate responses in real time, making it difficult to improve the user experience.

[1269] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1270] In this invention, the server includes: means for a user to input conditions; means for utilizing a generative AI model to generate a virtual personality based on the conditions; means for chatting between the virtual personality and the user; means for presenting multiple options during the chat and generating feedback based on the user's selection; means for providing the feedback to the user; means for displaying profile information of the virtual personality on a chat screen; and means for calling a generative AI model to analyze messages input by the user and generate appropriate responses. This allows users to continuously improve their communication skills through real-time interactions with virtual personalities based on their conditions.

[1271] "User" refers to a person who operates the system, inputs conditions, and interacts with a virtual personality.

[1272] "Conditions" refer to individual information such as age, sex, occupation, hobbies, and place of residence entered by the user.

[1273] A "virtual personality" refers to a virtual entity that can be used as a conversation partner and is generated based on the user's conditions using a generative AI model.

[1274] A "generative AI model" refers to an artificial intelligence model that generates a virtual personality using natural language processing based on conditions entered by the user.

[1275] "Chat" refers to a text-based interaction between a user and a virtual persona.

[1276] "Choices" refers to multiple response or reaction options that a user is presented with by a virtual persona during a chat.

[1277] "Feedback" refers to information such as suggestions and improvements provided based on the user's selections.

[1278] A "prompt" refers to an input instruction that causes a generative AI model to generate a specific response.

[1279] "Profile information" refers to information that a user can know, such as the virtual personality's personality, hobbies, and background.

[1280] "Session information" refers to the interaction history and associated data between a user and a virtual persona.

[1281] "Format check" refers to the process of checking whether the data entered by the user conforms to the specified format.

[1282] "HTTP response" refers to the data format returned from the server to the client (terminal).

[1283] This invention is a system that generates a virtual personality based on conditions input by a user and improves the user's communication skills through chatting with that virtual personality.

[1284] Hardware and Software Configuration

[1285] This system mainly consists of the following components:

[1286] 1. User's device: a device such as a personal computer, smartphone, or tablet.

[1287] 2. Server: Cloud server or local server.

[1288] 3. Generative AI models: Use natural language generation models such as GPT-3 and BERT.

[1289] Data processing and calculation

[1290] The system performs the following steps:

[1291] 1. The user enters the conditions

[1292] The user enters information such as age, gender, occupation, hobbies, and place of residence through the device interface. The device then performs a format check on the entered data to ensure there are no errors. If there are no errors, the data is serialized in JSON format and sent to the server.

[1293] 2. Acquire and analyze data

[1294] The server receives the condition data sent in JSON format, parses it, and extracts the user's condition information, which is then stored in temporary storage.

[1295] 3. Generate a virtual personality

[1296] The server sends a prompt to the generative AI model (e.g., GPT-3, BERT) based on the acquired condition information. The prompt includes the condition information entered by the user. For example, "Please generate a virtual personality based on the following conditions: age in his 20s, gender male, university student, tennis lover, residing in the Tokyo metropolitan area." The profile information of the generated virtual personality is saved in storage and linked to the user's session information.

[1297] 4. Send virtual personality information to the user device

[1298] The server sends the generated virtual personality data to the user's device as an HTTP response. The device analyzes this data and displays the virtual personality's profile (name, characteristics, hobbies, etc.) in a chat window.

[1299] 5. The user starts chatting with the virtual persona

[1300] The user types a message into the chat window, for example, "Hello! You said you like tennis. How often do you play?" The device then sends the typed message to the server.

[1301] 6. The virtual personality responds and offers options

[1302] The server analyzes the received message and calls the generative AI model again to generate a response from the virtual personality. For example, it generates a response like, "Hello! I play two or three times a week. How about you?" The generated response and multiple options (for example, "I play once a week," "Talk about another hobby," etc.) are sent to the user's device. The device displays the virtual personality's response and options in a chat window.

[1303] 7. The user selects an option

[1304] The user selects one of the options presented and confirms the choice by clicking or other operation. The device then sends information about the selected option to the server.

[1305] 8. Providing Feedback

[1306] The server generates feedback based on the user's choices, such as advice like, "When talking about different hobbies, it's important to find common interests." The generated feedback is sent to the user's device and displayed in the chat window.

[1307] 9. Record and track user growth

[1308] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for reference during the next session, which is used to support the user's continuous growth.

[1309] In this way, this system allows users to learn practical communication skills in a safe virtual environment and improve their ability to interact in the real world.

[1310] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1311] Step 1:

[1312] The user enters conditions (age, gender, occupation, hobbies, place of residence, etc.) using a device. The entered data is format-checked on the device to ensure there are no errors. If there are no errors, the data is serialized in JSON format.

[1313] Input: The user enters the conditions into the input form on the device.

[1314] Output: Serialized JSON formatted condition data

[1315] Step 2:

[1316] The device sends the condition data that has passed the format check to the server via an HTTP POST request. The server receives this request, parses the data, and extracts the user's condition information. The extracted data is saved in temporary storage.

[1317] Input: Condition data in serialized JSON format

[1318] Output: Parsed condition information data

[1319] Step 3:

[1320] The server then sends prompts to the generative AI model based on the extracted condition information. The prompts contain the user's condition information and prompt the model to generate speech content and a profile.

[1321] Input: Parsed condition information

[1322] Output: The prompt sent to the generative AI model

[1323] Step 4:

[1324] A generative AI model (e.g., GPT-3, BERT) generates a virtual personality profile based on the prompt text. This profile information is received on the server side, linked to the user's session information, and stored in storage.

[1325] Input: prompt statement

[1326] Output: Profile information of the generated virtual personality

[1327] Step 5:

[1328] The server sends the profile information of the virtual personality to the user's device as an HTTP response. The device receives this data, analyzes it, and displays the profile information (name, characteristics, hobbies, etc.) in the chat window.

[1329] Input: Profile information of the generated virtual personality

[1330] Output: Profile information displayed in the chat window

[1331] Step 6:

[1332] The user enters a message in the chat window. The device sends the message to the server, which receives the message and calls the generative AI model again to generate a response from the virtual personality.

[1333] Input: The message entered by the user

[1334] Output: A response message from the generative AI model

[1335] Step 7:

[1336] The server sends the generated response message and multiple options to the user's terminal, which receives it and displays the response message and options in a chat window.

[1337] Input: Response message from generative AI model, multiple choice

[1338] Output: Response message and options displayed in the chat window

[1339] Step 8:

[1340] The user selects one of the options presented and confirms the choice by clicking or other operation. The device then sends information about the selected option to the server.

[1341] Input: The choice selected by the user

[1342] Output: Choice information sent to the server

[1343] Step 9:

[1344] The server receives and analyzes the selection information and generates feedback, which is then sent back to the user's terminal and displayed in the terminal's chat window.

[1345] Input: Information about the option selected by the user

[1346] Output: Generated feedback

[1347] Step 10:

[1348] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for reference during the next session, to support the user's continuous growth.

[1349] Input: Chat session data

[1350] Output: Session information stored in the database

[1351] (Application example 1)

[1352] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1353] Security operators and guards are required to have advanced communication skills in order to respond quickly and appropriately to emergencies and daily security activities, but there is a lack of appropriate training methods and tools to efficiently improve these skills.

[1354] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1355] In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for generating a virtual personality based on a situation related to security activities, and means for communicating between the virtual personality and the user regarding an emergency. This enables security operators and security guards to efficiently improve the communication skills required for their work through virtual conversations based on actual situations.

[1356] The "means for user input" is a digital input interface that allows a user to provide specific requirements and situational information to the system.

[1357] The "means for generating a virtual personality based on the above conditions" is a technology that uses artificial intelligence to create a virtual conversation partner based on input condition information.

[1358] The "means for chatting between the virtual personality and the user" is a communication function that allows the generated virtual personality and the user to exchange text messages in real time.

[1359] "Means for presenting multiple options during chat and generating feedback based on the user's selection" is a mechanism that provides the user with several options during a conversation and generates an appropriate response or evaluation based on the selection.

[1360] The "means for providing the feedback to the user" is an output function for displaying or notifying the generated feedback or evaluation to the user.

[1361] The "means for generating a virtual personality based on a situation related to security activities" is a technology for generating a virtual personality based on a specific scenario related to security activities or emergency situations.

[1362] "Means for communication between the virtual personality and the user regarding an emergency" refers to a communication function for giving instructions and sharing information in an emergency through virtual dialogue.

[1363] This invention utilizes a virtual dialogue system to improve the effective communication skills of security operators and guards in emergency situations. The system is composed of a user, a terminal, and a server.

[1364] System Overview

[1365] The user uses a device to input specific conditions (for example, "Occupation: Security Operator," "Emergency: Fire," "Location: Office Building," "Instructions: Evacuation Instructions," etc.). The device checks the format of this condition data and then sends it to the server in JSON format. The server generates a virtual personality using a generative AI model (for example, GPT-2) based on the received condition information. The server then saves the virtual personality data and session information in storage and provides it to the user.

[1366] The user starts chatting with the virtual persona on the device. The message entered by the user is sent to the server, which uses an AI model to generate an appropriate response. The generated response and multiple options presented to the user are sent to the device and displayed in a chat window. The user selects one of the options, and feedback is generated and provided to the user based on that selection.

[1367] The system also includes a means to record the user's progress and use it for future sessions. This data supports the user's continuous improvement.

[1368] Hardware and software used

[1369] Hardware: Smartphone

[1370] Software: Python, generative AI model (GPT-2)

[1371] Explanation of data processing or data calculation

[1372] 1. Enter conditions and submit

[1373] The user inputs conditions such as "occupation: security operator," "emergency situation: fire," "location: office building," and "instructions: evacuation instructions."

[1374] The terminal checks the format of the condition data, and if there are no errors, sends it to the server in JSON format.

[1375] 2. Creation and provision of virtual personalities

[1376] The server generates a virtual personality using a generative AI model (GPT-2) based on the received condition information.

[1377] Data on the generated virtual personality is sent back to the user's terminal.

[1378] 3. Interacting with Virtual Personalities

[1379] The user inputs a message such as "Hello, there is an emergency. There is a fire. Please give us your instructions."

[1380] The server analyzes the received message using an AI model and generates an appropriate response.

[1381] The generated responses and options are displayed on the user's terminal.

[1382] 4. Providing Feedback

[1383] The server generates and provides appropriate feedback to the user based on the user's selection.

[1384] Provide specific advice and improvements to help users improve their skills.

[1385] Examples of concrete examples and prompts

[1386] Examples:

[1387] Enter the conditions: "Occupation: Security Operator", "Emergency: Fire", "Location: Office Building", "Instructions: Evacuation Instructions"

[1388] Initial message: "Hello, emergency. Fire. Your instructions needed."

[1389] Example prompt sentence:

[1390] Conditions: "Occupation: Security Operator", "Emergency: Fire", "Location: Office Building", "Instructions: Evacuation"

[1391] Dialogue start message: "Hello, emergency. Fire. Your instructions needed."

[1392] In this way, users can safely practice their communication skills in a scenario that closely resembles real-world work.

[1393] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1394] Step 1:

[1395] The user inputs conditions. Using their smartphone, the user inputs conditions such as "Occupation: Security Operator," "Emergency: Fire," "Location: Office Building," and "Instructions: Evacuation Instructions." The device checks the format of the input data to ensure there are no errors. After checking, the device serializes the condition data into JSON format and sends it to the server.

[1396] Step 2:

[1397] The server receives the data. The server receives the condition data in JSON format sent from the device. The server parses the data, extracts the condition information, and prepares it as input data for use in the next processing step.

[1398] Step 3:

[1399] The server generates a virtual personality. Based on the extracted condition information, the server invokes a generative AI model (e.g., GPT-2) to generate the virtual personality. This generative model generates appropriate personality attributes based on the condition information and constructs data for the virtual personality. The data for the generated virtual personality is stored in the server's storage and linked to the user's session information.

[1400] Step 4:

[1401] The server sends information about the virtual personality to the user's device. The server then sends the generated virtual personality data to the user's device as an HTTP response. The device receives this data, analyzes it, and displays the virtual personality's profile information in the chat window.

[1402] Step 5:

[1403] The user starts chatting with the virtual persona. The user types a message into the chat window on the device: "Hello, there's an emergency. There's a fire. Please give us your instructions." The device sends the typed message to the server. The server receives the message, analyzes it, and calls the AI ​​model to generate an appropriate response from the virtual persona.

[1404] Step 6:

[1405] The server generates a response and presents the user with multiple options. The server uses an AI model to generate a response for the virtual personality, and then generates multiple options to present to the user. For example, a response such as "Please remain calm and head for the emergency exit" is generated. The generated response and options are sent to the user's device as an HTTP response. The device receives this and displays it in a chat window.

[1406] Step 7:

[1407] The user selects an option. The user selects one of the options presented and enters the selection information. The selection information is sent to the server via the device. The server again uses the AI ​​model to generate the next response or feedback based on the selected option information.

[1408] Step 8:

[1409] The server generates feedback and provides it to the user. The server generates feedback for the selected option and provides the user with suggestions and improvements. For example, the server might generate feedback such as "It is important to communicate instructions specifically and concisely." The generated feedback is sent to the user's device as an HTTP response and displayed in the chat window.

[1410] Step 9:

[1411] The server records the user's progress and uses it for the next session. The server records the chat session data and saves it in a database to use for the next session. This recorded data is used by the user to continuously improve their communication skills.

[1412] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1413] This system generates a virtual personality based on user input and improves the user's communication skills through chat with the virtual personality. It also incorporates an emotion engine that recognizes the user's emotions and provides feedback to improve the effectiveness of the chat.

[1414] System Overview

[1415] The system includes the following main means:

[1416] 1. A way for users to enter conditions

[1417] 2. Means for generating a virtual personality based on the above conditions

[1418] 3. Means for chatting between the virtual personality and the user

[1419] 4. A way to present multiple options during a chat and generate feedback based on the user's selection

[1420] 5. Means for providing said feedback to the user

[1421] 6. Means of recognizing user emotions using an emotion engine

[1422] 7. A way to track user progress and use it in the next session

[1423] 8. A means of formatting user-entered condition data

[1424] Explanation of program processing

[1425] 1. The user enters the conditions

[1426] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) on the device.

[1427] Terminal: Checks the format of the entered data and sends it to the server if there are no errors.

[1428] 2. Send the data to the server

[1429] Terminal: Serialize the condition data in JSON format and send it to the server as an HTTP POST request.

[1430] Server: Receives the request and parses the data to extract condition information.

[1431] 3. The server generates a virtual personality

[1432] Server: Sends a request to a generative AI model (e.g., GPT-3 or BERT) based on condition information to generate a virtual personality.

[1433] Server: Stores the virtual personality data (name, age, personality, hobbies, etc.) in storage and links it to the user's session information.

[1434] 4. Send virtual personality information to the user's device

[1435] Server: Sends the generated virtual personality data to the user's device as an HTTP response.

[1436] Terminal: Analyzes the virtual personality's profile information and displays it in the chat window.

[1437] 5. The user starts chatting with the virtual persona

[1438] User: Type a message in the chat window.

[1439] Terminal: Sends the entered message to the server.

[1440] Server: Analyzes the message and invokes the AI ​​model for the virtual personality to generate an appropriate response.

[1441] 6. The virtual personality responds and offers options

[1442] Server: Uses AI models to generate responses for the virtual personality, and also generates multiple options to present to the user.

[1443] Server: Sends the generated responses and options to the user's terminal.

[1444] Terminal: Displays the virtual personality's responses and options in the chat window.

[1445] 7. The user selects an option

[1446] User: Select one of the options presented and send the selection to the terminal.

[1447] Terminal: Sends the selection information to the server.

[1448] Server: Generates the next response or feedback based on the selected information.

[1449] 8. Providing Feedback

[1450] Server: Generates feedback on the choices and provides suggestions and improvements to the user.

[1451] Server: Sends the generated feedback to the user's device.

[1452] Terminal: Show feedback in the chat window.

[1453] 9. The Emotional Engine

[1454] Server: Analyzes the text entered by the user using the emotion engine and extracts emotion information.

[1455] Server: Adjusts the virtual personality's responses based on emotional information and generates dialogue that takes the user's emotional state into account.

[1456] 10. Record and track user growth

[1457] Server: Records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions. This data is used to support the user's ongoing growth.

[1458] Specific examples

[1459] Step 1: User enters criteria

[1460] User: Enters the following criteria into the input form on the device: "Male in his 20s, university student, tennis lover, living in Tokyo."

[1461] Terminal: Checks the format of the condition data, and if there are no errors, sends it to the server.

[1462] Step 2: Send the data to the server

[1463] Terminal: Serialize the condition data into JSON format and send it to the server as an HTTP POST request.

[1464] Server: Parse the data and extract the condition information.

[1465] Step 3: The server generates a virtual personality

[1466] Server: Based on the condition information, a virtual personality "Yamada-kun" is generated using a generative AI model.

[1467] Server: Stores the virtual personality data in storage and links it to session information.

[1468] Step 4: Send virtual personality information to the user's device

[1469] Server: Sends the generated virtual personality data to the user's device.

[1470] Terminal: Display the virtual personality's profile information in the chat window.

[1471] Step 5: The user starts chatting with the virtual persona

[1472] User: Type in the chat window, "Hi! You said you like tennis. How often do you play?"

[1473] Terminal: Sends the entered message to the server.

[1474] Server: Parses the message and invokes the AI ​​model to generate a response.

[1475] Step 6: The virtual persona responds and offers options

[1476] Server: Using an AI model, it generates the virtual personality's response, "Hello! I play two or three times a week. What about you?", while also generating options such as "Continue talking about tennis," "Go back to my introduction," or "Talk about another hobby."

[1477] Server: Sends the response and options to the user's terminal.

[1478] Terminal: Show responses and options in the chat window.

[1479] Step 7: User selects a choice

[1480] User: Select "Talk about another hobby" from the options.

[1481] Terminal: Sends the selection information to the server.

[1482] Server: Generates the next response or feedback.

[1483] Step 8: Provide feedback

[1484] Server: Generate feedback for the choice: "Talking about other hobbies is a good opportunity to learn new things about each other."

[1485] Server: Sends feedback to the user's device.

[1486] Terminal: Show feedback in the chat window.

[1487] Step 9: Emotional Engine in Action

[1488] Server: Using an emotion engine, the text entered by the user is analyzed and emotional information such as "Looks fun" or "Very interested" is extracted.

[1489] Server: Based on the emotional information, the virtual personality's responses are adjusted to generate more friendly dialogue.

[1490] Step 10: Record and track user growth

[1491] Server: Records chat session data and stores it in a database for future sessions.

[1492] In this way, users can safely hone their communication skills in a virtual environment, understand their own emotional state through the emotion engine, and gain confidence in real-life face-to-face conversations.

[1493] The processing flow will be explained below.

[1494] Step 1:

[1495] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) into the condition input form on the device.

[1496] Step 2:

[1497] Terminal: Checks the format of the entered data to ensure that the format and required fields are correct. If there are any errors, an error message is displayed prompting the user to make corrections.

[1498] Step 3:

[1499] Terminal: Condition data that passes the format check is serialized into JSON format and sent to the server as an HTTP POST request.

[1500] Step 4:

[1501] Server: Receives the HTTP POST request and parses the data to extract user criteria information.

[1502] Step 5:

[1503] Server: Based on the extracted condition information, a request is sent to a generative AI model (e.g., GPT-3 or BERT) to generate a virtual personality.

[1504] Step 6:

[1505] Server: The generated virtual personality data (name, age, personality, hobbies, etc.) is stored in a storage and linked to the user's session information.

[1506] Step 7:

[1507] Server: Converts the virtual personality data into JSON format and sends it to the user's device as an HTTP response.

[1508] Step 8:

[1509] Terminal: Analyzes the received virtual personality data and displays profile information in the chat window.

[1510] Step 9:

[1511] User: Type a message in the chat window.

[1512] Step 10:

[1513] Terminal: Sends the entered message to the server.

[1514] Step 11:

[1515] Server: Analyzes the received message and calls the AI ​​model for the virtual personality to generate an appropriate response.

[1516] Step 12:

[1517] Server: Generates multiple options along with the responses of the generated virtual personality.

[1518] Step 13:

[1519] Server: Sends the generated responses and options to the user's terminal.

[1520] Step 14:

[1521] Terminal: A chat window displays the virtual personality's responses and the options presented.

[1522] Step 15:

[1523] User: Select one of the displayed options and send the selected information to the terminal.

[1524] Step 16:

[1525] Terminal: Sends the selection information to the server.

[1526] Step 17:

[1527] Server: Analyzes the selection information and generates the next response or feedback based on it.

[1528] Step 18:

[1529] Server: Sends the generated feedback and the next response to the user's terminal.

[1530] Step 19:

[1531] Terminal: Show feedback and next response in the chat window.

[1532] Step 20:

[1533] Server: Using the emotion engine, analyzes the message entered by the user and extracts emotional information.

[1534] Step 21:

[1535] Server: Based on the emotional information, the server appropriately adjusts the responses of the virtual personality and generates a dialogue that takes into account the user's emotional state.

[1536] Step 22:

[1537] Server: Sends the adjusted response to the user's device, and displays emotional information as feedback.

[1538] Step 23:

[1539] Terminal: Responses containing emotional information are displayed in the chat window, allowing users to check their own emotional state.

[1540] Step 24:

[1541] Server: Records all data from the chat session (inputs, responses, options, feedback, emotional information, etc.) and stores it in a database for future sessions.

[1542] Example 2

[1543] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1544] In modern society, providing a training environment for users to improve their communication skills is an important issue. In particular, there is a need for a system that provides real-time feedback while taking into account the user's emotional state. There is also a need for a means to record the user's progress over the long term and provide dialogue based on that record.

[1545] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1546] In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for presenting multiple options during the chat and generating feedback based on the user's selection, means for providing the feedback to the user, means for analyzing the user's emotions, means for adjusting the virtual personality's responses based on the emotion information, means for generating appropriate responses and suggestions for the user's selections, means for recording chat session data and using it in the next session, and means for formatting the condition data entered by the user. This allows users to train their communication skills in a virtual environment with peace of mind, understand their own emotional state through the emotion engine, and gain confidence in actual conversations.

[1547] A "user" is an individual who utilizes the system to chat with a virtual personality and improve their communication skills.

[1548] "Conditions" refers to information such as age, gender, occupation, and hobbies entered by the user.

[1549] A "virtual personality" is a character that can be used as a conversation partner, generated by a generative AI model based on conditions entered by the user.

[1550] "Chat" refers to the exchange of messages between a user and a virtual persona.

[1551] "Choices" refers to multiple options presented to a user during a chat.

[1552] "Feedback" refers to suggestions and improvements generated based on user selections.

[1553] "Emotion" refers to the psychological state that the emotion engine analyzes and extracts from the message entered by the user.

[1554] An "emotion engine" is software or algorithms that analyze user-entered text and extract emotional information.

[1555] A "response" refers to a message that a virtual personality returns in response to a user's message.

[1556] A "session" refers to a series of chat exchanges between a user and a virtual persona.

[1557] "Format check" refers to the process of checking whether the condition data entered by the user is in the correct format.

[1558] A "generative AI model" refers to an artificial intelligence model that generates a virtual personality based on given conditions.

[1559] A "prompt sentence" refers to an input sentence that requests the generative AI model to generate a virtual personality based on certain conditions.

[1560] This system generates a virtual personality based on user input and improves the user's communication skills through chat with the virtual personality. It also incorporates an emotion engine that recognizes the user's emotions and provides feedback to improve the effectiveness of the chat.

[1561] The system includes the following main means:

[1562] 1. A way for users to enter conditions

[1563] 2. Means for generating a virtual personality based on the above conditions

[1564] 3. Means for chatting between the virtual personality and the user

[1565] 4. A way to present multiple options during a chat and generate feedback based on the user's selection

[1566] 5. Means for providing said feedback to the user

[1567] 6. Means of recognizing user emotions using an emotion engine

[1568] 7. A way to track user progress and use it in the next session

[1569] 8. A method for format checking user-entered condition data

[1570] The program for this system uses the following hardware and software:

[1571] Terminal: A device that accepts user input and sends data to a server. Examples include PCs, smartphones, and tablets.

[1572] Server: A central computer system that processes data, generates virtual personalities, manages chats, analyzes emotions, and provides feedback. Examples include virtual machines on a cloud service or dedicated servers.

[1573] Generative AI model: An artificial intelligence model that generates virtual personalities based on conditions. Examples include OpenAI's GPT-3 and BERT.

[1574] Emotion engine: A software component that analyzes text entered by a user and extracts emotional information. It is implemented using a natural language processing library or an AI engine.

[1575] The following example is a concrete example of the flow of operations.

[1576] Example 1: User enters criteria

[1577] The user enters the following criteria into the input form on the terminal: "Male in his 20s, university student, tennis lover, living in Tokyo."

[1578] The terminal checks the format of the input condition data, and if there are no errors, sends it to the server.

[1579] Example prompt sentence:

[1580] "Generate a hobby profile for a male in his 20s, a university student, who likes tennis and lives in Tokyo."

[1581] Example 2: Server generates virtual personality

[1582] The server generates a virtual personality "Yamada-kun" using a generative AI model based on the condition information and links it to the session information.

[1583] Example 3: Chatting with a virtual personality

[1584] The user types into the chat window, "Hi! You said you like tennis. How often do you play?"

[1585] The server uses an emotion engine to analyze the text entered by the user and extract emotional information such as "it looks fun" or "it looks very interesting."

[1586] The server uses an AI model to generate a response and then presents the user with multiple options.

[1587] In this way, this system, which operates in cooperation between users, devices, and servers, provides an environment in which users can improve their communication skills with peace of mind, and by providing appropriate feedback based on emotion analysis, users can proceed with conversations with confidence in real-life situations.

[1588] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1589] Step 1:

[1590] The user enters the conditions

[1591] The user inputs the target conditions (e.g., age, gender, occupation, hobbies, etc.) into the input form provided on the terminal. For example, the user inputs "male in his 20s, university student, tennis lover, living in Tokyo."

[1592] The terminal receives the input data (age, gender, occupation, hobbies, etc.) and performs a format check. If any errors are found, the terminal notifies the user of the information.

[1593] Input: Condition data entered by the user in the input form

[1594] Output: Correctly formatted condition data (after format check)

[1595] Step 2:

[1596] Sending data to the server

[1597] The terminal serializes the condition data that has passed the format check into JSON format.

[1598] The device sends the serialized JSON data to the server as an HTTP POST request using the endpoint " / create_personality".

[1599] The server receives the request and parses the data to extract the necessary condition information.

[1600] Input: JSON format of condition data sent from the terminal

[1601] Output: Server-side parsed condition data

[1602] Step 3:

[1603] The server generates a virtual personality

[1604] The server sends a request to a generative AI model (e.g., GPT-3 or BERT) based on the parsed condition information. Specifically, the prompt statement "Generate a hobby profile for a male in his 20s, a university student, who likes tennis and lives in Tokyo" is passed to the generative AI model.

[1605] The server retrieves the virtual personality data (such as name, age, personality, hobbies, etc.) returned by the generative AI model and stores it in internal storage (e.g., a NoSQL database). This data is associated with the user's session information.

[1606] Input: Parsed condition data

[1607] Output: Virtual personality data returned from the generative AI model

[1608] Step 4:

[1609] Send virtual personality information to the user's device

[1610] The server sends the generated virtual personality data to the user's device as an HTTP response. Specifically, the response is composed of the user's session ID and the data.

[1611] The device analyzes the received virtual personality data and displays it in the chat window. For example, it might display information like "Yamada, male in his 20s, university student, tennis lover, living in Tokyo."

[1612] Input: Virtual personality data from the server

[1613] Output: Virtual personality profile information displayed in the chat window

[1614] Step 5:

[1615] The user initiates a chat with a virtual persona

[1616] The user types a message into the chat window, for example, "Hi! I saw you said you like tennis. How often do you play?"

[1617] The terminal sends the entered message to the server by sending a POST request to the endpoint " / chat".

[1618] The server analyzes the received message and processes the text content using its internal natural language processing engine.

[1619] Input: A chat message from the user

[1620] Output: Analysis results from the natural language processing engine

[1621] Step 6:

[1622] The virtual personality responds and offers options

[1623] The server uses a natural language processing engine to analyze the user's message and generate a response from the virtual personality, such as "Hello! I play two to three times a week."

[1624] The server generates multiple options for the user (e.g., "Continue talking about tennis," "Return to self-introduction," "Talk about another hobby").

[1625] The server sends the generated responses and options to the user's terminal.

[1626] The terminal displays the virtual personality's responses and options in a chat window.

[1627] Input: User's chat message and choices

[1628] Output: Response message and options from the virtual personality

[1629] Step 7:

[1630] The user selects an option

[1631] The user selects one of the presented options and transmits the selected information to the terminal.

[1632] The device sends the selection information to the server. Specifically, it sends a request including the option ID to the endpoint " / select_option".

[1633] The server analyzes the received selection information and generates the following response or feedback:

[1634] Input: User selection information

[1635] Output: The next response or feedback

[1636] Step 8:

[1637] Providing feedback

[1638] The server generates feedback based on the user's selection, for example, "Talking about other hobbies is a good opportunity to discover new aspects of each other."

[1639] The server transmits the generated feedback to the user's terminal.

[1640] The device displays the feedback in a chat window.

[1641] Input: User selection information

[1642] Output: Feedback message

[1643] Step 9:

[1644] The Emotional Engine in Action

[1645] The server uses an emotion engine to analyze the text entered by the user, extracting emotion information such as "fun" or "interesting."

[1646] The server adjusts the responses of the virtual personality based on the emotional information and generates a dialogue that takes into account the user's emotional state.

[1647] Input: User's chat message

[1648] Output: Emotional information and tailored response message

[1649] Step 10:

[1650] Record and track user growth

[1651] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions. This data is used to support the user's ongoing growth.

[1652] Input: Chat session data

[1653] Output: Session data stored in the database

[1654] As described above, users, devices, and servers work together to operate the system efficiently, improving users' communication skills and providing appropriate feedback based on emotion analysis.

[1655] (Application example 2)

[1656] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1657] Efficient work instruction and worker safety training are important issues in modern manufacturing sites. In particular, quickly and reliably learning new procedures and technologies is essential for improving productivity. However, traditional methods require face-to-face instruction, and the quality of instruction depends on the skill and experience of the instructor, making it difficult to provide consistent instruction. It is also difficult to provide feedback that reflects the emotions and level of understanding of workers.

[1658] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for presenting multiple options during the chat and generating feedback based on the user's selection, means for providing the feedback to the user, means for extracting emotional information from data input by the user using an emotion engine, means for adjusting responses and feedback based on the user's emotional information, and means installed in industrial machinery for generating a training virtual personality and providing guidance to workers. This enables real-time guidance and feedback according to the emotional state of the worker.

[1659] The "means for the user to input conditions" is an interface device that allows the user to input conditions for generating a virtual personality.

[1660] The "means for generating a virtual personality" is a system that generates a virtual personality with specific attributes and characteristics based on conditions entered by the user.

[1661] The "means for chatting" is a communication interface for sending and receiving messages between the user and the virtual personality.

[1662] The "means for presenting options and generating feedback" is a function that presents multiple options to the user during a chat and provides feedback based on the selection.

[1663] A "means for providing feedback to a user" is an interface device for presenting the generated feedback to a user.

[1664] The "means for extracting emotional information from user input data using an emotion engine" is a function for analyzing data input by a user and identifying an emotional state from that data.

[1665] The "means for adjusting responses and feedback based on the user's emotional information" is a function that appropriately adjusts the responses and feedback of the virtual personality based on the emotional information extracted by the emotion engine.

[1666] "Means to be installed in industrial machines, generate educational virtual personalities, and provide guidance to workers" refers to a system that is installed in industrial machines and provides education and guidance to workers through virtual personalities.

[1667] The following describes an embodiment of the present invention. This system generates a virtual personality based on conditions entered by a user, and provides training to workers through the virtual personality installed on industrial machines. This will be described in detail below.

[1668] First, the user inputs conditions using an interface device, such as the worker's years of experience, new work procedures, language, etc. The input conditions are serialized by the terminal and sent to the server as an HTTP POST request.

[1669] Next, based on the condition data received by the server, a generative AI model is used to generate a virtual personality. This generative AI model uses natural language processing models such as GPT-3 and BERT. The generated virtual personality has attributes and characteristics that are suitable for the worker.

[1670] After the virtual personality is generated, the server installs the data into the industrial machine. The installed virtual personality then interacts with the worker through chat. During the chat, the worker is presented with multiple options, and feedback is generated based on the options. The feedback is then provided to the worker via their terminal.

[1671] This system incorporates an emotion engine that extracts emotional information from user input data. The server uses the emotion engine to analyze the user's emotional state and adjusts the virtual personality's responses and feedback based on the emotional information.

[1672] This process allows for real-time guidance and feedback that takes into account the worker's emotional state. User progress data is also recorded and used for the next session.

[1673] The hardware used includes industrial machines (e.g., KUKA, FANUC), tablets and smart glasses for workers, and software including Flask (a web application framework), generative AI models (e.g., GPT-3, BERT), and emotion recognition modules.

[1674] example

[1675] Consider a scenario where a worker is learning a new safety procedure. For example, the following conditions are entered:

[1676] Worker experience: Beginner

[1677] New work procedure: Machine maintenance

[1678] Language: Japanese

[1679] The server generates a virtual personality based on these conditions. The virtual personality is designed to provide simple and easy-to-understand instruction for beginners. During the dialogue, if the worker asks, "Which part of the machine should I check first?", the virtual personality generates an appropriate response and presents specific options. The prompt text in this case is as follows:

[1680] "Worker: What part of the machine should I check first?

[1681] Virtual personality: First, it is important to check the power source of the machine. Please choose the next step from the options below:

[1682] 1. Turn off the power to the power unit

[1683] 2. Check the oil in the power parts

[1684] 3. Check with other workers

[1685] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1686] Step 1:

[1687] The user inputs conditions such as the worker's years of experience, new work procedures, and language on the interface device. The terminal checks the format of this input data, and if there are no errors, it serializes it and sends it to the server. The input includes the worker's experience, procedures, and language, and the output generates condition data in JSON format.

[1688] Step 2:

[1689] The server receives the condition data. The server receives the condition data sent from the device, parses it, and extracts the condition information. The input is JSON-formatted condition data, and the parsed condition information is obtained as output.

[1690] Step 3:

[1691] The server generates a virtual personality. Based on the extracted condition information, the server sends a request to a generative AI model (e.g., GPT-3, BERT) to generate a virtual personality. The data of the generated virtual personality is stored on the server and linked to the user's session information. The condition information is input, and the data of the generated virtual personality is obtained as output.

[1692] Step 4:

[1693] The server installs the virtual personality information into the industrial machine. The server sends the generated virtual personality data to the industrial machine and installs it. The input is the virtual personality data, and the output is the virtual personality installed in the industrial machine.

[1694] Step 5:

[1695] A user starts a chat with a virtual persona. A worker starts a chat with a virtual persona using a tablet or smart glasses. The worker types a message in the chat window, and the device sends the message to the server. There is a user message as input, and a message sent to the server as output.

[1696] Step 6:

[1697] The server generates responses and options for the virtual personality. The server analyzes the received message and generates appropriate responses and options using a generative AI model. It also analyzes the user's emotional information using an emotion engine. The input is the user message, and the output is the virtual personality's responses, options, and emotional information.

[1698] Step 7:

[1699] The server sends responses, options, and feedback to the user. The server then sends the generated responses, options, and feedback based on the emotional information to the user's device. The inputs are the virtual personality's responses, options, and emotional information, and the output is the data sent to the device.

[1700] Step 8:

[1701] The user selects an option. They choose one from multiple options displayed in the chat window and send the selection information to the terminal. The option data is input, and the selection information is sent to the server as output.

[1702] Step 9:

[1703] The server generates the next response or feedback based on the selection. The server generates the next response or feedback based on the selection information and sends it to the device. The selection information is the input, and the next response or feedback is sent to the device as the output.

[1704] Step 10:

[1705] The emotion engine analyzes the user's input. The server uses the emotion engine to analyze the user's input data and extract emotional information. The user's input data is the input, and emotional information is obtained as the output.

[1706] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1707] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1708] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1709] [Fourth embodiment]

[1710] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1711] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1712] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1713] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1714] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1715] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1716] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1717] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1718] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1719] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1720] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1721] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1722] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1723] The present invention is a system that generates a virtual personality based on conditions input by a user and improves the user's communication skills through chat with that virtual personality.

[1724] System Overview

[1725] The system includes a means for a user to input conditions and generate a virtual personality based on the conditions, a means for chatting between the generated virtual personality and the user, a means for presenting multiple options during the chat, a means for generating feedback based on the user's selection, and a means for providing the feedback to the user. It also includes a means for recording the user's progress and utilizing the progress in the next session.

[1726] Explanation of program processing

[1727] 1. The user enters the conditions

[1728] User: Uses a device to enter the target conditions (age, gender, place of origin, company, position, personality, family structure, address, preferences, etc.).

[1729] Terminal: Checks the format of the entered data to make sure there are no errors. If there are no errors, the data is sent to the server.

[1730] 2. Send the data to the server

[1731] Terminal: Sends condition data serialized in JSON format to the server.

[1732] Server: Receives the request, parses the data, and obtains the user's criteria information.

[1733] 3. The server generates a virtual personality

[1734] Server: Based on the acquired condition information, a virtual personality is generated using a generative AI model (e.g., GPT-3, BERT). The generated virtual personality data is stored in storage and linked to the user's session information.

[1735] 4. Send virtual personality information to the user's device

[1736] Server: Sends the generated virtual personality data to the user's device as an HTTP response.

[1737] Terminal: Analyzes the virtual personality's profile information and displays it in the chat window.

[1738] 5. The user starts chatting with the virtual persona

[1739] User: Type a message in the chat window.

[1740] Terminal: Sends the entered message to the server.

[1741] Server: Analyzes the message and invokes the AI ​​model for the virtual personality to generate an appropriate response.

[1742] 6. The virtual personality responds and offers options

[1743] Server: Uses AI models to generate responses for the virtual personality, and also generates multiple options to present to the user.

[1744] Server: Sends the generated responses and options to the user's terminal.

[1745] Terminal: Displays the virtual personality's responses and options in the chat window.

[1746] 7. The user selects an option

[1747] User: Choose one of the options presented.

[1748] Terminal: Sends the selected option information to the server.

[1749] Server: Generates the next response or feedback based on the option information.

[1750] 8. Providing Feedback

[1751] Server: Generates feedback on the choices and provides suggestions and improvements to the user.

[1752] Server: Sends the generated feedback to the user's device.

[1753] Terminal: Show feedback in the chat window.

[1754] 9. Record and track user growth

[1755] Server: Records chat session data and stores it in a database for future sessions. This data is used to support the continued growth of our users.

[1756] Specific examples

[1757] Step 1: User enters criteria

[1758] User: Enter the target criteria as "male in his 20s, university student, tennis lover, living in Tokyo."

[1759] Terminal: Checks the format of the condition data, and if there are no errors, sends it to the server.

[1760] Step 2: Send the data to the server

[1761] Terminal: Converts the condition data into JSON format and sends it to the server.

[1762] Server: Receives data and extracts condition information.

[1763] Step 3: The server generates a virtual personality

[1764] Server: Generates a virtual personality using a generative AI model based on condition information.

[1765] Server: Stores the data of the generated virtual personality in storage and links it to session information.

[1766] Step 4: Send virtual personality information to the user's device

[1767] Server: Sends virtual personality data to the user's device.

[1768] Terminal: Display the virtual personality's profile in the chat window.

[1769] Step 5: The user starts chatting with the virtual persona

[1770] User: Type the message "Hi! You said you like tennis. How often do you play?" into the chat window.

[1771] Terminal: Sends a message to the server.

[1772] Server: Parses the message and invokes the AI ​​model to generate a response.

[1773] Step 6: The virtual persona responds and offers options

[1774] Server: The virtual personality responds with "Hello! I play two or three times a week. Who's Tanaka?" and generates multiple options.

[1775] Server: Sends the response and options to the user's terminal.

[1776] Terminal: Show responses and options in the chat window.

[1777] Step 7: User selects a choice

[1778] User: Select "Talk about another hobby" from the options.

[1779] Terminal: Sends the selection information to the server.

[1780] Server: Generates the next response or feedback.

[1781] Step 8: Provide feedback

[1782] Server: Generate feedback "When talking about different hobbies, it's important to explore common interests."

[1783] Server: Sends feedback to the user's device.

[1784] Terminal: Display feedback in the chat window.

[1785] Step 9: Record and track user growth

[1786] Server: Records chat session data and stores it in a database for future sessions.

[1787] In this way, users can comfortably practice their communication skills in a virtual environment and gain confidence in real-life face-to-face conversations.

[1788] The processing flow will be explained below.

[1789] Step 1:

[1790] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) into the condition input form on the device.

[1791] Step 2:

[1792] Terminal: Check the format of the entered data to ensure that the data format is correct and that required fields are not missing.

[1793] Step 3:

[1794] Terminal: Condition data that passes the format check is serialized into JSON format and sent to the server as an HTTP POST request.

[1795] Step 4:

[1796] Server: Analyzes the received request and extracts the condition data sent.

[1797] Step 5:

[1798] Server: Using the condition data, it sends a request to a generative AI model (e.g., GPT-3 or BERT) to generate a virtual personality.

[1799] Step 6:

[1800] Server: Stores the generated virtual personality data (such as name, age, personality, hobbies, etc.) in storage and links it to the user's session information.

[1801] Step 7:

[1802] Server: Sends the virtual personality data to the user's terminal as an HTTP response.

[1803] Step 8:

[1804] Terminal: Analyzes the received virtual personality data and displays profile information in the chat window.

[1805] Step 9:

[1806] User: Type a message in the chat window.

[1807] Step 10:

[1808] Terminal: Sends the entered message to the server.

[1809] Step 11:

[1810] Server: Analyzes the received message and generates an appropriate reply by calling an AI model for the virtual personality to respond.

[1811] Step 12:

[1812] Server: Generates multiple options to present to the user along with the responses of the generated virtual personality.

[1813] Step 13:

[1814] Server: Sends the virtual personality's responses and options to the user's terminal.

[1815] Step 14:

[1816] Terminal: Displays the virtual personality's replies and options in the chat window.

[1817] Step 15:

[1818] User: Select one of the options presented and send the selection to the terminal.

[1819] Step 16:

[1820] Terminal: Sends the selection information to the server.

[1821] Step 17:

[1822] Server: Generates the next response or feedback based on the selected information.

[1823] Step 18:

[1824] Server: Sends the generated feedback and the next response to the user's device.

[1825] Step 19:

[1826] Terminal: Show feedback and next response in the chat window.

[1827] Step 20:

[1828] Server: Records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions.

[1829] Example 1

[1830] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1831] Previous systems for improving communication skills had the problem of making it difficult for users to receive appropriate support. Specifically, it was difficult to generate a virtual personality based on the user's individual conditions, and there was a lack of means to continuously support the user's growth through dialogue with that virtual personality. Furthermore, there was a lack of technology for format checking of user input data and generating appropriate responses in real time, making it difficult to improve the user experience.

[1832] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1833] In this invention, the server includes: means for a user to input conditions; means for utilizing a generative AI model to generate a virtual personality based on the conditions; means for chatting between the virtual personality and the user; means for presenting multiple options during the chat and generating feedback based on the user's selection; means for providing the feedback to the user; means for displaying profile information of the virtual personality on a chat screen; and means for calling a generative AI model to analyze messages input by the user and generate appropriate responses. This allows users to continuously improve their communication skills through real-time interactions with virtual personalities based on their conditions.

[1834] "User" refers to a person who operates the system, inputs conditions, and interacts with a virtual personality.

[1835] "Conditions" refer to individual information such as age, sex, occupation, hobbies, and place of residence entered by the user.

[1836] A "virtual personality" refers to a virtual entity that can be used as a conversation partner and is generated based on the user's conditions using a generative AI model.

[1837] A "generative AI model" refers to an artificial intelligence model that generates a virtual personality using natural language processing based on conditions entered by the user.

[1838] "Chat" refers to a text-based interaction between a user and a virtual persona.

[1839] "Choices" refers to multiple response or reaction options that a user is presented with by a virtual persona during a chat.

[1840] "Feedback" refers to information such as suggestions and improvements provided based on the user's selections.

[1841] A "prompt" refers to an input instruction that causes a generative AI model to generate a specific response.

[1842] "Profile information" refers to information that a user can know, such as the virtual personality's personality, hobbies, and background.

[1843] "Session information" refers to the interaction history and associated data between a user and a virtual persona.

[1844] "Format check" refers to the process of checking whether the data entered by the user conforms to the specified format.

[1845] "HTTP response" refers to the data format returned from the server to the client (terminal).

[1846] This invention is a system that generates a virtual personality based on conditions input by a user and improves the user's communication skills through chatting with that virtual personality.

[1847] Hardware and Software Configuration

[1848] This system mainly consists of the following components:

[1849] 1. User's device: a device such as a personal computer, smartphone, or tablet.

[1850] 2. Server: Cloud server or local server.

[1851] 3. Generative AI models: Use natural language generation models such as GPT-3 and BERT.

[1852] Data processing and calculation

[1853] The system performs the following steps:

[1854] 1. The user enters the conditions

[1855] The user enters information such as age, gender, occupation, hobbies, and place of residence through the device interface. The device then performs a format check on the entered data to ensure there are no errors. If there are no errors, the data is serialized in JSON format and sent to the server.

[1856] 2. Acquire and analyze data

[1857] The server receives the condition data sent in JSON format, parses it, and extracts the user's condition information, which is then stored in temporary storage.

[1858] 3. Generate a virtual personality

[1859] The server sends a prompt to the generative AI model (e.g., GPT-3, BERT) based on the acquired condition information. The prompt includes the condition information entered by the user. For example, "Please generate a virtual personality based on the following conditions: age in his 20s, gender male, university student, tennis lover, residing in the Tokyo metropolitan area." The profile information of the generated virtual personality is saved in storage and linked to the user's session information.

[1860] 4. Send virtual personality information to the user device

[1861] The server sends the generated virtual personality data to the user's device as an HTTP response. The device analyzes this data and displays the virtual personality's profile (name, characteristics, hobbies, etc.) in a chat window.

[1862] 5. The user starts chatting with the virtual persona

[1863] The user types a message into the chat window, for example, "Hello! You said you like tennis. How often do you play?" The device then sends the typed message to the server.

[1864] 6. The virtual personality responds and offers options

[1865] The server analyzes the received message and calls the generative AI model again to generate a response from the virtual personality. For example, it generates a response like, "Hello! I play two or three times a week. How about you?" The generated response and multiple options (for example, "I play once a week," "Talk about another hobby," etc.) are sent to the user's device. The device displays the virtual personality's response and options in a chat window.

[1866] 7. The user selects an option

[1867] The user selects one of the options presented and confirms the choice by clicking or other operation. The device then sends information about the selected option to the server.

[1868] 8. Providing Feedback

[1869] The server generates feedback based on the user's choices, such as advice like, "When talking about different hobbies, it's important to find common interests." The generated feedback is sent to the user's device and displayed in the chat window.

[1870] 9. Record and track user growth

[1871] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for reference during the next session, which is used to support the user's continuous growth.

[1872] In this way, this system allows users to learn practical communication skills in a safe virtual environment and improve their ability to interact in the real world.

[1873] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1874] Step 1:

[1875] The user enters conditions (age, gender, occupation, hobbies, place of residence, etc.) using a device. The entered data is format-checked on the device to ensure there are no errors. If there are no errors, the data is serialized in JSON format.

[1876] Input: The user enters the conditions into the input form on the device.

[1877] Output: Serialized JSON formatted condition data

[1878] Step 2:

[1879] The device sends the condition data that has passed the format check to the server via an HTTP POST request. The server receives this request, parses the data, and extracts the user's condition information. The extracted data is saved in temporary storage.

[1880] Input: Condition data in serialized JSON format

[1881] Output: Parsed condition information data

[1882] Step 3:

[1883] The server then sends prompts to the generative AI model based on the extracted condition information. The prompts contain the user's condition information and prompt the model to generate speech content and a profile.

[1884] Input: Parsed condition information

[1885] Output: The prompt sent to the generative AI model

[1886] Step 4:

[1887] A generative AI model (e.g., GPT-3, BERT) generates a virtual personality profile based on the prompt text. This profile information is received on the server side, linked to the user's session information, and stored in storage.

[1888] Input: prompt statement

[1889] Output: Profile information of the generated virtual personality

[1890] Step 5:

[1891] The server sends the profile information of the virtual personality to the user's device as an HTTP response. The device receives this data, analyzes it, and displays the profile information (name, characteristics, hobbies, etc.) in the chat window.

[1892] Input: Profile information of the generated virtual personality

[1893] Output: Profile information displayed in the chat window

[1894] Step 6:

[1895] The user enters a message in the chat window. The device sends the message to the server, which receives the message and calls the generative AI model again to generate a response from the virtual personality.

[1896] Input: The message entered by the user

[1897] Output: A response message from the generative AI model

[1898] Step 7:

[1899] The server sends the generated response message and multiple options to the user's terminal, which receives it and displays the response message and options in a chat window.

[1900] Input: Response message from generative AI model, multiple choice

[1901] Output: Response message and options displayed in the chat window

[1902] Step 8:

[1903] The user selects one of the options presented and confirms the choice by clicking or other operation. The device then sends information about the selected option to the server.

[1904] Input: The choice selected by the user

[1905] Output: Choice information sent to the server

[1906] Step 9:

[1907] The server receives and analyzes the selection information and generates feedback, which is then sent back to the user's terminal and displayed in the terminal's chat window.

[1908] Input: Information about the option selected by the user

[1909] Output: Generated feedback

[1910] Step 10:

[1911] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for reference during the next session, to support the user's continuous growth.

[1912] Input: Chat session data

[1913] Output: Session information stored in the database

[1914] (Application example 1)

[1915] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1916] Security operators and guards are required to have advanced communication skills in order to respond quickly and appropriately to emergencies and daily security activities, but there is a lack of appropriate training methods and tools to efficiently improve these skills.

[1917] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1918] In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for generating a virtual personality based on a situation related to security activities, and means for communicating between the virtual personality and the user regarding an emergency. This enables security operators and security guards to efficiently improve the communication skills required for their work through virtual conversations based on actual situations.

[1919] The "means for user input" is a digital input interface that allows a user to provide specific requirements and situational information to the system.

[1920] The "means for generating a virtual personality based on the above conditions" is a technology that uses artificial intelligence to create a virtual conversation partner based on input condition information.

[1921] The "means for chatting between the virtual personality and the user" is a communication function that allows the generated virtual personality and the user to exchange text messages in real time.

[1922] "Means for presenting multiple options during chat and generating feedback based on the user's selection" is a mechanism that provides the user with several options during a conversation and generates an appropriate response or evaluation based on the selection.

[1923] The "means for providing the feedback to the user" is an output function for displaying or notifying the generated feedback or evaluation to the user.

[1924] The "means for generating a virtual personality based on a situation related to security activities" is a technology for generating a virtual personality based on a specific scenario related to security activities or emergency situations.

[1925] "Means for communication between the virtual personality and the user regarding an emergency" refers to a communication function for giving instructions and sharing information in an emergency through virtual dialogue.

[1926] This invention utilizes a virtual dialogue system to improve the effective communication skills of security operators and guards in emergency situations. The system is composed of a user, a terminal, and a server.

[1927] System Overview

[1928] The user uses a device to input specific conditions (for example, "Occupation: Security Operator," "Emergency: Fire," "Location: Office Building," "Instructions: Evacuation Instructions," etc.). The device checks the format of this condition data and then sends it to the server in JSON format. The server generates a virtual personality using a generative AI model (for example, GPT-2) based on the received condition information. The server then saves the virtual personality data and session information in storage and provides it to the user.

[1929] The user starts chatting with the virtual persona on the device. The message entered by the user is sent to the server, which uses an AI model to generate an appropriate response. The generated response and multiple options presented to the user are sent to the device and displayed in a chat window. The user selects one of the options, and feedback is generated and provided to the user based on that selection.

[1930] The system also includes a means to record the user's progress and use it for future sessions. This data supports the user's continuous improvement.

[1931] Hardware and software used

[1932] Hardware: Smartphone

[1933] Software: Python, generative AI model (GPT-2)

[1934] Explanation of data processing or data calculation

[1935] 1. Enter conditions and submit

[1936] The user inputs conditions such as "occupation: security operator," "emergency situation: fire," "location: office building," and "instructions: evacuation instructions."

[1937] The terminal checks the format of the condition data, and if there are no errors, sends it to the server in JSON format.

[1938] 2. Creation and provision of virtual personalities

[1939] The server generates a virtual personality using a generative AI model (GPT-2) based on the received condition information.

[1940] Data on the generated virtual personality is sent back to the user's terminal.

[1941] 3. Interacting with Virtual Personalities

[1942] The user inputs a message such as "Hello, there is an emergency. There is a fire. Please give us your instructions."

[1943] The server analyzes the received message using an AI model and generates an appropriate response.

[1944] The generated responses and options are displayed on the user's terminal.

[1945] 4. Providing Feedback

[1946] The server generates and provides appropriate feedback to the user based on the user's selection.

[1947] Provide specific advice and improvements to help users improve their skills.

[1948] Examples of concrete examples and prompts

[1949] Examples:

[1950] Enter the conditions: "Occupation: Security Operator", "Emergency: Fire", "Location: Office Building", "Instructions: Evacuation Instructions"

[1951] Initial message: "Hello, emergency. Fire. Your instructions needed."

[1952] Example prompt sentence:

[1953] Conditions: "Occupation: Security Operator", "Emergency: Fire", "Location: Office Building", "Instructions: Evacuation"

[1954] Dialogue start message: "Hello, emergency. Fire. Your instructions needed."

[1955] In this way, users can safely practice their communication skills in a scenario that closely resembles real-world work.

[1956] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1957] Step 1:

[1958] The user inputs conditions. Using their smartphone, the user inputs conditions such as "Occupation: Security Operator," "Emergency: Fire," "Location: Office Building," and "Instructions: Evacuation Instructions." The device checks the format of the input data to ensure there are no errors. After checking, the device serializes the condition data into JSON format and sends it to the server.

[1959] Step 2:

[1960] The server receives the data. The server receives the condition data in JSON format sent from the device. The server parses the data, extracts the condition information, and prepares it as input data for use in the next processing step.

[1961] Step 3:

[1962] The server generates a virtual personality. Based on the extracted condition information, the server invokes a generative AI model (e.g., GPT-2) to generate the virtual personality. This generative model generates appropriate personality attributes based on the condition information and constructs data for the virtual personality. The data for the generated virtual personality is stored in the server's storage and linked to the user's session information.

[1963] Step 4:

[1964] The server sends information about the virtual personality to the user's device. The server then sends the generated virtual personality data to the user's device as an HTTP response. The device receives this data, analyzes it, and displays the virtual personality's profile information in the chat window.

[1965] Step 5:

[1966] The user starts chatting with the virtual persona. The user types a message into the chat window on the device: "Hello, there's an emergency. There's a fire. Please give us your instructions." The device sends the typed message to the server. The server receives the message, analyzes it, and calls the AI ​​model to generate an appropriate response from the virtual persona.

[1967] Step 6:

[1968] The server generates a response and presents the user with multiple options. The server uses an AI model to generate a response for the virtual personality, and then generates multiple options to present to the user. For example, a response such as "Please remain calm and head for the emergency exit" is generated. The generated response and options are sent to the user's device as an HTTP response. The device receives this and displays it in a chat window.

[1969] Step 7:

[1970] The user selects an option. The user selects one of the options presented and enters the selection information. The selection information is sent to the server via the device. The server again uses the AI ​​model to generate the next response or feedback based on the selected option information.

[1971] Step 8:

[1972] The server generates feedback and provides it to the user. The server generates feedback for the selected option and provides the user with suggestions and improvements. For example, the server might generate feedback such as "It is important to communicate instructions specifically and concisely." The generated feedback is sent to the user's device as an HTTP response and displayed in the chat window.

[1973] Step 9:

[1974] The server records the user's progress and uses it for the next session. The server records the chat session data and saves it in a database to use for the next session. This recorded data is used by the user to continuously improve their communication skills.

[1975] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1976] This system generates a virtual personality based on user input and improves the user's communication skills through chat with the virtual personality. It also incorporates an emotion engine that recognizes the user's emotions and provides feedback to improve the effectiveness of the chat.

[1977] System Overview

[1978] The system includes the following main means:

[1979] 1. A way for users to enter conditions

[1980] 2. Means for generating a virtual personality based on the above conditions

[1981] 3. Means for chatting between the virtual personality and the user

[1982] 4. A way to present multiple options during a chat and generate feedback based on the user's selection

[1983] 5. Means for providing said feedback to the user

[1984] 6. Means of recognizing user emotions using an emotion engine

[1985] 7. A way to track user progress and use it in the next session

[1986] 8. A means of formatting user-entered condition data

[1987] Explanation of program processing

[1988] 1. The user enters the conditions

[1989] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) on the device.

[1990] Terminal: Checks the format of the entered data and sends it to the server if there are no errors.

[1991] 2. Send the data to the server

[1992] Terminal: Serialize the condition data in JSON format and send it to the server as an HTTP POST request.

[1993] Server: Receives the request and parses the data to extract condition information.

[1994] 3. The server generates a virtual personality

[1995] Server: Sends a request to a generative AI model (e.g., GPT-3 or BERT) based on condition information to generate a virtual personality.

[1996] Server: Stores the virtual personality data (name, age, personality, hobbies, etc.) in storage and links it to the user's session information.

[1997] 4. Send virtual personality information to the user's device

[1998] Server: Sends the generated virtual personality data to the user's device as an HTTP response.

[1999] Terminal: Analyzes the virtual personality's profile information and displays it in the chat window.

[2000] 5. The user starts chatting with the virtual persona

[2001] User: Type a message in the chat window.

[2002] Terminal: Sends the entered message to the server.

[2003] Server: Analyzes the message and invokes the AI ​​model for the virtual personality to generate an appropriate response.

[2004] 6. The virtual personality responds and offers options

[2005] Server: Uses AI models to generate responses for the virtual personality, and also generates multiple options to present to the user.

[2006] Server: Sends the generated responses and options to the user's terminal.

[2007] Terminal: Displays the virtual personality's responses and options in the chat window.

[2008] 7. The user selects an option

[2009] User: Select one of the options presented and send the selection to the terminal.

[2010] Terminal: Sends the selection information to the server.

[2011] Server: Generates the next response or feedback based on the selected information.

[2012] 8. Providing Feedback

[2013] Server: Generates feedback on the choices and provides suggestions and improvements to the user.

[2014] Server: Sends the generated feedback to the user's device.

[2015] Terminal: Show feedback in the chat window.

[2016] 9. The Emotional Engine

[2017] Server: Analyzes the text entered by the user using the emotion engine and extracts emotion information.

[2018] Server: Adjusts the virtual personality's responses based on emotional information and generates dialogue that takes the user's emotional state into account.

[2019] 10. Record and track user growth

[2020] Server: Records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions. This data is used to support the user's ongoing growth.

[2021] Specific examples

[2022] Step 1: User enters criteria

[2023] User: Enters the following criteria into the input form on the device: "Male in his 20s, university student, tennis lover, living in Tokyo."

[2024] Terminal: Checks the format of the condition data, and if there are no errors, sends it to the server.

[2025] Step 2: Send the data to the server

[2026] Terminal: Serialize the condition data into JSON format and send it to the server as an HTTP POST request.

[2027] Server: Parse the data and extract the condition information.

[2028] Step 3: The server generates a virtual personality

[2029] Server: Based on the condition information, a virtual personality "Yamada-kun" is generated using a generative AI model.

[2030] Server: Stores the virtual personality data in storage and links it to session information.

[2031] Step 4: Send virtual personality information to the user's device

[2032] Server: Sends the generated virtual personality data to the user's device.

[2033] Terminal: Display the virtual personality's profile information in the chat window.

[2034] Step 5: The user starts chatting with the virtual persona

[2035] User: Type in the chat window, "Hi! You said you like tennis. How often do you play?"

[2036] Terminal: Sends the entered message to the server.

[2037] Server: Parses the message and invokes the AI ​​model to generate a response.

[2038] Step 6: The virtual persona responds and offers options

[2039] Server: Using an AI model, it generates the virtual personality's response, "Hello! I play two or three times a week. What about you?", while also generating options such as "Continue talking about tennis," "Go back to my introduction," or "Talk about another hobby."

[2040] Server: Sends the response and options to the user's terminal.

[2041] Terminal: Show responses and options in the chat window.

[2042] Step 7: User selects a choice

[2043] User: Select "Talk about another hobby" from the options.

[2044] Terminal: Sends the selection information to the server.

[2045] Server: Generates the next response or feedback.

[2046] Step 8: Provide feedback

[2047] Server: Generate feedback for the choice: "Talking about other hobbies is a good opportunity to learn new things about each other."

[2048] Server: Sends feedback to the user's device.

[2049] Terminal: Show feedback in the chat window.

[2050] Step 9: Emotional Engine in Action

[2051] Server: Using an emotion engine, the text entered by the user is analyzed and emotional information such as "Looks fun" or "Very interested" is extracted.

[2052] Server: Based on the emotional information, the virtual personality's responses are adjusted to generate more friendly dialogue.

[2053] Step 10: Record and track user growth

[2054] Server: Records chat session data and stores it in a database for future sessions.

[2055] In this way, users can safely hone their communication skills in a virtual environment, understand their own emotional state through the emotion engine, and gain confidence in real-life face-to-face conversations.

[2056] The processing flow will be explained below.

[2057] Step 1:

[2058] User: Enter the target conditions (age, gender, place of origin, job title, personality, family structure, address, preferences, etc.) into the condition input form on the device.

[2059] Step 2:

[2060] Terminal: Checks the format of the entered data to ensure that the format and required fields are correct. If there are any errors, an error message is displayed prompting the user to make corrections.

[2061] Step 3:

[2062] Terminal: Condition data that passes the format check is serialized into JSON format and sent to the server as an HTTP POST request.

[2063] Step 4:

[2064] Server: Receives the HTTP POST request and parses the data to extract user criteria information.

[2065] Step 5:

[2066] Server: Based on the extracted condition information, a request is sent to a generative AI model (e.g., GPT-3 or BERT) to generate a virtual personality.

[2067] Step 6:

[2068] Server: The generated virtual personality data (name, age, personality, hobbies, etc.) is stored in a storage and linked to the user's session information.

[2069] Step 7:

[2070] Server: Converts the virtual personality data into JSON format and sends it to the user's device as an HTTP response.

[2071] Step 8:

[2072] Terminal: Analyzes the received virtual personality data and displays profile information in the chat window.

[2073] Step 9:

[2074] User: Type a message in the chat window.

[2075] Step 10:

[2076] Terminal: Sends the entered message to the server.

[2077] Step 11:

[2078] Server: Analyzes the received message and calls the AI ​​model for the virtual personality to generate an appropriate response.

[2079] Step 12:

[2080] Server: Generates multiple options along with the responses of the generated virtual personality.

[2081] Step 13:

[2082] Server: Sends the generated responses and options to the user's terminal.

[2083] Step 14:

[2084] Terminal: A chat window displays the virtual personality's responses and the options presented.

[2085] Step 15:

[2086] User: Select one of the displayed options and send the selected information to the terminal.

[2087] Step 16:

[2088] Terminal: Sends the selection information to the server.

[2089] Step 17:

[2090] Server: Analyzes the selection information and generates the next response or feedback based on it.

[2091] Step 18:

[2092] Server: Sends the generated feedback and the next response to the user's terminal.

[2093] Step 19:

[2094] Terminal: Show feedback and next response in the chat window.

[2095] Step 20:

[2096] Server: Using the emotion engine, analyzes the message entered by the user and extracts emotional information.

[2097] Step 21:

[2098] Server: Based on the emotional information, the server appropriately adjusts the responses of the virtual personality and generates a dialogue that takes into account the user's emotional state.

[2099] Step 22:

[2100] Server: Sends the adjusted response to the user's device, and displays emotional information as feedback.

[2101] Step 23:

[2102] Terminal: Responses containing emotional information are displayed in the chat window, allowing users to check their own emotional state.

[2103] Step 24:

[2104] Server: Records all data from the chat session (inputs, responses, options, feedback, emotional information, etc.) and stores it in a database for future sessions.

[2105] Example 2

[2106] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2107] In modern society, providing a training environment for users to improve their communication skills is an important issue. In particular, there is a need for a system that provides real-time feedback while taking into account the user's emotional state. There is also a need for a means to record the user's progress over the long term and provide dialogue based on that record.

[2108] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2109] In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for presenting multiple options during the chat and generating feedback based on the user's selection, means for providing the feedback to the user, means for analyzing the user's emotions, means for adjusting the virtual personality's responses based on the emotion information, means for generating appropriate responses and suggestions for the user's selections, means for recording chat session data and using it in the next session, and means for formatting the condition data entered by the user. This allows users to train their communication skills in a virtual environment with peace of mind, understand their own emotional state through the emotion engine, and gain confidence in actual conversations.

[2110] A "user" is an individual who utilizes the system to chat with a virtual personality and improve their communication skills.

[2111] "Conditions" refers to information such as age, gender, occupation, and hobbies entered by the user.

[2112] A "virtual personality" is a character that can be used as a conversation partner, generated by a generative AI model based on conditions entered by the user.

[2113] "Chat" refers to the exchange of messages between a user and a virtual persona.

[2114] "Choices" refers to multiple options presented to a user during a chat.

[2115] "Feedback" refers to suggestions and improvements generated based on user selections.

[2116] "Emotion" refers to the psychological state that the emotion engine analyzes and extracts from the message entered by the user.

[2117] An "emotion engine" is software or algorithms that analyze user-entered text and extract emotional information.

[2118] A "response" refers to a message that a virtual personality returns in response to a user's message.

[2119] A "session" refers to a series of chat exchanges between a user and a virtual persona.

[2120] "Format check" refers to the process of checking whether the condition data entered by the user is in the correct format.

[2121] A "generative AI model" refers to an artificial intelligence model that generates a virtual personality based on given conditions.

[2122] A "prompt sentence" refers to an input sentence that requests the generative AI model to generate a virtual personality based on certain conditions.

[2123] This system generates a virtual personality based on user input and improves the user's communication skills through chat with the virtual personality. It also incorporates an emotion engine that recognizes the user's emotions and provides feedback to improve the effectiveness of the chat.

[2124] The system includes the following main means:

[2125] 1. A way for users to enter conditions

[2126] 2. Means for generating a virtual personality based on the above conditions

[2127] 3. Means for chatting between the virtual personality and the user

[2128] 4. A way to present multiple options during a chat and generate feedback based on the user's selection

[2129] 5. Means for providing said feedback to the user

[2130] 6. Means of recognizing user emotions using an emotion engine

[2131] 7. A way to track user progress and use it in the next session

[2132] 8. A method for format checking user-entered condition data

[2133] The program for this system uses the following hardware and software:

[2134] Terminal: A device that accepts user input and sends data to a server. Examples include PCs, smartphones, and tablets.

[2135] Server: A central computer system that processes data, generates virtual personalities, manages chats, analyzes emotions, and provides feedback. Examples include virtual machines on a cloud service or dedicated servers.

[2136] Generative AI model: An artificial intelligence model that generates virtual personalities based on conditions. Examples include OpenAI's GPT-3 and BERT.

[2137] Emotion engine: A software component that analyzes text entered by a user and extracts emotional information. It is implemented using a natural language processing library or an AI engine.

[2138] The following example is a concrete example of the flow of operations.

[2139] Example 1: User enters criteria

[2140] The user enters the following criteria into the input form on the terminal: "Male in his 20s, university student, tennis lover, living in Tokyo."

[2141] The terminal checks the format of the input condition data, and if there are no errors, sends it to the server.

[2142] Example prompt sentence:

[2143] "Generate a hobby profile for a male in his 20s, a university student, who likes tennis and lives in Tokyo."

[2144] Example 2: Server generates virtual personality

[2145] The server generates a virtual personality "Yamada-kun" using a generative AI model based on the condition information and links it to the session information.

[2146] Example 3: Chatting with a virtual personality

[2147] The user types into the chat window, "Hi! You said you like tennis. How often do you play?"

[2148] The server uses an emotion engine to analyze the text entered by the user and extract emotional information such as "it looks fun" or "it looks very interesting."

[2149] The server uses an AI model to generate a response and then presents the user with multiple options.

[2150] In this way, this system, which operates in cooperation between users, devices, and servers, provides an environment in which users can improve their communication skills with peace of mind, and by providing appropriate feedback based on emotion analysis, users can proceed with conversations with confidence in real-life situations.

[2151] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2152] Step 1:

[2153] The user enters the conditions

[2154] The user inputs the target conditions (e.g., age, gender, occupation, hobbies, etc.) into the input form provided on the terminal. For example, the user inputs "male in his 20s, university student, tennis lover, living in Tokyo."

[2155] The terminal receives the input data (age, gender, occupation, hobbies, etc.) and performs a format check. If any errors are found, the terminal notifies the user of the information.

[2156] Input: Condition data entered by the user in the input form

[2157] Output: Correctly formatted condition data (after format check)

[2158] Step 2:

[2159] Sending data to the server

[2160] The terminal serializes the condition data that has passed the format check into JSON format.

[2161] The device sends the serialized JSON data to the server as an HTTP POST request using the endpoint " / create_personality".

[2162] The server receives the request and parses the data to extract the necessary condition information.

[2163] Input: JSON format of condition data sent from the terminal

[2164] Output: Server-side parsed condition data

[2165] Step 3:

[2166] The server generates a virtual personality

[2167] The server sends a request to a generative AI model (e.g., GPT-3 or BERT) based on the parsed condition information. Specifically, the prompt statement "Generate a hobby profile for a male in his 20s, a university student, who likes tennis and lives in Tokyo" is passed to the generative AI model.

[2168] The server retrieves the virtual personality data (such as name, age, personality, hobbies, etc.) returned by the generative AI model and stores it in internal storage (e.g., a NoSQL database). This data is associated with the user's session information.

[2169] Input: Parsed condition data

[2170] Output: Virtual personality data returned from the generative AI model

[2171] Step 4:

[2172] Send virtual personality information to the user's device

[2173] The server sends the generated virtual personality data to the user's device as an HTTP response. Specifically, the response is composed of the user's session ID and the data.

[2174] The device analyzes the received virtual personality data and displays it in the chat window. For example, it might display information like "Yamada, male in his 20s, university student, tennis lover, living in Tokyo."

[2175] Input: Virtual personality data from the server

[2176] Output: Virtual personality profile information displayed in the chat window

[2177] Step 5:

[2178] The user initiates a chat with a virtual persona

[2179] The user types a message into the chat window, for example, "Hi! I saw you said you like tennis. How often do you play?"

[2180] The terminal sends the entered message to the server by sending a POST request to the endpoint " / chat".

[2181] The server analyzes the received message and processes the text content using its internal natural language processing engine.

[2182] Input: A chat message from the user

[2183] Output: Analysis results from the natural language processing engine

[2184] Step 6:

[2185] The virtual personality responds and offers options

[2186] The server uses a natural language processing engine to analyze the user's message and generate a response from the virtual personality, such as "Hello! I play two to three times a week."

[2187] The server generates multiple options for the user (e.g., "Continue talking about tennis," "Return to self-introduction," "Talk about another hobby").

[2188] The server sends the generated responses and options to the user's terminal.

[2189] The terminal displays the virtual personality's responses and options in a chat window.

[2190] Input: User's chat message and choices

[2191] Output: Response message and options from the virtual personality

[2192] Step 7:

[2193] The user selects an option

[2194] The user selects one of the presented options and transmits the selected information to the terminal.

[2195] The device sends the selection information to the server. Specifically, it sends a request including the option ID to the endpoint " / select_option".

[2196] The server analyzes the received selection information and generates the following response or feedback:

[2197] Input: User selection information

[2198] Output: The next response or feedback

[2199] Step 8:

[2200] Providing feedback

[2201] The server generates feedback based on the user's selection, for example, "Talking about other hobbies is a good opportunity to discover new aspects of each other."

[2202] The server transmits the generated feedback to the user's terminal.

[2203] The device displays the feedback in a chat window.

[2204] Input: User selection information

[2205] Output: Feedback message

[2206] Step 9:

[2207] The Emotional Engine in Action

[2208] The server uses an emotion engine to analyze the text entered by the user, extracting emotion information such as "fun" or "interesting."

[2209] The server adjusts the responses of the virtual personality based on the emotional information and generates a dialogue that takes into account the user's emotional state.

[2210] Input: User's chat message

[2211] Output: Emotional information and tailored response message

[2212] Step 10:

[2213] Record and track user growth

[2214] The server records chat session data (user input, choices, feedback, etc.) and stores it in a database for future sessions. This data is used to support the user's ongoing growth.

[2215] Input: Chat session data

[2216] Output: Session data stored in the database

[2217] As described above, users, devices, and servers work together to operate the system efficiently, improving users' communication skills and providing appropriate feedback based on emotion analysis.

[2218] (Application example 2)

[2219] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2220] Efficient work instruction and worker safety training are important issues in modern manufacturing sites. In particular, quickly and reliably learning new procedures and technologies is essential for improving productivity. However, traditional methods require face-to-face instruction, and the quality of instruction depends on the skill and experience of the instructor, making it difficult to provide consistent instruction. It is also difficult to provide feedback that reflects the emotions and level of understanding of workers.

[2221] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input conditions, means for generating a virtual personality based on the conditions, means for chatting between the virtual personality and the user, means for presenting multiple options during the chat and generating feedback based on the user's selection, means for providing the feedback to the user, means for extracting emotional information from data input by the user using an emotion engine, means for adjusting responses and feedback based on the user's emotional information, and means installed in industrial machinery for generating a training virtual personality and providing guidance to workers. This enables real-time guidance and feedback according to the emotional state of the worker.

[2222] The "means for the user to input conditions" is an interface device that allows the user to input conditions for generating a virtual personality.

[2223] The "means for generating a virtual personality" is a system that generates a virtual personality with specific attributes and characteristics based on conditions entered by the user.

[2224] The "means for chatting" is a communication interface for sending and receiving messages between the user and the virtual personality.

[2225] The "means for presenting options and generating feedback" is a function that presents multiple options to the user during a chat and provides feedback based on the selection.

[2226] A "means for providing feedback to a user" is an interface device for presenting the generated feedback to a user.

[2227] The "means for extracting emotional information from user input data using an emotion engine" is a function for analyzing data input by a user and identifying an emotional state from that data.

[2228] The "means for adjusting responses and feedback based on the user's emotional information" is a function that appropriately adjusts the responses and feedback of the virtual personality based on the emotional information extracted by the emotion engine.

[2229] "Means to be installed in industrial machines, generate educational virtual personalities, and provide guidance to workers" refers to a system that is installed in industrial machines and provides education and guidance to workers through virtual personalities.

[2230] The following describes an embodiment of the present invention. This system generates a virtual personality based on conditions entered by a user, and provides training to workers through the virtual personality installed on industrial machines. This will be described in detail below.

[2231] First, the user inputs conditions using an interface device, such as the worker's years of experience, new work procedures, language, etc. The input conditions are serialized by the terminal and sent to the server as an HTTP POST request.

[2232] Next, based on the condition data received by the server, a generative AI model is used to generate a virtual personality. This generative AI model uses natural language processing models such as GPT-3 and BERT. The generated virtual personality has attributes and characteristics that are suitable for the worker.

[2233] After the virtual personality is generated, the server installs the data into the industrial machine. The installed virtual personality then interacts with the worker through chat. During the chat, the worker is presented with multiple options, and feedback is generated based on the options. The feedback is then provided to the worker via their terminal.

[2234] This system incorporates an emotion engine that extracts emotional information from user input data. The server uses the emotion engine to analyze the user's emotional state and adjusts the virtual personality's responses and feedback based on the emotional information.

[2235] This process allows for real-time guidance and feedback that takes into account the worker's emotional state. User progress data is also recorded and used for the next session.

[2236] The hardware used includes industrial machines (e.g., KUKA, FANUC), tablets and smart glasses for workers, and software including Flask (a web application framework), generative AI models (e.g., GPT-3, BERT), and emotion recognition modules.

[2237] example

[2238] Consider a scenario where a worker is learning a new safety procedure. For example, the following conditions are entered:

[2239] Worker experience: Beginner

[2240] New work procedure: Machine maintenance

[2241] Language: Japanese

[2242] The server generates a virtual personality based on these conditions. The virtual personality is designed to provide simple and easy-to-understand instruction for beginners. During the dialogue, if the worker asks, "Which part of the machine should I check first?", the virtual personality generates an appropriate response and presents specific options. The prompt text in this case is as follows:

[2243] "Worker: What part of the machine should I check first?

[2244] Virtual personality: First, it is important to check the power source of the machine. Please choose the next step from the options below:

[2245] 1. Turn off the power to the power unit

[2246] 2. Check the oil in the power parts

[2247] 3. Check with other workers

[2248] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2249] Step 1:

[2250] The user inputs conditions such as the worker's years of experience, new work procedures, and language on the interface device. The terminal checks the format of this input data, and if there are no errors, it serializes it and sends it to the server. The input includes the worker's experience, procedures, and language, and the output generates condition data in JSON format.

[2251] Step 2:

[2252] The server receives the condition data. The server receives the condition data sent from the device, parses it, and extracts the condition information. The input is JSON-formatted condition data, and the parsed condition information is obtained as output.

[2253] Step 3:

[2254] The server generates a virtual personality. Based on the extracted condition information, the server sends a request to a generative AI model (e.g., GPT-3, BERT) to generate a virtual personality. The data of the generated virtual personality is stored on the server and linked to the user's session information. The condition information is input, and the data of the generated virtual personality is obtained as output.

[2255] Step 4:

[2256] The server installs the virtual personality information into the industrial machine. The server sends the generated virtual personality data to the industrial machine and installs it. The input is the virtual personality data, and the output is the virtual personality installed in the industrial machine.

[2257] Step 5:

[2258] A user starts a chat with a virtual persona. A worker starts a chat with a virtual persona using a tablet or smart glasses. The worker types a message in the chat window, and the device sends the message to the server. There is a user message as input, and a message sent to the server as output.

[2259] Step 6:

[2260] The server generates responses and options for the virtual personality. The server analyzes the received message and generates appropriate responses and options using a generative AI model. It also analyzes the user's emotional information using an emotion engine. The input is the user message, and the output is the virtual personality's responses, options, and emotional information.

[2261] Step 7:

[2262] The server sends responses, options, and feedback to the user. The server then sends the generated responses, options, and feedback based on the emotional information to the user's device. The inputs are the virtual personality's responses, options, and emotional information, and the output is the data sent to the device.

[2263] Step 8:

[2264] The user selects an option. They choose one from multiple options displayed in the chat window and send the selection information to the terminal. The option data is input, and the selection information is sent to the server as output.

[2265] Step 9:

[2266] The server generates the next response or feedback based on the selection. The server generates the next response or feedback based on the selection information and sends it to the device. The selection information is the input, and the next response or feedback is sent to the device as the output.

[2267] Step 10:

[2268] The emotion engine analyzes the user's input. The server uses the emotion engine to analyze the user's input data and extract emotional information. The user's input data is the input, and emotional information is obtained as the output.

[2269] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2270] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2271] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2272] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2273] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2274] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2275] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2276] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2277] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2278] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2279] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2280] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2281] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2282] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2283] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2284] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2285] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2286] As an example of a system configure...

Claims

1. a means for a user to input a condition; means for generating a virtual personality based on the conditions; means for chatting between the virtual personality and a user; means for presenting multiple options during chat and generating feedback based on the user's selection; A system including means for providing said feedback to a user.

2. 10. The system of claim 1, further comprising means for recording a user's progress and utilizing that progress in subsequent sessions.

3. 2. The system according to claim 1, further comprising means for checking the format of the condition data entered by the user.

Images