System

Abstract

To provide a system allowing users to train their own ideal behavior and utterances.SOLUTION: A system includes means for: storing information relating to an ideal human image set by a user; collecting at least one of behavior and an utterance of the user as data; generating a prompt sentence for generating instructions to custom-change at least one of the behavior and the utterance of the user based on the information of the ideal human image set by the user and at least one of the collected data about the user's behavior and utterance; using the generated prompt sentence and the generated AI model to generate a proposal message including at least one of the behavior and the utterance obtained by custom-changing at least one of the behavior and the utterance of the user; and presenting the generated proposal message to the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] There is a problem that people who lack the confidence to express their true selves due to trauma from human relationships or lack of education, etc., have difficulty becoming that ideal person, even if they have a specific ideal image for themselves. [Means for solving the problem]

[0005] The system projects the ideal image of a person set by the user, custom-changes the user's actions and speech based on that ideal image, and presents the custom-changed actions and speech to the user, providing repeated training to encourage the user to get closer to their ideal image. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16]FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention includes a means for projecting an ideal human image set by a user, a means for custom-changing the user's actions and comments based on the ideal human image, and a means for presenting the custom-changed actions and comments to the user and repeatedly training them. Specifically, the user sets an ideal human image on an application. For example, if a "compassionate person" is set as the ideal human image, the system custom-changes the user's actions and comments based on the "compassionate person". The system then presents the custom-changed actions and comments to the user and repeatedly trains them, encouraging the user to get closer to their ideal image.

[0029] "Example 2"

[0030] The system of the present invention includes a means for projecting an ideal human image set by a user, a means for custom-changing the user's actions and comments based on the ideal human image, and a means for presenting the custom-changed actions and comments to the user and repeatedly training them. Specifically, the user sets an ideal human image on an application. For example, if a "compassionate person" is set as the ideal human image, the system custom-changes the user's actions and comments based on the "compassionate person". The system then presents the custom-changed actions and comments to the user and repeatedly trains them, encouraging the user to get closer to their ideal image.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The user sets the ideal person in the application. For example, the user sets "a caring person" as the ideal person.

[0034] Step 2: The system custom-changes the user's behavior and speech based on the ideal image of a person. Specifically, the system analyzes the user's behavior and speech based on the image of a "compassionate person" and generates appropriate behavior and speech.

[0035] Step 3: The system presents the customized actions and statements to the user, for example, providing advice to the user to behave more considerately towards others.

[0036] Step 4: The user repeatedly practices the actions and statements suggested, which allows the user to get closer to their ideal self.

[0037] Example 1

[0038] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0039] In conventional self-improvement systems, it is difficult to custom change specific actions and speech based on the ideal image of a person set by the user, and there is a lack of effective means to record the progress of the repetitive training and provide feedback. Therefore, there is an issue that effective support for users to get closer to their ideal image is not provided.

[0040] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0041] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and comments based on the ideal human image using a generative AI model, a means for presenting the custom-changed actions and comments to the user and training them repeatedly, and a means for recording the progress of the repeated training and providing feedback. This enables the user to learn and practice specific actions and comments to approach their ideal image, and to effectively manage the progress.

[0042] "Ideal person" refers to the specific person image or personality traits that users should aim to be.

[0043] A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate data and custom change user behavior and speech.

[0044] "Custom change" refers to modifying a user's behavior and words based on an ideal image of a person.

[0045] "Repetitive training" refers to the user repeatedly practicing the custom-changed actions or utterances presented to them.

[0046] "Feedback" refers to evaluations and advice provided to a user based on their progress in repeated training.

[0047] "Recording progress" refers to saving the content and results of the repetitive training performed by the user as data.

[0048] "Presenting" refers to showing the user the custom-changed actions or utterances generated by the server.

[0049] This invention is a system that promotes the user to approach his / her ideal image by customizing actions and speech based on the ideal image of a person set by the user and repeatedly training the user. A specific embodiment of this system will be described below.

[0050] Hardware and Software Used

[0051] Hardware

[0052] Device: A device operated by a user, such as a smartphone, computer, or tablet.

[0053] Server: A server for processing data and running generative AI models.

[0054] software

[0055] Application: Software that provides an interface for users to define their ideal person image and review their custom-changed behaviors and statements.

[0056] Generative AI model: An algorithm for customizing a user's actions and speech based on an ideal image of a human. Specifically, we use generative AI models such as GPT-4 (registered trademark) from OpenAI (registered trademark).

[0057] Data processing and calculation

[0058] Users define their ideal image of a person

[0059] The user starts the application and sets the ideal image of the person. For example, the user sets "a considerate person" as the ideal image of the person.

[0060] The device sends the configuration data to the server.

[0061] The terminal transmits data of the ideal human image set by the user to the server. This data includes information about the ideal human image selected by the user.

[0062] The server analyzes the data using a generative AI model

[0063] The server uses a generative AI model based on the received data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comments data and custom-changes it based on the ideal image of a person.

[0064] The server generates custom-changed actions and statements

[0065] The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements of a "compassionate person."

[0066] Send the server-generated data to the device

[0067] The server transmits the generated custom-changed behavior and speech data to the terminal, which includes examples of specific behaviors and speech that the user should practice.

[0068] The device presents the data to the user

[0069] The terminal presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

[0070] Users perform repetitive training

[0071] The user then practices the proposed custom-changed actions and utterances, and the application tracks the user's progress and provides feedback as needed.

[0072] Examples

[0073] For example, if a user sets a "compassionate person" as their ideal person, the following prompt sentence is input into the generative AI model.

[0074] Examples of prompt statements

[0075] "Please provide advice to help users be considerate when speaking with friends."

[0076] Based on this prompt, the generative AI model generates specific examples of utterances that users can use when talking to their friends, such as "When a friend is in trouble, say, 'Let me know if there's anything I can do to help you.'"

[0077] In this way, users can learn and practice behaviors and speech based on their ideal image of a person.

[0078] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0079] Step 1:

[0080] Users define their ideal image of a person

[0081] The user launches the application and sets the ideal person by clicking the "Set Ideal Person" button on the application interface, selecting "A considerate person" from the drop-down menu, and pressing the "Set" button.

[0082] Input: User's choice of ideal person (e.g. "a caring person")

[0083] Output: Ideal human image configuration data

[0084] Step 2:

[0085] The device sends the configuration data to the server.

[0086] The device transmits the user's ideal person profile data to a server, where the transmission is secure using the HTTPS protocol.

[0087] Input: Ideal human image configuration data

[0088] Output: Configuration data sent to the server.

[0089] Step 3:

[0090] The server analyzes the data using a generative AI model

[0091] The server uses a generative AI model based on the received configuration data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comment data (e.g., chat logs and SNS posts) and custom changes it based on the ideal image of a person.

[0092] Input: Configuration data, user's past actions and statements

[0093] Output: Custom-changed behavior and speech data

[0094] Step 4:

[0095] The server generates custom-changed actions and statements

[0096] The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements of a "compassionate person."

[0097] Input: Custom-changed action or speech data

[0098] Output: Generated examples of specific actions and statements

[0099] Step 5:

[0100] Send the server-generated data to the device

[0101] The server transmits the generated custom-changed behavior and speech data to the terminal. This transmission is also performed using the HTTPS protocol.

[0102] Input: Examples of specific actions or statements generated

[0103] Output: Data sent to the terminal

[0104] Step 6:

[0105] The device presents the data to the user

[0106] The terminal presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

[0107] Input: Data sent to the terminal

[0108] Output: Examples of specific actions or statements presented to the user

[0109] Step 7:

[0110] Users perform repetitive training

[0111] The user then practices the proposed custom-changed actions and statements, and the application records the user's progress and provides feedback as needed. For example, the user might enter into the application, "Today, I made a kind comment to a friend."

[0112] Input: User's practice results

[0113] Output: Progress data recorded and feedback provided.

[0114] (Application example 1)

[0115] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0116] In conventional customer service training systems, it was difficult to customize specific actions and comments for staff to acquire the ideal customer service attitude, making it difficult to provide effective training. In addition, there was a lack of repetitive training for staff to get closer to the ideal customer service attitude, making it difficult for them to maintain the ideal attitude when actually serving customers.

[0117] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and remarks based on the ideal human image, a means for presenting the custom-changed actions and remarks to the user and repeatedly training them, a means for generating actions and remarks based on the ideal human image using a generative AI model, and a means for presenting the generated actions and remarks to the user as prompt sentences. This enables the staff to concretely understand the ideal customer service attitude and maintain the ideal attitude in actual customer service through repetitive training.

[0118] A "user" is an individual who uses the system to learn to behave and speak in a way that is consistent with the ideal image of a human being.

[0119] The "ideal person" refers to the specific character and attitude that the user should aim for, and serves as the standard by which the system will custom change behavior and speech.

[0120] "Means of projection" is a function that displays the ideal image of a person set by the user visually or as text on the system.

[0121] "Means for custom change" is a function that allows a user to change their current behavior and words based on their ideal image of the person.

[0122] "Means of presentation" is a function for displaying custom-changed actions and comments to the user.

[0123] "Means for repeated training" is a function that allows users to repeatedly practice custom-changed actions and utterances.

[0124] A "generative AI model" is an artificial intelligence model that generates actions and statements based on the ideal human image set by the user.

[0125] A "prompt" is a text-based instruction that presents to the user actions and statements that the generative AI model has generated based on an ideal image of a human being.

[0126] The system for carrying out the present invention customizes actions and speech based on an ideal human image set by the user, and presents the customized changes to the user for repetitive training. A specific embodiment of this system will be described below.

[0127] System Configuration

[0128] The system consists of the following main components:

[0129] 1. User device: A device such as a smartphone or tablet that is used by the user to define their ideal image of the person and check their custom-changed actions and statements.

[0130] 2. Server: Processes user input data and uses a generative AI model to generate actions and statements based on the ideal image of a human.

[0131] 3. Generative AI models: Using generative AI models such as OpenAI's GPT-3 (registered trademark), actions and statements are generated based on the ideal human image set by the user.

[0132] Program Processing

[0133] The server receives the ideal human image and current behavior and speech data sent from the user's device, and inputs them as prompts into the generative AI model. The generative AI model generates behavior and speech based on the ideal human image and returns the results to the server. The server then sends the generated behavior and speech to the user's device and presents them to the user.

[0134] Hardware and Software Used

[0135] Hardware: Smartphones, tablets, servers

[0136] Software: Python, OpenAI API, User Interface Application

[0137] Data processing and calculation

[0138] The server sends the user's ideal image of a person and their current actions and comments as prompts to the generative AI model. The generative AI model generates the ideal actions and comments based on the prompts and returns the results to the server. The server then sends the generated actions and comments to the user's device and presents them to the user.

[0139] Examples

[0140] For example, if a user sets "caring customer service" as their ideal person and their current behavior is "treating customers in a rude manner," the prompt text would be as follows:

[0141] Example prompt:

[0142] Ideal person: Attentive customer service

[0143] User behavior: Being rude to customers

[0144] Acting on the Ideal Person:

[0145] Based on this prompt, the generative AI model generates actions and utterances appropriate for "considerate customer service" and presents the results to the user. By repeatedly practicing the actions and utterances presented, the user can get closer to their ideal customer service attitude.

[0146] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0147] Step 1:

[0148] The user sets the ideal person image using the terminal. The user starts the application and inputs the ideal person image (e.g., "caring customer service"). The input data is sent from the terminal to the server.

[0149] Input: Ideal human image

[0150] Output: Ideal human image data sent to the server

[0151] Step 2:

[0152] The server receives the ideal human image data sent by the user. The server stores this data and uses it for the next process.

[0153] Input: Ideal human image data

[0154] Output: Saved ideal human image data

[0155] Step 3:

[0156] The user inputs the current behavior or speech using the terminal. The user inputs the current behavior (e.g., "I treat customers rudely") and sends it from the terminal to the server.

[0157] Input: Current action or statement

[0158] Output: Current action and speech data sent to the server

[0159] Step 4:

[0160] The server receives the current behavior and speech data sent by the user, and stores this data for further processing.

[0161] Input: Current action or speech data

[0162] Output: Saved current actions and speech data

[0163] Step 5:

[0164] The server generates a prompt by combining the stored ideal human image data with the current behavior and speech data. The generated prompt is then input into the generative AI model.

[0165] Input: Ideal person image data, current behavior and speech data

[0166] Output: Generated prompt statement

[0167] Step 6:

[0168] The server sends the generated prompt text to the generative AI model, which then generates the ideal behavior or speech based on the prompt text.

[0169] Input: Generated prompt text

[0170] Output: Ideal actions and statements generated by the generative AI model

[0171] Step 7:

[0172] The server receives the ideal actions and statements returned by the generative AI model, and then transmits this data to the user's device.

[0173] Input: Ideal actions and statements generated by a generative AI model

[0174] Output: Ideal behavior and speech data sent to the user's device

[0175] Step 8:

[0176] The user device receives the ideal behavior and speech data sent from the server and presents it to the user. The user confirms the presented behavior and speech and performs repeated training.

[0177] Input: Ideal behavior and speech data

[0178] Output: The desired actions or statements presented to the user

[0179] Through the above processing steps, the user can learn how to act and speak based on the ideal image of a person, and through repetitive training, come closer to that ideal attitude.

[0180] Example 2

[0181] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0182] In modern society, many people aim to improve themselves and become closer to their ideal image, but the process is difficult and requires continuous effort. In addition, there is a problem that effective training is difficult due to a lack of specific guidelines for actions and statements for self-improvement. Furthermore, there is no appropriate means to customize users' actions and statements based on the ideal image of a person, making it difficult for users to get closer to their ideal image.

[0183] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0184] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and comments based on the ideal human image, a means for presenting the custom-changed actions and comments to the user, a means for the user to repeatedly train the presented actions and comments, a means for collecting the user's actions and comments, and a means for analyzing the collected actions and comments and customizing them using a generation AI model. This allows the user to obtain specific guidelines for actions and comments and to approach their ideal image through continuous training.

[0185] "Ideal person" refers to the specific person image or personality traits that users should aim to be.

[0186] "Means of projection" refers to the function that allows the user to display the ideal image of the person they have set in visual or text form.

[0187] "Means for custom change" refers to the function that allows a user to change their current behavior and words based on their ideal image of the person.

[0188] "Means of presentation" refers to a function for notifying or displaying custom-changed actions or statements to the user.

[0189] "Means of repetitive training" refers to a function that allows the user to train themselves to become closer to their ideal image of a person by repeatedly carrying out suggested actions and statements.

[0190] "Means of collection" refers to the function for recording user actions and statements and storing them as data.

[0191] "Means of analysis" refers to the function of analyzing collected user behavior and comment data and extracting information for custom changes based on an ideal image of a person.

[0192] A "generative AI model" refers to an artificial intelligence model that converts a user's actions and words based on an ideal image of a human being.

[0193] This invention is a system that custom changes actions and speech based on an ideal human image set by a user, presents it to the user, and conducts repetitive training. A specific embodiment of this system will be described below.

[0194] Hardware and Software Used

[0195] Hardware

[0196] Device: The device used by a user, such as a smartphone, tablet, or computer.

[0197] Server: A remote server for processing and storing data.

[0198] software

[0199] Application: Software that allows users to define their ideal person image and check their custom-changed behavior and statements.

[0200] Generative AI model: An artificial intelligence model (e.g., GPT-4) that transforms user actions and statements based on an ideal image of a human being.

[0201] System Operation

[0202] Setting the ideal image of a person

[0203] The user starts the application and goes to the settings screen. There, they input their ideal image of the person they want to be. For example, they might input "a caring person."

[0204] Collection of user actions and comments

[0205] The device records the user's daily actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media. The collected data is sent to a server in real time.

[0206] Custom changes to actions and statements

[0207] The server analyzes the received user behavior and speech data. Using a generative AI model (e.g. GPT-4), the user's behavior and speech are custom-changed based on the ideal image of a person. For example, if a user says, "I'm busy today, so I can't help you," the generative AI model converts this to "I'm busy today, but please let me know if there's anything I can do to help you."

[0208] Displaying custom-changed actions and statements

[0209] The server transmits the custom-changed actions and utterances to the terminal. The terminal notifies the user of the change. The user confirms the notification and actually uses the custom-changed actions and utterances.

[0210] Repetitive training

[0211] The user repeatedly executes the customized actions and statements suggested to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts. In this way, the user trains to become closer to their ideal image of a person.

[0212] Examples

[0213] Setting the ideal image of a person

[0214] The user enters "compassionate person" in the application's settings screen and clicks the save button.

[0215] Collection of user actions and comments

[0216] The device will record the voice of users' conversations with friends and convert them into text, and also collect text data posted by users on social media.

[0217] Custom changes to actions and statements

[0218] The server inputs the collected text data into a generative AI model, which uses prompts to transform the user's speech into an ideal human image.

[0219] Displaying custom-changed actions and statements

[0220] The server sends the customized utterances output by the generative AI model to the device, which then displays a notification to the user and suggests something like, "I'm busy today, but please let me know if there's anything I can do to help you."

[0221] Repetitive training

[0222] The user uses the utterances suggested by the server in daily conversations and social media posts. For example, in a conversation with a friend, the user might say, "I'm busy today, but please let me know if there's anything I can do to help you." By repeating this, the user will naturally be able to act and speak as a "considerate person."

[0223] Examples of prompt statements

[0224] Here is an example of a prompt to be input to the generative AI model:

[0225] User currently says: "I'm busy today, I can't help you."

[0226] Ideal person: "A considerate person"

[0227] Prompt for generative AI model: "Transform the user's utterance into a compassionate person's utterance."

[0228] By inputting this prompt into a generative AI model, the user's speech is custom altered based on the ideal image of a person.

[0229] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0230] Step 1:

[0231] The user starts the application and moves to the setting screen. There, the user inputs the ideal image of the person. For example, the user inputs "a considerate person."

[0232] Input: The ideal person image entered by the user on the settings screen.

[0233] Output: The ideal person image is saved in the application.

[0234] Step 2:

[0235] The device records the user's daily actions and comments, for example, the contents of conversations the user has had with friends and the contents of posts made to social networking sites.

[0236] Input: What the user does or says.

[0237] Output: Collected behavioral and speech data is stored on the device and transmitted to the server in real time.

[0238] Step 3:

[0239] The server analyzes the received user behavior and speech data, and uses a generative AI model (e.g. GPT-4) to custom-change the user's behavior and speech based on the ideal image of a person.

[0240] Input: Collected behavioral and speech data, ideal human image.

[0241] Output: Custom changed actions and statements.

[0242] Specific operation: The server inputs the collected text data into the generative AI model and converts the user's utterances using prompt sentences.

[0243] Step 4:

[0244] The server transmits the custom-changed actions and utterances to the terminal. The terminal notifies the user of the change. The user confirms the notification and actually uses the custom-changed actions and utterances.

[0245] Input: Custom-changed actions and statements.

[0246] Output: The custom changed actions or statements presented to the user.

[0247] Specific operation: The device displays the custom-changed message received from the server as a notification.

[0248] Step 5:

[0249] The user repeatedly executes the proposed custom-changed actions and utterances, for example, using utterances suggested by the generative AI model in daily conversations and social media posts.

[0250] Input: The custom-changed actions or statements presented.

[0251] Output: Custom changes to actions and statements made by the user.

[0252] Specific operation: The user uses utterances suggested by the server in conversations with friends and in social media posts. By repeating this process, the user trains to become closer to the ideal person.

[0253] (Application example 2)

[0254] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0255] In conventional customer service work, there were limited training methods for staff to acquire the ideal customer service attitude, and it was difficult to provide effective training. In addition, there was a lack of specific guidelines and feedback for staff to practice the ideal customer service attitude, making it difficult to improve customer service quality. Furthermore, there was a need for an efficient method of repeated training to help staff get closer to their ideal image.

[0256] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0257] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and utterances based on the ideal human image, a means for presenting the custom-changed actions and utterances to the user and repeatedly training them, a means for training the ideal customer service attitude using smart glasses worn by the staff performing customer service work, and a means for generating custom-changed utterances based on the ideal human image using a generation AI model. This enables the staff to efficiently train and practice the ideal customer service attitude.

[0258] "User" means an individual or organization that uses the System.

[0259] The "ideal person" is a model of the behavior and attitude that users should strive to be.

[0260] A "projection means" is a device or software for visually or audibly displaying the ideal human image set by the user.

[0261] A "means of custom change" is a device or software that changes a user's behavior and speech based on an ideal image of a person.

[0262] A "means for presenting" is a device or software for displaying custom-changed actions or utterances to a user.

[0263] A "means for repeated training" is a device or software that allows a user to repeatedly practice custom-changed actions or utterances.

[0264] "Customer service" refers to the provision of products and services to customers.

[0265] "Smart glasses" are glasses-type devices that have the function of displaying information.

[0266] A "generative AI model" is an algorithm that uses artificial intelligence to generate text and actions.

[0267] "Custom-changed statements" are statements made by users that have been altered based on their ideal image of a person.

[0268] The system for implementing this invention projects the ideal human image set by the user, custom changes the user's actions and remarks, and repeatedly trains them. Specifically, the ideal customer service attitude can be trained using smart glasses worn by staff who serve customers.

[0269] System Configuration

[0270] The system consists of the following main components:

[0271] 1. User device: The device on which the user sets their ideal image. This can be a smartphone or tablet.

[0272] 2. Smart glasses: A device worn by staff performing customer service duties that has the function of visually displaying information.

[0273] 3. Server: Runs the generative AI model and processes data to customise user behaviour and speech.

[0274] 4. Generative AI model: An algorithm that generates custom-changed utterances based on the user’s ideal image of a person.

[0275] Program Processing

[0276] The server receives the ideal person image set by the user and custom changes the user's behavior and speech based on that image. Specifically, the process is as follows:

[0277] 1. Setting the ideal person image: Using the user terminal, the user sets the ideal person image. For example, the ideal image can be set as "kind and polite customer service."

[0278] 2. Enter user utterances: Enter actual utterances made by users. For example, "Welcome, is there anything you're looking for?"

[0279] 3. Executing the generative AI model: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image.

[0280] 4. Present customized utterances: Present customized utterances to the user through smart glasses, for example, "Welcome, we will help you find the product you are looking for."

[0281] 5. Repetitive training: The user repeatedly practices the custom-changed utterances presented to them.

[0282] Hardware and Software Used

[0283] Hardware: Smart glasses (e.g. Google® Glass®), user devices (smartphones, tablets)

[0284] Software: Generative AI models (e.g. OpenAI API), data processing servers

[0285] Examples

[0286] For example, the ideal customer service staff member may be described as "friendly and polite" and the following prompt may be used:

[0287] Example prompt:

[0288] Ideal person: Kind and polite customer service

[0289] User says: Hey there, is there anything you're looking for?

[0290] Custom statements based on the ideal person:

[0291] By inputting this prompt into the generative AI model, a custom-changed utterance based on the ideal customer service attitude is generated, for example, "Welcome, we will help you find the product you are looking for."

[0292] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0293] Step 1:

[0294] The user sets the ideal image of the person.

[0295] Input: The user inputs the ideal image of a person using a device (smartphone or tablet). For example, "kind and polite customer service."

[0296] Data processing: The terminal sends the inputted ideal image of a person to the server.

[0297] Output: The ideal person image is saved on the server.

[0298] Step 2:

[0299] The user types in the actual utterance.

[0300] Input: What the user actually says when using the terminal, for example, "Hello, is there something you're looking for?"

[0301] Data processing: The terminal sends the entered comments to the server.

[0302] Output: The user's comments are saved on the server.

[0303] Step 3:

[0304] The server runs the generative AI model.

[0305] Input: The server retrieves the stored ideal person profile and the user's statements.

[0306] Data calculation: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image. The prompt sentence is input to the generative AI model to obtain the custom-changed utterances.

[0307] Output: The generated custom-changed utterances are saved on the server.

[0308] Step 4:

[0309] The custom-changed statement is presented to the user.

[0310] Input: The server gets the generated custom changed utterance.

[0311] Data processing: The server sends the custom-changed utterances to the smart glasses.

[0312] Output: The custom-changed speech is displayed on the smart glasses.

[0313] Step 5:

[0314] The user performs repetitive training.

[0315] Input: The user sees the custom-changed utterance displayed on the smart glasses.

[0316] Specific behavior: The user repeatedly practices the utterances shown to them, for example, "Welcome, we are happy to help you find the product you are looking for."

[0317] Output: The user's customer service attitude becomes closer to the ideal person.

[0318] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0319] "Example 1"

[0320] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine analyzes the user's emotional state in real time. Specifically, the emotion state is estimated from the user's facial expression, tone of voice, choice of words, etc. In addition, this emotion engine tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training for the user to approach his / her ideal image according to the user's emotional state. For example, when the user is feeling stressed, training that promotes relaxation is provided. On the other hand, when the user is feeling joy, training that promotes behavior to share that joy is provided. This allows the user to train to approach his / her ideal image in a manner appropriate to his / her emotional state.

[0321] "Example 2"

[0322] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine analyzes the user's emotional state in real time. Specifically, the emotion state is estimated from the user's facial expression, tone of voice, choice of words, etc. In addition, this emotion engine tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training for the user to approach his / her ideal image according to the user's emotional state. For example, when the user is feeling stressed, training that promotes relaxation is provided. On the other hand, when the user is feeling joy, training that promotes behavior to share that joy is provided. This allows the user to train to approach his / her ideal image in a manner appropriate to his / her emotional state.

[0323] The process flow of each embodiment will be described below.

[0324] "Example 1"

[0325] Step 1: The user boots up the system and sets their ideal self. This ideal self specifies how the user will act and speak.

[0326] Step 2: The emotion engine analyzes the user's emotional state in real time, inferring it from the user's facial expressions, tone of voice, choice of words, etc.

[0327] Step 3: The emotion engine tracks changes in the user's emotional state, allowing it to tailor the iterative training to help the user achieve their ideal self.

[0328] Step 4: The user undergoes repetitive training to get closer to their ideal self. This training is adjusted according to the user's emotional state. For example, when the user is feeling stressed, training is provided to promote relaxation. On the other hand, when the user is feeling joy, training is provided to encourage behavior that encourages sharing that joy.

[0329] Example 1

[0330] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0331] With conventional systems, it was difficult to custom change behavior and speech based on the ideal person image set by the user, and the training content was not adjusted according to the user's emotional state, so there was an issue of a lack of effective support to help users get closer to their ideal image.

[0332] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0333] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, and a means for adjusting the training content based on data obtained from the emotion engine. This makes it possible to effectively support the user in getting closer to his or her ideal image.

[0334] A "user" refers to an individual who aims to use the system to become closer to their ideal image of a person.

[0335] An "ideal person" is a specific definition of the personality and behavioral characteristics that a user aspires to be.

[0336] "Means of projection" refers to the ability to display the user's ideal image of the person on the system in visual or text form.

[0337] "Means of custom change" refers to the function that changes and adjusts a user's actions and comments based on an ideal image of a person.

[0338] "Means of presentation" refers to the function of notifying or displaying custom-changed actions or comments to the user.

[0339] "Means for repeated training" refers to a function that allows the user to repeatedly practice based on the custom changes presented to them.

[0340] The "emotion engine" refers to a function that analyzes a user's emotional state in real time based on their facial expressions, tone of voice, choice of words, etc.

[0341] "Means for adjusting training content" refers to a function for changing and adjusting the training content according to the user's emotional state based on data obtained from the emotion engine.

[0342] This invention is a system that supports the user in getting closer to his / her ideal image through repetitive training by customizing actions and speech based on the ideal image of the person set by the user. A specific embodiment of this system will be described below.

[0343] Hardware and Software Used

[0344] Hardware

[0345] Device: The smartphone, tablet, or PC used by the user.

[0346] Server: Cloud server for data processing and storage.

[0347] Camera and microphone: Devices for capturing your facial expressions and tone of voice in real time.

[0348] software

[0349] Application: Provides an interface for users to define their ideal person image and review their custom-changed behaviors and statements.

[0350] Emotion engine: A software module for analyzing the user's emotional state in real time.

[0351] Generative AI model: An AI model that custom changes a user's behavior and speech based on an ideal image of a human being.

[0352] Data processing and calculation

[0353] Users define their ideal image of a person

[0354] The user launches the application and selects their ideal image of the person they want to be, for example, selecting "a caring person."

[0355] The server collects user behavior data

[0356] The server collects real-time input data, voice data, and text messages from users' smartphones and PCs, including what users chat with friends and what they say to their voice assistants.

[0357] The server analyzes the data and makes custom changes

[0358] The server analyzes the collected data using natural language processing (NLP) techniques. For example, if the user says "thank you" infrequently, the server will suggest that the user say "thank you" more frequently.

[0359] The server presents custom-changed actions and comments to the user

[0360] The server presents the custom-changed actions and comments to the user through the notification function of the application. For example, it sends a notification such as "Try saying 'Thank you' the next time you talk to your friend."

[0361] Users perform repetitive training

[0362] Based on the suggestions from the server, the user is made conscious of saying "thank you" more frequently in their daily lives. By repeating this, the user becomes closer to being a "considerate person."

[0363] Emotion engine analyzes the user's emotional state

[0364] The emotion engine uses the user's smartphone camera and microphone to analyze facial expressions and tone of voice in real time. For example, if a user is smiling while speaking, the emotion engine will determine that the user is feeling happy.

[0365] The server adjusts the training content

[0366] The server adjusts the training content according to the user's emotional state based on the data obtained from the emotion engine. For example, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax."

[0367] Examples

[0368] Setting an ideal human image

[0369] A user opens the application and sets "a compassionate person" as their ideal person.

[0370] Examples of custom changes

[0371] The server suggests that the user say "thank you" more frequently when conversing with friends.

[0372] Sentiment Engine Analysis

[0373] If a user smiles during a conversation, the emotion engine determines that the user is happy.

[0374] Adjusting training content

[0375] If the user is feeling stressed, the server offers deep breathing and meditation exercises.

[0376] Examples of prompt statements

[0377] "If the user sets the ideal person as a 'kind person,' please suggest what kind of actions and words should be custom changed."

[0378] The system supports users' growth by providing them with specific actions and statements to help them become closer to their ideal self, and by providing training tailored to their emotional state.

[0379] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0380] Step 1:

[0381] Users define their ideal image of a person

[0382] The user starts the application and sets the ideal person. Specifically, the user selects the ideal person from options such as "a considerate person" on the application's settings screen and saves the settings. The input is the ideal person selected by the user, and the output is the data of the set ideal person.

[0383] Step 2:

[0384] The server collects user behavior data

[0385] The server collects input data, voice data, and text messages from users' smartphones and PCs in real time. Specifically, it collects what users chat with their friends and what they say to their voice assistants. The input is user behavior data, and the output is a database of collected behavior data.

[0386] Step 3:

[0387] The server analyzes the data and makes custom changes

[0388] The server analyzes the collected data using natural language processing (NLP) technology. Specifically, it uses a generative AI model to change the user's behavior and utterances based on the ideal human image. For example, if the user says "thank you" infrequently, the server will suggest that the user say "thank you" more frequently. The input is the collected behavioral data, and the output is a custom-changed behavior or utterance suggestion.

[0389] Step 4:

[0390] The server presents custom-changed actions and comments to the user

[0391] The server presents the custom-changed actions and utterances to the user using the notification function of the application. Specifically, it sends a notification such as "Try saying 'Thank you' the next time you talk to your friend." The input is the custom-changed action or utterance suggestion, and the output is the notification to the user.

[0392] Step 5:

[0393] Users perform repetitive training

[0394] Based on the suggestions from the server, the user is conscious of saying "thank you" more frequently in their daily lives. Specifically, they increase the opportunities to say "thank you" in conversations with friends and family. The input is the suggestions from the server, and the output is the user's actual behavior.

[0395] Step 6:

[0396] Emotion engine analyzes the user's emotional state

[0397] The emotion engine uses the camera and microphone of the user's smartphone to analyze facial expressions and tone of voice in real time. Specifically, if the user is smiling and talking, the emotion engine determines that the user is feeling happy. The input is the user's facial expressions and tone of voice, and the output is the analyzed emotional state data.

[0398] Step 7:

[0399] The server adjusts the training content

[0400] The server adjusts the training content according to the user's emotional state based on the data obtained from the emotion engine. For example, if the user is feeling stressed, the server provides training such as "Take a deep breath and relax." The input is the analyzed emotional state data, and the output is the adjusted training content.

[0401] (Application example 1)

[0402] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0403] Conventional customer service training systems lacked specific feedback to help staff develop ideal customer service attitudes, and the training content was not adjusted according to the emotional state of the staff, making it difficult to provide effective training.

[0404] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0405] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, a means for analyzing the user's emotional state in real time, a means for adjusting the training content according to the user's emotional state, and a means for providing feedback to the user using a smart device, thereby enabling staff to effectively acquire the ideal customer service attitude.

[0406] A "user" is an individual who uses the system to train themselves to act and speak in a way that reflects their ideal image of a person.

[0407] The "ideal person" is a model of specific behaviors and attitudes that users should aim for.

[0408] A "projection means" is a method or device for visually or aurally presenting the ideal human image set by the user.

[0409] A "means for custom change" is a method or device for changing a user's actions and words based on an ideal image of a person.

[0410] The "means for repetitive training" refers to a method or device for repeatedly practicing custom-changed actions and speech in order for the user to become closer to the ideal image of a person.

[0411] "Means for analyzing emotional state in real time" refers to a method or device for inferring emotions from a user's facial expressions, tone of voice, choice of words, etc.

[0412] A "means for adjusting training content" is a method or device for changing the training content in response to the user's emotional state.

[0413] A "smart device" is a device that has a built-in camera, microphone, speaker, etc. and is used to provide feedback to the user.

[0414] A "means for providing feedback" is a method or device for providing a user with advice or instructions regarding ideal actions or statements.

[0415] As an embodiment of the present invention, a training system is provided for helping staff working in brick-and-mortar stores acquire an ideal customer service attitude. The system includes a means for projecting an ideal human image set by the user, custom-changing the user's actions and remarks, and repeatedly training the user. It also includes a means for analyzing the user's emotional state in real time and adjusting the training content.

[0416] Hardware and Software Used

[0417] Hardware

[0418] Smart glasses: Equipped with a camera, microphone, and speaker, they capture what the user does and says and provide feedback.

[0419] Server: A central processing unit for data processing and analysis.

[0420] software

[0421] OpenCV: A library for capturing and processing camera images.

[0422] EmotionRecognizer: A library for estimating emotions from facial expressions.

[0423] SpeechRecognition: A speech recognition library.

[0424] TextToSpeech: A library for converting text to speech.

[0425] IdealBehaviorModel: A model that generates ideal behaviors and statements.

[0426] Data processing and calculation

[0427] Camera footage capture

[0428] We use the camera in the smart glasses to capture the user's facial expressions and actions in real time. We use OpenCV to acquire the video data and EmotionRecognizer to analyze the emotional state.

[0429] Speech Recognition

[0430] It uses the microphone in the smart glasses to capture what the user is saying, and uses the SpeechRecognition library to convert the speech data into text.

[0431] Feedback Generation

[0432] IdealBehaviorModel is used to generate ideal behaviors and utterances based on the user's emotional state and utterances. The generated feedback is converted to speech using the TextToSpeech library and provided to the user through the smart glasses' speaker.

[0433] Examples

[0434] scenario

[0435] Staff working in physical stores will undergo training to provide "considerate customer service."

[0436] situation

[0437] When customers have questions about products, staff will respond appropriately.

[0438] feedback

[0439] If staff are feeling stressed, the system will provide relaxation advice.

[0440] Examples of prompt statements

[0441] If the user is aiming to be a "caring customer," tell them how to respond to customer questions. If the user is stressed, offer them some relaxation advice.

[0442] In this way, customer service training can be carried out effectively in physical stores.

[0443] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0444] Step 1:

[0445] A user wears the smart glasses and starts the system. The camera in the smart glasses captures the user's facial expressions and actions in real time, and the microphone captures the user's speech. The input is the camera video and audio data, and the output is the captured video and audio data.

[0446] Step 2:

[0447] The server uses OpenCV to process the video data sent from the smart glasses. Specifically, it extracts the user's facial expressions from the video data and analyzes their emotional state using EmotionRecognizer. The input is the captured video data, and the output is the user's emotional state.

[0448] Step 3:

[0449] The server uses the SpeechRecognition library to convert the voice data sent from the smart glasses into text. The input is the captured voice data and the output is the text data.

[0450] Step 4:

[0451] The server uses IdealBehaviorModel to generate ideal actions and utterances based on the user's emotional state and text data. The input is the user's emotional state and text data, and the output is the text of the ideal actions and utterances.

[0452] Step 5:

[0453] The server converts the generated text of the ideal actions and utterances into speech using the TextToSpeech library. The input is the text of the ideal actions and utterances, and the output is the speech data.

[0454] Step 6:

[0455] The voice data sent from the server is provided to the user through the speaker of the smart glasses. The input is the voice data, and the output is the voice feedback that the user hears.

[0456] Step 7:

[0457] The user modifies his / her actions or utterances based on the provided feedback and again captures new data through the smart glasses' camera and microphone, which repeats the process from step 1 to step 6. The input is the video and audio data of the modified actions or utterances, and the output is the new feedback.

[0458] Example 2

[0459] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0460] Conventional systems have the problem that it is difficult for users to customize specific actions and comments to get closer to their ideal image of a person, and the training content is not adjusted according to the user's emotional state, making it difficult to achieve effective training. Furthermore, the system lacked the function to analyze the user's emotional state in real time and adjust the training content based on that, making it difficult to get closer to the ideal image while reducing the user's psychological burden.

[0461] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0462] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, a means for analyzing the user's emotional state in real time, and a means for adjusting the training content according to the emotional state. This allows the user to receive effective training to get closer to his or her ideal image.

[0463] "Ideal person" refers to the type of person or personality traits that a user should aspire to.

[0464] "Means of projection" refers to devices or software that visually or audibly display the ideal human image set by the user.

[0465] "Means of custom change" refers to algorithms or programs that change a user's behavior and words based on an ideal image of a person.

[0466] "Means of presentation" refers to devices or software for notifying the user of custom-changed actions or statements.

[0467] "Means of repetitive training" refers to a program or system that allows a user to repeatedly practice custom-changed actions or utterances.

[0468] "Means of analyzing emotional state in real time" refers to devices or software that infer a user's emotional state by analyzing their facial expressions, tone of voice, choice of words, etc.

[0469] "Means for adjusting training content" refers to algorithms or programs for changing the content or method of training in response to the user's emotional state.

[0470] This invention is a system that supports the user in getting closer to his / her ideal image through repetitive training by customizing actions and speech based on the ideal image of the person set by the user. A specific embodiment of this system will be described below.

[0471] System Configuration

[0472] Users define their ideal image of a person

[0473] Using the application on the device, the user sets the ideal person they want to be. For example, they can select "a caring person." This setting takes into account the user's trauma in relationships and education.

[0474] The server collects user actions and comments.

[0475] The servers collect what users do and say in real time, including any text or voice data they type, such as when they talk to friends in a chat app or to a voice assistant.

[0476] Analyze the data collected by the server

[0477] The server analyzes the collected data. Specifically, it uses natural language processing (NLP) techniques to understand what the user is saying. Here, it uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4.

[0478] The server custom-changes actions and comments based on the ideal image of a person.

[0479] Based on the analysis results, the server will customize the user's actions and comments to match the ideal person. For example, "I'm busy today, so I can't help you" will be changed to "I'm busy today, but please let me know if there's anything I can do to help you."

[0480] The server presents the custom changes to the user

[0481] The server presents the custom-changed actions and comments to the user, who can review them and modify them as necessary.

[0482] The server uses an emotion engine to analyze the user's emotional state.

[0483] The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it analyzes the user's facial expressions, tone of voice, choice of words, etc. It uses Microsoft(R) Azure(R)'s Emotion API and Affectiva's SDK.

[0484] The server adjusts the training content according to the user's emotional state.

[0485] The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, the server provides training to promote relaxation, and if the user is feeling joy, the server provides training to promote behavior such as sharing that joy.

[0486] The user performs the training

[0487] The user performs the training content presented by the server, and undergoes repetitive training to get closer to their ideal self.

[0488] Examples

[0489] Setting the ideal image of a person

[0490] A user opens the application, selects "Define your ideal person" and chooses "A caring person."

[0491] Collection of actions and statements

[0492] The application collects in the background what you chat with your friends.

[0493] Analyzing data

[0494] The server analyzes chat contents collected by the server using Google Cloud Natural Language API to perform sentiment analysis. It also uses OpenAI's GPT-4 to understand what users say and generate appropriate responses.

[0495] Examples of custom changes

[0496] If the user says "I'm busy today and can't help you," the server changes this to "I'm busy today, but let me know if there's anything I can do to help."

[0497] Sentiment Engine Analysis

[0498] When a user speaks into the camera, the server uses Microsoft Azure's Emotion API to determine whether the user is feeling stressed based on their facial expressions and tone of voice.

[0499] Adjusting training content

[0500] When a user feels stressed, the server provides relaxation training such as "Take a deep breath and relax."

[0501] Running the training

[0502] The user performs the relaxation training presented by the server.

[0503] Examples of prompt statements

[0504] "If a user sets the ideal person as a 'kind person,' how would you customize their actions and words?"

[0505] This system allows users to receive effective training to get closer to their ideal image.

[0506] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0507] Step 1:

[0508] Users define their ideal image of a person

[0509] The user starts the application on the device and sets the ideal person. Specifically, the user selects "Set ideal person" from the application menu and selects the ideal person from options such as "a considerate person." The input is the ideal person selected by the user, and the output is the setting information.

[0510] Step 2:

[0511] The server collects user actions and comments.

[0512] The server collects the user's actions and comments in real time. Specifically, it collects the content of the user's conversations with friends on chat apps and the content of what the user says to the voice assistant. The input is the user's text data and voice data, and the output is the collected data on the actions and comments.

[0513] Step 3:

[0514] Analyze the data collected by the server

[0515] The server analyzes the collected data. Specifically, it uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4 to understand what users say and perform sentiment analysis and key phrase extraction. The input is the collected behavioral and utterance data, and the output is the analysis results.

[0516] Step 4:

[0517] The server custom-changes actions and comments based on the ideal image of a person.

[0518] Based on the analysis results, the server custom-changes the user's actions and comments to match the ideal person. For example, "I'm busy today, so I can't help you" is changed to "I'm busy today, but please let me know if there's anything I can do to help you." The input is the analysis results, and the output is the custom-changed actions and comments.

[0519] Step 5:

[0520] The server presents the custom changes to the user

[0521] The server presents the custom-changed actions and statements to the user, who can then review and modify them as necessary. The input is the custom-changed actions and statements, and the output is the content presented to the user.

[0522] Step 6:

[0523] The server uses an emotion engine to analyze the user's emotional state.

[0524] The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it uses Microsoft Azure's Emotion API and Affectiva's SDK to analyze the user's facial expressions, tone of voice, choice of words, etc. The input is the user's facial expressions, tone of voice, and choice of words, and the output is the analysis of the user's emotional state.

[0525] Step 7:

[0526] The server adjusts the training content according to the user's emotional state.

[0527] The server adjusts the training content according to the user's emotional state. For example, if the user feels stressed, it provides training to promote relaxation, and if the user feels joy, it provides training to promote behavior to share that joy. The input is the analysis result of the emotional state, and the output is the adjusted training content.

[0528] Step 8:

[0529] The user performs the training

[0530] The user executes the training content presented by the server. This allows the user to perform repeated training to approach his / her ideal image. The input is the adjusted training content, and the output is the result of the user's execution.

[0531] (Application example 2)

[0532] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0533] In conventional customer service work, it is difficult for staff to maintain an ideal customer service attitude, and there is a problem of customer satisfaction decreasing, especially when the staff's emotional state is unstable. Also, there is a lack of effective training methods to help staff get closer to their ideal image.

[0534] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0535] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and remarks based on the ideal human image, a means for presenting the custom-changed actions and remarks to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, a means for adjusting the training content based on the emotion engine, and a means for displaying on the smart glasses. This allows staff to receive appropriate training according to their emotional state while maintaining their ideal customer service attitude.

[0536] "User" means an individual or organization that uses the System.

[0537] The "ideal person" is a model of the behavior and attitude that users should strive to be.

[0538] A "means of projection" is a device or software that visually or audibly displays the ideal human image set by the user.

[0539] "Means of custom change" are algorithms or programs that change a user's behavior and words based on an ideal image of a person.

[0540] "Means of presentation" refers to a display device or notification system that notifies the user of custom-changed actions or comments.

[0541] "Means for repeated training" is a function that allows users to repeatedly practice their custom-changed actions and statements in order to get closer to their ideal image of the person they want to be.

[0542] An "emotion engine" is software or hardware for analyzing a user's emotional state in real time.

[0543] The "means for adjusting training content" is a function for changing the content of training provided to the user based on the analysis results of the emotion engine.

[0544] "Smart glasses" are wearable devices that can visually display information to users when worn by the user.

[0545] The system for implementing this invention includes a series of means for projecting an ideal human image set by the user, custom changing the user's actions and speech, and repeatedly training it.Furthermore, it has a function of analyzing the user's emotional state in real time using an emotion engine and adjusting the training content.

[0546] Hardware and Software Used

[0547] Hardware: Smart glasses, camera

[0548] Software: OpenCV (face detection), EmotionRecognizer, BehaviorSuggestion

[0549] System Configuration

[0550] 1. A means to project the ideal image of a person set by the user:

[0551] Users can set their ideal image of the person through the application. For example, if the ideal is "kind and polite customer service," the system will customize the user's behavior and speech based on this information.

[0552] 2. Ways to customise user behaviour and comments:

[0553] The system runs algorithms to change the user's current behavior and speech based on the ideal person, for example making suggestions to promote relaxation if the user is feeling stressed.

[0554] 3. Present custom-changed actions and statements to users and train them repeatedly:

[0555] The customized actions and utterances are displayed on the smart glasses' display, allowing the user to repeatedly practice their ideal actions and utterances.

[0556] 4. Emotion Engine:

[0557] The camera captures the user's facial expressions and uses EmotionRecognizer to analyze the user's emotional state in real time, providing training content according to the user's emotional state.

[0558] 5. Ways to adjust training:

[0559] Based on the analysis results of the emotion engine, the BehaviorSuggestion engine suggests appropriate actions and statements. For example, if the user is feeling happy, it provides training to encourage behavior such as sharing that happiness.

[0560] 6. Means of display on smart glasses:

[0561] Suggested actions and statements are displayed in real time on the smart glasses' display, helping users maintain the ideal attitude while serving customers.

[0562] Examples

[0563] Scenario: Staff in a brick-and-mortar store are fitted with smart glasses to help them maintain a "friendly and courteous" attitude while serving customers.

[0564] Emotional state: If staff are feeling stressed, suggestions to promote relaxation (e.g., "Take a deep breath and relax").

[0565] Action Suggestion: Suggest a kind statement to customers such as, "Please let me know if you need any help."

[0566] Examples of prompt statements

[0567] Develop an application that changes actions and comments in real time based on the ideal customer service attitude set by the user, and displays them on the smart glasses display. Include a function to analyze the user's emotional state using an emotion engine and provide appropriate training.

[0568] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0569] Step 1:

[0570] The user starts the application and sets the ideal image of the person. The user inputs the ideal image, such as "kind and polite customer service." This input data is sent to the server and saved.

[0571] Step 2:

[0572] The server executes an algorithm that custom-changes actions and utterances based on the user's ideal person image. It receives the ideal person image and the user's current behavior data as input, and generates custom-changed actions and utterances as output.

[0573] Step 3:

[0574] The server sends the customized actions and comments to the smart glasses. The smart glasses receive them and display them on the display. The user can then see them and practice the ideal actions and comments.

[0575] Step 4:

[0576] The camera captures the user's facial expressions and transmits them to the server in real time. The server uses EmotionRecognizer to analyze the user's emotional state from the input facial expression data and generates emotional state data as output.

[0577] Step 5:

[0578] The server adjusts the training content based on the emotional state data. The BehaviorSuggestion engine takes the emotional state data and the ideal human image as input, suggests appropriate actions and utterances, and generates the adjusted training content as output.

[0579] Step 6:

[0580] The server transmits the adjusted training content to the smart glasses, which receive it and display it on the display. The user can then view it and practice the training according to their emotional state.

[0581] Step 7:

[0582] As the user practices the training, the server tracks changes in their emotional state and adjusts the training content as necessary, allowing the user to receive optimal training to become the ideal person they want to be.

[0583] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0584] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0585] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0586] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0587] [Second embodiment]

[0588] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0589] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0590] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0591] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0592] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0593] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0594] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0595] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0596] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0597] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0598] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0599] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0600] "Example 1"

[0601] The system of the present invention includes a means for projecting an ideal human image set by a user, a means for custom-changing the user's actions and comments based on the ideal human image, and a means for presenting the custom-changed actions and comments to the user and repeatedly training them. Specifically, the user sets an ideal human image on an application. For example, if a "compassionate person" is set as the ideal human image, the system custom-changes the user's actions and comments based on the "compassionate person". The system then presents the custom-changed actions and comments to the user and repeatedly trains them, encouraging the user to get closer to their ideal image.

[0602] "Example 2"

[0603] The system of the present invention includes a means for projecting an ideal human image set by a user, a means for custom-changing the user's actions and comments based on the ideal human image, and a means for presenting the custom-changed actions and comments to the user and repeatedly training them. Specifically, the user sets an ideal human image on an application. For example, if a "compassionate person" is set as the ideal human image, the system custom-changes the user's actions and comments based on the "compassionate person". The system then presents the custom-changed actions and comments to the user and repeatedly trains them, encouraging the user to get closer to their ideal image.

[0604] The process flow of each embodiment will be described below.

[0605] "Example 1"

[0606] Step 1: The user sets the ideal person in the application. For example, the user sets "a caring person" as the ideal person.

[0607] Step 2: The system custom-changes the user's behavior and speech based on the ideal image of a person. Specifically, the system analyzes the user's behavior and speech based on the image of a "compassionate person" and generates appropriate behavior and speech.

[0608] Step 3: The system presents the customized actions and statements to the user, for example, providing advice to the user to behave more considerately towards others.

[0609] Step 4: The user repeatedly practices the actions and statements suggested, which allows the user to get closer to their ideal self.

[0610] Example 1

[0611] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0612] In conventional self-improvement systems, it is difficult to custom change specific actions and speech based on the ideal image of a person set by the user, and there is a lack of effective means to record the progress of the repetitive training and provide feedback. Therefore, there is an issue that effective support for users to get closer to their ideal image is not provided.

[0613] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0614] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and comments based on the ideal human image using a generative AI model, a means for presenting the custom-changed actions and comments to the user and training them repeatedly, and a means for recording the progress of the repeated training and providing feedback. This enables the user to learn and practice specific actions and comments to approach their ideal image, and to effectively manage the progress.

[0615] "Ideal person" refers to the specific person image or personality traits that users should aim to be.

[0616] A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate data and custom change user behavior and speech.

[0617] "Custom change" refers to modifying a user's behavior and words based on an ideal image of a person.

[0618] "Repetitive training" refers to the user repeatedly practicing the custom-changed actions or utterances presented to them.

[0619] "Feedback" refers to evaluations and advice provided to a user based on their progress in repeated training.

[0620] "Recording progress" refers to saving the content and results of the repetitive training performed by the user as data.

[0621] "Presenting" refers to showing the user the custom-changed actions or utterances generated by the server.

[0622] This invention is a system that promotes the user to approach his / her ideal image by customizing actions and speech based on the ideal image of a person set by the user and repeatedly training the user. A specific embodiment of this system will be described below.

[0623] Hardware and Software Used

[0624] Hardware

[0625] Device: A device operated by a user, such as a smartphone, computer, or tablet.

[0626] Server: A server for processing data and running generative AI models.

[0627] software

[0628] Application: Software that provides an interface for users to define their ideal person image and review their custom-changed behaviors and statements.

[0629] Generative AI model: An algorithm for customizing a user's actions and speech based on an ideal human image. Specifically, we use generative AI models such as OpenAI's GPT-4.

[0630] Data processing and calculation

[0631] Users define their ideal image of a person

[0632] The user starts the application and sets the ideal image of the person. For example, the user sets "a considerate person" as the ideal image of the person.

[0633] The device sends the configuration data to the server.

[0634] The terminal transmits data of the ideal human image set by the user to the server. This data includes information about the ideal human image selected by the user.

[0635] The server analyzes the data using a generative AI model

[0636] The server uses a generative AI model based on the received data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comments data and custom-changes it based on the ideal image of a person.

[0637] The server generates custom-changed actions and statements

[0638] The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements of a "compassionate person."

[0639] Send the server-generated data to the device

[0640] The server transmits the generated custom-changed behavior and speech data to the terminal, which includes examples of specific behaviors and speech that the user should practice.

[0641] The device presents the data to the user

[0642] The terminal presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

[0643] Users perform repetitive training

[0644] The user then practices the proposed custom-changed actions and utterances, and the application tracks the user's progress and provides feedback as needed.

[0645] Examples

[0646] For example, if a user sets a "compassionate person" as their ideal person, the following prompt sentence is input into the generative AI model.

[0647] Examples of prompt statements

[0648] "Please provide advice to help users be considerate when speaking with friends."

[0649] Based on this prompt, the generative AI model generates specific examples of utterances that users can use when talking to their friends, such as "When a friend is in trouble, say, 'Let me know if there's anything I can do to help you.'"

[0650] In this way, users can learn and practice behaviors and speech based on their ideal image of a person.

[0651] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0652] Step 1:

[0653] Users define their ideal image of a person

[0654] The user launches the application and sets the ideal person by clicking the "Set Ideal Person" button on the application interface, selecting "A considerate person" from the drop-down menu, and pressing the "Set" button.

[0655] Input: User's choice of ideal person (e.g. "a caring person")

[0656] Output: Ideal human image configuration data

[0657] Step 2:

[0658] The device sends the configuration data to the server.

[0659] The device transmits the user's ideal person profile data to a server, where the transmission is secure using the HTTPS protocol.

[0660] Input: Ideal human image configuration data

[0661] Output: Configuration data sent to the server.

[0662] Step 3:

[0663] The server analyzes the data using a generative AI model

[0664] The server uses a generative AI model based on the received configuration data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comment data (e.g., chat logs and SNS posts) and custom changes it based on the ideal image of a person.

[0665] Input: Configuration data, user's past actions and statements

[0666] Output: Custom-changed behavior and speech data

[0667] Step 4:

[0668] The server generates custom-changed actions and statements

[0669] The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements of a "compassionate person."

[0670] Input: Custom-changed action or speech data

[0671] Output: Generated examples of specific actions and statements

[0672] Step 5:

[0673] Send the server-generated data to the device

[0674] The server transmits the generated custom-changed behavior and speech data to the terminal. This transmission is also performed using the HTTPS protocol.

[0675] Input: Examples of specific actions or statements generated

[0676] Output: Data sent to the terminal

[0677] Step 6:

[0678] The device presents the data to the user

[0679] The terminal presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

[0680] Input: Data sent to the terminal

[0681] Output: Examples of specific actions or statements presented to the user

[0682] Step 7:

[0683] Users perform repetitive training

[0684] The user then practices the proposed custom-changed actions and statements, and the application records the user's progress and provides feedback as needed. For example, the user might enter into the application, "Today, I made a kind comment to a friend."

[0685] Input: User's practice results

[0686] Output: Progress data recorded and feedback provided.

[0687] (Application example 1)

[0688] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0689] In conventional customer service training systems, it was difficult to customize specific actions and comments for staff to acquire the ideal customer service attitude, making it difficult to provide effective training. In addition, there was a lack of repetitive training for staff to get closer to the ideal customer service attitude, making it difficult for them to maintain the ideal attitude when actually serving customers.

[0690] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and remarks based on the ideal human image, a means for presenting the custom-changed actions and remarks to the user and repeatedly training them, a means for generating actions and remarks based on the ideal human image using a generative AI model, and a means for presenting the generated actions and remarks to the user as prompt sentences. This enables the staff to concretely understand the ideal customer service attitude and maintain the ideal attitude in actual customer service through repetitive training.

[0691] A "user" is an individual who uses the system to learn to behave and speak in a way that is consistent with the ideal image of a human being.

[0692] The "ideal person" refers to the specific character and attitude that the user should aim for, and serves as the standard by which the system will custom change behavior and speech.

[0693] "Means of projection" is a function that displays the ideal image of a person set by the user visually or as text on the system.

[0694] "Means for custom change" is a function that allows a user to change their current behavior and words based on their ideal image of the person.

[0695] "Means of presentation" is a function for displaying custom-changed actions and comments to the user.

[0696] "Means for repeated training" is a function that allows users to repeatedly practice custom-changed actions and utterances.

[0697] A "generative AI model" is an artificial intelligence model that generates actions and statements based on the ideal human image set by the user.

[0698] A "prompt" is a text-based instruction that presents to the user actions and statements that the generative AI model has generated based on an ideal image of a human being.

[0699] The system for carrying out the present invention customizes actions and speech based on an ideal human image set by the user, and presents the customized changes to the user for repetitive training. A specific embodiment of this system will be described below.

[0700] System Configuration

[0701] The system consists of the following main components:

[0702] 1. User device: A device such as a smartphone or tablet that the user uses to define their ideal image of the person and check their custom-changed actions and statements.

[0703] 2. Server: Processes user input data and uses a generative AI model to generate actions and statements based on the ideal image of a human.

[0704] 3. Generative AI models: Using generative AI models such as OpenAI's GPT-3, actions and statements are generated based on the ideal human image set by the user.

[0705] Program Processing

[0706] The server receives the ideal human image and current behavior and speech data sent from the user's device, and inputs them as prompts into the generative AI model. The generative AI model generates behavior and speech based on the ideal human image and returns the results to the server. The server then sends the generated behavior and speech to the user's device and presents them to the user.

[0707] Hardware and Software Used

[0708] Hardware: Smartphones, tablets, servers

[0709] Software: Python, OpenAI API, User Interface Application

[0710] Data processing and calculation

[0711] The server sends the user's ideal image of a person and their current actions and comments as prompts to the generative AI model. The generative AI model generates the ideal actions and comments based on the prompts and returns the results to the server. The server then sends the generated actions and comments to the user's device and presents them to the user.

[0712] Examples

[0713] For example, if a user sets "caring customer service" as their ideal person and their current behavior is "treating customers in a rude manner," the prompt text would be as follows:

[0714] Example prompt:

[0715] Ideal person: Attentive customer service

[0716] User behavior: Being rude to customers

[0717] Acting on the Ideal Person:

[0718] Based on this prompt, the generative AI model generates actions and utterances appropriate for "considerate customer service" and presents the results to the user. By repeatedly practicing the actions and utterances presented, the user can get closer to their ideal customer service attitude.

[0719] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0720] Step 1:

[0721] The user sets the ideal person image using the terminal. The user starts the application and inputs the ideal person image (e.g., "caring customer service"). The input data is sent from the terminal to the server.

[0722] Input: Ideal human image

[0723] Output: Ideal human image data sent to the server

[0724] Step 2:

[0725] The server receives the ideal human image data sent by the user. The server stores this data and uses it for the next process.

[0726] Input: Ideal human image data

[0727] Output: Saved ideal human image data

[0728] Step 3:

[0729] The user inputs the current behavior or speech using the terminal. The user inputs the current behavior (e.g., "I treat customers rudely") and sends it from the terminal to the server.

[0730] Input: Current action or statement

[0731] Output: Current action and speech data sent to the server

[0732] Step 4:

[0733] The server receives the current behavior and speech data sent by the user, and stores this data for further processing.

[0734] Input: Current action or speech data

[0735] Output: Saved current actions and speech data

[0736] Step 5:

[0737] The server generates a prompt by combining the stored ideal human image data with the current behavior and speech data. The generated prompt is then input into the generative AI model.

[0738] Input: Ideal person image data, current behavior and speech data

[0739] Output: Generated prompt statement

[0740] Step 6:

[0741] The server sends the generated prompt text to the generative AI model, which then generates the ideal behavior or speech based on the prompt text.

[0742] Input: Generated prompt text

[0743] Output: Ideal actions and statements generated by the generative AI model

[0744] Step 7:

[0745] The server receives the ideal actions and statements returned by the generative AI model, and then transmits this data to the user's device.

[0746] Input: Ideal actions and statements generated by a generative AI model

[0747] Output: Ideal behavior and speech data sent to the user's device

[0748] Step 8:

[0749] The user device receives the ideal behavior and speech data sent from the server and presents it to the user. The user confirms the presented behavior and speech and performs repeated training.

[0750] Input: Ideal behavior and speech data

[0751] Output: The desired actions or statements presented to the user

[0752] Through the above processing steps, the user can learn how to act and speak based on the ideal image of a person, and through repetitive training, come closer to that ideal attitude.

[0753] Example 2

[0754] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0755] In modern society, many people aim to improve themselves and become closer to their ideal image, but the process is difficult and requires continuous effort. In addition, there is a problem that effective training is difficult due to a lack of specific guidelines for actions and statements for self-improvement. Furthermore, there is no appropriate means to customize users' actions and statements based on the ideal image of a person, making it difficult for users to get closer to their ideal image.

[0756] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0757] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and comments based on the ideal human image, a means for presenting the custom-changed actions and comments to the user, a means for the user to repeatedly train the presented actions and comments, a means for collecting the user's actions and comments, and a means for analyzing the collected actions and comments and customizing them using a generation AI model. This allows the user to obtain specific guidelines for actions and comments and to approach their ideal image through continuous training.

[0758] "Ideal person" refers to the specific person image or personality traits that users should aim to be.

[0759] "Means of projection" refers to the function that allows the user to display the ideal image of the person they have set in visual or text form.

[0760] "Means for custom change" refers to the function that allows a user to change their current behavior and words based on their ideal image of the person.

[0761] "Means of presentation" refers to a function for notifying or displaying custom-changed actions or statements to the user.

[0762] "Means of repetitive training" refers to a function that allows the user to train themselves to become closer to their ideal image of a person by repeatedly carrying out suggested actions and statements.

[0763] "Means of collection" refers to the function for recording user actions and statements and storing them as data.

[0764] "Means of analysis" refers to the function of analyzing collected user behavior and comment data and extracting information for custom changes based on an ideal image of a person.

[0765] A "generative AI model" refers to an artificial intelligence model that converts a user's actions and words based on an ideal image of a human being.

[0766] This invention is a system that custom changes actions and speech based on an ideal human image set by a user, presents it to the user, and conducts repetitive training. A specific embodiment of this system will be described below.

[0767] Hardware and Software Used

[0768] Hardware

[0769] Device: The device used by a user, such as a smartphone, tablet, or computer.

[0770] Server: A remote server for processing and storing data.

[0771] software

[0772] Application: Software that allows users to define their ideal person image and check their custom-changed behavior and statements.

[0773] Generative AI model: An artificial intelligence model (e.g., GPT-4) that transforms user actions and statements based on an ideal image of a human being.

[0774] System Operation

[0775] Setting the ideal image of a person

[0776] The user starts the application and goes to the settings screen. There, they input their ideal image of the person they want to be. For example, they might input "a caring person."

[0777] Collection of user actions and comments

[0778] The device records the user's daily actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media. The collected data is sent to a server in real time.

[0779] Custom changes to actions and statements

[0780] The server analyzes the received user behavior and speech data. Using a generative AI model (e.g. GPT-4), the user's behavior and speech are custom-changed based on the ideal image of a person. For example, if a user says, "I'm busy today, so I can't help you," the generative AI model converts this to "I'm busy today, but please let me know if there's anything I can do to help you."

[0781] Displaying custom-changed actions and statements

[0782] The server transmits the custom-changed actions and utterances to the terminal. The terminal notifies the user of the change. The user confirms the notification and actually uses the custom-changed actions and utterances.

[0783] Repetitive training

[0784] The user repeatedly executes the customized actions and statements suggested to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts. In this way, the user trains to become closer to their ideal image of a person.

[0785] Examples

[0786] Setting the ideal image of a person

[0787] The user enters "compassionate person" in the application's settings screen and clicks the save button.

[0788] Collection of user actions and comments

[0789] The device will record the voice of users' conversations with friends and convert them into text, and also collect text data posted by users on social media.

[0790] Custom changes to actions and statements

[0791] The server inputs the collected text data into a generative AI model, which uses prompts to transform the user's speech into an ideal human image.

[0792] Displaying custom-changed actions and statements

[0793] The server sends the customized utterances output by the generative AI model to the device, which then displays a notification to the user and suggests something like, "I'm busy today, but please let me know if there's anything I can do to help you."

[0794] Repetitive training

[0795] The user uses the utterances suggested by the server in daily conversations and social media posts. For example, in a conversation with a friend, the user might say, "I'm busy today, but please let me know if there's anything I can do to help you." By repeating this, the user will naturally be able to act and speak as a "considerate person."

[0796] Examples of prompt statements

[0797] Here is an example of a prompt to be input to the generative AI model:

[0798] User currently says: "I'm busy today, I can't help you."

[0799] Ideal person: "A considerate person"

[0800] Prompt for generative AI model: "Transform the user's utterance into a compassionate person's utterance."

[0801] By inputting this prompt into a generative AI model, the user's speech is custom altered based on the ideal image of a person.

[0802] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0803] Step 1:

[0804] The user starts the application and moves to the setting screen. There, the user inputs the ideal image of the person. For example, the user inputs "a considerate person."

[0805] Input: The ideal person image entered by the user on the settings screen.

[0806] Output: The ideal person image is saved in the application.

[0807] Step 2:

[0808] The device records the user's daily actions and comments, for example, the contents of conversations the user has had with friends and the contents of posts made to social networking sites.

[0809] Input: What the user does or says.

[0810] Output: Collected behavioral and speech data is stored on the device and transmitted to the server in real time.

[0811] Step 3:

[0812] The server analyzes the received user behavior and speech data, and uses a generative AI model (e.g. GPT-4) to custom-change the user's behavior and speech based on the ideal image of a person.

[0813] Input: Collected behavioral and speech data, ideal human image.

[0814] Output: Custom changed actions and statements.

[0815] Specific operation: The server inputs the collected text data into the generative AI model and converts the user's utterances using prompt sentences.

[0816] Step 4:

[0817] The server transmits the custom-changed actions and utterances to the terminal. The terminal notifies the user of the change. The user confirms the notification and actually uses the custom-changed actions and utterances.

[0818] Input: Custom-changed actions and statements.

[0819] Output: The custom changed actions or statements presented to the user.

[0820] Specific operation: The device displays the custom-changed message received from the server as a notification.

[0821] Step 5:

[0822] The user repeatedly executes the proposed custom-changed actions and utterances, for example, using utterances suggested by the generative AI model in daily conversations and social media posts.

[0823] Input: The custom-changed actions or statements presented.

[0824] Output: Custom changes to actions and statements made by the user.

[0825] Specific operation: The user uses utterances suggested by the server in conversations with friends and in social media posts. By repeating this process, the user trains to become closer to the ideal person.

[0826] (Application example 2)

[0827] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0828] In conventional customer service work, there were limited training methods for staff to acquire the ideal customer service attitude, and it was difficult to provide effective training. In addition, there was a lack of specific guidelines and feedback for staff to practice the ideal customer service attitude, making it difficult to improve customer service quality. Furthermore, there was a need for an efficient method of repeated training to help staff get closer to their ideal image.

[0829] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0830] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and utterances based on the ideal human image, a means for presenting the custom-changed actions and utterances to the user and repeatedly training them, a means for training the ideal customer service attitude using smart glasses worn by the staff performing customer service work, and a means for generating custom-changed utterances based on the ideal human image using a generation AI model. This enables the staff to efficiently train and practice the ideal customer service attitude.

[0831] "User" means an individual or organization that uses the System.

[0832] The "ideal person" is a model of the behavior and attitude that users should strive to be.

[0833] A "projection means" is a device or software for visually or audibly displaying the ideal human image set by the user.

[0834] A "means of custom change" is a device or software that changes a user's behavior and speech based on an ideal image of a person.

[0835] A "means for presenting" is a device or software for displaying custom-changed actions or utterances to a user.

[0836] A "means for repeated training" is a device or software that allows a user to repeatedly practice custom-changed actions or utterances.

[0837] "Customer service" refers to the provision of products and services to customers.

[0838] "Smart glasses" are glasses-type devices that have the function of displaying information.

[0839] A "generative AI model" is an algorithm that uses artificial intelligence to generate text and actions.

[0840] "Custom-changed statements" are statements made by users that have been altered based on their ideal image of a person.

[0841] The system for implementing this invention projects the ideal human image set by the user, custom changes the user's actions and remarks, and repeatedly trains them. Specifically, the ideal customer service attitude can be trained using smart glasses worn by staff who serve customers.

[0842] System Configuration

[0843] The system consists of the following main components:

[0844] 1. User device: The device on which the user sets their ideal image. This can be a smartphone or tablet.

[0845] 2. Smart glasses: A device worn by staff performing customer service duties that has the function of visually displaying information.

[0846] 3. Server: Runs the generative AI model and processes data to customise user behaviour and speech.

[0847] 4. Generative AI model: An algorithm that generates custom-changed utterances based on the user’s ideal image of a person.

[0848] Program Processing

[0849] The server receives the ideal person image set by the user and custom changes the user's behavior and speech based on that image. Specifically, the process is as follows:

[0850] 1. Setting the ideal person image: Using the user terminal, the user sets the ideal person image. For example, the ideal image can be set as "kind and polite customer service."

[0851] 2. Enter user utterances: Enter actual utterances made by users. For example, "Welcome, is there anything you're looking for?"

[0852] 3. Executing the generative AI model: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image.

[0853] 4. Present customized utterances: Present customized utterances to the user through smart glasses, for example, "Welcome, we will help you find the product you are looking for."

[0854] 5. Repetitive training: The user repeatedly practices the custom-changed utterances presented to them.

[0855] Hardware and Software Used

[0856] Hardware: Smart glasses (e.g. Google Glass), user devices (smartphones, tablets)

[0857] Software: Generative AI models (e.g. OpenAI API), data processing servers

[0858] Examples

[0859] For example, the ideal customer service staff member may be described as "friendly and polite" and the following prompt may be used:

[0860] Example prompt:

[0861] Ideal person: Kind and polite customer service

[0862] User says: Hey there, is there anything you're looking for?

[0863] Custom statements based on the ideal person:

[0864] By inputting this prompt into the generative AI model, a custom-changed utterance based on the ideal customer service attitude is generated. For example, a utterance like, "Welcome, we will help you find the product you are looking for" is generated.

[0865] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0866] Step 1:

[0867] The user sets the ideal image of the person.

[0868] Input: The user inputs the ideal image of a person using a device (smartphone or tablet). For example, "kind and polite customer service."

[0869] Data processing: The terminal sends the inputted ideal image of a person to the server.

[0870] Output: The ideal person image is saved on the server.

[0871] Step 2:

[0872] The user types in the actual utterance.

[0873] Input: What the user actually says when using the terminal, for example, "Hello, is there something you're looking for?"

[0874] Data processing: The terminal sends the entered comments to the server.

[0875] Output: The user's comments are saved on the server.

[0876] Step 3:

[0877] The server runs the generative AI model.

[0878] Input: The server retrieves the stored ideal person profile and the user's statements.

[0879] Data calculation: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image. The prompt sentence is input to the generative AI model to obtain the custom-changed utterances.

[0880] Output: The generated custom-changed utterances are saved on the server.

[0881] Step 4:

[0882] The custom-changed statement is presented to the user.

[0883] Input: The server gets the generated custom changed utterance.

[0884] Data processing: The server sends the custom-changed utterances to the smart glasses.

[0885] Output: The custom-changed speech is displayed on the smart glasses.

[0886] Step 5:

[0887] The user performs repetitive training.

[0888] Input: The user sees the custom-changed utterance displayed on the smart glasses.

[0889] Specific behavior: The user repeatedly practices the utterances shown to them, for example, "Welcome, we are happy to help you find the product you are looking for."

[0890] Output: The user's customer service attitude becomes closer to the ideal person.

[0891] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0892] "Example 1"

[0893] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine analyzes the user's emotional state in real time. Specifically, the emotion state is estimated from the user's facial expression, tone of voice, choice of words, etc. In addition, this emotion engine tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training for the user to approach his / her ideal image according to the user's emotional state. For example, when the user is feeling stressed, training that promotes relaxation is provided. On the other hand, when the user is feeling joy, training that promotes behavior to share that joy is provided. This allows the user to train to approach his / her ideal image in a manner appropriate to his / her emotional state.

[0894] "Example 2"

[0895] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine analyzes the user's emotional state in real time. Specifically, the emotion state is estimated from the user's facial expression, tone of voice, choice of words, etc. In addition, this emotion engine tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training for the user to approach his / her ideal image according to the user's emotional state. For example, when the user is feeling stressed, training that promotes relaxation is provided. On the other hand, when the user is feeling joy, training that promotes behavior to share that joy is provided. This allows the user to train to approach his / her ideal image in a manner appropriate to his / her emotional state.

[0896] The process flow of each embodiment will be described below.

[0897] "Example 1"

[0898] Step 1: The user boots up the system and sets their ideal self. This ideal self specifies how the user will act and speak.

[0899] Step 2: The emotion engine analyzes the user's emotional state in real time, inferring it from the user's facial expressions, tone of voice, choice of words, etc.

[0900] Step 3: The emotion engine tracks changes in the user's emotional state, allowing it to tailor the iterative training to help the user achieve their ideal self.

[0901] Step 4: The user undergoes repetitive training to get closer to their ideal self. This training is adjusted according to the user's emotional state. For example, when the user is feeling stressed, training is provided to promote relaxation. On the other hand, when the user is feeling joy, training is provided to encourage behavior that encourages sharing that joy.

[0902] Example 1

[0903] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0904] With conventional systems, it was difficult to custom change behavior and speech based on the ideal person image set by the user, and the training content was not adjusted according to the user's emotional state, so there was an issue of a lack of effective support to help users get closer to their ideal image.

[0905] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0906] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, and a means for adjusting the training content based on data obtained from the emotion engine. This makes it possible to effectively support the user in getting closer to his or her ideal image.

[0907] A "user" refers to an individual who aims to use the system to become closer to their ideal image of a person.

[0908] An "ideal person" is a specific definition of the personality and behavioral characteristics that a user aspires to be.

[0909] "Means of projection" refers to the ability to display the user's ideal image of the person on the system in visual or text form.

[0910] "Means of custom change" refers to the function that changes and adjusts a user's actions and comments based on an ideal image of a person.

[0911] "Means of presentation" refers to the function of notifying or displaying custom-changed actions or comments to the user.

[0912] "Means for repeated training" refers to a function that allows the user to repeatedly practice based on the custom changes presented to them.

[0913] The "emotion engine" refers to a function that analyzes a user's emotional state in real time based on their facial expressions, tone of voice, choice of words, etc.

[0914] "Means for adjusting training content" refers to a function for changing and adjusting the training content according to the user's emotional state based on data obtained from the emotion engine.

[0915] This invention is a system that supports the user in getting closer to his / her ideal image through repetitive training by customizing actions and speech based on the ideal image of the person set by the user. A specific embodiment of this system will be described below.

[0916] Hardware and Software Used

[0917] Hardware

[0918] Device: The smartphone, tablet, or PC used by the user.

[0919] Server: Cloud server for data processing and storage.

[0920] Camera and microphone: Devices for capturing your facial expressions and tone of voice in real time.

[0921] software

[0922] Application: Provides an interface for users to define their ideal person image and review their custom-changed behaviors and statements.

[0923] Emotion engine: A software module for analyzing the user's emotional state in real time.

[0924] Generative AI model: An AI model that custom changes a user's behavior and speech based on an ideal image of a human being.

[0925] Data processing and calculation

[0926] Users define their ideal image of a person

[0927] The user launches the application and selects their ideal image of the person they want to be, for example, selecting "a caring person."

[0928] The server collects user behavior data

[0929] The server collects real-time input data, voice data, and text messages from users' smartphones and PCs, including what users chat with friends and what they say to their voice assistants.

[0930] The server analyzes the data and makes custom changes

[0931] The server analyzes the collected data using natural language processing (NLP) techniques. For example, if the user says "thank you" infrequently, the server will suggest that the user say "thank you" more frequently.

[0932] The server presents custom-changed actions and comments to the user

[0933] The server presents the custom-changed actions and comments to the user through the notification function of the application. For example, it sends a notification such as "Try saying 'Thank you' the next time you talk to your friend."

[0934] Users perform repetitive training

[0935] Based on the suggestions from the server, the user is made conscious of saying "thank you" more frequently in their daily lives. By repeating this, the user becomes closer to being a "considerate person."

[0936] Emotion engine analyzes the user's emotional state

[0937] The emotion engine uses the user's smartphone camera and microphone to analyze facial expressions and tone of voice in real time. For example, if a user is smiling while speaking, the emotion engine will determine that the user is feeling happy.

[0938] The server adjusts the training content

[0939] The server adjusts the training content according to the user's emotional state based on the data obtained from the emotion engine. For example, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax."

[0940] Examples

[0941] Setting an ideal human image

[0942] A user opens the application and sets "a compassionate person" as their ideal person.

[0943] Examples of custom changes

[0944] The server suggests that the user say "thank you" more frequently when conversing with friends.

[0945] Sentiment Engine Analysis

[0946] If a user smiles during a conversation, the emotion engine determines that the user is happy.

[0947] Adjusting training content

[0948] If the user is feeling stressed, the server offers deep breathing and meditation exercises.

[0949] Examples of prompt statements

[0950] "If the user sets the ideal person as a 'kind person,' please suggest what kind of actions and words should be custom changed."

[0951] The system supports users' growth by providing them with specific actions and statements to help them become closer to their ideal self, and by providing training tailored to their emotional state.

[0952] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0953] Step 1:

[0954] Users define their ideal image of a person

[0955] The user starts the application and sets the ideal person. Specifically, the user selects the ideal person from options such as "a considerate person" on the application's settings screen and saves the settings. The input is the ideal person selected by the user, and the output is the data of the set ideal person.

[0956] Step 2:

[0957] The server collects user behavior data

[0958] The server collects input data, voice data, and text messages from users' smartphones and PCs in real time. Specifically, it collects what users chat with their friends and what they say to their voice assistants. The input is user behavior data, and the output is a database of collected behavior data.

[0959] Step 3:

[0960] The server analyzes the data and makes custom changes

[0961] The server analyzes the collected data using natural language processing (NLP) technology. Specifically, it uses a generative AI model to change the user's behavior and utterances based on the ideal human image. For example, if the user says "thank you" infrequently, the server will suggest that the user say "thank you" more frequently. The input is the collected behavioral data, and the output is a custom-changed behavior or utterance suggestion.

[0962] Step 4:

[0963] The server presents custom-changed actions and comments to the user

[0964] The server presents the custom-changed actions and utterances to the user using the notification function of the application. Specifically, it sends a notification such as "Try saying 'Thank you' the next time you talk to your friend." The input is the custom-changed action or utterance suggestion, and the output is the notification to the user.

[0965] Step 5:

[0966] Users perform repetitive training

[0967] Based on the suggestions from the server, the user is conscious of saying "thank you" more frequently in their daily lives. Specifically, they increase the opportunities to say "thank you" in conversations with friends and family. The input is the suggestions from the server, and the output is the user's actual behavior.

[0968] Step 6:

[0969] Emotion engine analyzes the user's emotional state

[0970] The emotion engine uses the camera and microphone of the user's smartphone to analyze facial expressions and tone of voice in real time. Specifically, if the user is smiling and talking, the emotion engine determines that the user is feeling happy. The input is the user's facial expressions and tone of voice, and the output is the analyzed emotional state data.

[0971] Step 7:

[0972] The server adjusts the training content

[0973] The server adjusts the training content according to the user's emotional state based on the data obtained from the emotion engine. For example, if the user is feeling stressed, the server provides training such as "Take a deep breath and relax." The input is the analyzed emotional state data, and the output is the adjusted training content.

[0974] (Application example 1)

[0975] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0976] Conventional customer service training systems lacked specific feedback to help staff develop ideal customer service attitudes, and the training content was not adjusted according to the emotional state of the staff, making it difficult to provide effective training.

[0977] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0978] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, a means for analyzing the user's emotional state in real time, a means for adjusting the training content according to the user's emotional state, and a means for providing feedback to the user using a smart device, thereby enabling staff to effectively acquire the ideal customer service attitude.

[0979] A "user" is an individual who uses the system to train themselves to act and speak in a way that reflects their ideal image of a person.

[0980] The "ideal person" is a model of specific behaviors and attitudes that users should aim for.

[0981] A "projection means" is a method or device for visually or aurally presenting the ideal human image set by the user.

[0982] A "means for custom change" is a method or device for changing a user's actions and words based on an ideal image of a person.

[0983] The "means for repetitive training" refers to a method or device for repeatedly practicing custom-changed actions and speech in order for the user to become closer to the ideal image of a person.

[0984] "Means for analyzing emotional state in real time" refers to a method or device for inferring emotions from a user's facial expressions, tone of voice, choice of words, etc.

[0985] A "means for adjusting training content" is a method or device for changing the training content in response to the user's emotional state.

[0986] A "smart device" is a device that has a built-in camera, microphone, speaker, etc. and is used to provide feedback to the user.

[0987] A "means for providing feedback" is a method or device for providing a user with advice or instructions regarding ideal actions or statements.

[0988] As an embodiment of the present invention, a training system is provided for helping staff working in brick-and-mortar stores acquire an ideal customer service attitude. The system includes a means for projecting an ideal human image set by the user, custom-changing the user's actions and remarks, and repeatedly training the user. It also includes a means for analyzing the user's emotional state in real time and adjusting the training content.

[0989] Hardware and Software Used

[0990] Hardware

[0991] Smart glasses: Equipped with a camera, microphone, and speaker, they capture what the user does and says and provide feedback.

[0992] Server: A central processing unit for data processing and analysis.

[0993] software

[0994] OpenCV: A library for capturing and processing camera images.

[0995] EmotionRecognizer: A library for estimating emotions from facial expressions.

[0996] SpeechRecognition: A speech recognition library.

[0997] TextToSpeech: A library for converting text to speech.

[0998] IdealBehaviorModel: A model that generates ideal behaviors and statements.

[0999] Data processing and calculation

[1000] Camera footage capture

[1001] We use the camera in the smart glasses to capture the user's facial expressions and actions in real time. We use OpenCV to acquire the video data and EmotionRecognizer to analyze the emotional state.

[1002] Speech Recognition

[1003] It uses the microphone in the smart glasses to capture what the user is saying, and uses the SpeechRecognition library to convert the speech data into text.

[1004] Feedback Generation

[1005] IdealBehaviorModel is used to generate ideal behaviors and utterances based on the user's emotional state and utterances. The generated feedback is converted to speech using the TextToSpeech library and provided to the user through the smart glasses' speaker.

[1006] Examples

[1007] scenario

[1008] Staff working in physical stores will undergo training to provide "considerate customer service."

[1009] situation

[1010] When customers have questions about products, staff will respond appropriately.

[1011] feedback

[1012] If staff are feeling stressed, the system will provide relaxation advice.

[1013] Examples of prompt statements

[1014] If the user is aiming to be a "caring customer," tell them how to respond to customer questions. If the user is stressed, offer them some relaxation advice.

[1015] In this way, customer service training can be carried out effectively in physical stores.

[1016] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1017] Step 1:

[1018] A user puts on the smart glasses and starts the system. The camera in the smart glasses captures the user's facial expressions and actions in real time, and the microphone captures the user's speech. The input is the camera video and audio data, and the output is the captured video and audio data.

[1019] Step 2:

[1020] The server uses OpenCV to process the video data sent from the smart glasses. Specifically, it extracts the user's facial expressions from the video data and analyzes the user's emotional state using EmotionRecognizer. The input is the captured video data and the output is the user's emotional state.

[1021] Step 3:

[1022] The server uses the SpeechRecognition library to convert the voice data sent from the smart glasses into text. The input is the captured voice data and the output is the text data.

[1023] Step 4:

[1024] The server uses IdealBehaviorModel to generate ideal actions and utterances based on the user's emotional state and text data. The input is the user's emotional state and text data, and the output is the text of the ideal actions and utterances.

[1025] Step 5:

[1026] The server converts the generated text of the ideal actions and utterances into speech using the TextToSpeech library. The input is the text of the ideal actions and utterances, and the output is the speech data.

[1027] Step 6:

[1028] The voice data sent from the server is provided to the user through the speaker of the smart glasses. The input is the voice data, and the output is the voice feedback that the user hears.

[1029] Step 7:

[1030] The user modifies his / her actions or utterances based on the provided feedback and again captures new data through the smart glasses' camera and microphone, which repeats the process from step 1 to step 6. The input is the video and audio data of the modified actions or utterances, and the output is the new feedback.

[1031] Example 2

[1032] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1033] Conventional systems have the problem that it is difficult for users to customize specific actions and comments to get closer to their ideal image of a person, and the training content is not adjusted according to the user's emotional state, making it difficult to achieve effective training. Furthermore, the system lacked the function to analyze the user's emotional state in real time and adjust the training content based on that, making it difficult to get closer to the ideal image while reducing the user's psychological burden.

[1034] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1035] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, a means for analyzing the user's emotional state in real time, and a means for adjusting the training content according to the emotional state. This allows the user to receive effective training to get closer to his or her ideal image.

[1036] "Ideal person" refers to the type of person or personality traits that a user should aspire to.

[1037] "Means of projection" refers to devices or software that visually or audibly display the ideal human image set by the user.

[1038] "Means of custom change" refers to algorithms or programs that change a user's behavior and words based on an ideal image of a person.

[1039] "Means of presentation" refers to devices or software for notifying the user of custom-changed actions or statements.

[1040] "Means of repetitive training" refers to a program or system that allows a user to repeatedly practice custom-changed actions or utterances.

[1041] "Means of analyzing emotional state in real time" refers to devices or software that infer a user's emotional state by analyzing their facial expressions, tone of voice, choice of words, etc.

[1042] "Means for adjusting training content" refers to algorithms or programs for changing the content or method of training in response to the user's emotional state.

[1043] This invention is a system that supports the user in getting closer to his / her ideal image through repetitive training by customizing actions and speech based on the ideal image of the person set by the user. A specific embodiment of this system will be described below.

[1044] System Configuration

[1045] Users define their ideal image of a person

[1046] Using the application on the device, the user sets the ideal person they want to be. For example, they can select "a caring person." This setting takes into account the user's trauma in relationships and education.

[1047] The server collects user actions and comments.

[1048] The servers collect what users do and say in real time, including any text or voice data they type, such as when they talk to friends in a chat app or to a voice assistant.

[1049] Analyze the data collected by the server

[1050] The server analyzes the collected data. Specifically, it uses natural language processing (NLP) techniques to understand what the user is saying. Here, it uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4.

[1051] The server custom-changes actions and comments based on the ideal image of a person.

[1052] Based on the analysis results, the server will customize the user's actions and comments to match the ideal person. For example, "I'm busy today, so I can't help you" will be changed to "I'm busy today, but please let me know if there's anything I can do to help you."

[1053] The server presents the custom changes to the user

[1054] The server presents the custom-changed actions and comments to the user, who can review them and modify them as necessary.

[1055] The server uses an emotion engine to analyze the user's emotional state.

[1056] The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it analyzes the user's facial expressions, tone of voice, choice of words, etc. It uses Microsoft Azure's Emotion API and Affectiva's SDK.

[1057] The server adjusts the training content according to the user's emotional state.

[1058] The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, the server provides training to promote relaxation, and if the user is feeling joy, the server provides training to promote behavior such as sharing that joy.

[1059] The user performs the training

[1060] The user performs the training content presented by the server, and undergoes repetitive training to get closer to their ideal self.

[1061] Examples

[1062] Setting the ideal image of a person

[1063] A user opens the application, selects "Define your ideal person" and chooses "A caring person."

[1064] Collection of actions and statements

[1065] The application collects in the background what you chat with your friends.

[1066] Analyzing data

[1067] The server analyzes chat contents collected by the server using Google Cloud Natural Language API to perform sentiment analysis. It also uses OpenAI's GPT-4 to understand what users say and generate appropriate responses.

[1068] Examples of custom changes

[1069] If the user says "I'm busy today and can't help you," the server changes this to "I'm busy today, but let me know if there's anything I can do to help."

[1070] Sentiment Engine Analysis

[1071] When a user speaks into the camera, the server uses Microsoft Azure's Emotion API to determine whether the user is feeling stressed based on their facial expressions and tone of voice.

[1072] Adjusting training content

[1073] When a user feels stressed, the server provides relaxation training such as "Take a deep breath and relax."

[1074] Running the training

[1075] The user performs the relaxation training presented by the server.

[1076] Examples of prompt statements

[1077] "If a user sets the ideal person as a 'kind person,' how would you customize their actions and words?"

[1078] This system allows users to receive effective training to get closer to their ideal image.

[1079] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1080] Step 1:

[1081] Users define their ideal image of a person

[1082] The user starts the application on the device and sets the ideal person. Specifically, the user selects "Set ideal person" from the application menu and selects the ideal person from options such as "a considerate person." The input is the ideal person selected by the user, and the output is the setting information.

[1083] Step 2:

[1084] The server collects user actions and comments.

[1085] The server collects the user's actions and comments in real time. Specifically, it collects the content of the user's conversations with friends on chat apps and the content of what the user says to the voice assistant. The input is the user's text data and voice data, and the output is the collected data on the actions and comments.

[1086] Step 3:

[1087] Analyze the data collected by the server

[1088] The server analyzes the collected data. Specifically, it uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4 to understand what users say and perform sentiment analysis and key phrase extraction. The input is the collected behavioral and utterance data, and the output is the analysis results.

[1089] Step 4:

[1090] The server custom-changes actions and comments based on the ideal image of a person.

[1091] Based on the analysis results, the server custom-changes the user's actions and comments to match the ideal person. For example, "I'm busy today, so I can't help you" is changed to "I'm busy today, but please let me know if there's anything I can do to help you." The input is the analysis results, and the output is the custom-changed actions and comments.

[1092] Step 5:

[1093] The server presents the custom changes to the user

[1094] The server presents the custom-changed actions and statements to the user, who can then review and modify them as necessary. The input is the custom-changed actions and statements, and the output is the content presented to the user.

[1095] Step 6:

[1096] The server uses an emotion engine to analyze the user's emotional state.

[1097] The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it uses Microsoft Azure's Emotion API and Affectiva's SDK to analyze the user's facial expressions, tone of voice, choice of words, etc. The input is the user's facial expressions, tone of voice, and choice of words, and the output is the analysis of the user's emotional state.

[1098] Step 7:

[1099] The server adjusts the training content according to the user's emotional state.

[1100] The server adjusts the training content according to the user's emotional state. For example, if the user feels stressed, it provides training to promote relaxation, and if the user feels joy, it provides training to promote behavior to share that joy. The input is the analysis result of the emotional state, and the output is the adjusted training content.

[1101] Step 8:

[1102] The user performs the training

[1103] The user executes the training content presented by the server. This allows the user to perform repeated training to approach his / her ideal image. The input is the adjusted training content, and the output is the result of the user's execution.

[1104] (Application example 2)

[1105] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1106] In conventional customer service work, it is difficult for staff to maintain an ideal customer service attitude, and there is a problem of customer satisfaction decreasing, especially when the staff's emotional state is unstable. Also, there is a lack of effective training methods to help staff get closer to their ideal image.

[1107] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1108] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and remarks based on the ideal human image, a means for presenting the custom-changed actions and remarks to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, a means for adjusting the training content based on the emotion engine, and a means for displaying on the smart glasses. This allows staff to receive appropriate training according to their emotional state while maintaining their ideal customer service attitude.

[1109] "User" means an individual or organization that uses the System.

[1110] The "ideal person" is a model of the behavior and attitude that users should strive to be.

[1111] A "means of projection" is a device or software that visually or audibly displays the ideal human image set by the user.

[1112] "Means of custom change" are algorithms or programs that change a user's behavior and words based on an ideal image of a person.

[1113] "Means of presentation" refers to a display device or notification system that notifies the user of custom-changed actions or comments.

[1114] "Means for repeated training" is a function that allows users to repeatedly practice their custom-changed actions and statements in order to get closer to their ideal image of the person they want to be.

[1115] An "emotion engine" is software or hardware for analyzing a user's emotional state in real time.

[1116] The "means for adjusting training content" is a function for changing the content of training provided to the user based on the analysis results of the emotion engine.

[1117] "Smart glasses" are wearable devices that can visually display information to users when worn by the user.

[1118] The system for implementing this invention includes a series of means for projecting an ideal human image set by the user, custom changing the user's actions and speech, and repeatedly training it.Furthermore, it has a function of analyzing the user's emotional state in real time using an emotion engine and adjusting the training content.

[1119] Hardware and Software Used

[1120] Hardware: Smart glasses, camera

[1121] Software: OpenCV (face detection), EmotionRecognizer, BehaviorSuggestion

[1122] System Configuration

[1123] 1. A means to project the ideal image of a person set by the user:

[1124] Users can set their ideal image of the person through the application. For example, if the ideal is "kind and polite customer service," the system will customize the user's behavior and speech based on this information.

[1125] 2. Ways to customise user behaviour and comments:

[1126] The system runs algorithms to change the user's current behavior and speech based on the ideal person, for example making suggestions to promote relaxation if the user is feeling stressed.

[1127] 3. Present custom-changed actions and statements to users and train them repeatedly:

[1128] The customized actions and utterances are displayed on the smart glasses' display, allowing the user to repeatedly practice their ideal actions and utterances.

[1129] 4. Emotion Engine:

[1130] The camera captures the user's facial expressions and uses EmotionRecognizer to analyze the user's emotional state in real time, providing training content according to the user's emotional state.

[1131] 5. Ways to adjust training content:

[1132] Based on the analysis results of the emotion engine, the BehaviorSuggestion engine suggests appropriate actions and statements. For example, if the user is feeling happy, it provides training to encourage behavior such as sharing that happiness.

[1133] 6. Means of display on smart glasses:

[1134] Suggested actions and statements are displayed in real time on the smart glasses' display, helping users maintain the ideal attitude while serving customers.

[1135] Examples

[1136] Scenario: Staff in a brick-and-mortar store are fitted with smart glasses to help them maintain a "friendly and courteous" attitude while serving customers.

[1137] Emotional state: If staff are feeling stressed, suggestions to promote relaxation (e.g., "Take a deep breath and relax").

[1138] Action Suggestion: Suggest a kind statement to customers such as, "Please let me know if you need any help."

[1139] Examples of prompt statements

[1140] Develop an application that changes actions and comments in real time based on the ideal customer service attitude set by the user, and displays them on the smart glasses display. Include a function to analyze the user's emotional state using an emotion engine and provide appropriate training.

[1141] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1142] Step 1:

[1143] The user starts the application and sets the ideal image of the person. The user inputs the ideal image, such as "kind and polite customer service." This input data is sent to the server and saved.

[1144] Step 2:

[1145] The server executes an algorithm that custom-changes actions and utterances based on the user's ideal person image. It receives the ideal person image and the user's current behavior data as input, and generates custom-changed actions and utterances as output.

[1146] Step 3:

[1147] The server sends the customized actions and comments to the smart glasses. The smart glasses receive them and display them on the display. The user can then see them and practice the ideal actions and comments.

[1148] Step 4:

[1149] The camera captures the user's facial expressions and transmits them to the server in real time. The server uses EmotionRecognizer to analyze the user's emotional state from the input facial expression data and generates emotional state data as output.

[1150] Step 5:

[1151] The server adjusts the training content based on the emotional state data. The BehaviorSuggestion engine takes the emotional state data and the ideal human image as input, suggests appropriate actions and utterances, and generates the adjusted training content as output.

[1152] Step 6:

[1153] The server transmits the adjusted training content to the smart glasses, which receive it and display it on the display. The user can then view it and practice the training according to their emotional state.

[1154] Step 7:

[1155] As the user practices the training, the server tracks changes in their emotional state and adjusts the training content as necessary, allowing the user to receive optimal training to become the ideal person they want to be.

[1156] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1157] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1158] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1159] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1160] [Third embodiment]

[1161] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1162] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1163] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1164] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1165] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1166] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1167] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1168] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1169] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1170] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1171] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1172] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1173] "Example 1"

[1174] The system of the present invention includes a means for projecting an ideal human image set by a user, a means for custom-changing the user's actions and comments based on the ideal human image, and a means for presenting the custom-changed actions and comments to the user and repeatedly training them. Specifically, the user sets an ideal human image on an application. For example, if a "compassionate person" is set as the ideal human image, the system custom-changes the user's actions and comments based on the "compassionate person". The system then presents the custom-changed actions and comments to the user and repeatedly trains them, encouraging the user to get closer to their ideal image.

[1175] "Example 2"

[1176] The system of the present invention includes a means for projecting an ideal human image set by a user, a means for custom-changing the user's actions and comments based on the ideal human image, and a means for presenting the custom-changed actions and comments to the user and repeatedly training them. Specifically, the user sets an ideal human image on an application. For example, if a "compassionate person" is set as the ideal human image, the system custom-changes the user's actions and comments based on the "compassionate person". The system then presents the custom-changed actions and comments to the user and repeatedly trains them, encouraging the user to get closer to their ideal image.

[1177] The process flow of each embodiment will be described below.

[1178] "Example 1"

[1179] Step 1: The user sets the ideal person in the application. For example, the user sets "a caring person" as the ideal person.

[1180] Step 2: The system custom-changes the user's behavior and speech based on the ideal image of a person. Specifically, the system analyzes the user's behavior and speech based on the image of a "compassionate person" and generates appropriate behavior and speech.

[1181] Step 3: The system presents the customized actions and statements to the user, for example, providing advice to the user to behave more considerately towards others.

[1182] Step 4: The user repeatedly practices the actions and statements suggested, which allows the user to get closer to their ideal self.

[1183] Example 1

[1184] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1185] In conventional self-improvement systems, it is difficult to custom change specific actions and speech based on the ideal image of a person set by the user, and there is a lack of effective means to record the progress of the repetitive training and provide feedback. Therefore, there is an issue that effective support for users to get closer to their ideal image is not provided.

[1186] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1187] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and comments based on the ideal human image using a generative AI model, a means for presenting the custom-changed actions and comments to the user and training them repeatedly, and a means for recording the progress of the repeated training and providing feedback. This enables the user to learn and practice specific actions and comments to approach their ideal image, and to effectively manage the progress.

[1188] "Ideal person" refers to the specific person image or personality traits that users should aim to be.

[1189] A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate data and custom change user behavior and speech.

[1190] "Custom change" refers to modifying a user's behavior and words based on an ideal image of a person.

[1191] "Repetitive training" refers to the user repeatedly practicing the custom-changed actions or utterances presented to them.

[1192] "Feedback" refers to evaluations and advice provided to a user based on their progress in repeated training.

[1193] "Recording progress" refers to saving the content and results of the repetitive training performed by the user as data.

[1194] "Presenting" refers to showing the user the custom-changed actions or utterances generated by the server.

[1195] This invention is a system that promotes the user to approach his / her ideal image by customizing actions and speech based on the ideal image of a person set by the user and repeatedly training the user. A specific embodiment of this system will be described below.

[1196] Hardware and Software Used

[1197] Hardware

[1198] Device: A device operated by a user, such as a smartphone, computer, or tablet.

[1199] Server: A server for processing data and running generative AI models.

[1200] software

[1201] Application: Software that provides an interface for users to define their ideal person image and review their custom-changed behaviors and statements.

[1202] Generative AI model: An algorithm for customizing a user's actions and speech based on an ideal human image. Specifically, we use generative AI models such as OpenAI's GPT-4.

[1203] Data processing and calculation

[1204] Users define their ideal image of a person

[1205] The user starts the application and sets the ideal image of the person. For example, the user sets "a considerate person" as the ideal image of the person.

[1206] The device sends the configuration data to the server.

[1207] The terminal transmits data of the ideal human image set by the user to the server. This data includes information about the ideal human image selected by the user.

[1208] The server analyzes the data using a generative AI model

[1209] The server uses a generative AI model based on the received data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comments data and custom-changes it based on the ideal image of a person.

[1210] The server generates custom-changed actions and statements

[1211] The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements of a "compassionate person."

[1212] Send the server-generated data to the device

[1213] The server transmits the generated custom-changed behavior and speech data to the terminal, which includes examples of specific behaviors and speech that the user should practice.

[1214] The device presents the data to the user

[1215] The terminal presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

[1216] Users perform repetitive training

[1217] The user then practices the proposed custom-changed actions and utterances, and the application tracks the user's progress and provides feedback as needed.

[1218] Examples

[1219] For example, if a user sets a "compassionate person" as their ideal person, the following prompt sentence is input into the generative AI model.

[1220] Examples of prompt statements

[1221] "Please provide advice to help users be considerate when speaking with friends."

[1222] Based on this prompt, the generative AI model generates specific examples of utterances that users can use when talking to their friends, such as "When a friend is in trouble, say, 'Let me know if there's anything I can do to help you.'"

[1223] In this way, users can learn and practice behaviors and speech based on their ideal image of a person.

[1224] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1225] Step 1:

[1226] Users define their ideal image of a person

[1227] The user launches the application and sets the ideal person by clicking the "Set Ideal Person" button on the application interface, selecting "A considerate person" from the drop-down menu, and pressing the "Set" button.

[1228] Input: User's choice of ideal person (e.g. "a caring person")

[1229] Output: Ideal human image configuration data

[1230] Step 2:

[1231] The device sends the configuration data to the server.

[1232] The device transmits the user's ideal person profile data to a server, where the transmission is secure using the HTTPS protocol.

[1233] Input: Ideal human image configuration data

[1234] Output: Configuration data sent to the server.

[1235] Step 3:

[1236] The server analyzes the data using a generative AI model

[1237] The server uses a generative AI model based on the received configuration data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comment data (e.g., chat logs and SNS posts) and custom changes it based on the ideal image of a person.

[1238] Input: Configuration data, user's past actions and statements

[1239] Output: Custom-changed behavior and speech data

[1240] Step 4:

[1241] The server generates custom-changed actions and statements

[1242] The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements of a "compassionate person."

[1243] Input: Custom-changed action or speech data

[1244] Output: Generated examples of specific actions and statements

[1245] Step 5:

[1246] Send the server-generated data to the device

[1247] The server transmits the generated custom-changed behavior and speech data to the terminal. This transmission is also performed using the HTTPS protocol.

[1248] Input: Examples of specific actions or statements generated

[1249] Output: Data sent to the terminal

[1250] Step 6:

[1251] The device presents the data to the user

[1252] The terminal presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

[1253] Input: Data sent to the terminal

[1254] Output: Examples of specific actions or statements presented to the user

[1255] Step 7:

[1256] Users perform repetitive training

[1257] The user then practices the proposed custom-changed actions and statements, and the application records the user's progress and provides feedback as needed. For example, the user might enter into the application, "Today, I made a kind comment to a friend."

[1258] Input: User's practice results

[1259] Output: Progress data recorded and feedback provided.

[1260] (Application example 1)

[1261] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1262] In conventional customer service training systems, it was difficult to customize specific actions and comments for staff to acquire the ideal customer service attitude, making it difficult to provide effective training. In addition, there was a lack of repetitive training for staff to get closer to the ideal customer service attitude, making it difficult for them to maintain the ideal attitude when actually serving customers.

[1263] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and remarks based on the ideal human image, a means for presenting the custom-changed actions and remarks to the user and repeatedly training them, a means for generating actions and remarks based on the ideal human image using a generative AI model, and a means for presenting the generated actions and remarks to the user as prompt sentences. This enables the staff to concretely understand the ideal customer service attitude and maintain the ideal attitude in actual customer service through repetitive training.

[1264] A "user" is an individual who uses the system to learn to behave and speak in a way that is consistent with the ideal image of a human being.

[1265] The "ideal person" refers to the specific character and attitude that the user should aim for, and serves as the standard by which the system will custom change behavior and speech.

[1266] "Means of projection" is a function that displays the ideal image of a person set by the user visually or as text on the system.

[1267] "Means for custom change" is a function that allows a user to change their current behavior and words based on their ideal image of the person.

[1268] "Means of presentation" is a function for displaying custom-changed actions and comments to the user.

[1269] "Means for repeated training" is a function that allows users to repeatedly practice custom-changed actions and utterances.

[1270] A "generative AI model" is an artificial intelligence model that generates actions and statements based on the ideal human image set by the user.

[1271] A "prompt" is a text-based instruction that presents to the user actions and statements that the generative AI model has generated based on an ideal image of a human being.

[1272] The system for carrying out the present invention customizes actions and speech based on an ideal human image set by the user, and presents the customized changes to the user for repetitive training. A specific embodiment of this system will be described below.

[1273] System Configuration

[1274] The system consists of the following main components:

[1275] 1. User device: A device such as a smartphone or tablet that is used by the user to define their ideal image of the person and check their custom-changed actions and statements.

[1276] 2. Server: Processes user input data and uses a generative AI model to generate actions and statements based on the ideal image of a human.

[1277] 3. Generative AI models: Using generative AI models such as OpenAI's GPT-3, actions and statements are generated based on the ideal human image set by the user.

[1278] Program Processing

[1279] The server receives the ideal human image and current behavior and speech data sent from the user's device, and inputs them as prompts into the generative AI model. The generative AI model generates behavior and speech based on the ideal human image and returns the results to the server. The server then sends the generated behavior and speech to the user's device and presents them to the user.

[1280] Hardware and Software Used

[1281] Hardware: Smartphones, tablets, servers

[1282] Software: Python, OpenAI API, User Interface Application

[1283] Data processing and calculation

[1284] The server sends the user's ideal image of a person and their current actions and comments as prompts to the generative AI model. The generative AI model generates the ideal actions and comments based on the prompts and returns the results to the server. The server then sends the generated actions and comments to the user's device and presents them to the user.

[1285] Examples

[1286] For example, if a user sets "caring customer service" as their ideal person and their current behavior is "treating customers in a rude manner," the prompt text would be as follows:

[1287] Example prompt:

[1288] Ideal person: Attentive customer service

[1289] User behavior: Being rude to customers

[1290] Acting on the Ideal Person:

[1291] Based on this prompt, the generative AI model generates actions and utterances appropriate for "considerate customer service" and presents the results to the user. By repeatedly practicing the actions and utterances presented, the user can get closer to their ideal customer service attitude.

[1292] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1293] Step 1:

[1294] The user sets the ideal person image using the terminal. The user starts the application and inputs the ideal person image (e.g., "caring customer service"). The input data is sent from the terminal to the server.

[1295] Input: Ideal human image

[1296] Output: Ideal human image data sent to the server

[1297] Step 2:

[1298] The server receives the ideal human image data sent by the user. The server stores this data and uses it for the next process.

[1299] Input: Ideal human image data

[1300] Output: Saved ideal human image data

[1301] Step 3:

[1302] The user inputs the current behavior or speech using the terminal. The user inputs the current behavior (e.g., "I treat customers rudely") and sends it from the terminal to the server.

[1303] Input: Current action or statement

[1304] Output: Current action and speech data sent to the server

[1305] Step 4:

[1306] The server receives the current behavior and speech data sent by the user, and stores this data for further processing.

[1307] Input: Current action or speech data

[1308] Output: Saved current actions and speech data

[1309] Step 5:

[1310] The server generates a prompt by combining the stored ideal human image data with the current behavior and speech data. The generated prompt is then input into the generative AI model.

[1311] Input: Ideal person image data, current behavior and speech data

[1312] Output: Generated prompt statement

[1313] Step 6:

[1314] The server sends the generated prompt text to the generative AI model, which then generates the ideal behavior or speech based on the prompt text.

[1315] Input: Generated prompt text

[1316] Output: Ideal actions and statements generated by the generative AI model

[1317] Step 7:

[1318] The server receives the ideal actions and statements returned by the generative AI model, and then transmits this data to the user's device.

[1319] Input: Ideal actions and statements generated by a generative AI model

[1320] Output: Ideal behavior and speech data sent to the user's device

[1321] Step 8:

[1322] The user device receives the ideal behavior and speech data sent from the server and presents it to the user. The user confirms the presented behavior and speech and performs repeated training.

[1323] Input: Ideal behavior and speech data

[1324] Output: The desired actions or statements presented to the user

[1325] Through the above processing steps, the user can learn how to act and speak based on the ideal image of a person, and through repetitive training, come closer to that ideal attitude.

[1326] Example 2

[1327] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1328] In modern society, many people aim to improve themselves and become closer to their ideal image, but the process is difficult and requires continuous effort. In addition, there is a problem that effective training is difficult due to a lack of specific guidelines for actions and statements for self-improvement. Furthermore, there is no appropriate means to customize users' actions and statements based on the ideal image of a person, making it difficult for users to get closer to their ideal image.

[1329] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1330] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and comments based on the ideal human image, a means for presenting the custom-changed actions and comments to the user, a means for the user to repeatedly train the presented actions and comments, a means for collecting the user's actions and comments, and a means for analyzing the collected actions and comments and customizing them using a generation AI model. This allows the user to obtain specific guidelines for actions and comments and to approach their ideal image through continuous training.

[1331] "Ideal person" refers to the specific person image or personality traits that users should aim to be.

[1332] "Means of projection" refers to the function that allows the user to display the ideal image of the person they have set in visual or text form.

[1333] "Means for custom change" refers to the function that allows a user to change their current behavior and words based on their ideal image of the person.

[1334] "Means of presentation" refers to a function for notifying or displaying custom-changed actions or statements to the user.

[1335] "Means of repetitive training" refers to a function that allows the user to train themselves to become closer to their ideal image of a person by repeatedly carrying out suggested actions and statements.

[1336] "Means of collection" refers to the function for recording user actions and statements and storing them as data.

[1337] "Means of analysis" refers to the function of analyzing collected user behavior and comment data and extracting information for custom changes based on an ideal image of a person.

[1338] A "generative AI model" refers to an artificial intelligence model that converts a user's actions and words based on an ideal image of a human being.

[1339] This invention is a system that custom changes actions and speech based on an ideal human image set by a user, presents it to the user, and conducts repetitive training. A specific embodiment of this system will be described below.

[1340] Hardware and Software Used

[1341] Hardware

[1342] Device: The device used by a user, such as a smartphone, tablet, or computer.

[1343] Server: A remote server for processing and storing data.

[1344] software

[1345] Application: Software that allows users to define their ideal person image and check their custom-changed behavior and statements.

[1346] Generative AI model: An artificial intelligence model (e.g., GPT-4) that transforms user actions and statements based on an ideal image of a human being.

[1347] System Operation

[1348] Setting the ideal image of a person

[1349] The user starts the application and goes to the settings screen. There, they input their ideal image of the person they want to be. For example, they might input "a caring person."

[1350] Collection of user actions and comments

[1351] The device records the user's daily actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media. The collected data is sent to a server in real time.

[1352] Custom changes to actions and statements

[1353] The server analyzes the received user behavior and speech data. Using a generative AI model (e.g. GPT-4), the user's behavior and speech are custom-changed based on the ideal image of a person. For example, if a user says, "I'm busy today, so I can't help you," the generative AI model converts this to "I'm busy today, but please let me know if there's anything I can do to help you."

[1354] Displaying custom-changed actions and statements

[1355] The server transmits the custom-changed actions and utterances to the terminal. The terminal notifies the user of the change. The user confirms the notification and actually uses the custom-changed actions and utterances.

[1356] Repetitive training

[1357] The user repeatedly executes the customized actions and statements suggested to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts. In this way, the user trains to become closer to their ideal image of a person.

[1358] Examples

[1359] Setting the ideal image of a person

[1360] The user enters "compassionate person" in the application's settings screen and clicks the save button.

[1361] Collection of user actions and comments

[1362] The device will record the voice of users' conversations with friends and convert them into text, and also collect text data posted by users on social media.

[1363] Custom changes to actions and statements

[1364] The server inputs the collected text data into a generative AI model, which uses prompts to transform the user's speech into an ideal human image.

[1365] Displaying custom-changed actions and statements

[1366] The server sends the customized utterances output by the generative AI model to the device, which then displays a notification to the user and suggests something like, "I'm busy today, but please let me know if there's anything I can do to help you."

[1367] Repetitive training

[1368] The user uses the utterances suggested by the server in daily conversations and social media posts. For example, in a conversation with a friend, the user might say, "I'm busy today, but please let me know if there's anything I can do to help you." By repeating this, the user will naturally be able to act and speak as a "considerate person."

[1369] Examples of prompt statements

[1370] Here is an example of a prompt to be input to the generative AI model:

[1371] User currently says: "I'm busy today, I can't help you."

[1372] Ideal person: "A considerate person"

[1373] Prompt for generative AI model: "Transform the user's utterance into a compassionate person's utterance."

[1374] By inputting this prompt into a generative AI model, the user's speech is custom altered based on the ideal image of a person.

[1375] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1376] Step 1:

[1377] The user starts the application and moves to the setting screen. There, the user inputs the ideal image of the person. For example, the user inputs "a considerate person."

[1378] Input: The ideal person image entered by the user on the settings screen.

[1379] Output: The ideal person image is saved in the application.

[1380] Step 2:

[1381] The device records the user's daily actions and comments, for example, the contents of conversations the user has had with friends and the contents of posts made to social networking sites.

[1382] Input: What the user does or says.

[1383] Output: Collected behavioral and speech data is stored on the device and transmitted to the server in real time.

[1384] Step 3:

[1385] The server analyzes the received user behavior and speech data, and uses a generative AI model (e.g. GPT-4) to custom-change the user's behavior and speech based on the ideal image of a person.

[1386] Input: Collected behavioral and speech data, ideal human image.

[1387] Output: Custom changed actions and statements.

[1388] Specific operation: The server inputs the collected text data into the generative AI model and converts the user's utterances using prompt sentences.

[1389] Step 4:

[1390] The server transmits the custom-changed actions and utterances to the terminal. The terminal notifies the user of the change. The user confirms the notification and actually uses the custom-changed actions and utterances.

[1391] Input: Custom-changed actions and statements.

[1392] Output: The custom changed actions or statements presented to the user.

[1393] Specific operation: The device displays the custom-changed message received from the server as a notification.

[1394] Step 5:

[1395] The user repeatedly executes the proposed custom-changed actions and utterances, for example, using utterances suggested by the generative AI model in daily conversations and social media posts.

[1396] Input: The custom-changed actions or statements presented.

[1397] Output: Custom changes to actions and statements made by the user.

[1398] Specific operation: The user uses utterances suggested by the server in conversations with friends and in social media posts. By repeating this process, the user trains to become closer to the ideal person.

[1399] (Application example 2)

[1400] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1401] In conventional customer service work, there were limited training methods for staff to acquire the ideal customer service attitude, and it was difficult to provide effective training. In addition, there was a lack of specific guidelines and feedback for staff to practice the ideal customer service attitude, making it difficult to improve customer service quality. Furthermore, there was a need for an efficient method of repeated training to help staff get closer to their ideal image.

[1402] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1403] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and utterances based on the ideal human image, a means for presenting the custom-changed actions and utterances to the user and repeatedly training them, a means for training the ideal customer service attitude using smart glasses worn by the staff performing customer service work, and a means for generating custom-changed utterances based on the ideal human image using a generation AI model. This enables the staff to efficiently train and practice the ideal customer service attitude.

[1404] "User" means an individual or organization that uses the System.

[1405] The "ideal person" is a model of the behavior and attitude that users should strive to be.

[1406] A "projection means" is a device or software for visually or audibly displaying the ideal human image set by the user.

[1407] A "means of custom change" is a device or software that changes a user's behavior and speech based on an ideal image of a person.

[1408] A "means for presenting" is a device or software for displaying custom-changed actions or utterances to a user.

[1409] A "means for repeated training" is a device or software that allows a user to repeatedly practice custom-changed actions or utterances.

[1410] "Customer service" refers to the provision of products and services to customers.

[1411] "Smart glasses" are glasses-type devices that have the function of displaying information.

[1412] A "generative AI model" is an algorithm that uses artificial intelligence to generate text and actions.

[1413] "Custom-changed statements" are statements made by users that have been altered based on their ideal image of a person.

[1414] The system for implementing this invention projects the ideal human image set by the user, custom changes the user's actions and remarks, and repeatedly trains them. Specifically, the ideal customer service attitude can be trained using smart glasses worn by staff who serve customers.

[1415] System Configuration

[1416] The system consists of the following main components:

[1417] 1. User device: The device on which the user sets their ideal image. This can be a smartphone or tablet.

[1418] 2. Smart glasses: A device worn by staff performing customer service duties that has the function of visually displaying information.

[1419] 3. Server: Runs the generative AI model and processes data to customise user behaviour and speech.

[1420] 4. Generative AI model: An algorithm that generates custom-changed utterances based on the user’s ideal image of a person.

[1421] Program Processing

[1422] The server receives the ideal person image set by the user and custom changes the user's behavior and speech based on that image. Specifically, the process is as follows:

[1423] 1. Setting the ideal person image: Using the user terminal, the user sets the ideal person image. For example, the ideal image can be set as "kind and polite customer service."

[1424] 2. Enter user utterances: Enter actual utterances made by users. For example, "Welcome, is there anything you're looking for?"

[1425] 3. Executing the generative AI model: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image.

[1426] 4. Present customized utterances: Present customized utterances to the user through smart glasses, for example, "Welcome, we will help you find the product you are looking for."

[1427] 5. Repetitive training: The user repeatedly practices the custom-changed utterances presented to them.

[1428] Hardware and Software Used

[1429] Hardware: Smart glasses (e.g. Google Glass), user devices (smartphones, tablets)

[1430] Software: Generative AI models (e.g. OpenAI API), data processing servers

[1431] Examples

[1432] For example, the ideal customer service staff member may be described as "friendly and polite" and the following prompt may be used:

[1433] Example prompt:

[1434] Ideal person: Kind and polite customer service

[1435] User says: Hey there, is there anything you're looking for?

[1436] Custom statements based on the ideal person:

[1437] By inputting this prompt into the generative AI model, a custom-changed utterance based on the ideal customer service attitude is generated, for example, "Welcome, we will help you find the product you are looking for."

[1438] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1439] Step 1:

[1440] The user sets the ideal image of the person.

[1441] Input: The user inputs the ideal image of a person using a device (smartphone or tablet). For example, "kind and polite customer service."

[1442] Data processing: The terminal sends the inputted ideal image of a person to the server.

[1443] Output: The ideal person image is saved on the server.

[1444] Step 2:

[1445] The user types in the actual utterance.

[1446] Input: What the user actually says when using the terminal, for example, "Hello, is there something you're looking for?"

[1447] Data processing: The terminal sends the entered comments to the server.

[1448] Output: The user's comments are saved on the server.

[1449] Step 3:

[1450] The server runs the generative AI model.

[1451] Input: The server retrieves the stored ideal person profile and the user's statements.

[1452] Data calculation: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image. The prompt sentence is input to the generative AI model to obtain the custom-changed utterances.

[1453] Output: The generated custom-changed utterances are saved on the server.

[1454] Step 4:

[1455] The custom-changed statement is presented to the user.

[1456] Input: The server gets the generated custom changed utterance.

[1457] Data processing: The server sends the custom-changed utterances to the smart glasses.

[1458] Output: The custom-changed speech is displayed on the smart glasses.

[1459] Step 5:

[1460] The user performs repetitive training.

[1461] Input: The user sees the custom-changed utterance displayed on the smart glasses.

[1462] Specific behavior: The user repeatedly practices the utterances shown to them, for example, "Welcome, we are happy to help you find the product you are looking for."

[1463] Output: The user's customer service attitude becomes closer to the ideal person.

[1464] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1465] "Example 1"

[1466] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine analyzes the user's emotional state in real time. Specifically, the emotion state is estimated from the user's facial expression, tone of voice, choice of words, etc. In addition, this emotion engine tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training for the user to approach his / her ideal image according to the user's emotional state. For example, when the user is feeling stressed, training that promotes relaxation is provided. On the other hand, when the user is feeling joy, training that promotes behavior to share that joy is provided. This allows the user to train to approach his / her ideal image in a manner appropriate to his / her emotional state.

[1467] "Example 2"

[1468] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine analyzes the user's emotional state in real time. Specifically, the emotion state is estimated from the user's facial expression, tone of voice, choice of words, etc. In addition, this emotion engine tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training for the user to approach his / her ideal image according to the user's emotional state. For example, when the user is feeling stressed, training that promotes relaxation is provided. On the other hand, when the user is feeling joy, training that promotes behavior to share that joy is provided. This allows the user to train to approach his / her ideal image in a manner appropriate to his / her emotional state.

[1469] The process flow of each embodiment will be described below.

[1470] "Example 1"

[1471] Step 1: The user boots up the system and sets their ideal self. This ideal self specifies how the user will act and speak.

[1472] Step 2: The emotion engine analyzes the user's emotional state in real time, inferring it from the user's facial expressions, tone of voice, choice of words, etc.

[1473] Step 3: The emotion engine tracks changes in the user's emotional state, allowing it to tailor the iterative training to help the user achieve their ideal self.

[1474] Step 4: The user undergoes repetitive training to get closer to their ideal self. This training is adjusted according to the user's emotional state. For example, when the user is feeling stressed, training is provided to promote relaxation. On the other hand, when the user is feeling joy, training is provided to encourage behavior that encourages sharing that joy.

[1475] Example 1

[1476] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1477] With conventional systems, it was difficult to custom change behavior and speech based on the ideal person image set by the user, and the training content was not adjusted according to the user's emotional state, so there was an issue of a lack of effective support to help users get closer to their ideal image.

[1478] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1479] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, and a means for adjusting the training content based on data obtained from the emotion engine. This makes it possible to effectively support the user in getting closer to his or her ideal image.

[1480] A "user" refers to an individual who aims to use the system to become closer to their ideal image of a person.

[1481] An "ideal person" is a specific definition of the personality and behavioral characteristics that a user aspires to be.

[1482] "Means of projection" refers to the ability to display the user's ideal image of the person on the system in visual or text form.

[1483] "Means of custom change" refers to the function that changes and adjusts a user's actions and comments based on an ideal image of a person.

[1484] "Means of presentation" refers to the function of notifying or displaying custom-changed actions or comments to the user.

[1485] "Means for repeated training" refers to a function that allows the user to repeatedly practice based on the custom changes presented to them.

[1486] The "emotion engine" refers to a function that analyzes a user's emotional state in real time based on their facial expressions, tone of voice, choice of words, etc.

[1487] "Means for adjusting training content" refers to a function for changing and adjusting the training content according to the user's emotional state based on data obtained from the emotion engine.

[1488] This invention is a system that supports the user in getting closer to his / her ideal image through repetitive training by customizing actions and speech based on the ideal image of the person set by the user. A specific embodiment of this system will be described below.

[1489] Hardware and Software Used

[1490] Hardware

[1491] Device: The smartphone, tablet, or PC used by the user.

[1492] Server: Cloud server for data processing and storage.

[1493] Camera and microphone: Devices for capturing your facial expressions and tone of voice in real time.

[1494] software

[1495] Application: Provides an interface for users to define their ideal person image and review their custom-changed behaviors and statements.

[1496] Emotion engine: A software module for analyzing the user's emotional state in real time.

[1497] Generative AI model: An AI model that custom changes a user's behavior and speech based on an ideal image of a human being.

[1498] Data processing and calculation

[1499] Users define their ideal image of a person

[1500] The user launches the application and selects their ideal image of the person they want to be, for example, selecting "a caring person."

[1501] The server collects user behavior data

[1502] The server collects real-time input data, voice data, and text messages from users' smartphones and PCs, including what users chat with friends and what they say to their voice assistants.

[1503] The server analyzes the data and makes custom changes

[1504] The server analyzes the collected data using natural language processing (NLP) techniques. For example, if the user says "thank you" infrequently, the server will suggest that the user say "thank you" more frequently.

[1505] The server presents custom-changed actions and comments to the user

[1506] The server presents the custom-changed actions and comments to the user through the notification function of the application. For example, it sends a notification such as "Try saying 'Thank you' the next time you talk to your friend."

[1507] Users perform repetitive training

[1508] Based on the suggestions from the server, the user is made conscious of saying "thank you" more frequently in their daily lives. By repeating this, the user becomes closer to being a "considerate person."

[1509] Emotion engine analyzes the user's emotional state

[1510] The emotion engine uses the user's smartphone camera and microphone to analyze facial expressions and tone of voice in real time. For example, if a user is smiling while speaking, the emotion engine will determine that the user is feeling happy.

[1511] The server adjusts the training content

[1512] The server adjusts the training content according to the user's emotional state based on the data obtained from the emotion engine. For example, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax."

[1513] Examples

[1514] Setting an ideal human image

[1515] A user opens the application and sets "a compassionate person" as their ideal person.

[1516] Examples of custom changes

[1517] The server suggests that the user say "thank you" more frequently when conversing with friends.

[1518] Sentiment Engine Analysis

[1519] If a user smiles during a conversation, the emotion engine determines that the user is happy.

[1520] Adjusting training content

[1521] If the user is feeling stressed, the server offers deep breathing and meditation exercises.

[1522] Examples of prompt statements

[1523] "If the user sets the ideal person as a 'kind person,' please suggest what kind of actions and words should be custom changed."

[1524] The system supports users' growth by providing them with specific actions and statements to help them become closer to their ideal self, and by providing training tailored to their emotional state.

[1525] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1526] Step 1:

[1527] Users define their ideal image of a person

[1528] The user starts the application and sets the ideal person. Specifically, the user selects the ideal person from options such as "a considerate person" on the application's settings screen and saves the settings. The input is the ideal person selected by the user, and the output is the data of the set ideal person.

[1529] Step 2:

[1530] The server collects user behavior data

[1531] The server collects input data, voice data, and text messages from users' smartphones and PCs in real time. Specifically, it collects what users chat with their friends and what they say to their voice assistants. The input is user behavior data, and the output is a database of collected behavior data.

[1532] Step 3:

[1533] The server analyzes the data and makes custom changes

[1534] The server analyzes the collected data using natural language processing (NLP) technology. Specifically, it uses a generative AI model to change the user's behavior and utterances based on the ideal human image. For example, if the user says "thank you" infrequently, the server will suggest that the user say "thank you" more frequently. The input is the collected behavioral data, and the output is a custom-changed behavior or utterance suggestion.

[1535] Step 4:

[1536] The server presents custom-changed actions and comments to the user

[1537] The server presents the custom-changed actions and utterances to the user using the notification function of the application. Specifically, it sends a notification such as "Try saying 'Thank you' the next time you talk to your friend." The input is the custom-changed action or utterance suggestion, and the output is the notification to the user.

[1538] Step 5:

[1539] Users perform repetitive training

[1540] Based on the suggestions from the server, the user is conscious of saying "thank you" more frequently in their daily lives. Specifically, they increase the opportunities to say "thank you" in conversations with friends and family. The input is the suggestions from the server, and the output is the user's actual behavior.

[1541] Step 6:

[1542] Emotion engine analyzes the user's emotional state

[1543] The emotion engine uses the camera and microphone of the user's smartphone to analyze facial expressions and tone of voice in real time. Specifically, if the user is smiling and talking, the emotion engine determines that the user is feeling happy. The input is the user's facial expressions and tone of voice, and the output is the analyzed emotional state data.

[1544] Step 7:

[1545] The server adjusts the training content

[1546] The server adjusts the training content according to the user's emotional state based on the data obtained from the emotion engine. For example, if the user is feeling stressed, the server provides training such as "Take a deep breath and relax." The input is the analyzed emotional state data, and the output is the adjusted training content.

[1547] (Application example 1)

[1548] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1549] Conventional customer service training systems lacked specific feedback to help staff develop ideal customer service attitudes, and the training content was not adjusted according to the emotional state of the staff, making it difficult to provide effective training.

[1550] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1551] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, a means for analyzing the user's emotional state in real time, a means for adjusting the training content according to the user's emotional state, and a means for providing feedback to the user using a smart device, thereby enabling staff to effectively acquire the ideal customer service attitude.

[1552] A "user" is an individual who uses the system to train themselves to act and speak in a way that reflects their ideal image of a person.

[1553] The "ideal person" is a model of specific behaviors and attitudes that users should aim for.

[1554] A "projection means" is a method or device for visually or aurally presenting the ideal human image set by the user.

[1555] A "means for custom change" is a method or device for changing a user's actions and words based on an ideal image of a person.

[1556] The "means for repetitive training" refers to a method or device for repeatedly practicing custom-changed actions and speech in order for the user to become closer to the ideal image of a person.

[1557] "Means for analyzing emotional state in real time" refers to a method or device for inferring emotions from a user's facial expressions, tone of voice, choice of words, etc.

[1558] A "means for adjusting training content" is a method or device for changing the training content in response to the user's emotional state.

[1559] A "smart device" is a device that has a built-in camera, microphone, speaker, etc. and is used to provide feedback to the user.

[1560] A "means for providing feedback" is a method or device for providing a user with advice or instructions regarding ideal actions or statements.

[1561] As an embodiment of the present invention, a training system is provided for helping staff working in brick-and-mortar stores acquire an ideal customer service attitude. The system includes a means for projecting an ideal human image set by the user, custom-changing the user's actions and remarks, and repeatedly training the user. It also includes a means for analyzing the user's emotional state in real time and adjusting the training content.

[1562] Hardware and Software Used

[1563] Hardware

[1564] Smart glasses: Equipped with a camera, microphone, and speaker, they capture what the user does and says and provide feedback.

[1565] Server: A central processing unit for data processing and analysis.

[1566] software

[1567] OpenCV: A library for capturing and processing camera images.

[1568] EmotionRecognizer: A library for estimating emotions from facial expressions.

[1569] SpeechRecognition: A speech recognition library.

[1570] TextToSpeech: A library for converting text to speech.

[1571] IdealBehaviorModel: A model that generates ideal behaviors and statements.

[1572] Data processing and calculation

[1573] Camera footage capture

[1574] We use the camera in the smart glasses to capture the user's facial expressions and actions in real time. We use OpenCV to acquire the video data and EmotionRecognizer to analyze the emotional state.

[1575] Speech Recognition

[1576] It uses the microphone in the smart glasses to capture what the user is saying, and uses the SpeechRecognition library to convert the speech data into text.

[1577] Feedback Generation

[1578] IdealBehaviorModel is used to generate ideal behaviors and utterances based on the user's emotional state and utterances. The generated feedback is converted to speech using the TextToSpeech library and provided to the user through the smart glasses' speaker.

[1579] Examples

[1580] scenario

[1581] Staff working in physical stores will undergo training to provide "considerate customer service."

[1582] situation

[1583] When customers have questions about products, staff will respond appropriately.

[1584] feedback

[1585] If staff are feeling stressed, the system will provide relaxation advice.

[1586] Examples of prompt statements

[1587] If the user is aiming to be a "caring customer," tell them how to respond to customer questions. If the user is stressed, offer them some relaxation advice.

[1588] In this way, customer service training can be carried out effectively in physical stores.

[1589] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1590] Step 1:

[1591] A user puts on the smart glasses and starts the system. The camera in the smart glasses captures the user's facial expressions and actions in real time, and the microphone captures the user's speech. The input is the camera video and audio data, and the output is the captured video and audio data.

[1592] Step 2:

[1593] The server uses OpenCV to process the video data sent from the smart glasses. Specifically, it extracts the user's facial expressions from the video data and analyzes the user's emotional state using EmotionRecognizer. The input is the captured video data and the output is the user's emotional state.

[1594] Step 3:

[1595] The server uses the SpeechRecognition library to convert the voice data sent from the smart glasses into text. The input is the captured voice data and the output is the text data.

[1596] Step 4:

[1597] The server uses IdealBehaviorModel to generate ideal actions and utterances based on the user's emotional state and text data. The input is the user's emotional state and text data, and the output is the text of the ideal actions and utterances.

[1598] Step 5:

[1599] The server converts the generated text of the ideal actions and utterances into speech using the TextToSpeech library. The input is the text of the ideal actions and utterances, and the output is the speech data.

[1600] Step 6:

[1601] The voice data sent from the server is provided to the user through the speaker of the smart glasses. The input is the voice data, and the output is the voice feedback that the user hears.

[1602] Step 7:

[1603] The user modifies his / her actions or utterances based on the provided feedback and again captures new data through the smart glasses' camera and microphone, which repeats the process from step 1 to step 6. The input is the video and audio data of the modified actions or utterances, and the output is the new feedback.

[1604] Example 2

[1605] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1606] Conventional systems have the problem that it is difficult for users to customize specific actions and comments to get closer to their ideal image of a person, and the training content is not adjusted according to the user's emotional state, making it difficult to achieve effective training. Furthermore, the system lacked the function to analyze the user's emotional state in real time and adjust the training content based on that, making it difficult to get closer to the ideal image while reducing the user's psychological burden.

[1607] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1608] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, a means for analyzing the user's emotional state in real time, and a means for adjusting the training content according to the emotional state. This allows the user to receive effective training to get closer to his or her ideal image.

[1609] "Ideal person" refers to the type of person or personality traits that a user should aspire to.

[1610] "Means of projection" refers to devices or software that visually or audibly display the ideal human image set by the user.

[1611] "Means of custom change" refers to algorithms or programs that change a user's behavior and words based on an ideal image of a person.

[1612] "Means of presentation" refers to devices or software for notifying the user of custom-changed actions or statements.

[1613] "Means of repetitive training" refers to a program or system that allows a user to repeatedly practice custom-changed actions or utterances.

[1614] "Means of analyzing emotional state in real time" refers to devices or software that infer a user's emotional state by analyzing their facial expressions, tone of voice, choice of words, etc.

[1615] "Means for adjusting training content" refers to algorithms or programs for changing the content or method of training in response to the user's emotional state.

[1616] This invention is a system that supports the user in getting closer to his / her ideal image through repetitive training by customizing actions and speech based on the ideal image of the person set by the user. A specific embodiment of this system will be described below.

[1617] System Configuration

[1618] Users define their ideal image of a person

[1619] Using the application on the device, the user sets the ideal person they want to be. For example, they can select "a caring person." This setting takes into account the user's trauma in relationships and education.

[1620] The server collects user actions and comments.

[1621] The servers collect what users do and say in real time, including any text or voice data they type, such as when they talk to friends in a chat app or to a voice assistant.

[1622] Analyze the data collected by the server

[1623] The server analyzes the collected data. Specifically, it uses natural language processing (NLP) techniques to understand what the user is saying. Here, it uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4.

[1624] The server custom-changes actions and comments based on the ideal image of a person.

[1625] Based on the analysis results, the server will customize the user's actions and comments to match the ideal person. For example, "I'm busy today, so I can't help you" will be changed to "I'm busy today, but please let me know if there's anything I can do to help you."

[1626] The server presents the custom changes to the user

[1627] The server presents the custom-changed actions and comments to the user, who can review them and modify them as necessary.

[1628] The server uses an emotion engine to analyze the user's emotional state.

[1629] The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it analyzes the user's facial expressions, tone of voice, choice of words, etc. It uses Microsoft Azure's Emotion API and Affectiva's SDK.

[1630] The server adjusts the training content according to the user's emotional state.

[1631] The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, the server provides training to promote relaxation, and if the user is feeling joy, the server provides training to promote behavior such as sharing that joy.

[1632] The user performs the training

[1633] The user performs the training content presented by the server, and undergoes repetitive training to get closer to their ideal self.

[1634] Examples

[1635] Setting the ideal image of a person

[1636] A user opens the application, selects "Define your ideal person" and chooses "A caring person."

[1637] Collection of actions and statements

[1638] The application collects in the background what you chat with your friends.

[1639] Analyzing data

[1640] The server analyzes chat contents collected by the server using Google Cloud Natural Language API to perform sentiment analysis. It also uses OpenAI's GPT-4 to understand what users say and generate appropriate responses.

[1641] Examples of custom changes

[1642] If the user says "I'm busy today and can't help you," the server changes this to "I'm busy today, but let me know if there's anything I can do to help."

[1643] Sentiment Engine Analysis

[1644] When a user speaks into the camera, the server uses Microsoft Azure's Emotion API to determine whether the user is feeling stressed based on their facial expressions and tone of voice.

[1645] Adjusting training content

[1646] When a user feels stressed, the server provides relaxation training such as "Take a deep breath and relax."

[1647] Running the training

[1648] The user performs the relaxation training presented by the server.

[1649] Examples of prompt statements

[1650] "If a user sets the ideal person as a 'kind person,' how would you customize their actions and words?"

[1651] This system allows users to receive effective training to get closer to their ideal image.

[1652] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1653] Step 1:

[1654] Users define their ideal image of a person

[1655] The user starts the application on the device and sets the ideal person. Specifically, the user selects "Set ideal person" from the application menu and selects the ideal person from options such as "a considerate person." The input is the ideal person selected by the user, and the output is the setting information.

[1656] Step 2:

[1657] The server collects user actions and comments.

[1658] The server collects the user's actions and comments in real time. Specifically, it collects the content of the user's conversations with friends on chat apps and the content of what the user says to the voice assistant. The input is the user's text data and voice data, and the output is the collected data on the actions and comments.

[1659] Step 3:

[1660] Analyze the data collected by the server

[1661] The server analyzes the collected data. Specifically, it uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4 to understand what users say and perform sentiment analysis and key phrase extraction. The input is the collected behavioral and utterance data, and the output is the analysis results.

[1662] Step 4:

[1663] The server custom-changes actions and comments based on the ideal image of a person.

[1664] Based on the analysis results, the server custom-changes the user's actions and comments to match the ideal person. For example, "I'm busy today, so I can't help you" is changed to "I'm busy today, but please let me know if there's anything I can do to help you." The input is the analysis results, and the output is the custom-changed actions and comments.

[1665] Step 5:

[1666] The server presents the custom changes to the user

[1667] The server presents the custom-changed actions and statements to the user, who can then review and modify them as necessary. The input is the custom-changed actions and statements, and the output is the content presented to the user.

[1668] Step 6:

[1669] The server uses an emotion engine to analyze the user's emotional state.

[1670] The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it uses Microsoft Azure's Emotion API and Affectiva's SDK to analyze the user's facial expressions, tone of voice, choice of words, etc. The input is the user's facial expressions, tone of voice, and choice of words, and the output is the analysis of the user's emotional state.

[1671] Step 7:

[1672] The server adjusts the training content according to the user's emotional state.

[1673] The server adjusts the training content according to the user's emotional state. For example, if the user feels stressed, it provides training to promote relaxation, and if the user feels joy, it provides training to promote behavior to share that joy. The input is the analysis result of the emotional state, and the output is the adjusted training content.

[1674] Step 8:

[1675] The user performs the training

[1676] The user executes the training content presented by the server. This allows the user to perform repeated training to approach his / her ideal image. The input is the adjusted training content, and the output is the result of the user's execution.

[1677] (Application example 2)

[1678] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1679] In conventional customer service work, it is difficult for staff to maintain an ideal customer service attitude, and there is a problem of customer satisfaction decreasing, especially when the staff's emotional state is unstable. Also, there is a lack of effective training methods to help staff get closer to their ideal image.

[1680] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1681] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and remarks based on the ideal human image, a means for presenting the custom-changed actions and remarks to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, a means for adjusting the training content based on the emotion engine, and a means for displaying on the smart glasses. This allows staff to receive appropriate training according to their emotional state while maintaining their ideal customer service attitude.

[1682] "User" means an individual or organization that uses the System.

[1683] The "ideal person" is a model of the behavior and attitude that users should strive to be.

[1684] A "means of projection" is a device or software that visually or audibly displays the ideal human image set by the user.

[1685] "Means of custom change" are algorithms or programs that change a user's behavior and words based on an ideal image of a person.

[1686] "Means of presentation" refers to a display device or notification system that notifies the user of custom-changed actions or comments.

[1687] "Means for repeated training" is a function that allows users to repeatedly practice their custom-changed actions and statements in order to get closer to their ideal image of the person they want to be.

[1688] An "emotion engine" is software or hardware for analyzing a user's emotional state in real time.

[1689] The "means for adjusting training content" is a function for changing the content of training provided to the user based on the analysis results of the emotion engine.

[1690] "Smart glasses" are wearable devices that can visually display information to users when worn by the user.

[1691] The system for implementing this invention includes a series of means for projecting an ideal human image set by the user, custom changing the user's actions and speech, and repeatedly training it.Furthermore, it has a function of analyzing the user's emotional state in real time using an emotion engine and adjusting the training content.

[1692] Hardware and Software Used

[1693] Hardware: Smart glasses, camera

[1694] Software: OpenCV (face detection), EmotionRecognizer, BehaviorSuggestion

[1695] System Configuration

[1696] 1. A means to project the ideal image of a person set by the user:

[1697] Users can set their ideal image of the person through the application. For example, if the ideal is "kind and polite customer service," the system will customize the user's behavior and speech based on this information.

[1698] 2. Ways to customise user behaviour and comments:

[1699] The system runs algorithms to change the user's current behavior and speech based on the ideal person, for example making suggestions to promote relaxation if the user is feeling stressed.

[1700] 3. Present custom-changed actions and statements to users and train them repeatedly:

[1701] The customized actions and utterances are displayed on the smart glasses' display, allowing the user to repeatedly practice their ideal actions and utterances.

[1702] 4. Emotion Engine:

[1703] The camera captures the user's facial expressions and uses EmotionRecognizer to analyze the user's emotional state in real time, providing training content according to the user's emotional state.

[1704] 5. Ways to adjust training:

[1705] Based on the analysis results of the emotion engine, the BehaviorSuggestion engine suggests appropriate actions and statements. For example, if the user is feeling happy, it provides training to encourage behavior such as sharing that happiness.

[1706] 6. Means of display on smart glasses:

[1707] Suggested actions and statements are displayed in real time on the smart glasses' display, helping users maintain the ideal attitude while serving customers.

[1708] Examples

[1709] Scenario: Staff in a brick-and-mortar store are fitted with smart glasses to help them maintain a "friendly and courteous" attitude while serving customers.

[1710] Emotional state: If staff are feeling stressed, suggestions to promote relaxation (e.g., "Take a deep breath and relax").

[1711] Action Suggestion: Suggest a kind statement to customers such as, "Please let me know if you need any help."

[1712] Examples of prompt statements

[1713] Develop an application that changes actions and comments in real time based on the ideal customer service attitude set by the user, and displays them on the smart glasses display. Include a function to analyze the user's emotional state using an emotion engine and provide appropriate training.

[1714] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1715] Step 1:

[1716] The user starts the application and sets the ideal image of the person. The user inputs the ideal image, such as "kind and polite customer service." This input data is sent to the server and saved.

[1717] Step 2:

[1718] The server executes an algorithm that custom-changes actions and utterances based on the user's ideal person image. It receives the ideal person image and the user's current behavior data as input, and generates custom-changed actions and utterances as output.

[1719] Step 3:

[1720] The server sends the customized actions and comments to the smart glasses. The smart glasses receive them and display them on the display. The user can then see them and practice the ideal actions and comments.

[1721] Step 4:

[1722] The camera captures the user's facial expressions and transmits them to the server in real time. The server uses EmotionRecognizer to analyze the user's emotional state from the input facial expression data and generates emotional state data as output.

[1723] Step 5:

[1724] The server adjusts the training content based on the emotional state data. The BehaviorSuggestion engine takes the emotional state data and the ideal human image as input, suggests appropriate actions and utterances, and generates the adjusted training content as output.

[1725] Step 6:

[1726] The server transmits the adjusted training content to the smart glasses, which receive it and display it on the display. The user can then view it and practice the training according to their emotional state.

[1727] Step 7:

[1728] As the user practices the training, the server tracks changes in their emotional state and adjusts the training content as necessary, allowing the user to receive optimal training to become the ideal person they want to be.

[1729] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1730] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1731] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1732] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1733] [Fourth embodiment]

[1734] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1735] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1736] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1737] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1738] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1739] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1740] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1741] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1742] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1743] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1744] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1745] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1746] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1747] "Example 1"

[1748] The system of the present invention includes a means for projecting an ideal human image set by a user, a means for custom-changing the user's actions and comments based on the ideal human image, and a means for presenting the custom-changed actions and comments to the user and repeatedly training them. Specifically, the user sets an ideal human image on an application. For example, if a "compassionate person" is set as the ideal human image, the system custom-changes the user's actions and comments based on the "compassionate person". The system then presents the custom-changed actions and comments to the user and repeatedly trains them, encouraging the user to get closer to their ideal image.

[1749] "Example 2"

[1750] The system of the present invention includes a means for projecting an ideal human image set by a user, a means for custom-changing the user's actions and comments based on the ideal human image, and a means for presenting the custom-changed actions and comments to the user and repeatedly training them. Specifically, the user sets an ideal human image on an application. For example, if a "compassionate person" is set as the ideal human image, the system custom-changes the user's actions and comments based on the "compassionate person". The system then presents the custom-changed actions and comments to the user and repeatedly trains them, encouraging the user to get closer to their ideal image.

[1751] The process flow of each embodiment will be described below.

[1752] "Example 1"

[1753] Step 1: The user sets the ideal person in the application. For example, the user sets "a caring person" as the ideal person.

[1754] Step 2: The system custom-changes the user's behavior and speech based on the ideal image of a person. Specifically, the system analyzes the user's behavior and speech based on the image of a "compassionate person" and generates appropriate behavior and speech.

[1755] Step 3: The system presents the customized actions and statements to the user, for example, providing advice to the user to behave more considerately towards others.

[1756] Step 4: The user repeatedly practices the actions and statements suggested, which allows the user to get closer to their ideal self.

[1757] Example 1

[1758] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1759] In conventional self-improvement systems, it is difficult to custom change specific actions and speech based on the ideal image of a person set by the user, and there is a lack of effective means to record the progress of the repetitive training and provide feedback. Therefore, there is an issue that effective support for users to get closer to their ideal image is not provided.

[1760] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1761] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and comments based on the ideal human image using a generative AI model, a means for presenting the custom-changed actions and comments to the user and training them repeatedly, and a means for recording the progress of the repeated training and providing feedback. This enables the user to learn and practice specific actions and comments to approach their ideal image, and to effectively manage the progress.

[1762] "Ideal person" refers to the specific person image or personality traits that users should aim to be.

[1763] A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate data and custom change user behavior and speech.

[1764] "Custom change" refers to modifying a user's behavior and words based on an ideal image of a person.

[1765] "Repetitive training" refers to the user repeatedly practicing the custom-changed actions or utterances presented to them.

[1766] "Feedback" refers to evaluations and advice provided to a user based on their progress in repeated training.

[1767] "Recording progress" refers to saving the content and results of the repetitive training performed by the user as data.

[1768] "Presenting" refers to showing the user the custom-changed actions or utterances generated by the server.

[1769] This invention is a system that promotes the user to approach his / her ideal image by customizing actions and speech based on the ideal image of a person set by the user and repeatedly training the user. A specific embodiment of this system will be described below.

[1770] Hardware and Software Used

[1771] Hardware

[1772] Device: A device operated by a user, such as a smartphone, computer, or tablet.

[1773] Server: A server for processing data and running generative AI models.

[1774] software

[1775] Application: Software that provides an interface for users to define their ideal person image and review their custom-changed behaviors and statements.

[1776] Generative AI model: An algorithm for customizing a user's actions and speech based on an ideal human image. Specifically, we use generative AI models such as OpenAI's GPT-4.

[1777] Data processing and calculation

[1778] Users define their ideal image of a person

[1779] The user starts the application and sets the ideal image of the person. For example, the user sets "a considerate person" as the ideal image of the person.

[1780] The device sends the configuration data to the server.

[1781] The terminal transmits data of the ideal human image set by the user to the server. This data includes information about the ideal human image selected by the user.

[1782] The server analyzes the data using a generative AI model

[1783] The server uses a generative AI model based on the received data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comments data and custom-changes it based on the ideal image of a person.

[1784] The server generates custom-changed actions and statements

[1785] The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements of a "compassionate person."

[1786] Send the server-generated data to the device

[1787] The server transmits the generated custom-changed behavior and speech data to the terminal, which includes examples of specific behaviors and speech that the user should practice.

[1788] The device presents the data to the user

[1789] The terminal presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

[1790] Users perform repetitive training

[1791] The user then practices the proposed custom-changed actions and utterances, and the application tracks the user's progress and provides feedback as needed.

[1792] Examples

[1793] For example, if a user sets a "compassionate person" as their ideal person, the following prompt sentence is input into the generative AI model.

[1794] Examples of prompt statements

[1795] "Please provide advice to help users be considerate when speaking with friends."

[1796] Based on this prompt, the generative AI model generates specific examples of utterances that users can use when talking to their friends, such as "When a friend is in trouble, say, 'Let me know if there's anything I can do to help you.'"

[1797] In this way, users can learn and practice behaviors and speech based on their ideal image of a person.

[1798] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1799] Step 1:

[1800] Users define their ideal image of a person

[1801] The user launches the application and sets the ideal person by clicking the "Set Ideal Person" button on the application interface, selecting "A considerate person" from the drop-down menu, and pressing the "Set" button.

[1802] Input: User's choice of ideal person (e.g. "a caring person")

[1803] Output: Ideal human image configuration data

[1804] Step 2:

[1805] The device sends the configuration data to the server.

[1806] The device transmits the user's ideal person profile data to a server, where the transmission is secure using the HTTPS protocol.

[1807] Input: Ideal human image configuration data

[1808] Output: Configuration data sent to the server.

[1809] Step 3:

[1810] The server analyzes the data using a generative AI model

[1811] The server uses a generative AI model based on the received configuration data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comment data (e.g., chat logs and SNS posts) and custom changes it based on the ideal image of a person.

[1812] Input: Configuration data, user's past actions and statements

[1813] Output: Custom-changed behavior and speech data

[1814] Step 4:

[1815] The server generates custom-changed actions and statements

[1816] The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements of a "compassionate person."

[1817] Input: Custom-changed action or speech data

[1818] Output: Generated examples of specific actions and statements

[1819] Step 5:

[1820] Send the server-generated data to the device

[1821] The server transmits the generated custom-changed behavior and speech data to the terminal. This transmission is also performed using the HTTPS protocol.

[1822] Input: Examples of specific actions or statements generated

[1823] Output: Data sent to the terminal

[1824] Step 6:

[1825] The device presents the data to the user

[1826] The terminal presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

[1827] Input: Data sent to the terminal

[1828] Output: Examples of specific actions or statements presented to the user

[1829] Step 7:

[1830] Users perform repetitive training

[1831] The user then practices the proposed custom-changed actions and statements, and the application records the user's progress and provides feedback as needed. For example, the user might enter into the application, "Today, I made a kind comment to a friend."

[1832] Input: User's practice results

[1833] Output: Progress data recorded and feedback provided.

[1834] (Application example 1)

[1835] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1836] In conventional customer service training systems, it was difficult to customize specific actions and comments for staff to acquire the ideal customer service attitude, making it difficult to provide effective training. In addition, there was a lack of repetitive training for staff to get closer to the ideal customer service attitude, making it difficult for them to maintain the ideal attitude when actually serving customers.

[1837] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and remarks based on the ideal human image, a means for presenting the custom-changed actions and remarks to the user and repeatedly training them, a means for generating actions and remarks based on the ideal human image using a generative AI model, and a means for presenting the generated actions and remarks to the user as prompt sentences. This enables the staff to concretely understand the ideal customer service attitude and maintain the ideal attitude in actual customer service through repetitive training.

[1838] A "user" is an individual who uses the system to learn to behave and speak in a way that is consistent with the ideal image of a human being.

[1839] The "ideal person" refers to the specific character and attitude that the user should aim for, and serves as the standard by which the system will custom change behavior and speech.

[1840] "Means of projection" is a function that displays the ideal image of a person set by the user visually or as text on the system.

[1841] "Means for custom change" is a function that allows a user to change their current behavior and words based on their ideal image of the person.

[1842] "Means of presentation" is a function for displaying custom-changed actions and comments to the user.

[1843] "Means for repeated training" is a function that allows users to repeatedly practice custom-changed actions and utterances.

[1844] A "generative AI model" is an artificial intelligence model that generates actions and statements based on the ideal human image set by the user.

[1845] A "prompt" is a text-based instruction that presents to the user actions and statements that the generative AI model has generated based on an ideal image of a human being.

[1846] The system for carrying out the present invention customizes actions and speech based on an ideal human image set by the user, and presents the customized changes to the user for repetitive training. A specific embodiment of this system will be described below.

[1847] System Configuration

[1848] The system consists of the following main components:

[1849] 1. User device: A device such as a smartphone or tablet that the user uses to define their ideal image of the person and check their custom-changed actions and statements.

[1850] 2. Server: Processes user input data and uses a generative AI model to generate actions and statements based on the ideal image of a human.

[1851] 3. Generative AI models: Using generative AI models such as OpenAI's GPT-3, actions and statements are generated based on the ideal human image set by the user.

[1852] Program Processing

[1853] The server receives the ideal human image and current behavior and speech data sent from the user's device, and inputs them as prompts into the generative AI model. The generative AI model generates behavior and speech based on the ideal human image and returns the results to the server. The server then sends the generated behavior and speech to the user's device and presents them to the user.

[1854] Hardware and Software Used

[1855] Hardware: Smartphones, tablets, servers

[1856] Software: Python, OpenAI API, User Interface Application

[1857] Data processing and calculation

[1858] The server sends the user's ideal image of a person and their current actions and comments as prompts to the generative AI model. The generative AI model generates the ideal actions and comments based on the prompts and returns the results to the server. The server then sends the generated actions and comments to the user's device and presents them to the user.

[1859] Examples

[1860] For example, if a user sets "caring customer service" as their ideal person and their current behavior is "treating customers in a rude manner," the prompt text would be as follows:

[1861] Example prompt:

[1862] Ideal person: Attentive customer service

[1863] User behavior: Being rude to customers

[1864] Acting on the Ideal Person:

[1865] Based on this prompt, the generative AI model generates actions and utterances appropriate for "considerate customer service" and presents the results to the user. By repeatedly practicing the actions and utterances presented, the user can get closer to their ideal customer service attitude.

[1866] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1867] Step 1:

[1868] The user sets the ideal person image using the terminal. The user starts the application and inputs the ideal person image (e.g., "caring customer service"). The input data is sent from the terminal to the server.

[1869] Input: Ideal human image

[1870] Output: Ideal human image data sent to the server

[1871] Step 2:

[1872] The server receives the ideal human image data sent by the user. The server stores this data and uses it for the next process.

[1873] Input: Ideal human image data

[1874] Output: Saved ideal human image data

[1875] Step 3:

[1876] The user inputs the current behavior or speech using the terminal. The user inputs the current behavior (e.g., "I treat customers rudely") and sends it from the terminal to the server.

[1877] Input: Current action or statement

[1878] Output: Current action and speech data sent to the server

[1879] Step 4:

[1880] The server receives the current behavior and speech data sent by the user, and stores this data for further processing.

[1881] Input: Current action or speech data

[1882] Output: Saved current actions and speech data

[1883] Step 5:

[1884] The server generates a prompt by combining the stored ideal human image data with the current behavior and speech data. The generated prompt is then input into the generative AI model.

[1885] Input: Ideal person image data, current behavior and speech data

[1886] Output: Generated prompt statement

[1887] Step 6:

[1888] The server sends the generated prompt text to the generative AI model, which then generates the ideal behavior or speech based on the prompt text.

[1889] Input: Generated prompt text

[1890] Output: Ideal actions and statements generated by the generative AI model

[1891] Step 7:

[1892] The server receives the ideal actions and statements returned by the generative AI model, and then transmits this data to the user's device.

[1893] Input: Ideal actions and statements generated by a generative AI model

[1894] Output: Ideal behavior and speech data sent to the user's device

[1895] Step 8:

[1896] The user device receives the ideal behavior and speech data sent from the server and presents it to the user. The user confirms the presented behavior and speech and performs repeated training.

[1897] Input: Ideal behavior and speech data

[1898] Output: The desired actions or statements presented to the user

[1899] Through the above processing steps, the user can learn how to act and speak based on the ideal image of a person, and through repetitive training, come closer to that ideal attitude.

[1900] Example 2

[1901] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1902] In modern society, many people aim to improve themselves and become closer to their ideal image, but the process is difficult and requires continuous effort. In addition, there is a problem that effective training is difficult due to a lack of specific guidelines for actions and statements for self-improvement. Furthermore, there is no appropriate means to customize users' actions and statements based on the ideal image of a person, making it difficult for users to get closer to their ideal image.

[1903] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1904] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and comments based on the ideal human image, a means for presenting the custom-changed actions and comments to the user, a means for the user to repeatedly train the presented actions and comments, a means for collecting the user's actions and comments, and a means for analyzing the collected actions and comments and customizing them using a generation AI model. This allows the user to obtain specific guidelines for actions and comments and to approach their ideal image through continuous training.

[1905] "Ideal person" refers to the specific person image or personality traits that users should aim to be.

[1906] "Means of projection" refers to the function that allows the user to display the ideal image of the person they have set in visual or text form.

[1907] "Means for custom change" refers to the function that allows a user to change their current behavior and words based on their ideal image of the person.

[1908] "Means of presentation" refers to a function for notifying or displaying custom-changed actions or statements to the user.

[1909] "Means of repetitive training" refers to a function that allows the user to train themselves to become closer to their ideal image of a person by repeatedly carrying out suggested actions and statements.

[1910] "Means of collection" refers to the function for recording user actions and statements and storing them as data.

[1911] "Means of analysis" refers to the function of analyzing collected user behavior and comment data and extracting information for custom changes based on an ideal image of a person.

[1912] A "generative AI model" refers to an artificial intelligence model that converts a user's actions and words based on an ideal image of a human being.

[1913] This invention is a system that custom changes actions and speech based on an ideal human image set by a user, presents it to the user, and conducts repetitive training. A specific embodiment of this system will be described below.

[1914] Hardware and Software Used

[1915] Hardware

[1916] Device: The device used by a user, such as a smartphone, tablet, or computer.

[1917] Server: A remote server for processing and storing data.

[1918] software

[1919] Application: Software that allows users to define their ideal person image and check their custom-changed behavior and statements.

[1920] Generative AI model: An artificial intelligence model (e.g., GPT-4) that transforms user actions and statements based on an ideal image of a human being.

[1921] System Operation

[1922] Setting the ideal image of a person

[1923] The user starts the application and goes to the settings screen. There, they input their ideal image of the person they want to be. For example, they might input "a caring person."

[1924] Collection of user actions and comments

[1925] The device records the user's daily actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media. The collected data is sent to a server in real time.

[1926] Custom changes to actions and statements

[1927] The server analyzes the received user behavior and speech data. Using a generative AI model (e.g. GPT-4), the user's behavior and speech are custom-changed based on the ideal image of a person. For example, if a user says, "I'm busy today, so I can't help you," the generative AI model converts this to "I'm busy today, but please let me know if there's anything I can do to help you."

[1928] Displaying custom-changed actions and statements

[1929] The server transmits the custom-changed actions and utterances to the terminal. The terminal notifies the user of the change. The user confirms the notification and actually uses the custom-changed actions and utterances.

[1930] Repetitive training

[1931] The user repeatedly executes the customized actions and statements suggested to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts. In this way, the user trains to become closer to their ideal image of a person.

[1932] Examples

[1933] Setting the ideal image of a person

[1934] The user enters "compassionate person" in the application's settings screen and clicks the save button.

[1935] Collection of user actions and comments

[1936] The device will record the voice of users' conversations with friends and convert them into text, and also collect text data posted by users on social media.

[1937] Custom changes to actions and statements

[1938] The server inputs the collected text data into a generative AI model, which uses prompts to transform the user's speech into an ideal human image.

[1939] Displaying custom-changed actions and statements

[1940] The server sends the customized utterances output by the generative AI model to the device, which then displays a notification to the user and suggests something like, "I'm busy today, but please let me know if there's anything I can do to help you."

[1941] Repetitive training

[1942] The user uses the utterances suggested by the server in daily conversations and social media posts. For example, in a conversation with a friend, the user might say, "I'm busy today, but please let me know if there's anything I can do to help you." By repeating this, the user will naturally be able to act and speak as a "considerate person."

[1943] Examples of prompt statements

[1944] Here is an example of a prompt to be input to the generative AI model:

[1945] User currently says: "I'm busy today, I can't help you."

[1946] Ideal person: "A considerate person"

[1947] Prompt for generative AI model: "Transform the user's utterance into a compassionate person's utterance."

[1948] By inputting this prompt into a generative AI model, the user's speech is custom altered based on the ideal image of a person.

[1949] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1950] Step 1:

[1951] The user starts the application and moves to the setting screen. There, the user inputs the ideal image of the person. For example, the user inputs "a considerate person."

[1952] Input: The ideal person image entered by the user on the settings screen.

[1953] Output: The ideal person image is saved in the application.

[1954] Step 2:

[1955] The device records the user's daily actions and comments, for example, the contents of conversations the user has had with friends and the contents of posts made to social networking sites.

[1956] Input: What the user does or says.

[1957] Output: Collected behavioral and speech data is stored on the device and transmitted to the server in real time.

[1958] Step 3:

[1959] The server analyzes the received user behavior and speech data, and uses a generative AI model (e.g. GPT-4) to custom-change the user's behavior and speech based on the ideal image of a person.

[1960] Input: Collected behavioral and speech data, ideal human image.

[1961] Output: Custom changed actions and statements.

[1962] Specific operation: The server inputs the collected text data into the generative AI model and converts the user's utterances using prompt sentences.

[1963] Step 4:

[1964] The server transmits the custom-changed actions and utterances to the terminal. The terminal notifies the user of the change. The user confirms the notification and actually uses the custom-changed actions and utterances.

[1965] Input: Custom-changed actions and statements.

[1966] Output: The custom changed actions or statements presented to the user.

[1967] Specific operation: The device displays the custom-changed message received from the server as a notification.

[1968] Step 5:

[1969] The user repeatedly executes the proposed custom-changed actions and utterances, for example, using utterances suggested by the generative AI model in daily conversations and social media posts.

[1970] Input: The custom-changed actions or statements presented.

[1971] Output: Custom changes to actions and statements made by the user.

[1972] Specific operation: The user uses utterances suggested by the server in conversations with friends and in social media posts. By repeating this process, the user trains to become closer to the ideal person.

[1973] (Application example 2)

[1974] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1975] In conventional customer service work, there were limited training methods for staff to acquire the ideal customer service attitude, and it was difficult to provide effective training. In addition, there was a lack of specific guidelines and feedback for staff to practice the ideal customer service attitude, making it difficult to improve customer service quality. Furthermore, there was a need for an efficient method of repeated training to help staff get closer to their ideal image.

[1976] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1977] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and utterances based on the ideal human image, a means for presenting the custom-changed actions and utterances to the user and repeatedly training them, a means for training the ideal customer service attitude using smart glasses worn by the staff performing customer service work, and a means for generating custom-changed utterances based on the ideal human image using a generation AI model. This enables the staff to efficiently train and practice the ideal customer service attitude.

[1978] "User" means an individual or organization that uses the System.

[1979] The "ideal person" is a model of the behavior and attitude that users should strive to be.

[1980] A "projection means" is a device or software for visually or audibly displaying the ideal human image set by the user.

[1981] A "means of custom change" is a device or software that changes a user's behavior and speech based on an ideal image of a person.

[1982] A "means for presenting" is a device or software for displaying custom-changed actions or utterances to a user.

[1983] A "means for repeated training" is a device or software that allows a user to repeatedly practice custom-changed actions or utterances.

[1984] "Customer service" refers to the provision of products and services to customers.

[1985] "Smart glasses" are glasses-type devices that have the function of displaying information.

[1986] A "generative AI model" is an algorithm that uses artificial intelligence to generate text and actions.

[1987] "Custom-changed statements" are statements made by users that have been altered based on their ideal image of a person.

[1988] The system for implementing this invention projects the ideal human image set by the user, custom changes the user's actions and remarks, and repeatedly trains them. Specifically, the ideal customer service attitude can be trained using smart glasses worn by staff who serve customers.

[1989] System Configuration

[1990] The system consists of the following main components:

[1991] 1. User device: The device on which the user sets their ideal image. This can be a smartphone or tablet.

[1992] 2. Smart glasses: A device worn by staff performing customer service duties that has the function of visually displaying information.

[1993] 3. Server: Runs the generative AI model and processes data to customise user behaviour and speech.

[1994] 4. Generative AI model: An algorithm that generates custom-changed utterances based on the user’s ideal image of a person.

[1995] Program Processing

[1996] The server receives the ideal person image set by the user and custom changes the user's behavior and speech based on that image. Specifically, the process is as follows:

[1997] 1. Setting the ideal person image: Using the user terminal, the user sets the ideal person image. For example, the ideal image can be set as "kind and polite customer service."

[1998] 2. Enter user utterances: Enter actual utterances made by users. For example, "Welcome, is there anything you're looking for?"

[1999] 3. Executing the generative AI model: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image.

[2000] 4. Present customized utterances: Present customized utterances to the user through smart glasses, for example, "Welcome, we will help you find the product you are looking for."

[2001] 5. Repetitive training: The user repeatedly practices the custom-changed utterances presented to them.

[2002] Hardware and Software Used

[2003] Hardware: Smart glasses (e.g. Google Glass), user devices (smartphones, tablets)

[2004] Software: Generative AI models (e.g. OpenAI API), data processing servers

[2005] Examples

[2006] For example, the ideal customer service staff member may be described as "friendly and polite" and the following prompt may be used:

[2007] Example prompt:

[2008] Ideal person: Kind and polite customer service

[2009] User says: Hey there, is there anything you're looking for?

[2010] Custom statements based on the ideal person:

[2011] By inputting this prompt into the generative AI model, a custom-changed utterance based on the ideal customer service attitude is generated. For example, a utterance like, "Welcome, we will help you find the product you are looking for" is generated.

[2012] The flow of the specific process in the application example 2 will be described with reference to FIG.

[2013] Step 1:

[2014] The user sets the ideal image of the person.

[2015] Input: The user inputs the ideal image of a person using a device (smartphone or tablet). For example, "kind and polite customer service."

[2016] Data processing: The terminal sends the inputted ideal image of a person to the server.

[2017] Output: The ideal person image is saved on the server.

[2018] Step 2:

[2019] The user types in the actual utterance.

[2020] Input: What the user actually says when using the terminal, for example, "Hello, is there something you're looking for?"

[2021] Data processing: The terminal sends the entered comments to the server.

[2022] Output: The user's comments are saved on the server.

[2023] Step 3:

[2024] The server runs the generative AI model.

[2025] Input: The server retrieves the stored ideal person profile and the user's statements.

[2026] Data calculation: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image. The prompt sentence is input to the generative AI model to obtain the custom-changed utterances.

[2027] Output: The generated custom-changed utterances are saved on the server.

[2028] Step 4:

[2029] The custom-changed statement is presented to the user.

[2030] Input: The server gets the generated custom changed utterance.

[2031] Data processing: The server sends the custom-changed utterances to the smart glasses.

[2032] Output: The custom-changed speech is displayed on the smart glasses.

[2033] Step 5:

[2034] The user performs repetitive training.

[2035] Input: The user sees the custom-changed utterance displayed on the smart glasses.

[2036] Specific behavior: The user repeatedly practices the utterances shown to them, for example, "Welcome, we are happy to help you find the product you are looking for."

[2037] Output: The user's customer service attitude becomes closer to the ideal person.

[2038] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2039] "Example 1"

[2040] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine analyzes the user's emotional state in real time. Specifically, the emotion state is estimated from the user's facial expression, tone of voice, choice of words, etc. In addition, this emotion engine tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training for the user to approach his / her ideal image according to the user's emotional state. For example, when the user is feeling stressed, training that promotes relaxation is provided. On the other hand, when the user is feeling joy, training that promotes behavior to share that joy is provided. This allows the user to train to approach his / her ideal image in a manner appropriate to his / her emotional state.

[2041] "Example 2"

[2042] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine analyzes the user's emotional state in real time. Specifically, the emotion state is estimated from the user's facial expression, tone of voice, choice of words, etc. In addition, this emotion engine tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training for the user to approach his / her ideal image according to the user's emotional state. For example, when the user is feeling stressed, training that promotes relaxation is provided. On the other hand, when the user is feeling joy, training that promotes behavior to share that joy is provided. This allows the user to train to approach his / her ideal image in a manner appropriate to his / her emotional state.

[2043] The process flow of each embodiment will be described below.

[2044] "Example 1"

[2045] Step 1: The user boots the system and sets his ideal image. This ideal image is

[2046] It specifically shows how users will act and speak.

[2047] Step 2: The emotion engine analyzes the user's emotional state in real time, inferring it from the user's facial expressions, tone of voice, choice of words, etc.

[2048] Step 3: The emotion engine tracks changes in the user's emotional state, allowing it to tailor the iterative training to help the user achieve their ideal self.

[2049] Step 4: The user undergoes repetitive training to get closer to their ideal self. This training is adjusted according to the user's emotional state. For example, when the user is feeling stressed, training is provided to promote relaxation. On the other hand, when the user is feeling joy, training is provided to encourage behavior that encourages sharing that joy.

[2050] Example 1

[2051] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[2052] With conventional systems, it was difficult to custom change behavior and speech based on the ideal person image set by the user, and the training content was not adjusted according to the user's emotional state, so there was an issue of a lack of effective support to help users get closer to their ideal image.

[2053] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2054] In this invention, the server includes a means for projecting an ideal human image set by the user, a means for custom-changing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, and a means for adjusting the training content based on data obtained from the emotion engine. This makes it possible to effectively support the user in getting closer to his or her ideal image.

[2055] A "user" refers to an individual who aims to use the system to become closer to their ideal image of a person.

[2056] An "ideal person" is a specific definition of the personality and behavioral characteristics that a user aspires to be.

[2057] "Means of projection" refers to the ability to display the user's ideal image of the person on the system in visual or text form.

[2058] "Means of custom change" refers to the function that changes and adjusts a user's actions and comments based on an ideal image of a person.

[2059] "Means of presentation" refers to the function of notifying or displaying custom-changed actions or comments to the user.

[2060] "Means for repeated training" refers to a function that allows the user to repeatedly practice based on the custom changes presented to them.

[2061] The "emotion engine" refers to a function that analyzes a user's emotional state in real time based on their facial expressions, tone of voice, choice of words, etc.

[2062] "Means for adjusting training content" refers to a function for changing and adjusting the training content according to the user's emotional state based on data obtained from the emotion engine.

[2063] This invention is a system that supports the user in getting closer to his / her ideal image through repetitive training by customizing actions and speech based on the ideal image of the person set by the user. A specific embodiment of this system will be described below.

[2064] Hardware and Software Used

[2065] Hardware

[2066] Device: The smartphone, tablet, or PC used by the user.

[2067] Server: Cloud server for data processing and storage.

[2068] Camera and microphone: Devices for capturing your facial expressions and tone of voice in real time.

[2069] software

[2070] Application: Provides an interface for users to define their ideal person image and review their custom-changed behaviors and statements.

[2071] Emotion engine: A software module for analyzing the user's emotional state in real time.

[2072] Generative AI model: An AI model that custom changes a user's behavior and sp...

Claims

1. A means for storing information regarding an ideal human image set by a user; A means for collecting at least one of user actions and comments as data; A means for generating a prompt sentence for generating an instruction to custom change at least one of the user's actions and utterances based on information regarding the ideal human image set by the user and at least one of the collected data regarding the user's actions and utterances; A means for generating a suggested message including at least one of an action and an utterance that is a custom change of at least one of the user's action and utterance by using the generated prompt sentence and a generating AI model; and means for presenting the generated suggestion message to the user.

2. means for identifying an emotional state of the user using an emotion engine; The system of claim 1, wherein the means for generating a prompt sentence generates a prompt sentence that generates instructions to custom change at least one of the user's behavior and speech based on information about the ideal human image set by the user, at least one of collected data about the user's behavior and speech, and the identified emotional state of the user.

3. The system of claim 1 , wherein the means for presenting to the user is a display of smart glasses.

Images