system

Abstract

We provide the system. [Solution] In an analytical device, a means for identifying personality traits based on personality attribute data collected from the user, A means for generating multiple personalized pieces of advice based on identified personality traits, A means of presenting generated advice to the user and receiving a response from the user, A system including means for analyzing the received user response and adjusting the device's learning function to improve the accuracy of the advice generation means.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Problems in a marital relationship are delicate issues that are difficult for many married people to consult about, so the problem is that the opportunity to receive appropriate advice and guidance is limited. Under such circumstances, there is a need for specific and individualized support for spouses to understand their personality characteristics and improve their relationship with each other.

Means for Solving the Problems

[0005] This invention provides a system that generates multiple personalized pieces of advice based on personality attribute data collected from a user, by having an analysis device identify personality traits based on those traits. Furthermore, it has a function to present the generated advice to the user and receive responses from the user, and has a learning function that improves the accuracy of the advice generation means by analyzing the received response data. Through such means, it is possible to provide effective support for improving the individual relationship between spouses.

[0006] An "analytical device" is a device that has the function of extracting and processing specific information based on input data.

[0007] "Personality attribute data" refers to information that indicates the user's personality traits and behavioral tendencies.

[0008] "Personality traits" refer to the characteristics and features that define an individual's personality and behavioral patterns.

[0009] "Personalized advice" refers to advice tailored to the specific circumstances and personality of an individual.

[0010] "User" refers to an individual who uses the system.

[0011] "Response" refers to the input or reaction that a user gives to the system.

[0012] "Learning function" refers to a function that improves the performance and accuracy of a system by utilizing data acquired in the past. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0015] First, the language used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is an AI-powered system designed to support the improvement of marital relationships, aiming to provide personalized advice based on user input data. Specifically, the system identifies the personality traits of the couple through an analysis device and generates advice to support relationship building based on that information.

[0035] System configuration and operation

[0036] Users answer a personality test using their own devices. This test includes questions designed to provide a detailed understanding of the user's personality traits.

[0037] The device receives the responses from the user and sends them to the server as digital data.

[0038] The server uses an algorithm to analyze personality traits based on the received data. This analysis creates a personality profile for each user.

[0039] The server simultaneously references existing psychology databases to identify potential problems and areas for improvement based on personality traits. Based on the identified issues, it generates advice including multiple options for improving relationships.

[0040] The device presents the generated advice to the user, enabling them to implement it in their daily life.

[0041] The user tries out the suggested advice and returns the results and feedback to the system as a response.

[0042] The terminal sends the user's response to the server and initiates the feedback process.

[0043] The server analyzes the feedback and utilizes its learning capabilities to adjust its algorithms to improve the accuracy of future advice.

[0044] Specific example

[0045] For example, if a married couple uses this system, they might each take a personality assessment first, which could reveal that their individual personality traits "need harmony." Based on this result, the server generates specific advice such as "have a shared hobby once a week" or "understand differences of opinion." Users then put this advice into practice and provide feedback on the results and changes, which helps the system improve its ability to provide even more effective support.

[0046] This system allows couples to receive specific, actionable advice based on their individual personality traits, enabling them to improve their relationship.

[0047] The following describes the processing flow.

[0048] Step 1:

[0049] The user launches the application and accesses the personality test. They enter their answers to the test questions and press the "Submit" button when finished.

[0050] Step 2:

[0051] The device collects the user's response data and formats it as digital information. It then sends this data to the server.

[0052] Step 3:

[0053] The server receives the submitted response data and applies a personality assessment algorithm to analyze the user's personality traits. This process calculates MBTI and Big Five scores.

[0054] Step 4:

[0055] Based on the analysis results, the server creates personality profiles of the user and their partner, and, referencing these profiles with an existing psychological database, generates specific advice necessary for improving the relationship.

[0056] Step 5:

[0057] The server sends the generated advice to the terminal. This advice includes actionable steps and specific suggestions for improving relationships.

[0058] Step 6:

[0059] The device displays advice to the user and provides an interface to encourage its application in daily life.

[0060] Step 7:

[0061] Users follow the advice provided and record feedback on the results and effects within the application. This feedback includes the actions taken and the changes observed.

[0062] Step 8:

[0063] The device sends user feedback data to the server. This feedback is processed to make future advice more personalized and effective.

[0064] Step 9:

[0065] The server analyzes the feedback it receives and adjusts and improves the advice generation algorithm. This allows the system to evolve so that it can provide more personalized suggestions to future users.

[0066] (Example 1)

[0067] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0068] Traditional relationship-building support systems often lack the nuanced approach based on individual characteristics, and are limited to providing general advice. This makes it difficult to effectively improve relationships with others. In particular, there is a need for more personalized support that provides specific advice tailored to each individual's personality and circumstances.

[0069] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0070] In this invention, the server includes means for analyzing personal characteristics based on input data obtained from the user, means for generating personalized relationship improvement advice by referring to the analysis results and a set of psychological data, and means for notifying the user of the generated advice using a terminal and receiving the user's response. This makes it possible to provide advice tailored to individual situations and characteristics.

[0071] "User" refers to an individual who uses the system to input information or receive services.

[0072] "Input data" refers to information provided by the user and processed by the system.

[0073] "Personal characteristics" refer to the unique personality and behavioral traits of the user.

[0074] "Analysis" refers to calculations and processing performed using input data to identify individual characteristics.

[0075] A "psychological data collection" refers to a database of information and knowledge based on the psychological characteristics of humans.

[0076] "Advice" refers to information that includes specific actions or suggestions aimed at improving the relationship with the user.

[0077] A "terminal" refers to an electronic device used by a user to access a system.

[0078] A "server" refers to a computer system that stores and processes data and controls the sending and receiving of information between it and terminals.

[0079] "Calculation method" refers to the calculation methods and algorithms that a system uses to process data and generate advice.

[0080] "Adaptive" means that the system has the ability to adjust its operation and output as needed based on new information it receives from the user.

[0081] In this invention, the system primarily involves a server, terminal, and user exchanging data with each other to provide personalized advice aimed at improving marital relationships. The details are described below.

[0082] Hardware and software to use

[0083] The terminal is used by the user as a means of answering questions and receiving generated advice. This includes digital devices such as smartphones, tablets, and personal computers. Specific applications are installed on these terminals, and users access the system using these applications.

[0084] A server is a computing system that receives and processes data transmitted from terminals. The server implements algorithms for data analysis, analyzing user input data and generating advice for improving relationships.

[0085] The generative AI model operates on a server and is responsible for data analysis and advice generation. This allows it to create personalized advice based on a set of psychological data. The AI ​​model also has a feedback learning function, utilizing user response data to improve its prediction accuracy.

[0086] Specific actions and prompt examples

[0087] Users access the system via a smartphone or other device and take a personality test. The results of this test are sent to the server as digital data.

[0088] Based on the transmitted data, the server generates advice for improving relationships using an analysis algorithm and a generative AI model. The generated advice is sent to the relevant user's device and presented to the user.

[0089] For example, if a couple uses this system and a personality test reveals that they "need harmony," the server will generate advice such as, "Try to engage in a shared hobby together once a week." Users then put this advice into practice in their daily lives and send the results back to the system as feedback.

[0090] Example of a prompt:

[0091] "Based on the personality assessment results, please provide specific advice on how to improve our marital relationship. The assessment result is 'Harmony is needed.'"

[0092] In this way, the system provides advice to improve interpersonal relationships in a way that is beneficial to each individual user. Furthermore, through continuous feedback, the system's ability to provide even more effective advice improves.

[0093] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0094] Step 1:

[0095] The user launches the application on their device and starts the personality test. The input consists of answers to various questions. Specifically, the user answers multiple-choice questions and saves the answers as data on their device by pressing the submit button. As output, all the answers are recorded on the device as a single dataset.

[0096] Step 2:

[0097] The terminal receives the stored user response data, encrypts it, and sends it to the server. Specifically, the terminal uses a secure data transmission protocol to send encrypted data packets to the server over the internet. The input is the user's response data, and the output is a file in digital data format that reaches the server.

[0098] Step 3:

[0099] The server receives and decrypts transmitted data packets. It handles encrypted user response data as input. Based on this, it applies data analysis algorithms to analyze the user's characteristics. Specifically, the server uses a generative AI model to analyze the data and generate a user personality profile. The output is the analyzed personality data.

[0100] Step 4:

[0101] The server uses a generative AI model to generate advice for improving relationships based on analyzed personality traits. The input consists of analyzed personality trait data and a set of psychological data. Specifically, the server runs the AI ​​model, evaluates the attribute data, and forms prompt sentences that generate appropriate advice. The output is personalized advice.

[0102] Step 5:

[0103] The server sends the generated advice to the terminal. The input is pre-generated advice data. Specifically, the server uses a data transfer protocol to securely send the advice to the terminal. The output is the received advice displayed on the terminal.

[0104] Step 6:

[0105] Users review the advice displayed on their device and attempt to put it into practice in their daily lives. The input includes specific advice. The specific action involves the user taking new actions based on the provided advice to improve their relationships.

[0106] Step 7:

[0107] The user inputs the results of the execution as feedback into the terminal and sends that data to the server. The input includes feedback information regarding the results of implementing the advice. Specifically, the user fills out a feedback form and submits it. The output reaches the server as feedback.

[0108] Step 8:

[0109] The server analyzes the received feedback and modifies the calculation method of the system's generated AI model to improve its accuracy. The input is user feedback data. Specifically, the server retrains the generated AI model and performs a learning process based on the feedback. The output is an improved algorithm, which is used for future advice generation.

[0110] (Application Example 1)

[0111] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0112] In modern society, marital relationships frequently deteriorate due to a lack of communication and personality clashes. This can lead to a negative atmosphere within the home and, in some cases, ultimately result in divorce. To address this issue, couples need to understand each other's personality traits and accept appropriate advice to build a better relationship.

[0113] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0114] In this invention, the server includes means for identifying personality traits based on personality attribute data collected from the user, means for generating a plurality of personalized pieces of advice based on the identified personality traits, and means for collecting conversational data using a speech recognition device. This makes it possible for users to easily understand their own personality traits and receive necessary advice in real time.

[0115] A "user" is an individual who uses this system to undergo a personality assessment and receive advice.

[0116] "Personality attribute data" refers to information about a user's personality extracted from their responses to personality assessment tests and conversation data.

[0117] "Personality traits" refer to the characteristics and tendencies of an individual's personality, analyzed based on the user's personality attribute data.

[0118] "Advice" refers to specific suggestions for improving relationships, generated based on the user's personality traits.

[0119] A "speech recognition device" is a device that collects the user's conversation as audio data and converts it into text data.

[0120] "Natural language processing technology" is a technique that analyzes collected conversational data to understand the intentions and emotions of the users.

[0121] The system implementing this invention operates around a consumer robot equipped with voice recognition capabilities, which is placed in each home. First, the user undergoes a personality assessment through conversation with the robot. Voice data is collected through a high-performance microphone built into the robot and converted into text data using the Google® Speech-to-Text API. This converted data is then sent to a server.

[0122] The server analyzes text data using OpenAI's GPT model to generate user personality trait data. The analysis process utilizes natural language processing techniques to identify the user's personality using information obtained from the conversation. Based on this profile information, the server consults a psychology database to generate personalized advice for improving relationships.

[0123] The generated advice is sent from the server to the robot, which outputs it as voice. At this time, the robot clearly explains the suggested advice to the user. The user tries out the presented advice and provides feedback to the robot about the results and their impressions.

[0124] User feedback is also sent to the server via the robot. The server uses this feedback to adaptively train its algorithm, improving the accuracy of future advice. This process is designed to ensure users continuously receive specific and helpful advice for improving their relationships.

[0125] For example, if a user consults the robot saying, "We've been having a lot of disagreements lately," the robot might respond with advice such as, "How about setting aside some time to come to an agreement?" By having the user provide feedback on how they feel after implementing this advice day by day, the server's generated AI model learns to suggest more appropriate advice.

[0126] An example of a prompt for the generating AI model is: "Analyze the user's conversation log and generate advice that will help improve the relationship between the couple."

[0127] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0128] Step 1:

[0129] The user initiates a conversation with the robot. Voice data is collected through a high-performance microphone inside the robot. The input is voice data, and the output is text data converted by the Google Speech-to-Text API. This conversion process analyzes the characteristics of the voice signal and converts them into text information.

[0130] Step 2:

[0131] The terminal, or robot, sends this text data to the server. The input here is the text data generated in step 1, and the output is the digital data to be sent to the server. The data is delivered accurately to the server using a communication protocol.

[0132] Step 3:

[0133] The server performs natural language processing on the received text data. The input is text data of the user's conversation, and the output is user personality trait data extracted through analysis. A generative AI model is used to analyze the content of the text and perform data processing and calculations to extract the user's personality traits.

[0134] Step 4:

[0135] The server references a psychology database based on personality trait data and generates corresponding advice. The input is personality trait data, and the output is personalized advice. This process applies algorithms for database searching and optimal advice generation.

[0136] Step 5:

[0137] The server sends the generated advice to the robot. The input is the text data of the generated advice, and the output is the data to be sent to the robot. The advice is processed via a secure communication channel to ensure it reaches the robot safely.

[0138] Step 6:

[0139] The robot conveys the received advice to the user through a voice output device. The input is text data of the advice, and the output is voice information for the user. Speech synthesis technology is used to convert the text into a natural conversational style, which is then played through the speaker.

[0140] Step 7:

[0141] The user follows the robot's advice and provides voice feedback on the results and their impressions. The input is voice data of the user's actions, and the output is the robot's re-collection of voice data. This data is then converted back into text using speech recognition technology and prepared to be sent to the server.

[0142] Step 8:

[0143] The server receives feedback data, evaluates the effectiveness of the advice using a generated AI model, and learns from success stories and failure factors. The input is user feedback text data, and the output is adjustment data for algorithm training. This process aims to improve the accuracy of future advice generation.

[0144] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0145] This invention aims to support the improvement of marital relationships using an AI system equipped with emotion recognition capabilities. The system analyzes the user's personality traits and emotional state, and provides more precise and personalized advice.

[0146] System configuration and operation

[0147] Users participate in everyday interactions and personality tests using an application equipped with an emotion engine. The emotion engine evaluates the user's voice tone, facial expressions, and text data in real time to recognize their emotional state.

[0148] The terminal sends data obtained from the user (responses to personality assessments and emotional data) to an analysis device, which then formats the data.

[0149] After receiving data from the terminal, the server analyzes it to identify the user's personality traits and emotional profile. The emotion engine then incorporates the identified emotional states as influencing factors into the analysis.

[0150] The server generates advice aimed at improving relationships, taking into account both emotional data and personality traits. This advice is optimized for the user's current emotional state.

[0151] The device presents the generated advice to the user in real time. The content and tone of the advice are adjusted based on feedback from the emotion engine.

[0152] Users act on the advice presented in their daily lives and input the results as feedback into the system. This feedback is used to re-evaluate the user's emotional state.

[0153] The server analyzes the collected feedback and uses the system's learning capabilities to adjust the advice generation algorithm.

[0154] Specific example

[0155] For example, if a user's emotion engine detects voice patterns indicating stress and text messages indicating depression, the server will generate advice such as, "We recommend taking a walk in nature to refresh yourself," or "Let's make time for a relaxing conversation with your partner." If that advice is insufficient, encouraging messages will be added. In this way, the system approaches the user from both an emotional and personality perspective, enabling more practical and impactful support.

[0156] This invention allows couples to receive advice that takes their emotional state into consideration, helping them to improve their relationship more effectively.

[0157] The following describes the processing flow.

[0158] Step 1:

[0159] The user launches the application and inputs data through a personality test and daily activities. The emotion engine obtains the user's emotional state in real time from their voice tone, facial expressions, and text.

[0160] Step 2:

[0161] The terminal collects user input data and emotional state data, and formats this data into digital information. It then sends the data to the server.

[0162] Step 3:

[0163] The server analyzes the received data, calculates the user's personality traits, and identifies their emotional state. Simultaneously, it incorporates the evaluation results from the emotion engine to generate a personality profile that combines the emotional states.

[0164] Step 4:

[0165] Based on the identified personality profile and emotional state, the server generates specific advice for improving relationships. This advice will be tailored to the user's current emotional state.

[0166] Step 5:

[0167] The device displays the generated advice to the user. The advice has a message tone adjusted to take into account feedback from the emotion engine.

[0168] Step 6:

[0169] Users implement the advice in their daily lives and input feedback on its effects and their impressions into the application. This feedback includes changes in their emotional state.

[0170] Step 7:

[0171] The device sends the collected feedback to the server. The feedback data is used to re-evaluate the user's emotional state.

[0172] Step 8:

[0173] The server analyzes the feedback and modifies the advice generation algorithm to improve accuracy. In particular, adjustments that reflect the results of the emotion engine improve the quality of advice for subsequent sessions.

[0174] (Example 2)

[0175] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0176] The present invention aims to support effective relationship improvement in marital relationships and other interpersonal relationships by providing personalized advice that takes into account the user's emotional state and personality traits. However, conventional approaches have the problem of not adequately providing immediate advice based on the user's real-time emotional state, nor does it allow for adaptive learning through subsequent feedback.

[0177] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0178] In this invention, the server includes means for processing audio, video, and text data collected from the user and formatting them into a format suitable for analysis; means for analyzing the formatted data and generating a profile to identify the user's emotional state; and means for generating multiple personalized pieces of advice based on the generated profile, taking into account the emotional state and personality traits. This enables the provision of effective advice tailored to the user's current situation and supports continuous relationship improvement based on that advice.

[0179] "User" refers to the person who will receive analysis of their emotional state and advice using this system.

[0180] "Audio data" refers to acoustic information, including the user's speaking style and tone of voice.

[0181] "Video data" refers to visual information, including the user's facial expressions and gestures.

[0182] "Text data" refers to written characters or messages entered by the user.

[0183] "Formatting" refers to the process of converting and organizing collected raw data so that it can be easily analyzed.

[0184] A "profile" refers to a dataset that aggregates a user's emotional state and personality traits and represents them in an identifiable format.

[0185] "Personalized advice" refers to advice specifically created based on each user's emotional state and personality traits.

[0186] A "generative AI model" refers to an algorithm and system that uses data to create new outputs.

[0187] "Feedback" refers to the act of providing the system with information about the advice the user has taken and the subsequent changes in their emotional state.

[0188] "Learning function" refers to the process of improving the system's algorithms based on collected data and feedback.

[0189] This invention uses an AI system equipped with emotion recognition capabilities to support the improvement of users' interpersonal relationships. Specific embodiments are described below.

[0190] Users provide data through everyday conversations and personality tests using an application with a built-in emotion engine. This application records the user's voice with a microphone, captures facial expressions with a webcam, and accepts text-based input. All the data collected in this way is sent to the emotion engine, which evaluates the user's emotional state in real time.

[0191] The terminal formats the audio, video, and text data acquired from the user into an appropriate format and performs the necessary preprocessing for sentiment analysis. Specifically, this includes noise reduction of audio data and standardization of facial expression data. The formatted data is then sent to the server for analysis.

[0192] The server receives data sent from the terminal and uses a generative AI model to create a user's emotional profile. This profile includes the user's current emotional state and long-term personality traits. Based on the emotional profile, the server generates optimized advice for the user. This advice consists of helpful suggestions and encouraging messages tailored to the user's current situation.

[0193] For example, if a user sends a voice message indicating stress and text indicating depression, the server will generate advice such as "We recommend taking a walk in nature to refresh yourself" or "Make time for a relaxing conversation with your partner." This advice is delivered in a way that best suits the user's emotional state, and encouraging messages may be added as needed.

[0194] An example of an input prompt for the generating AI model might be: "The user's voice patterns indicate stress, and the text data shows signs of depression. Based on this information, generate the most appropriate relationship improvement advice for the user." This enables an intelligent approach tailored to the individual user's state.

[0195] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0196] Step 1:

[0197] Users provide audio, video, and text data through the application. Specifically, users speak into a microphone, capture their facial expressions with a webcam, and input text. This data serves as input for evaluating the user's emotional state.

[0198] Step 2:

[0199] The terminal receives data from the user and formats it. Audio data undergoes noise reduction, video data undergoes facial expression extraction and standardization, and text data undergoes keyword extraction. These processes result in a dataset converted into a format that is easy to analyze.

[0200] Step 3:

[0201] The terminal sends formatted data to the server. The output here is integrated data containing voice tone, facial expression patterns, and text keyword information.

[0202] Step 4:

[0203] The server analyzes the received data and uses a generative AI model to generate a user's emotional profile. This profile reflects both short-term emotional states and long-term personality traits and serves as input for subsequent processing.

[0204] Step 5:

[0205] The server generates personalized advice for the user based on their emotional profile. It uses the input as prompts for a generating AI model to form the advice. The advice might include suggestions for refreshing oneself or prompts for further dialogue.

[0206] Step 6:

[0207] The device receives advice generated from the server and presents it to the user. The content and tone of the advice are adjusted based on the user's latest emotional feedback. This ensures that the advice is delivered in a form that is most appropriate for the user.

[0208] Step 7:

[0209] The user enters feedback into the system regarding the advice they have received. This feedback includes information about the results of the actions taken and changes in their feelings, and is used in the next step.

[0210] Step 8:

[0211] The server analyzes the feedback and adjusts the generated AI model based on it. Through its learning function, it improves the advice generation algorithm and enhances the quality of subsequent outputs.

[0212] (Application Example 2)

[0213] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0214] In interpersonal relationships, particularly within families, communication can be hindered by individual emotional states, leading to a deterioration of relationships. There is a need to provide effective means to resolve this issue and promote relationship improvement.

[0215] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0216] In this invention, the server includes means for identifying an emotional state based on voice signals and facial expression data collected from the user, means for generating personalized advice based on the identified emotional state and personality traits, and means for presenting the generated advice to the user through voice and visual output and receiving feedback from the user. This enables effective communication and relationship improvement tailored to each individual's emotional state.

[0217] A "user" is an individual who provides voice and facial expression data to the system and receives advice as a result.

[0218] "Speech signals" are data that describes the characteristics of sound waves obtained from the user's speech.

[0219] "Facial expression data" refers to video information used to analyze the visual characteristics of a user's face.

[0220] "Emotional state" refers to the expression of the user's inner emotions, identified based on voice signals and facial expression data.

[0221] "Personality traits" are characteristic attributes related to the user's personality and individuality.

[0222] "Personalized advice" refers to specific and tailored behavioral guidelines for the user, generated based on identified emotional states and personality traits.

[0223] "Audio and visual output" refers to the audio and video formats used to convey generated information to the user.

[0224] "Feedback" refers to information that users send back to the server regarding their responses and results to the advice they receive.

[0225] "Learning function" refers to the ability to adjust the advice generation algorithm based on received feedback to improve the overall accuracy and effectiveness of the system.

[0226] This invention describes a specific embodiment of a system aimed at improving interpersonal relationships within the home. The system has a process of collecting voice signals and facial expression data from the user and identifying the emotional state based on them.

[0227] The server uses a platform equipped with high-precision sensors to process audio signals and facial expression data. For example, a microphone is used to analyze the tone of voice, and a camera is used to analyze facial expression data. This makes it possible to identify the user's emotional state in real time.

[0228] The server combines identified emotional states with pre-collected personality traits and uses a generative AI model like OpenAI to generate personalized advice. This generated advice is presented to the user via the device in audio and video formats. In this way, the user can interactively receive advice and take subsequent actions based on the results.

[0229] As a concrete example, if the system detects that one user in a married couple is experiencing stress, it analyzes their emotional state on a server and provides advice such as, "Why not take a walk in a nature park with your partner today to refresh yourselves?" By making such suggestions, the system can support improvements in everyday relationships.

[0230] An example of a prompt for a generative AI model is, "Please suggest an activity suitable for a relaxed emotional state."

[0231] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0232] Step 1:

[0233] The device collects the user's voice signals and facial expression data. Using a microphone and camera as sensors, it acquires audio and video in real time and formats them as digital data. The input is the user's physical voice and facial expressions, and the output is this digital data.

[0234] Step 2:

[0235] The terminal transmits the collected digital data to the server. The server performs speech recognition and facial expression analysis based on the received data. It analyzes the trends in speech tone and the facial features obtained from facial expressions, and performs data processing to identify the emotional state. The input is digitized audio and video data, and the output is an index indicating the user's emotional state.

[0236] Step 3:

[0237] The server integrates identified emotional states with already recorded personality traits and uses a generative AI model to create optimal advice. Here, prompts are set according to the emotional state, and the AI ​​model performs reasoning. The input is the emotional state and personality traits, and the output is a personalized advice message.

[0238] Step 4:

[0239] The server sends the generated advice to the terminal, which then presents it to the user as audio and visual output. Speech synthesis is used to provide information to the user in a natural way. The input is the advice message, and the output is the presentation of information verbally and visually.

[0240] Step 5:

[0241] The user acts based on the advice provided and inputs the results as feedback data into the terminal. The terminal forwards the feedback to the server. The input is feedback information from the user, and the output is an update to the system's history data.

[0242] Step 6:

[0243] The server analyzes feedback data, learns the advice generation algorithm, and adjusts parameters to improve accuracy. The feedback data is used as an evaluation metric for the algorithm and provides material for future improvements. The input is the feedback data, and the output is the improved algorithm model.

[0244] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0245] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0246] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0247] [Second Embodiment]

[0248] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0249] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0250] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0251] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0252] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0253] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0254] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0255] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0256] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0257] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0258] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0259] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0260] This invention is an AI-powered system designed to support the improvement of marital relationships, aiming to provide personalized advice based on user input data. Specifically, the system identifies the personality traits of the couple through an analysis device and generates advice to support relationship building based on that information.

[0261] System configuration and operation

[0262] Users answer a personality test using their own devices. This test includes questions designed to provide a detailed understanding of the user's personality traits.

[0263] The device receives the responses from the user and sends them to the server as digital data.

[0264] The server uses an algorithm to analyze personality traits based on the received data. This analysis creates a personality profile for each user.

[0265] The server simultaneously references existing psychology databases to identify potential problems and areas for improvement based on personality traits. Based on the identified issues, it generates advice including multiple options for improving relationships.

[0266] The device presents the generated advice to the user, enabling them to implement it in their daily life.

[0267] The user tries out the suggested advice and returns the results and feedback to the system as a response.

[0268] The terminal sends the user's response to the server and initiates the feedback process.

[0269] The server analyzes the feedback and utilizes its learning capabilities to adjust its algorithms to improve the accuracy of future advice.

[0270] Specific example

[0271] For example, if a married couple uses this system, they might each take a personality assessment first, which could reveal that their individual personality traits "need harmony." Based on this result, the server generates specific advice such as "have a shared hobby once a week" or "understand differences of opinion." Users then put this advice into practice and provide feedback on the results and changes, which helps the system improve its ability to provide even more effective support.

[0272] With this system, couples can receive specific and actionable advice based on their individual personality traits and improve their relationship.

[0273] The following is an explanation of the processing flow.

[0274] Step 1:

[0275] The user launches the application and accesses the personality diagnosis test. The user enters answers to the test questions and presses the "Send" button when finished.

[0276] Step 2:

[0277] The terminal collects the user's answer data and formats it as digital information. Then, this data is sent to the server.

[0278] Step 3:

[0279] The server receives the received answer data, applies a personality diagnosis algorithm to analyze the user's personality traits, and calculates scores for MBTI and the Big Five in this process.

[0280] Step 4:

[0281] Based on the analysis results, the server forms the user and partner's personality profiles, and while referring to the existing psychological database, generates specific advice necessary for relationship improvement.

[0282] Step 5:

[0283] The server sends the generated advice to the terminal. This advice includes actionable action steps and specific proposals for improving relationships.

[0284] Step 6:

[0285] The terminal provides an interface for displaying advice to the user and promoting its application in daily life.

[0286] Step 7:

[0287] The user executes the provided advice and records feedback on the results and effects within the application. This feedback includes the actions taken and the observed changes.

[0288] Step 8:

[0289] The terminal sends the feedback data from the user to the server. The feedback is processed so that the next advice is more personalized and effective.

[0290] Step 9:

[0291] The server analyzes the received feedback and adjusts and improves the advice generation algorithm. This evolves the system so that more personalized proposals can be made to future users.

[0292] (Example 1)

[0293] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0294] In conventional human relationship building support systems, there is often a lack of a detailed approach based on individual characteristics and the ability to provide only general advice is an issue. Therefore, there has been a problem that it is difficult to effectively improve relationships with others. In particular, there is a need to provide more personalized support by offering specific advice according to an individual's personality and situation.

[0295] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0296] In this invention, the server includes means for analyzing personal characteristics based on input data obtained from the user, means for generating personalized relationship improvement advice by referring to the analysis results and a set of psychological data, and means for notifying the user of the generated advice using a terminal and receiving the user's response. This makes it possible to provide advice tailored to individual situations and characteristics.

[0297] "User" refers to an individual who uses the system to input information or receive services.

[0298] "Input data" refers to information provided by the user and processed by the system.

[0299] "Personal characteristics" refer to the unique personality and behavioral traits of the user.

[0300] "Analysis" refers to calculations and processing performed using input data to identify individual characteristics.

[0301] A "psychological data collection" refers to a database of information and knowledge based on the psychological characteristics of humans.

[0302] "Advice" refers to information that includes specific actions or suggestions aimed at improving the relationship with the user.

[0303] A "terminal" refers to an electronic device used by a user to access a system.

[0304] A "server" refers to a computer system that stores and processes data and controls the sending and receiving of information between it and terminals.

[0305] "Calculation method" refers to the calculation methods and algorithms that a system uses to process data and generate advice.

[0306] "Adaptive" means that the system has the function of appropriately adjusting its operations and outputs based on new information obtained from the user.

[0307] In the system based on this invention, mainly the server, the terminal, and the user exchange data with each other, and provide individual advice aiming at improving the marital relationship. The details will be described below.

[0308] Hardware and Software to be Used

[0309] The terminal is used as a means for the user to answer questions and receive the generated advice. Digital devices such as smartphones, tablets, and personal computers correspond to this. Specific applications are installed on these terminals, and the user uses this application to access the system.

[0310] The server is a computer system for receiving and processing data transmitted from the terminal. An algorithm for data analysis is implemented in the server, which analyzes the input data of the user and generates advice for relationship improvement.

[0311] The generated AI model operates within the server and is in charge of data analysis and advice generation. Thereby, advice optimized for each user is created based on the psychological data set. This AI model has a feedback learning function and utilizes the response data from the user to improve its prediction accuracy.

[0312] Specific Operations and Prompt Sentence Examples

[0313] The user accesses the system via a terminal such as a smartphone and takes a personality diagnosis test. The results of this test are transmitted to the server as digital data.

[0314] Based on the transmitted data, the server generates advice for improving relationships using an analysis algorithm and a generative AI model. The generated advice is sent to the relevant user's device and presented to the user.

[0315] For example, if a couple uses this system and a personality test reveals that they "need harmony," the server will generate advice such as, "Try to engage in a shared hobby together once a week." Users then put this advice into practice in their daily lives and send the results back to the system as feedback.

[0316] Example of a prompt:

[0317] "Based on the personality assessment results, please provide specific advice on how to improve our marital relationship. The assessment result is 'Harmony is needed.'"

[0318] In this way, the system provides advice to improve interpersonal relationships in a way that is beneficial to each individual user. Furthermore, through continuous feedback, the system's ability to provide even more effective advice improves.

[0319] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0320] Step 1:

[0321] The user launches the application on their device and starts the personality test. The input consists of answers to various questions. Specifically, the user answers multiple-choice questions and saves the answers as data on their device by pressing the submit button. As output, all the answers are recorded on the device as a single dataset.

[0322] Step 2:

[0323] The terminal receives the stored user response data, encrypts it, and sends it to the server. Specifically, the terminal uses a secure data transmission protocol to send encrypted data packets to the server over the internet. The input is the user's response data, and the output is a file in digital data format that reaches the server.

[0324] Step 3:

[0325] The server receives and decrypts transmitted data packets. It handles encrypted user response data as input. Based on this, it applies data analysis algorithms to analyze the user's characteristics. Specifically, the server uses a generative AI model to analyze the data and generate a user personality profile. The output is the analyzed personality data.

[0326] Step 4:

[0327] The server uses a generative AI model to generate advice for improving relationships based on analyzed personality traits. The input consists of analyzed personality trait data and a set of psychological data. Specifically, the server runs the AI ​​model, evaluates the attribute data, and forms prompt sentences that generate appropriate advice. The output is personalized advice.

[0328] Step 5:

[0329] The server sends the generated advice to the terminal. The input is pre-generated advice data. Specifically, the server uses a data transfer protocol to securely send the advice to the terminal. The output is the received advice displayed on the terminal.

[0330] Step 6:

[0331] Users review the advice displayed on their device and attempt to put it into practice in their daily lives. The input includes specific advice. The specific action involves the user taking new actions based on the provided advice to improve their relationships.

[0332] Step 7:

[0333] The user inputs the results of the execution as feedback into the terminal and sends that data to the server. The input includes feedback information regarding the results of implementing the advice. Specifically, the user fills out a feedback form and submits it. The output reaches the server as feedback.

[0334] Step 8:

[0335] The server analyzes the received feedback and modifies the calculation method of the system's generated AI model to improve its accuracy. The input is user feedback data. Specifically, the server retrains the generated AI model and performs a learning process based on the feedback. The output is an improved algorithm, which is used for future advice generation.

[0336] (Application Example 1)

[0337] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0338] In modern society, marital relationships frequently deteriorate due to a lack of communication and personality clashes. This can lead to a negative atmosphere within the home and, in some cases, ultimately result in divorce. To address this issue, couples need to understand each other's personality traits and accept appropriate advice to build a better relationship.

[0339] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0340] In this invention, the server includes means for identifying personality traits based on personality attribute data collected from the user, means for generating a plurality of personalized pieces of advice based on the identified personality traits, and means for collecting conversational data using a speech recognition device. This makes it possible for users to easily understand their own personality traits and receive necessary advice in real time.

[0341] A "user" is an individual who uses this system to undergo a personality assessment and receive advice.

[0342] "Personality attribute data" refers to information about a user's personality extracted from their responses to personality assessment tests and conversation data.

[0343] "Personality traits" refer to the characteristics and tendencies of an individual's personality, analyzed based on the user's personality attribute data.

[0344] "Advice" refers to specific suggestions for improving relationships, generated based on the user's personality traits.

[0345] A "speech recognition device" is a device that collects the user's conversation as audio data and converts it into text data.

[0346] "Natural language processing technology" is a technique that analyzes collected conversational data to understand the intentions and emotions of the users.

[0347] The system implementing this invention operates around a consumer robot equipped with voice recognition capabilities, which is placed in each home. First, the user undergoes a personality assessment through conversation with the robot. Voice data is collected through a high-performance microphone built into the robot and converted into text data using the Google Speech-to-Text API. This converted data is then sent to a server.

[0348] The server uses OpenAI's GPT model to analyze text data and generate user personality trait data. The analysis process utilizes natural language processing techniques to identify the user's personality using information obtained from the conversation. Based on this profile information, the server consults a psychology database to generate personalized advice for improving relationships.

[0349] The generated advice is sent from the server to the robot, which outputs it as voice. At this time, the robot clearly explains the suggested advice to the user. The user tries out the presented advice and provides feedback to the robot about the results and their impressions.

[0350] User feedback is also sent to the server via the robot. The server uses this feedback to adaptively train its algorithm, improving the accuracy of future advice. This process is designed to ensure users continuously receive specific and helpful advice for improving their relationships.

[0351] For example, if a user consults the robot saying, "We've been having a lot of disagreements lately," the robot might respond with advice such as, "How about setting aside some time to come to an agreement?" By having the user provide feedback on how they feel after implementing this advice day by day, the server's generated AI model learns to suggest more appropriate advice.

[0352] An example of a prompt for the generating AI model is: "Analyze the user's conversation log and generate advice that will help improve the relationship between the couple."

[0353] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0354] Step 1:

[0355] The user initiates a conversation with the robot. Voice data is collected through a high-performance microphone inside the robot. The input is voice data, and the output is text data converted by the Google Speech-to-Text API. This conversion process analyzes the characteristics of the voice signal and converts them into text information.

[0356] Step 2:

[0357] The terminal, or robot, sends this text data to the server. The input here is the text data generated in step 1, and the output is the digital data to be sent to the server. The data is delivered accurately to the server using a communication protocol.

[0358] Step 3:

[0359] The server performs natural language processing on the received text data. The input is text data of the user's conversation, and the output is user personality trait data extracted through analysis. A generative AI model is used to analyze the content of the text and perform data processing and calculations to extract the user's personality traits.

[0360] Step 4:

[0361] The server references a psychology database based on personality trait data and generates corresponding advice. The input is personality trait data, and the output is personalized advice. This process applies algorithms for database searching and optimal advice generation.

[0362] Step 5:

[0363] The server sends the generated advice to the robot. The input is the text data of the generated advice, and the output is the data to be sent to the robot. The advice is processed via a secure communication channel to ensure it reaches the robot safely.

[0364] Step 6:

[0365] The robot conveys the received advice to the user through a voice output device. The input is text data of the advice, and the output is voice information for the user. Speech synthesis technology is used to convert the text into a natural conversational style, which is then played through the speaker.

[0366] Step 7:

[0367] The user follows the robot's advice and provides voice feedback on the results and their impressions. The input is voice data of the user's actions, and the output is the robot's re-collection of voice data. This data is then converted back into text using speech recognition technology and prepared to be sent to the server.

[0368] Step 8:

[0369] The server receives feedback data, evaluates the effectiveness of the advice using a generated AI model, and learns from success stories and failure factors. The input is user feedback text data, and the output is adjustment data for algorithm training. This process aims to improve the accuracy of future advice generation.

[0370] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0371] This invention aims to support the improvement of marital relationships using an AI system equipped with emotion recognition capabilities. The system analyzes the user's personality traits and emotional state, and provides more precise and personalized advice.

[0372] System configuration and operation

[0373] Users participate in everyday interactions and personality tests using an application equipped with an emotion engine. The emotion engine evaluates the user's voice tone, facial expressions, and text data in real time to recognize their emotional state.

[0374] The terminal sends data obtained from the user (responses to personality assessments and emotional data) to an analysis device, which then formats the data.

[0375] After receiving data from the terminal, the server analyzes it to identify the user's personality traits and emotional profile. The emotion engine then incorporates the identified emotional states as influencing factors into the analysis.

[0376] The server generates advice aimed at improving relationships, taking into account both emotional data and personality traits. This advice is optimized for the user's current emotional state.

[0377] The device presents the generated advice to the user in real time. The content and tone of the advice are adjusted based on feedback from the emotion engine.

[0378] Users act on the advice presented in their daily lives and input the results as feedback into the system. This feedback is used to re-evaluate the user's emotional state.

[0379] The server analyzes the collected feedback and uses the system's learning capabilities to adjust the advice generation algorithm.

[0380] Specific example

[0381] For example, if a user's emotion engine detects voice patterns indicating stress and text messages indicating depression, the server will generate advice such as, "We recommend taking a walk in nature to refresh yourself," or "Let's make time for a relaxing conversation with your partner." If that advice is insufficient, encouraging messages will be added. In this way, the system approaches the user from both an emotional and personality perspective, enabling more practical and impactful support.

[0382] This invention allows couples to receive advice that takes their emotional state into consideration, helping them to improve their relationship more effectively.

[0383] The following describes the processing flow.

[0384] Step 1:

[0385] The user launches the application and inputs data through a personality test and daily activities. The emotion engine obtains the user's emotional state in real time from their voice tone, facial expressions, and text.

[0386] Step 2:

[0387] The terminal collects user input data and emotional state data, and formats this data into digital information. It then sends the data to the server.

[0388] Step 3:

[0389] The server analyzes the received data, calculates the user's personality traits, and identifies their emotional state. Simultaneously, it incorporates the evaluation results from the emotion engine to generate a personality profile that combines the emotional states.

[0390] Step 4:

[0391] Based on the identified personality profile and emotional state, the server generates specific advice for improving relationships. This advice will be tailored to the user's current emotional state.

[0392] Step 5:

[0393] The device displays the generated advice to the user. The advice has a message tone adjusted to take into account feedback from the emotion engine.

[0394] Step 6:

[0395] Users implement the advice in their daily lives and input feedback on its effects and their impressions into the application. This feedback includes changes in their emotional state.

[0396] Step 7:

[0397] The device sends the collected feedback to the server. The feedback data is used to re-evaluate the user's emotional state.

[0398] Step 8:

[0399] The server analyzes the feedback and modifies the advice generation algorithm to improve accuracy. In particular, adjustments that reflect the results of the emotion engine improve the quality of advice for subsequent sessions.

[0400] (Example 2)

[0401] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0402] The present invention aims to support effective relationship improvement in marital relationships and other interpersonal relationships by providing personalized advice that takes into account the user's emotional state and personality traits. However, conventional approaches have the problem of not adequately providing immediate advice based on the user's real-time emotional state, nor does it allow for adaptive learning through subsequent feedback.

[0403] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0404] In this invention, the server includes means for processing audio, video, and text data collected from the user and formatting them into a format suitable for analysis; means for analyzing the formatted data and generating a profile to identify the user's emotional state; and means for generating multiple personalized pieces of advice based on the generated profile, taking into account the emotional state and personality traits. This enables the provision of effective advice tailored to the user's current situation and supports continuous relationship improvement based on that advice.

[0405] "User" refers to the person who will receive analysis of their emotional state and advice using this system.

[0406] "Audio data" refers to acoustic information, including the user's speaking style and tone of voice.

[0407] "Video data" refers to visual information, including the user's facial expressions and gestures.

[0408] "Text data" refers to written characters or messages entered by the user.

[0409] "Formatting" refers to the process of converting and organizing collected raw data so that it can be easily analyzed.

[0410] A "profile" refers to a dataset that aggregates a user's emotional state and personality traits and represents them in an identifiable format.

[0411] "Personalized advice" refers to advice specifically created based on each user's emotional state and personality traits.

[0412] A "generative AI model" refers to an algorithm and system that uses data to create new outputs.

[0413] "Feedback" refers to the act of providing the system with information about the advice the user has taken and the subsequent changes in their emotional state.

[0414] "Learning function" refers to the process of improving the system's algorithms based on collected data and feedback.

[0415] This invention uses an AI system equipped with emotion recognition capabilities to support the improvement of users' interpersonal relationships. Specific embodiments are described below.

[0416] Users provide data through everyday conversations and personality tests using an application with a built-in emotion engine. This application records the user's voice with a microphone, captures facial expressions with a webcam, and accepts text-based input. All the data collected in this way is sent to the emotion engine, which evaluates the user's emotional state in real time.

[0417] The terminal formats the audio, video, and text data acquired from the user into an appropriate format and performs the necessary preprocessing for sentiment analysis. Specifically, this includes noise reduction of audio data and standardization of facial expression data. The formatted data is then sent to the server for analysis.

[0418] The server receives data sent from the terminal and uses a generative AI model to create a user's emotional profile. This profile includes the user's current emotional state and long-term personality traits. Based on the emotional profile, the server generates optimized advice for the user. This advice consists of helpful suggestions and encouraging messages tailored to the user's current situation.

[0419] For example, if a user sends a voice message indicating stress and text indicating depression, the server will generate advice such as "We recommend taking a walk in nature to refresh yourself" or "Make time for a relaxing conversation with your partner." This advice is delivered in a way that best suits the user's emotional state, and encouraging messages may be added as needed.

[0420] An example of an input prompt for the generating AI model might be: "The user's voice patterns indicate stress, and the text data shows signs of depression. Based on this information, generate the most appropriate relationship improvement advice for the user." This enables an intelligent approach tailored to the individual user's state.

[0421] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0422] Step 1:

[0423] Users provide audio, video, and text data through the application. Specifically, users speak into a microphone, capture their facial expressions with a webcam, and input text. This data serves as input for evaluating the user's emotional state.

[0424] Step 2:

[0425] The terminal receives data from the user and formats it. Audio data undergoes noise reduction, video data undergoes facial expression extraction and standardization, and text data undergoes keyword extraction. These processes result in a dataset converted into a format that is easy to analyze.

[0426] Step 3:

[0427] The terminal sends formatted data to the server. The output here is integrated data containing voice tone, facial expression patterns, and text keyword information.

[0428] Step 4:

[0429] The server analyzes the received data and uses a generative AI model to generate a user's emotional profile. This profile reflects both short-term emotional states and long-term personality traits and serves as input for subsequent processing.

[0430] Step 5:

[0431] The server generates personalized advice for the user based on their emotional profile. It uses the input as prompts for a generating AI model to form the advice. The advice might include suggestions for refreshing oneself or prompts for further dialogue.

[0432] Step 6:

[0433] The device receives advice generated from the server and presents it to the user. The content and tone of the advice are adjusted based on the user's latest emotional feedback. This ensures that the advice is delivered in a form that is most appropriate for the user.

[0434] Step 7:

[0435] The user enters feedback into the system regarding the advice they have received. This feedback includes information about the results of the actions taken and changes in their feelings, and is used in the next step.

[0436] Step 8:

[0437] The server analyzes the feedback and adjusts the generated AI model based on it. Through its learning function, it improves the advice generation algorithm and enhances the quality of subsequent outputs.

[0438] (Application Example 2)

[0439] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0440] In interpersonal relationships, particularly within families, communication can be hindered by individual emotional states, leading to a deterioration of relationships. There is a need to provide effective means to resolve this issue and promote relationship improvement.

[0441] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0442] In this invention, the server includes means for identifying an emotional state based on voice signals and facial expression data collected from the user, means for generating personalized advice based on the identified emotional state and personality traits, and means for presenting the generated advice to the user through voice and visual output and receiving feedback from the user. This enables effective communication and relationship improvement tailored to each individual's emotional state.

[0443] A "user" is an individual who provides voice and facial expression data to the system and receives advice as a result.

[0444] "Speech signals" are data that describes the characteristics of sound waves obtained from the user's speech.

[0445] "Facial expression data" refers to video information used to analyze the visual characteristics of a user's face.

[0446] "Emotional state" refers to the expression of the user's inner emotions, identified based on voice signals and facial expression data.

[0447] "Personality traits" are characteristic attributes related to the user's personality and individuality.

[0448] "Personalized advice" refers to specific and tailored behavioral guidelines for the user, generated based on identified emotional states and personality traits.

[0449] "Audio and visual output" refers to the audio and video formats used to convey generated information to the user.

[0450] "Feedback" refers to information that users send back to the server regarding their responses and results to the advice they receive.

[0451] "Learning function" refers to the ability to adjust the advice generation algorithm based on received feedback to improve the overall accuracy and effectiveness of the system.

[0452] This invention describes a specific embodiment of a system aimed at improving interpersonal relationships within the home. The system has a process of collecting voice signals and facial expression data from the user and identifying the emotional state based on them.

[0453] The server uses a platform equipped with high-precision sensors to process audio signals and facial expression data. For example, a microphone is used to analyze the tone of voice, and a camera is used to analyze facial expression data. This makes it possible to identify the user's emotional state in real time.

[0454] The server combines identified emotional states with pre-collected personality traits and uses a generative AI model like OpenAI to generate personalized advice. This generated advice is presented to the user via the device in audio and video formats. In this way, the user can interactively receive advice and take subsequent actions based on the results.

[0455] As a concrete example, if the system detects that one user in a married couple is experiencing stress, it analyzes their emotional state on a server and provides advice such as, "Why not take a walk in a nature park with your partner today to refresh yourselves?" By making such suggestions, the system can support improvements in everyday relationships.

[0456] An example of a prompt for a generative AI model is, "Please suggest an activity suitable for a relaxed emotional state."

[0457] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0458] Step 1:

[0459] The device collects the user's voice signals and facial expression data. Using a microphone and camera as sensors, it acquires audio and video in real time and formats them as digital data. The input is the user's physical voice and facial expressions, and the output is this digital data.

[0460] Step 2:

[0461] The terminal transmits the collected digital data to the server. The server performs speech recognition and facial expression analysis based on the received data. It analyzes the trends in speech tone and the facial features obtained from facial expressions, and performs data processing to identify the emotional state. The input is digitized audio and video data, and the output is an index indicating the user's emotional state.

[0462] Step 3:

[0463] The server integrates identified emotional states with already recorded personality traits and uses a generative AI model to create optimal advice. Here, prompts are set according to the emotional state, and the AI ​​model performs reasoning. The input is the emotional state and personality traits, and the output is a personalized advice message.

[0464] Step 4:

[0465] The server sends the generated advice to the terminal, which then presents it to the user as audio and visual output. Speech synthesis is used to provide information to the user in a natural way. The input is the advice message, and the output is the presentation of information verbally and visually.

[0466] Step 5:

[0467] The user acts based on the advice provided and inputs the results as feedback data into the terminal. The terminal forwards the feedback to the server. The input is feedback information from the user, and the output is an update to the system's history data.

[0468] Step 6:

[0469] The server analyzes feedback data, learns the advice generation algorithm, and adjusts parameters to improve accuracy. The feedback data is used as an evaluation metric for the algorithm and provides material for future improvements. The input is the feedback data, and the output is the improved algorithm model.

[0470] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0471] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0472] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0473] [Third Embodiment]

[0474] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0475] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0476] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0477] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0478] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0479] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0480] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0481] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0482] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0483] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0484] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0485] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0486] This invention is an AI-powered system designed to support the improvement of marital relationships, aiming to provide personalized advice based on user input data. Specifically, the system identifies the personality traits of the couple through an analysis device and generates advice to support relationship building based on that information.

[0487] System configuration and operation

[0488] Users answer a personality test using their own devices. This test includes questions designed to provide a detailed understanding of the user's personality traits.

[0489] The device receives the responses from the user and sends them to the server as digital data.

[0490] The server uses an algorithm to analyze personality traits based on the received data. This analysis creates a personality profile for each user.

[0491] The server simultaneously references existing psychology databases to identify potential problems and areas for improvement based on personality traits. Based on the identified issues, it generates advice including multiple options for improving relationships.

[0492] The device presents the generated advice to the user, enabling them to implement it in their daily life.

[0493] The user tries out the suggested advice and returns the results and feedback to the system as a response.

[0494] The terminal sends the user's response to the server and initiates the feedback process.

[0495] The server analyzes the feedback and utilizes its learning capabilities to adjust its algorithms to improve the accuracy of future advice.

[0496] Specific example

[0497] For example, if a married couple uses this system, they might each take a personality assessment first, which could reveal that their individual personality traits "need harmony." Based on this result, the server generates specific advice such as "have a shared hobby once a week" or "understand differences of opinion." Users then put this advice into practice and provide feedback on the results and changes, which helps the system improve its ability to provide even more effective support.

[0498] This system allows couples to receive specific, actionable advice based on their individual personality traits, enabling them to improve their relationship.

[0499] The following describes the processing flow.

[0500] Step 1:

[0501] The user launches the application and accesses the personality test. They enter their answers to the test questions and press the "Submit" button when finished.

[0502] Step 2:

[0503] The device collects the user's response data and formats it as digital information. It then sends this data to the server.

[0504] Step 3:

[0505] The server receives the submitted response data and applies a personality assessment algorithm to analyze the user's personality traits. This process calculates MBTI and Big Five scores.

[0506] Step 4:

[0507] Based on the analysis results, the server creates personality profiles of the user and their partner, and, referencing these profiles with an existing psychological database, generates specific advice necessary for improving the relationship.

[0508] Step 5:

[0509] The server sends the generated advice to the terminal. This advice includes actionable steps and specific suggestions for improving relationships.

[0510] Step 6:

[0511] The device displays advice to the user and provides an interface to encourage its application in daily life.

[0512] Step 7:

[0513] Users follow the advice provided and record feedback on the results and effects within the application. This feedback includes the actions taken and the changes observed.

[0514] Step 8:

[0515] The device sends user feedback data to the server. This feedback is processed to make future advice more personalized and effective.

[0516] Step 9:

[0517] The server analyzes the feedback it receives and adjusts and improves the advice generation algorithm. This allows the system to evolve so that it can provide more personalized suggestions to future users.

[0518] (Example 1)

[0519] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0520] Traditional relationship-building support systems often lack the nuanced approach based on individual characteristics, and are limited to providing general advice. This makes it difficult to effectively improve relationships with others. In particular, there is a need for more personalized support that provides specific advice tailored to each individual's personality and circumstances.

[0521] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0522] In this invention, the server includes means for analyzing personal characteristics based on input data obtained from the user, means for generating personalized relationship improvement advice by referring to the analysis results and a set of psychological data, and means for notifying the user of the generated advice using a terminal and receiving the user's response. This makes it possible to provide advice tailored to individual situations and characteristics.

[0523] "User" refers to an individual who uses the system to input information or receive services.

[0524] "Input data" refers to information provided by the user and processed by the system.

[0525] "Personal characteristics" refer to the unique personality and behavioral traits of the user.

[0526] "Analysis" refers to calculations and processing performed using input data to identify individual characteristics.

[0527] A "psychological data collection" refers to a database of information and knowledge based on the psychological characteristics of humans.

[0528] "Advice" refers to information that includes specific actions or suggestions aimed at improving the relationship with the user.

[0529] A "terminal" refers to an electronic device used by a user to access a system.

[0530] A "server" refers to a computer system that stores and processes data and controls the sending and receiving of information between it and terminals.

[0531] "Calculation method" refers to the calculation methods and algorithms that a system uses to process data and generate advice.

[0532] "Adaptive" means that the system has the ability to adjust its operation and output as needed based on new information it receives from the user.

[0533] In this invention, the system primarily involves a server, terminal, and user exchanging data with each other to provide personalized advice aimed at improving marital relationships. The details are described below.

[0534] Hardware and software to use

[0535] The terminal is used by the user as a means of answering questions and receiving generated advice. This includes digital devices such as smartphones, tablets, and personal computers. Specific applications are installed on these terminals, and users access the system using these applications.

[0536] A server is a computing system that receives and processes data transmitted from terminals. The server implements algorithms for data analysis, analyzing user input data and generating advice for improving relationships.

[0537] The generative AI model operates on a server and is responsible for data analysis and advice generation. This allows it to create personalized advice based on a set of psychological data. The AI ​​model also has a feedback learning function, utilizing user response data to improve its prediction accuracy.

[0538] Specific actions and prompt examples

[0539] Users access the system via a smartphone or other device and take a personality test. The results of this test are sent to the server as digital data.

[0540] Based on the transmitted data, the server generates advice for improving relationships using an analysis algorithm and a generative AI model. The generated advice is sent to the relevant user's device and presented to the user.

[0541] For example, if a couple uses this system and a personality test reveals that they "need harmony," the server will generate advice such as, "Try to engage in a shared hobby together once a week." Users then put this advice into practice in their daily lives and send the results back to the system as feedback.

[0542] Example of a prompt:

[0543] "Based on the personality assessment results, please provide specific advice on how to improve our marital relationship. The assessment result is 'Harmony is needed.'"

[0544] In this way, the system provides advice to improve interpersonal relationships in a way that is beneficial to each individual user. Furthermore, through continuous feedback, the system's ability to provide even more effective advice improves.

[0545] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0546] Step 1:

[0547] The user launches the application on their device and starts the personality test. The input consists of answers to various questions. Specifically, the user answers multiple-choice questions and saves the answers as data on their device by pressing the submit button. As output, all the answers are recorded on the device as a single dataset.

[0548] Step 2:

[0549] The terminal receives the stored user response data, encrypts it, and sends it to the server. Specifically, the terminal uses a secure data transmission protocol to send encrypted data packets to the server over the internet. The input is the user's response data, and the output is a file in digital data format that reaches the server.

[0550] Step 3:

[0551] The server receives and decrypts transmitted data packets. It handles encrypted user response data as input. Based on this, it applies data analysis algorithms to analyze the user's characteristics. Specifically, the server uses a generative AI model to analyze the data and generate a user personality profile. The output is the analyzed personality data.

[0552] Step 4:

[0553] The server uses a generative AI model to generate advice for improving relationships based on analyzed personality traits. The input consists of analyzed personality trait data and a set of psychological data. Specifically, the server runs the AI ​​model, evaluates the attribute data, and forms prompt sentences that generate appropriate advice. The output is personalized advice.

[0554] Step 5:

[0555] The server sends the generated advice to the terminal. The input is pre-generated advice data. Specifically, the server uses a data transfer protocol to securely send the advice to the terminal. The output is the received advice displayed on the terminal.

[0556] Step 6:

[0557] Users review the advice displayed on their device and attempt to put it into practice in their daily lives. The input includes specific advice. The specific action involves the user taking new actions based on the provided advice to improve their relationships.

[0558] Step 7:

[0559] The user inputs the results of the execution as feedback into the terminal and sends that data to the server. The input includes feedback information regarding the results of implementing the advice. Specifically, the user fills out a feedback form and submits it. The output reaches the server as feedback.

[0560] Step 8:

[0561] The server analyzes the received feedback and modifies the calculation method of the system's generated AI model to improve its accuracy. The input is user feedback data. Specifically, the server retrains the generated AI model and performs a learning process based on the feedback. The output is an improved algorithm, which is used for future advice generation.

[0562] (Application Example 1)

[0563] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0564] In modern society, marital relationships frequently deteriorate due to a lack of communication and personality clashes. This can lead to a negative atmosphere within the home and, in some cases, ultimately result in divorce. To address this issue, couples need to understand each other's personality traits and accept appropriate advice to build a better relationship.

[0565] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0566] In this invention, the server includes means for identifying personality traits based on personality attribute data collected from the user, means for generating a plurality of personalized pieces of advice based on the identified personality traits, and means for collecting conversational data using a speech recognition device. This makes it possible for users to easily understand their own personality traits and receive necessary advice in real time.

[0567] A "user" is an individual who uses this system to undergo a personality assessment and receive advice.

[0568] "Personality attribute data" refers to information about a user's personality extracted from their responses to personality assessment tests and conversation data.

[0569] "Personality traits" refer to the characteristics and tendencies of an individual's personality, analyzed based on the user's personality attribute data.

[0570] "Advice" refers to specific suggestions for improving relationships, generated based on the user's personality traits.

[0571] A "speech recognition device" is a device that collects the user's conversation as audio data and converts it into text data.

[0572] "Natural language processing technology" is a technique that analyzes collected conversational data to understand the intentions and emotions of the users.

[0573] The system implementing this invention operates around a consumer robot equipped with voice recognition capabilities, which is placed in each home. First, the user undergoes a personality assessment through conversation with the robot. Voice data is collected through a high-performance microphone built into the robot and converted into text data using the Google Speech-to-Text API. This converted data is then sent to a server.

[0574] The server uses OpenAI's GPT model to analyze text data and generate user personality trait data. The analysis process utilizes natural language processing techniques to identify the user's personality using information obtained from the conversation. Based on this profile information, the server consults a psychology database to generate personalized advice for improving relationships.

[0575] The generated advice is sent from the server to the robot, which outputs it as voice. At this time, the robot clearly explains the suggested advice to the user. The user tries out the presented advice and provides feedback to the robot about the results and their impressions.

[0576] User feedback is also sent to the server via the robot. The server uses this feedback to adaptively train its algorithm, improving the accuracy of future advice. This process is designed to ensure users continuously receive specific and helpful advice for improving their relationships.

[0577] For example, if a user consults the robot saying, "We've been having a lot of disagreements lately," the robot might respond with advice such as, "How about setting aside some time to come to an agreement?" By having the user provide feedback on how they feel after implementing this advice day by day, the server's generated AI model learns to suggest more appropriate advice.

[0578] An example of a prompt for the generating AI model is: "Analyze the user's conversation log and generate advice that will help improve the relationship between the couple."

[0579] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0580] Step 1:

[0581] The user initiates a conversation with the robot. Voice data is collected through a high-performance microphone inside the robot. The input is voice data, and the output is text data converted by the Google Speech-to-Text API. This conversion process analyzes the characteristics of the voice signal and converts them into text information.

[0582] Step 2:

[0583] The terminal, or robot, sends this text data to the server. The input here is the text data generated in step 1, and the output is the digital data to be sent to the server. The data is delivered accurately to the server using a communication protocol.

[0584] Step 3:

[0585] The server performs natural language processing on the received text data. The input is text data of the user's conversation, and the output is user personality trait data extracted through analysis. A generative AI model is used to analyze the content of the text and perform data processing and calculations to extract the user's personality traits.

[0586] Step 4:

[0587] The server references a psychology database based on personality trait data and generates corresponding advice. The input is personality trait data, and the output is personalized advice. This process applies algorithms for database searching and optimal advice generation.

[0588] Step 5:

[0589] The server sends the generated advice to the robot. The input is the text data of the generated advice, and the output is the data to be sent to the robot. The advice is processed via a secure communication channel to ensure it reaches the robot safely.

[0590] Step 6:

[0591] The robot conveys the received advice to the user through a voice output device. The input is text data of the advice, and the output is voice information for the user. Speech synthesis technology is used to convert the text into a natural conversational style, which is then played through the speaker.

[0592] Step 7:

[0593] The user follows the robot's advice and provides voice feedback on the results and their impressions. The input is voice data of the user's actions, and the output is the robot's re-collection of voice data. This data is then converted back into text using speech recognition technology and prepared to be sent to the server.

[0594] Step 8:

[0595] The server receives feedback data, evaluates the effectiveness of the advice using a generated AI model, and learns from success stories and failure factors. The input is user feedback text data, and the output is adjustment data for algorithm training. This process aims to improve the accuracy of future advice generation.

[0596] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0597] This invention aims to support the improvement of marital relationships using an AI system equipped with emotion recognition capabilities. The system analyzes the user's personality traits and emotional state, and provides more precise and personalized advice.

[0598] System configuration and operation

[0599] Users participate in everyday interactions and personality tests using an application equipped with an emotion engine. The emotion engine evaluates the user's voice tone, facial expressions, and text data in real time to recognize their emotional state.

[0600] The terminal sends data obtained from the user (responses to personality assessments and emotional data) to an analysis device, which then formats the data.

[0601] After receiving data from the terminal, the server analyzes it to identify the user's personality traits and emotional profile. The emotion engine then incorporates the identified emotional states as influencing factors into the analysis.

[0602] The server generates advice aimed at improving relationships, taking into account both emotional data and personality traits. This advice is optimized for the user's current emotional state.

[0603] The device presents the generated advice to the user in real time. The content and tone of the advice are adjusted based on feedback from the emotion engine.

[0604] Users act on the advice presented in their daily lives and input the results as feedback into the system. This feedback is used to re-evaluate the user's emotional state.

[0605] The server analyzes the collected feedback and uses the system's learning capabilities to adjust the advice generation algorithm.

[0606] Specific example

[0607] For example, if a user's emotion engine detects voice patterns indicating stress and text messages indicating depression, the server will generate advice such as, "We recommend taking a walk in nature to refresh yourself," or "Let's make time for a relaxing conversation with your partner." If that advice is insufficient, encouraging messages will be added. In this way, the system approaches the user from both an emotional and personality perspective, enabling more practical and impactful support.

[0608] This invention allows couples to receive advice that takes their emotional state into consideration, helping them to improve their relationship more effectively.

[0609] The following describes the processing flow.

[0610] Step 1:

[0611] The user launches the application and inputs data through a personality test and daily activities. The emotion engine obtains the user's emotional state in real time from their voice tone, facial expressions, and text.

[0612] Step 2:

[0613] The terminal collects user input data and emotional state data, and formats this data into digital information. It then sends the data to the server.

[0614] Step 3:

[0615] The server analyzes the received data, calculates the user's personality traits, and identifies their emotional state. Simultaneously, it incorporates the evaluation results from the emotion engine to generate a personality profile that combines the emotional states.

[0616] Step 4:

[0617] Based on the identified personality profile and emotional state, the server generates specific advice for improving relationships. This advice will be tailored to the user's current emotional state.

[0618] Step 5:

[0619] The device displays the generated advice to the user. The advice has a message tone adjusted to take into account feedback from the emotion engine.

[0620] Step 6:

[0621] Users implement the advice in their daily lives and input feedback on its effects and their impressions into the application. This feedback includes changes in their emotional state.

[0622] Step 7:

[0623] The device sends the collected feedback to the server. The feedback data is used to re-evaluate the user's emotional state.

[0624] Step 8:

[0625] The server analyzes the feedback and modifies the advice generation algorithm to improve accuracy. In particular, adjustments that reflect the results of the emotion engine improve the quality of advice for subsequent sessions.

[0626] (Example 2)

[0627] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0628] The present invention aims to support effective relationship improvement in marital relationships and other interpersonal relationships by providing personalized advice that takes into account the user's emotional state and personality traits. However, conventional approaches have the problem of not adequately providing immediate advice based on the user's real-time emotional state, nor does it allow for adaptive learning through subsequent feedback.

[0629] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0630] In this invention, the server includes means for processing audio, video, and text data collected from the user and formatting them into a format suitable for analysis; means for analyzing the formatted data and generating a profile to identify the user's emotional state; and means for generating multiple personalized pieces of advice based on the generated profile, taking into account the emotional state and personality traits. This enables the provision of effective advice tailored to the user's current situation and supports continuous relationship improvement based on that advice.

[0631] "User" refers to the person who will receive analysis of their emotional state and advice using this system.

[0632] "Audio data" refers to acoustic information, including the user's speaking style and tone of voice.

[0633] "Video data" refers to visual information, including the user's facial expressions and gestures.

[0634] "Text data" refers to written characters or messages entered by the user.

[0635] "Formatting" refers to the process of converting and organizing collected raw data so that it can be easily analyzed.

[0636] A "profile" refers to a dataset that aggregates a user's emotional state and personality traits and represents them in an identifiable format.

[0637] "Personalized advice" refers to advice specifically created based on each user's emotional state and personality traits.

[0638] A "generative AI model" refers to an algorithm and system that uses data to create new outputs.

[0639] "Feedback" refers to the act of providing the system with information about the advice the user has taken and the subsequent changes in their emotional state.

[0640] "Learning function" refers to the process of improving the system's algorithms based on collected data and feedback.

[0641] This invention uses an AI system equipped with emotion recognition capabilities to support the improvement of users' interpersonal relationships. Specific embodiments are described below.

[0642] Users provide data through everyday conversations and personality tests using an application with a built-in emotion engine. This application records the user's voice with a microphone, captures facial expressions with a webcam, and accepts text-based input. All the data collected in this way is sent to the emotion engine, which evaluates the user's emotional state in real time.

[0643] The terminal formats the audio, video, and text data acquired from the user into an appropriate format and performs the necessary preprocessing for sentiment analysis. Specifically, this includes noise reduction of audio data and standardization of facial expression data. The formatted data is then sent to the server for analysis.

[0644] The server receives data sent from the terminal and uses a generative AI model to create a user's emotional profile. This profile includes the user's current emotional state and long-term personality traits. Based on the emotional profile, the server generates optimized advice for the user. This advice consists of helpful suggestions and encouraging messages tailored to the user's current situation.

[0645] For example, if a user sends a voice message indicating stress and text indicating depression, the server will generate advice such as "We recommend taking a walk in nature to refresh yourself" or "Make time for a relaxing conversation with your partner." This advice is delivered in a way that best suits the user's emotional state, and encouraging messages may be added as needed.

[0646] An example of an input prompt for the generating AI model might be: "The user's voice patterns indicate stress, and the text data shows signs of depression. Based on this information, generate the most appropriate relationship improvement advice for the user." This enables an intelligent approach tailored to the individual user's state.

[0647] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0648] Step 1:

[0649] Users provide audio, video, and text data through the application. Specifically, users speak into a microphone, capture their facial expressions with a webcam, and input text. This data serves as input for evaluating the user's emotional state.

[0650] Step 2:

[0651] The terminal receives data from the user and formats it. Audio data undergoes noise reduction, video data undergoes facial expression extraction and standardization, and text data undergoes keyword extraction. These processes result in a dataset converted into a format that is easy to analyze.

[0652] Step 3:

[0653] The terminal sends formatted data to the server. The output here is integrated data containing voice tone, facial expression patterns, and text keyword information.

[0654] Step 4:

[0655] The server analyzes the received data and uses a generative AI model to generate a user's emotional profile. This profile reflects both short-term emotional states and long-term personality traits and serves as input for subsequent processing.

[0656] Step 5:

[0657] The server generates personalized advice for the user based on their emotional profile. It uses the input as prompts for a generating AI model to form the advice. The advice might include suggestions for refreshing oneself or prompts for further dialogue.

[0658] Step 6:

[0659] The device receives advice generated from the server and presents it to the user. The content and tone of the advice are adjusted based on the user's latest emotional feedback. This ensures that the advice is delivered in a form that is most appropriate for the user.

[0660] Step 7:

[0661] The user enters feedback into the system regarding the advice they have received. This feedback includes information about the results of the actions taken and changes in their feelings, and is used in the next step.

[0662] Step 8:

[0663] The server analyzes the feedback and adjusts the generated AI model based on it. Through its learning function, it improves the advice generation algorithm and enhances the quality of subsequent outputs.

[0664] (Application Example 2)

[0665] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0666] In interpersonal relationships, particularly within families, communication can be hindered by individual emotional states, leading to a deterioration of relationships. There is a need to provide effective means to resolve this issue and promote relationship improvement.

[0667] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0668] In this invention, the server includes means for identifying an emotional state based on voice signals and facial expression data collected from the user, means for generating personalized advice based on the identified emotional state and personality traits, and means for presenting the generated advice to the user through voice and visual output and receiving feedback from the user. This enables effective communication and relationship improvement tailored to each individual's emotional state.

[0669] A "user" is an individual who provides voice and facial expression data to the system and receives advice as a result.

[0670] "Speech signals" are data that describes the characteristics of sound waves obtained from the user's speech.

[0671] "Facial expression data" refers to video information used to analyze the visual characteristics of a user's face.

[0672] "Emotional state" refers to the expression of the user's inner emotions, identified based on voice signals and facial expression data.

[0673] "Personality traits" are characteristic attributes related to the user's personality and individuality.

[0674] "Personalized advice" refers to specific and tailored behavioral guidelines for the user, generated based on identified emotional states and personality traits.

[0675] "Audio and visual output" refers to the audio and video formats used to convey generated information to the user.

[0676] "Feedback" refers to information that users send back to the server regarding their responses and results to the advice they receive.

[0677] "Learning function" refers to the ability to adjust the advice generation algorithm based on received feedback to improve the overall accuracy and effectiveness of the system.

[0678] This invention describes a specific embodiment of a system aimed at improving interpersonal relationships within the home. The system has a process of collecting voice signals and facial expression data from the user and identifying the emotional state based on them.

[0679] The server uses a platform equipped with high-precision sensors to process audio signals and facial expression data. For example, a microphone is used to analyze the tone of voice, and a camera is used to analyze facial expression data. This makes it possible to identify the user's emotional state in real time.

[0680] The server combines identified emotional states with pre-collected personality traits and uses a generative AI model like OpenAI to generate personalized advice. This generated advice is presented to the user via the device in audio and video formats. In this way, the user can interactively receive advice and take subsequent actions based on the results.

[0681] As a concrete example, if the system detects that one user in a married couple is experiencing stress, it analyzes their emotional state on a server and provides advice such as, "Why not take a walk in a nature park with your partner today to refresh yourselves?" By making such suggestions, the system can support improvements in everyday relationships.

[0682] An example of a prompt for a generative AI model is, "Please suggest an activity suitable for a relaxed emotional state."

[0683] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0684] Step 1:

[0685] The device collects the user's voice signals and facial expression data. Using a microphone and camera as sensors, it acquires audio and video in real time and formats them as digital data. The input is the user's physical voice and facial expressions, and the output is this digital data.

[0686] Step 2:

[0687] The terminal transmits the collected digital data to the server. The server performs speech recognition and facial expression analysis based on the received data. It analyzes the trends in speech tone and the facial features obtained from facial expressions, and performs data processing to identify the emotional state. The input is digitized audio and video data, and the output is an index indicating the user's emotional state.

[0688] Step 3:

[0689] The server integrates identified emotional states with already recorded personality traits and uses a generative AI model to create optimal advice. Here, prompts are set according to the emotional state, and the AI ​​model performs reasoning. The input is the emotional state and personality traits, and the output is a personalized advice message.

[0690] Step 4:

[0691] The server sends the generated advice to the terminal, which then presents it to the user as audio and visual output. Speech synthesis is used to provide information to the user in a natural way. The input is the advice message, and the output is the presentation of information verbally and visually.

[0692] Step 5:

[0693] The user acts based on the advice provided and inputs the results as feedback data into the terminal. The terminal forwards the feedback to the server. The input is feedback information from the user, and the output is an update to the system's history data.

[0694] Step 6:

[0695] The server analyzes feedback data, learns the advice generation algorithm, and adjusts parameters to improve accuracy. The feedback data is used as an evaluation metric for the algorithm and provides material for future improvements. The input is the feedback data, and the output is the improved algorithm model.

[0696] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0697] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0698] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0699] [Fourth Embodiment]

[0700] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0701] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0702] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0703] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0704] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0705] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0706] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0707] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0708] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0709] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0710] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0711] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0712] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0713] This invention is an AI-powered system designed to support the improvement of marital relationships, aiming to provide personalized advice based on user input data. Specifically, the system identifies the personality traits of the couple through an analysis device and generates advice to support relationship building based on that information.

[0714] System configuration and operation

[0715] Users answer a personality test using their own devices. This test includes questions designed to provide a detailed understanding of the user's personality traits.

[0716] The device receives the responses from the user and sends them to the server as digital data.

[0717] The server uses an algorithm to analyze personality traits based on the received data. This analysis creates a personality profile for each user.

[0718] The server simultaneously references existing psychology databases to identify potential problems and areas for improvement based on personality traits. Based on the identified issues, it generates advice including multiple options for improving relationships.

[0719] The device presents the generated advice to the user, enabling them to implement it in their daily life.

[0720] The user tries out the suggested advice and returns the results and feedback to the system as a response.

[0721] The terminal sends the user's response to the server and initiates the feedback process.

[0722] The server analyzes the feedback and utilizes its learning capabilities to adjust its algorithms to improve the accuracy of future advice.

[0723] Specific example

[0724] For example, if a married couple uses this system, they might each take a personality assessment first, which could reveal that their individual personality traits "need harmony." Based on this result, the server generates specific advice such as "have a shared hobby once a week" or "understand differences of opinion." Users then put this advice into practice and provide feedback on the results and changes, which helps the system improve its ability to provide even more effective support.

[0725] This system allows couples to receive specific, actionable advice based on their individual personality traits, enabling them to improve their relationship.

[0726] The following describes the processing flow.

[0727] Step 1:

[0728] The user launches the application and accesses the personality test. They enter their answers to the test questions and press the "Submit" button when finished.

[0729] Step 2:

[0730] The device collects the user's response data and formats it as digital information. It then sends this data to the server.

[0731] Step 3:

[0732] The server receives the submitted response data and applies a personality assessment algorithm to analyze the user's personality traits. This process calculates MBTI and Big Five scores.

[0733] Step 4:

[0734] Based on the analysis results, the server creates personality profiles of the user and their partner, and, referencing these profiles with an existing psychological database, generates specific advice necessary for improving the relationship.

[0735] Step 5:

[0736] The server sends the generated advice to the terminal. This advice includes actionable steps and specific suggestions for improving relationships.

[0737] Step 6:

[0738] The device displays advice to the user and provides an interface to encourage its application in daily life.

[0739] Step 7:

[0740] Users follow the advice provided and record feedback on the results and effects within the application. This feedback includes the actions taken and the changes observed.

[0741] Step 8:

[0742] The device sends user feedback data to the server. This feedback is processed to make future advice more personalized and effective.

[0743] Step 9:

[0744] The server analyzes the feedback it receives and adjusts and improves the advice generation algorithm. This allows the system to evolve so that it can provide more personalized suggestions to future users.

[0745] (Example 1)

[0746] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0747] Traditional relationship-building support systems often lack the nuanced approach based on individual characteristics, and are limited to providing general advice. This makes it difficult to effectively improve relationships with others. In particular, there is a need for more personalized support that provides specific advice tailored to each individual's personality and circumstances.

[0748] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0749] In this invention, the server includes means for analyzing personal characteristics based on input data obtained from the user, means for generating personalized relationship improvement advice by referring to the analysis results and a set of psychological data, and means for notifying the user of the generated advice using a terminal and receiving the user's response. This makes it possible to provide advice tailored to individual situations and characteristics.

[0750] "User" refers to an individual who uses the system to input information or receive services.

[0751] "Input data" refers to information provided by the user and processed by the system.

[0752] "Personal characteristics" refer to the unique personality and behavioral traits of the user.

[0753] "Analysis" refers to calculations and processing performed using input data to identify individual characteristics.

[0754] A "psychological data collection" refers to a database of information and knowledge based on the psychological characteristics of humans.

[0755] "Advice" refers to information that includes specific actions or suggestions aimed at improving the relationship with the user.

[0756] A "terminal" refers to an electronic device used by a user to access a system.

[0757] A "server" refers to a computer system that stores and processes data and controls the sending and receiving of information between it and terminals.

[0758] "Calculation method" refers to the calculation methods and algorithms that a system uses to process data and generate advice.

[0759] "Adaptive" means that the system has the ability to adjust its operation and output as needed based on new information it receives from the user.

[0760] In this invention, the system primarily involves a server, terminal, and user exchanging data with each other to provide personalized advice aimed at improving marital relationships. The details are described below.

[0761] Hardware and software to use

[0762] The terminal is used by the user as a means of answering questions and receiving generated advice. This includes digital devices such as smartphones, tablets, and personal computers. Specific applications are installed on these terminals, and users access the system using these applications.

[0763] A server is a computing system that receives and processes data transmitted from terminals. The server implements algorithms for data analysis, analyzing user input data and generating advice for improving relationships.

[0764] The generative AI model operates on a server and is responsible for data analysis and advice generation. This allows it to create personalized advice based on a set of psychological data. The AI ​​model also has a feedback learning function, utilizing user response data to improve its prediction accuracy.

[0765] Specific actions and prompt examples

[0766] Users access the system via a smartphone or other device and take a personality test. The results of this test are sent to the server as digital data.

[0767] Based on the transmitted data, the server generates advice for improving relationships using an analysis algorithm and a generative AI model. The generated advice is sent to the relevant user's device and presented to the user.

[0768] For example, if a couple uses this system and a personality test reveals that they "need harmony," the server will generate advice such as, "Try to engage in a shared hobby together once a week." Users then put this advice into practice in their daily lives and send the results back to the system as feedback.

[0769] Example of a prompt:

[0770] "Based on the personality assessment results, please provide specific advice on how to improve our marital relationship. The assessment result is 'Harmony is needed.'"

[0771] In this way, the system provides advice to improve interpersonal relationships in a way that is beneficial to each individual user. Furthermore, through continuous feedback, the system's ability to provide even more effective advice improves.

[0772] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0773] Step 1:

[0774] The user launches the application on their device and starts the personality test. The input consists of answers to various questions. Specifically, the user answers multiple-choice questions and saves the answers as data on their device by pressing the submit button. As output, all the answers are recorded on the device as a single dataset.

[0775] Step 2:

[0776] The terminal receives the stored user response data, encrypts it, and sends it to the server. Specifically, the terminal uses a secure data transmission protocol to send encrypted data packets to the server over the internet. The input is the user's response data, and the output is a file in digital data format that reaches the server.

[0777] Step 3:

[0778] The server receives and decrypts transmitted data packets. It handles encrypted user response data as input. Based on this, it applies data analysis algorithms to analyze the user's characteristics. Specifically, the server uses a generative AI model to analyze the data and generate a user personality profile. The output is the analyzed personality data.

[0779] Step 4:

[0780] The server uses a generative AI model to generate advice for improving relationships based on analyzed personality traits. The input consists of analyzed personality trait data and a set of psychological data. Specifically, the server runs the AI ​​model, evaluates the attribute data, and forms prompt sentences that generate appropriate advice. The output is personalized advice.

[0781] Step 5:

[0782] The server sends the generated advice to the terminal. The input is pre-generated advice data. Specifically, the server uses a data transfer protocol to securely send the advice to the terminal. The output is the received advice displayed on the terminal.

[0783] Step 6:

[0784] Users review the advice displayed on their device and attempt to put it into practice in their daily lives. The input includes specific advice. The specific action involves the user taking new actions based on the provided advice to improve their relationships.

[0785] Step 7:

[0786] The user inputs the results of the execution as feedback into the terminal and sends that data to the server. The input includes feedback information regarding the results of implementing the advice. Specifically, the user fills out a feedback form and submits it. The output reaches the server as feedback.

[0787] Step 8:

[0788] The server analyzes the received feedback and modifies the calculation method of the system's generated AI model to improve its accuracy. The input is user feedback data. Specifically, the server retrains the generated AI model and performs a learning process based on the feedback. The output is an improved algorithm, which is used for future advice generation.

[0789] (Application Example 1)

[0790] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0791] In modern society, marital relationships frequently deteriorate due to a lack of communication and personality clashes. This can lead to a negative atmosphere within the home and, in some cases, ultimately result in divorce. To address this issue, couples need to understand each other's personality traits and accept appropriate advice to build a better relationship.

[0792] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0793] In this invention, the server includes means for identifying personality traits based on personality attribute data collected from the user, means for generating a plurality of personalized pieces of advice based on the identified personality traits, and means for collecting conversational data using a speech recognition device. This makes it possible for users to easily understand their own personality traits and receive necessary advice in real time.

[0794] A "user" is an individual who uses this system to undergo a personality assessment and receive advice.

[0795] "Personality attribute data" refers to information about a user's personality extracted from their responses to personality assessment tests and conversation data.

[0796] "Personality traits" refer to the characteristics and tendencies of an individual's personality, analyzed based on the user's personality attribute data.

[0797] "Advice" refers to specific suggestions for improving relationships, generated based on the user's personality traits.

[0798] A "speech recognition device" is a device that collects the user's conversation as audio data and converts it into text data.

[0799] "Natural language processing technology" is a technique that analyzes collected conversational data to understand the intentions and emotions of the users.

[0800] The system implementing this invention operates around a consumer robot equipped with voice recognition capabilities, which is placed in each home. First, the user undergoes a personality assessment through conversation with the robot. Voice data is collected through a high-performance microphone built into the robot and converted into text data using the Google Speech-to-Text API. This converted data is then sent to a server.

[0801] The server uses OpenAI's GPT model to analyze text data and generate user personality trait data. The analysis process utilizes natural language processing techniques to identify the user's personality using information obtained from the conversation. Based on this profile information, the server consults a psychology database to generate personalized advice for improving relationships.

[0802] The generated advice is sent from the server to the robot, which outputs it as voice. At this time, the robot clearly explains the suggested advice to the user. The user tries out the presented advice and provides feedback to the robot about the results and their impressions.

[0803] User feedback is also sent to the server via the robot. The server uses this feedback to adaptively train its algorithm, improving the accuracy of future advice. This process is designed to ensure users continuously receive specific and helpful advice for improving their relationships.

[0804] For example, if a user consults the robot saying, "We've been having a lot of disagreements lately," the robot might respond with advice such as, "How about setting aside some time to come to an agreement?" By having the user provide feedback on how they feel after implementing this advice day by day, the server's generated AI model learns to suggest more appropriate advice.

[0805] An example of a prompt for the generating AI model is: "Analyze the user's conversation log and generate advice that will help improve the relationship between the couple."

[0806] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0807] Step 1:

[0808] The user initiates a conversation with the robot. Voice data is collected through a high-performance microphone inside the robot. The input is voice data, and the output is text data converted by the Google Speech-to-Text API. This conversion process analyzes the characteristics of the voice signal and converts them into text information.

[0809] Step 2:

[0810] The terminal, or robot, sends this text data to the server. The input here is the text data generated in step 1, and the output is the digital data to be sent to the server. The data is delivered accurately to the server using a communication protocol.

[0811] Step 3:

[0812] The server performs natural language processing on the received text data. The input is text data of the user's conversation, and the output is user personality trait data extracted through analysis. A generative AI model is used to analyze the content of the text and perform data processing and calculations to extract the user's personality traits.

[0813] Step 4:

[0814] The server references a psychology database based on personality trait data and generates corresponding advice. The input is personality trait data, and the output is personalized advice. This process applies algorithms for database searching and optimal advice generation.

[0815] Step 5:

[0816] The server sends the generated advice to the robot. The input is the text data of the generated advice, and the output is the data to be sent to the robot. The advice is processed via a secure communication channel to ensure it reaches the robot safely.

[0817] Step 6:

[0818] The robot conveys the received advice to the user through a voice output device. The input is text data of the advice, and the output is voice information for the user. Speech synthesis technology is used to convert the text into a natural conversational style, which is then played through the speaker.

[0819] Step 7:

[0820] The user follows the robot's advice and provides voice feedback on the results and their impressions. The input is voice data of the user's actions, and the output is the robot's re-collection of voice data. This data is then converted back into text using speech recognition technology and prepared to be sent to the server.

[0821] Step 8:

[0822] The server receives feedback data, evaluates the effectiveness of the advice using a generated AI model, and learns from success stories and failure factors. The input is user feedback text data, and the output is adjustment data for algorithm training. This process aims to improve the accuracy of future advice generation.

[0823] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0824] This invention aims to support the improvement of marital relationships using an AI system equipped with emotion recognition capabilities. The system analyzes the user's personality traits and emotional state, and provides more precise and personalized advice.

[0825] System configuration and operation

[0826] Users participate in everyday interactions and personality tests using an application equipped with an emotion engine. The emotion engine evaluates the user's voice tone, facial expressions, and text data in real time to recognize their emotional state.

[0827] The terminal sends data obtained from the user (responses to personality assessments and emotional data) to an analysis device, which then formats the data.

[0828] After receiving data from the terminal, the server analyzes it to identify the user's personality traits and emotional profile. The emotion engine then incorporates the identified emotional states as influencing factors into the analysis.

[0829] The server generates advice aimed at improving relationships, taking into account both emotional data and personality traits. This advice is optimized for the user's current emotional state.

[0830] The device presents the generated advice to the user in real time. The content and tone of the advice are adjusted based on feedback from the emotion engine.

[0831] Users act on the advice presented in their daily lives and input the results as feedback into the system. This feedback is used to re-evaluate the user's emotional state.

[0832] The server analyzes the collected feedback and uses the system's learning capabilities to adjust the advice generation algorithm.

[0833] Specific example

[0834] For example, if a user's emotion engine detects voice patterns indicating stress and text messages indicating depression, the server will generate advice such as, "We recommend taking a walk in nature to refresh yourself," or "Let's make time for a relaxing conversation with your partner." If that advice is insufficient, encouraging messages will be added. In this way, the system approaches the user from both an emotional and personality perspective, enabling more practical and impactful support.

[0835] This invention allows couples to receive advice that takes their emotional state into consideration, helping them to improve their relationship more effectively.

[0836] The following describes the processing flow.

[0837] Step 1:

[0838] The user launches the application and inputs data through a personality test and daily activities. The emotion engine obtains the user's emotional state in real time from their voice tone, facial expressions, and text.

[0839] Step 2:

[0840] The terminal collects user input data and emotional state data, and formats this data into digital information. It then sends the data to the server.

[0841] Step 3:

[0842] The server analyzes the received data, calculates the user's personality traits, and identifies their emotional state. Simultaneously, it incorporates the evaluation results from the emotion engine to generate a personality profile that combines the emotional states.

[0843] Step 4:

[0844] Based on the identified personality profile and emotional state, the server generates specific advice for improving relationships. This advice will be tailored to the user's current emotional state.

[0845] Step 5:

[0846] The device displays the generated advice to the user. The advice has a message tone adjusted to take into account feedback from the emotion engine.

[0847] Step 6:

[0848] Users implement the advice in their daily lives and input feedback on its effects and their impressions into the application. This feedback includes changes in their emotional state.

[0849] Step 7:

[0850] The device sends the collected feedback to the server. The feedback data is used to re-evaluate the user's emotional state.

[0851] Step 8:

[0852] The server analyzes the feedback and modifies the advice generation algorithm to improve accuracy. In particular, adjustments that reflect the results of the emotion engine improve the quality of advice for subsequent sessions.

[0853] (Example 2)

[0854] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0855] The present invention aims to support effective relationship improvement in marital relationships and other interpersonal relationships by providing personalized advice that takes into account the user's emotional state and personality traits. However, conventional approaches have the problem of not adequately providing immediate advice based on the user's real-time emotional state, nor does it allow for adaptive learning through subsequent feedback.

[0856] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0857] In this invention, the server includes means for processing audio, video, and text data collected from the user and formatting them into a format suitable for analysis; means for analyzing the formatted data and generating a profile to identify the user's emotional state; and means for generating multiple personalized pieces of advice based on the generated profile, taking into account the emotional state and personality traits. This enables the provision of effective advice tailored to the user's current situation and supports continuous relationship improvement based on that advice.

[0858] "User" refers to the person who will receive analysis of their emotional state and advice using this system.

[0859] "Audio data" refers to acoustic information, including the user's speaking style and tone of voice.

[0860] "Video data" refers to visual information, including the user's facial expressions and gestures.

[0861] "Text data" refers to written characters or messages entered by the user.

[0862] "Formatting" refers to the process of converting and organizing collected raw data so that it can be easily analyzed.

[0863] A "profile" refers to a dataset that aggregates a user's emotional state and personality traits and represents them in an identifiable format.

[0864] "Personalized advice" refers to advice specifically created based on each user's emotional state and personality traits.

[0865] A "generative AI model" refers to an algorithm and system that uses data to create new outputs.

[0866] "Feedback" refers to the act of providing the system with information about the advice the user has taken and the subsequent changes in their emotional state.

[0867] "Learning function" refers to the process of improving the system's algorithms based on collected data and feedback.

[0868] This invention uses an AI system equipped with emotion recognition capabilities to support the improvement of users' interpersonal relationships. Specific embodiments are described below.

[0869] Users provide data through everyday conversations and personality tests using an application with a built-in emotion engine. This application records the user's voice with a microphone, captures facial expressions with a webcam, and accepts text-based input. All the data collected in this way is sent to the emotion engine, which evaluates the user's emotional state in real time.

[0870] The terminal formats the audio, video, and text data acquired from the user into an appropriate format and performs the necessary preprocessing for sentiment analysis. Specifically, this includes noise reduction of audio data and standardization of facial expression data. The formatted data is then sent to the server for analysis.

[0871] The server receives data sent from the terminal and uses a generative AI model to create a user's emotional profile. This profile includes the user's current emotional state and long-term personality traits. Based on the emotional profile, the server generates optimized advice for the user. This advice consists of helpful suggestions and encouraging messages tailored to the user's current situation.

[0872] For example, if a user sends a voice message indicating stress and text indicating depression, the server will generate advice such as "We recommend taking a walk in nature to refresh yourself" or "Make time for a relaxing conversation with your partner." This advice is delivered in a way that best suits the user's emotional state, and encouraging messages may be added as needed.

[0873] An example of an input prompt for the generating AI model might be: "The user's voice patterns indicate stress, and the text data shows signs of depression. Based on this information, generate the most appropriate relationship improvement advice for the user." This enables an intelligent approach tailored to the individual user's state.

[0874] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0875] Step 1:

[0876] Users provide audio, video, and text data through the application. Specifically, users speak into a microphone, capture their facial expressions with a webcam, and input text. This data serves as input for evaluating the user's emotional state.

[0877] Step 2:

[0878] The terminal receives data from the user and formats it. Audio data undergoes noise reduction, video data undergoes facial expression extraction and standardization, and text data undergoes keyword extraction. These processes result in a dataset converted into a format that is easy to analyze.

[0879] Step 3:

[0880] The terminal sends formatted data to the server. The output here is integrated data containing voice tone, facial expression patterns, and text keyword information.

[0881] Step 4:

[0882] The server analyzes the received data and uses a generative AI model to generate a user's emotional profile. This profile reflects both short-term emotional states and long-term personality traits and serves as input for subsequent processing.

[0883] Step 5:

[0884] The server generates personalized advice for the user based on their emotional profile. It uses the input as prompts for a generating AI model to form the advice. The advice might include suggestions for refreshing oneself or prompts for further dialogue.

[0885] Step 6:

[0886] The device receives advice generated from the server and presents it to the user. The content and tone of the advice are adjusted based on the user's latest emotional feedback. This ensures that the advice is delivered in a form that is most appropriate for the user.

[0887] Step 7:

[0888] The user enters feedback into the system regarding the advice they have received. This feedback includes information about the results of the actions taken and changes in their feelings, and is used in the next step.

[0889] Step 8:

[0890] The server analyzes the feedback and adjusts the generated AI model based on it. Through its learning function, it improves the advice generation algorithm and enhances the quality of subsequent outputs.

[0891] (Application Example 2)

[0892] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0893] In interpersonal relationships, particularly within families, communication can be hindered by individual emotional states, leading to a deterioration of relationships. There is a need to provide effective means to resolve this issue and promote relationship improvement.

[0894] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0895] In this invention, the server includes means for identifying an emotional state based on voice signals and facial expression data collected from the user, means for generating personalized advice based on the identified emotional state and personality traits, and means for presenting the generated advice to the user through voice and visual output and receiving feedback from the user. This enables effective communication and relationship improvement tailored to each individual's emotional state.

[0896] A "user" is an individual who provides voice and facial expression data to the system and receives advice as a result.

[0897] "Speech signals" are data that describes the characteristics of sound waves obtained from the user's speech.

[0898] "Facial expression data" refers to video information used to analyze the visual characteristics of a user's face.

[0899] "Emotional state" refers to the expression of the user's inner emotions, identified based on voice signals and facial expression data.

[0900] "Personality traits" are characteristic attributes related to the user's personality and individuality.

[0901] "Personalized advice" refers to specific and tailored behavioral guidelines for the user, generated based on identified emotional states and personality traits.

[0902] "Audio and visual output" refers to the audio and video formats used to convey generated information to the user.

[0903] "Feedback" refers to information that users send back to the server regarding their responses and results to the advice they receive.

[0904] "Learning function" refers to the ability to adjust the advice generation algorithm based on received feedback to improve the overall accuracy and effectiveness of the system.

[0905] This invention describes a specific embodiment of a system aimed at improving interpersonal relationships within the home. The system has a process of collecting voice signals and facial expression data from the user and identifying the emotional state based on them.

[0906] The server uses a platform equipped with high-precision sensors to process audio signals and facial expression data. For example, a microphone is used to analyze the tone of voice, and a camera is used to analyze facial expression data. This makes it possible to identify the user's emotional state in real time.

[0907] The server combines identified emotional states with pre-collected personality traits and uses a generative AI model like OpenAI to generate personalized advice. This generated advice is presented to the user via the device in audio and video formats. In this way, the user can interactively receive advice and take subsequent actions based on the results.

[0908] As a concrete example, if the system detects that one user in a married couple is experiencing stress, it analyzes their emotional state on a server and provides advice such as, "Why not take a walk in a nature park with your partner today to refresh yourselves?" By making such suggestions, the system can support improvements in everyday relationships.

[0909] An example of a prompt for a generative AI model is, "Please suggest an activity suitable for a relaxed emotional state."

[0910] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0911] Step 1:

[0912] The device collects the user's voice signals and facial expression data. Using a microphone and camera as sensors, it acquires audio and video in real time and formats them as digital data. The input is the user's physical voice and facial expressions, and the output is this digital data.

[0913] Step 2:

[0914] The terminal transmits the collected digital data to the server. The server performs speech recognition and facial expression analysis based on the received data. It analyzes the trends in speech tone and the facial features obtained from facial expressions, and performs data processing to identify the emotional state. The input is digitized audio and video data, and the output is an index indicating the user's emotional state.

[0915] Step 3:

[0916] The server integrates identified emotional states with already recorded personality traits and uses a generative AI model to create optimal advice. Here, prompts are set according to the emotional state, and the AI ​​model performs reasoning. The input is the emotional state and personality traits, and the output is a personalized advice message.

[0917] Step 4:

[0918] The server sends the generated advice to the terminal, which then presents it to the user as audio and visual output. Speech synthesis is used to provide information to the user in a natural way. The input is the advice message, and the output is the presentation of information verbally and visually.

[0919] Step 5:

[0920] The user acts based on the advice provided and inputs the results as feedback data into the terminal. The terminal forwards the feedback to the server. The input is feedback information from the user, and the output is an update to the system's history data.

[0921] Step 6:

[0922] The server analyzes feedback data, learns the advice generation algorithm, and adjusts parameters to improve accuracy. The feedback data is used as an evaluation metric for the algorithm and provides material for future improvements. The input is the feedback data, and the output is the improved algorithm model.

[0923] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0924] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0925] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0926] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0927] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0928] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0929] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0930] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0931] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0932] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0933] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0934] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0935] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0936] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0937] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0938] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0939] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0940] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0941] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0942] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0943] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0944] The following is further disclosed regarding the embodiments described above.

[0945] (Claim 1)

[0946] In an analytical device, a means for identifying personality traits based on personality attribute data collected from the user,

[0947] A means for generating multiple personalized pieces of advice based on identified personality traits,

[0948] A means of presenting generated advice to the user and receiving a response from the user,

[0949] A system including means for analyzing the received user response and adjusting the device's learning function to improve the accuracy of the advice generation means.

[0950] (Claim 2)

[0951] The system according to claim 1, wherein the personalized advice is for supporting the formation of interpersonal relationships.

[0952] (Claim 3)

[0953] The system according to claim 1, wherein the adjustment of the learning function is dynamically performed based on the user's response data received.

[0954] "Example 1"

[0955] (Claim 1)

[0956] A means of analyzing personal characteristics based on input data obtained from users,

[0957] A means for generating advice for improving individual relationships by referring to the analysis results and the psychological data set,

[0958] A means of notifying the user of the generated advice using a terminal and receiving the user's response,

[0959] Means for modifying the system's calculation method to improve the accuracy of the generation means based on user feedback received,

[0960] A system that includes means for a terminal to transmit information to a data server, and for the server to store and process that information.

[0961] (Claim 2)

[0962] The system according to claim 1, wherein the aforementioned advice is intended to help improve interpersonal relationships.

[0963] (Claim 3)

[0964] The system according to claim 1, wherein the calculation method is modified adaptively based on response information provided by the user.

[0965] "Application Example 1"

[0966] (Claim 1)

[0967] A means of identifying personality traits based on personality attribute data collected from users,

[0968] A means for generating multiple personalized pieces of advice based on identified personality traits,

[0969] A means of presenting generated advice to the user and receiving a response from the user,

[0970] A means for analyzing the user's response received and adjusting the device's learning function to improve the accuracy of the advice generation means,

[0971] A means of collecting conversation data using a speech recognition device,

[0972] A method for analyzing collected conversation data using natural language processing technology,

[0973] A means of dynamically updating the user's personality traits based on the analysis results,

[0974] A system that includes this.

[0975] (Claim 2)

[0976] The system according to claim 1, wherein the personalized advice is intended to support the formation of interpersonal relationships and is capable of real-time voice output.

[0977] (Claim 3)

[0978] The system according to claim 1, wherein the adjustment of the learning function is dynamically performed based on the user's response data received, and further adaptively adjusted based on the results of voice data analysis.

[0979] "Example 2 of combining an emotion engine"

[0980] (Claim 1)

[0981] A means of processing audio, video, and text data collected from users and formatting them into a format suitable for analysis,

[0982] A means for analyzing formatted data and generating a profile to identify the user's emotional state,

[0983] A means for generating multiple personalized pieces of advice that take into account emotional states and personality traits based on the generated profile,

[0984] A means of presenting generated advice to the user and dynamically adjusting its content and tone based on emotional feedback,

[0985] A system that includes means for analyzing user feedback and adjusting learning capabilities to improve the advice generation algorithm using a generative AI model.

[0986] (Claim 2)

[0987] The system according to claim 1, wherein the personalized advice is intended to support the improvement of interpersonal relationships.

[0988] (Claim 3)

[0989] The system according to claim 1, wherein the adjustment of the learning function is dynamically performed based on user feedback data received.

[0990] "Application example 2 when combining with an emotional engine"

[0991] (Claim 1)

[0992] A means of identifying emotional states based on audio signals and facial expression data collected from the user,

[0993] A means for generating personalized advice based on identified emotional states and personality traits,

[0994] A means of presenting generated advice to the user through audio and visual output, and receiving feedback from the user,

[0995] A system including means for adjusting the learning function of the device to improve the accuracy of the advice generation means based on the feedback received.

[0996] (Claim 2)

[0997] The system according to claim 1, wherein the personalized advice is for supporting the improvement of interpersonal relationships.

[0998] (Claim 3)

[0999] The system according to claim 1, wherein the adjustment of the learning function is performed dynamically based on the received feedback data. [Explanation of symbols]

[1000] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. In an analytical device, a means for identifying personality traits based on personality attribute data collected from the user, A means for generating multiple personalized pieces of advice based on identified personality traits, A means of presenting generated advice to the user and receiving a response from the user, A system including means for analyzing the received user response and adjusting the device's learning function to improve the accuracy of the advice generation means.

2. The system according to claim 1, wherein the personalized advice is for supporting the formation of interpersonal relationships.

3. The system according to claim 1, wherein the adjustment of the learning function is dynamically performed based on the user's response data received.

Images