system

Abstract

We provide the system. [Solution] A means of collecting user profile data and generating a detailed profile based on values, personality, and lifestyle, A means of capturing and analyzing conversational and nonverbal communication during a date in real time, A means of suggesting personalized advice and activities based on the analysis results, A means of collecting feedback from users after a date and updating their profile, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional matching services lack support in relationship building after matching. Therefore, there is a problem that many users cannot maintain a relationship due to nervousness and lack of communication at the first meeting. Also, in the current social situation where face-to-face communication opportunities are decreasing, users are required to improve their communication skills. For this reason, an effective system for supporting relationship building during and after dating is needed.

Means for Solving the Problems

[0005] This invention is characterized by collecting user profile data and generating a detailed profile based on it. Furthermore, during dates, it captures conversational and nonverbal communication data in real time and analyzes it with AI, providing personalized advice and activity suggestions based on the results. After dates, it collects feedback from users and uses that information to update their profiles, thus providing continuous support. This makes it easier for users to communicate effectively during dates and receive support to develop their relationships.

[0006] "User profile data" refers to data that includes information about a user's values, personality, and lifestyle.

[0007] A "detailed profile" is information that details a user's characteristics, generated based on the collected user profile data.

[0008] "Real-time capture" means instantly recording conversation and nonverbal communication data during a date.

[0009] "Nonverbal communication" refers to means of communication that do not involve words, such as body language, facial expressions, and tone of voice.

[0010] "AI analysis" is the process of using artificial intelligence technology to evaluate and process collected data and derive useful information.

[0011] "Personalized advice" refers to advice and suggestions provided individually based on the user's profile and communication during dating.

[0012] "Activity suggestions" refer to recommending actions and activities that align with users' common interests and characteristics.

[0013] "Feedback" refers to information including opinions and impressions that users provide after a dating experience.

[0014] "Updating a profile" refers to modifying or adding to an existing user profile based on new information obtained through feedback. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disk (e.g., hard disk), or magnetic tape, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is a system that supports relationship building based on user profile data. Specifically, users provide basic information and lifestyle details via their devices, which the server analyzes to generate a detailed profile. This profile is used not only for appropriate matching between users but also for providing support during dates and suggesting activities.

[0037] The server receives conversation and nonverbal communication data from the device during the date. This data is processed in real time and analyzed by AI. Based on the analysis, the server provides the user with appropriate advice and topics through the device. As a result, users can communicate more confidently and smoothly during their dates.

[0038] Furthermore, the server collects feedback after each date and updates the user profile based on that feedback. This process helps users become aware of areas for improvement in future dates and communication, enabling them to sustainably develop their relationships.

[0039] As a concrete example, suppose user A registers information about their values ​​and hobbies. Based on this information, the server suggests user B, who is a good match for user A, and sets up a date. On the day of the date, the device monitors user A's conversation and sends it to the server. When the conversation stalls, the server suggests topics related to shared hobbies to help user A proceed more naturally. After the date, user A provides feedback through the device, and the server reflects this information in their profile, enabling it to make better suggestions for the next date. In this way, the present invention provides an environment in which users can confidently deepen their interpersonal interactions.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] Users use a device to input data about their basic information, values, hobbies, and lifestyle. Once the information is entered, the device sends it to the server.

[0043] Step 2:

[0044] The server generates a detailed profile based on the received user data. This profile includes the user's characteristics and interests, which helps in matching them with suitable dating partners.

[0045] Step 3:

[0046] The server uses the generated profile to select the most suitable match for the user and notifies the user of a date proposal. The user receives the notification and confirms and accepts the date, time, and activity details.

[0047] Step 4:

[0048] On the day of the date, the device captures the user's conversation in real time. The captured audio and nonverbal communication data is immediately sent to the server.

[0049] Step 5:

[0050] The server uses AI to analyze the received dating data and detect potential conversational pauses or awkward moments that the user might encounter. If necessary, it provides advice and topic suggestions to the user via the device.

[0051] Step 6:

[0052] After the date ends, the device sends a feedback form to the user. The user fills out the feedback and reflects on their date experience.

[0053] Step 7:

[0054] The server collects user feedback and incorporates it into profile updates. This allows for more improved advice and suggestions for future dates.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] In modern society, many people experience difficulties in interpersonal interactions. In particular, finding someone who shares their interests and values, and maintaining smooth communication during interactions, present unique challenges when building new relationships. Furthermore, providing real-time communication support and fostering continuous relationship development based on feedback is not easy. Effective solutions to these challenges are needed.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes means for collecting user attribute data and generating detailed attribute information based on interests, personality, and lifestyle; means for recording and analyzing conversations and nonverbal communications in real time during interactions; means for proposing customized advice and activities based on the analysis results; and means for collecting feedback from users after interactions and updating attribute information. This enables optimal matching between users, as well as real-time support for smooth interactions and support for the development of continuous relationships.

[0060] A "user" refers to an individual who uses the system and is the entity that provides their attribute data.

[0061] "Attribute data" refers to data that includes information about a user's interests, personality, and lifestyle.

[0062] "Detailed attribute information" refers to information that represents the user's characteristics, generated based on collected attribute data.

[0063] "In conversation" refers to a state in which a user is communicating with another individual.

[0064] "Conversation" refers to the exchange of information, either verbally or in writing, and is a part of the communication that takes place between users.

[0065] "Nonverbal communication" refers to communication that takes place through means other than words, such as gestures and facial expressions.

[0066] "Real-time recording" refers to instantly collecting and saving information while a conversation is in progress.

[0067] "Analyzing" refers to the process of analyzing collected data to extract meaning and patterns.

[0068] "Customized advice" refers to advice optimized for the user's characteristics and circumstances based on analyzed data.

[0069] "Activity suggestions" refer to instructions that present and encourage users to take on activities based on their interests and commonalities.

[0070] This invention provides a system to support relationship building based on user attribute data. The system starts operating when the user sends basic information and lifestyle data collected via a terminal to a server. The server analyzes this data using Python®-based data analysis tools and machine learning libraries such as Scikit-learn to generate detailed attribute information.

[0071] The server performs user matching based on the generated attribute information. Here, machine learning algorithms are used to select the most suitable partner based on the user's interests and personality. This process can be accelerated by leveraging common cloud computing platforms.

[0072] The device used by the user during the conversation records the conversation content and nonverbal communication in real time and sends it to the server. The server analyzes this data using natural language processing libraries such as NLTK and SpaCy. Based on the analysis results, the server provides the user with customized advice and suggestions for activities based on shared interests through the device.

[0073] As a concrete example, when user A registers information about their interests and personality, the server suggests user B, who is a good match for user A. On the day of the date, the device monitors user A's conversation and sends data to the server. If the conversation stalls, the server displays a prompt on user A's device such as, "Try talking about music, which is a common hobby you share with user B."

[0074] After the date ends, user A provides feedback to the server via their device. The server stores this feedback in a database and uses it to improve future matching suggestions, thereby continuously supporting the relationship between users.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] Users use a terminal to input basic information and lifestyle data. This input data includes name, age, hobbies, values, etc. This information is sent from the terminal to the server. The entered data is stored in the server's database in its original form.

[0078] Step 2:

[0079] The server processes the received user attribute data using a Python-based data analysis tool to generate detailed attribute information. This process extracts feature vectors from the input data and performs clustering or classification using a machine learning library (e.g., Scikit-learn). The output is a profile based on the user's interests and personality. This profile is used for matching in the next step.

[0080] Step 3:

[0081] The server matches users with other registered users based on the generated profile information. The input is the generated profile information, which is used to evaluate compatibility using a machine learning algorithm. The output is a list of the best match. The server analyzes this information, selects the best candidate (e.g., User B), and proposes it to the user via the terminal.

[0082] Step 4:

[0083] On the day of the interaction, the terminal records the user's conversation in real time and sends it to the server. The input consists of voice data and other communication data. The server uses speech recognition technology to convert this data into text and performs analysis using natural language processing libraries (e.g., NLTK, SpaCy). The output provides analysis results regarding the flow and emotions of the conversation. Specifically, it identifies the tone of the conversation and any pauses.

[0084] Step 5:

[0085] The server provides the user with appropriate advice and activity suggestions based on the analysis results. The input is the result of the conversation analysis obtained in the previous step, and the output is expressed as a prompt. This allows the terminal to display customized suggestions, such as "Try talking about cooking, which is a common hobby of yours."

[0086] Step 6:

[0087] Users provide feedback after the interaction ends. The input consists of information regarding their satisfaction with the interaction and areas for improvement. The terminal sends this data to the server, which analyzes the data and updates the profile. The output is the updated user profile information, which improves the accuracy of future matching.

[0088] (Application Example 1)

[0089] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0090] Conventional interpersonal relationship support systems merely collect user profiles, but they are insufficient in providing real-time support during conversations or specific content based on shared interests. Furthermore, a mechanism is needed to smoothly provide common topics and activities in order to deepen mutual understanding between users.

[0091] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0092] In this invention, the server includes means for collecting user attribute data and generating a profile, means for collecting and analyzing conversational and nonverbal communication in real time during a dialogue, and means for presenting advice and activities tailored based on the analysis results. This enables the user to receive appropriate support during the dialogue and to communicate smoothly.

[0093] "User attribute data" refers to distinctive information related to individual users, including elements such as values, personality, and lifestyle.

[0094] A "profile" is a detailed representation of a person based on their attribute data, and is used as digital data for analysis and recommendations.

[0095] "Conversation and nonverbal communication in a dialogue" refers to various forms of communication, including written or spoken communication, and visual or physical responses.

[0096] "Collecting and analyzing data in real time" refers to the process of acquiring data instantly the moment it is generated and analyzing it immediately to obtain results quickly.

[0097] "Adjusted advice and actions" refer to advice and suggested actions that are optimized based on analyzed data and are tailored to the user's needs.

[0098] "Content related to common interests" refers to content related to shared interests among users, and is provided to facilitate conversations and activities.

[0099] "Information that contributes to facilitating dialogue" refers to data and suggestions that are useful for smoothing and deepening interactions between users, and are intended to support communication.

[0100] This system utilizes user terminals, servers, and AI technology to provide functions that support real-time communication between users.

[0101] The user's device collects attribute data from individual users and sends it to the server. This attribute data includes information such as values, personality, and lifestyle. The server then generates a detailed profile. Using this profile, the server employs an AI model to suggest optimal advice and activities for the user. This AI model utilizes generative AI such as OpenAI's GPT-4®.

[0102] Furthermore, during the conversation, the user's device collects conversation content and nonverbal communication data in real time and sends it to the server. The server analyzes this data and provides information that is of common interest or that might pique new interest. This uses a natural language processing library (e.g., spaCy), and Apache® Kafka is used for real-time data analysis.

[0103] As a concrete example, if user A and user B share a common hobby, the server will provide users with recommendations for events and movies related to that hobby via their devices. Furthermore, if a user develops a new interest, for example, if the topic of the tea ceremony comes up, the server can immediately provide relevant video links. In this process, the prompt message used would be, "Users A and B share a common hobby: Japanese cuisine. Please provide relevant topics and content to suggest when the conversation lulls."

[0104] Thus, the present invention facilitates smoother communication between users and enables deeper dialogue based on shared interests and new information.

[0105] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0106] Step 1:

[0107] The user's device receives attribute data from the user as input. This includes information about values, personality, and lifestyle. The device organizes this data and prepares it for transmission to the connected server. The server receives this data, analyzes the user's attribute data, and generates a profile.

[0108] Step 2:

[0109] The server extracts common interests and potential topics based on the generated user profiles. Using a generative AI model, it creates prompts to suggest optimal advice and activities among users. These prompts are input into the AI ​​model, which then outputs suggestions.

[0110] Step 3:

[0111] Once the date begins, the user's device collects conversation content and nonverbal communication in real time. This data is sent to a server, which uses a natural language processing library to analyze the conversation. Based on the input conversation data, common topics and interests are further refined, and information that can be presented to the user is prepared.

[0112] Step 4:

[0113] Based on the analysis, the server suggests tailored advice and activities to the user. For example, it may output information about new movies or events and display it as a notification on the device. Also, if the conversation is interrupted, it will use the AI ​​model again to suggest topics to resume the conversation.

[0114] Step 5:

[0115] After a date, the user submits feedback to the server via their device. This feedback is important data for the system and is used to improve the accuracy of the profile. The server uses this to update the profile and enable more helpful suggestions for the next date.

[0116] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0117] This invention is a system that provides support during dating by taking into account the user's profile data and emotional state. The user inputs basic information and lifestyle data through a terminal, and the server generates a detailed profile based on this information. After profile generation, the server suggests a matched partner to the user and prompts them to prepare for the date.

[0118] During the date, the device is equipped with sensors for voice and non-voice input, which capture conversation and nonverbal communication data in real time. At this time, an emotion engine operates and analyzes the user's emotional state using the captured data. Based on the results of the emotion engine's analysis, the server understands the user's emotional state.

[0119] Based on the analysis of emotional states and profiles, the server suggests personalized advice and activities based on shared interests to the user via the device. This makes it easier for users to communicate in a way that aligns with their emotions, leading to more fulfilling dates.

[0120] For example, if the emotion engine detects that user A is feeling nervous during a date, the server will suggest relaxing activities and present easy-to-talk-about topics through the device. In this way, user A can continue the date in a more relaxed state.

[0121] After a date, users provide feedback using their devices, which is collected by a server. The server updates the user's profile based on the feedback and sentiment engine data, enabling further advice for future dates. This allows users to continuously improve and grow with each date. Through these features, the present invention provides comprehensive support for users to deepen their relationships.

[0122] The following describes the processing flow.

[0123] Step 1:

[0124] Users use their devices to input data about their basic information, values, hobbies, and lifestyle. This information is then transmitted to the server via the device.

[0125] Step 2:

[0126] The server generates a detailed profile based on the received user data. This generated profile is used to determine the best match for the user.

[0127] Step 3:

[0128] The server suggests the most suitable match for the user's profile and notifies the user of the date details. The user reviews the schedule and activities and accepts.

[0129] Step 4:

[0130] At the start of the date, the device uses voice sensors and a camera to capture conversation and nonverbal communication data in real time. The captured data is sent to a server.

[0131] Step 5:

[0132] The server utilizes an emotion engine to analyze captured data and detect the user's emotional state. This analysis allows the server to understand the user's emotional state, such as whether they are stressed or relaxed.

[0133] Step 6:

[0134] Based on the analysis results of the emotion engine, the server selects appropriate advice and activities for the user and presents them to the user through the device. This allows the user to respond appropriately to the situation.

[0135] Step 7:

[0136] After the date ends, the device sends a feedback form to the user to collect information about their experience. The user fills out the feedback and sends it to the server via the device.

[0137] Step 8:

[0138] The server integrates feedback data with emotional data from the emotion engine and updates the user profile. This allows for the provision of suggestions for improvement and new advice for the next date.

[0139] (Example 2)

[0140] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0141] In modern society, while the evolution of digital technology has made building relationships online commonplace, deepening mutual understanding can be difficult in face-to-face communication. Especially during initial encounters or early stages of interaction, tension and misunderstandings are more likely to occur, highlighting the need for methods to improve the quality of dates. Furthermore, being able to understand a user's emotional state in real time and provide advice and activities based on that understanding can lead to better communication.

[0142] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0143] In this invention, the server includes means for collecting user characteristic information and generating a detailed profile based on values, personality, and lifestyle; means for acquiring voice and non-voice communication during a date in real time and analyzing the emotional state; and means for suggesting personalized advice and activities based on shared interests based on the analysis results. This enables communication support that is tailored to the user's emotions and interests.

[0144] A "user" refers to anyone who uses the system and provides profile data.

[0145] "Characteristic information" refers to information such as the user's values, personality, and lifestyle, and is used to generate profiles.

[0146] A "profile" refers to a collection of detailed data generated based on a user's characteristics.

[0147] "Voice communication" refers to conversations and exchanges of spoken language between users.

[0148] "Non-verbal communication" refers to interactions between users that do not involve voice, such as gestures and facial expressions.

[0149] "Acquiring information in real time" means obtaining information immediately at the time it is received.

[0150] "Emotional state" refers to the user's psychological state or mood, and is analyzed by the emotion engine.

[0151] "Analysis" refers to the process of processing information based on acquired data to derive meaning or results.

[0152] "Advice" refers to personalized advice and suggestions provided to the user.

[0153] "Activities" refer to specific activities suggested based on user interactions and interests.

[0154] A "generative AI model" refers to an algorithm that uses artificial intelligence to generate information tailored to the user.

[0155] A "prompt statement" refers to a specific instruction or command that causes a generative AI model to operate.

[0156] "Support" refers to assistance or assistance that helps users communicate smoothly.

[0157] To implement this invention, the user first inputs their personal information using a terminal. This terminal can be a smartphone or a personal computer. This information will be used later to create a profile and includes the user's values, personality, and lifestyle.

[0158] The server receives characteristic information sent by the user and generates a detailed profile based on it. This profile generation uses a generative AI model, which optimizes the profile content using prompts. For example, the prompt might read, "Generate a profile based on user A's hobbies."

[0159] During the date, the device uses sensors for voice and non-voice input to capture the user's voice and non-voice communication in real time. The server analyzes this data and activates an emotion engine to understand the user's emotional state. Emotions, including the user's level of tension, are assessed.

[0160] Based on the analysis results, the server suggests advice and activities based on shared interests for the user. This helps support communication during the date. For example, if the user is nervous, the server suggests "relaxing activities" and provides a topic such as "talking about recent travel experiences."

[0161] After a date ends, users provide feedback using their devices. This includes their satisfaction level and areas for improvement. The server collects this feedback appropriately and uses it to update their profile and improve the accuracy of future suggestions. In this way, the system can provide users with a meaningful dating experience.

[0162] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0163] Step 1:

[0164] The user uses a terminal to enter their personal information, such as name, age, hobbies, and areas of interest. This information is sent to the server and used as input data for generating the next profile.

[0165] Step 2:

[0166] The server uses a generative AI model to generate a detailed profile based on the characteristic information received from the user. In this process, the system receives a prompt such as, "Generate a profile based on user A's hobbies." The generative AI model processes this prompt and generates profile data that reflects the user's values ​​and personality. This output forms the basis for matching and providing advice.

[0167] Step 3:

[0168] During the date, the device acquires audio and non-audio data in real time using sensors. The microphone and camera operate, capturing language, facial expressions, and other information. This data is sent to a server as input, where it is analyzed by an emotion engine to determine the user's emotional state. The output is user emotion evaluation data.

[0169] Step 4:

[0170] Based on emotion assessment data and profiles, the server suggests personalized advice and activities for the user. For example, if the analysis indicates that the user is feeling stressed, specific suggestions such as "take a break in a relaxing place" or "talk about recent movies" will be provided. These suggestions are delivered to the user through their device.

[0171] Step 5:

[0172] After the date, the user returns their device to provide feedback. This feedback includes writing down their impressions and feelings about the date. This feedback data is sent to the server and integrated with previous sentiment evaluation data to form the basis for updating the profile. The updated profile contributes to improving and optimizing advice for future dates.

[0173] (Application Example 2)

[0174] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0175] In today's world, improving the quality of interpersonal relationships and communication is crucial. However, face-to-face communication often causes tension and anxiety, and this problem is particularly pronounced in new environments and interactions with new people. Furthermore, providing personalized advice based on each user's different values ​​and characteristics is a challenge. Effective means of addressing these challenges are needed.

[0176] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0177] In this invention, the server includes means for collecting the user's biometric data and generating a detailed user profile based on characteristics and lifestyle; means for acquiring and analyzing dialogue and nonverbal communication within the virtual space in real time; and means for proposing customized advice and activities based on the analysis results. This enables effective communication in a less stressful virtual environment and allows for the proposal of advice and activities optimized for each user.

[0178] "User biometric data" refers to information and signals related to the user's body, specifically including digitizable information such as heart rate, skin temperature, and movement.

[0179] "Characteristics" refer to a user's individuality and tendencies, indicating their unique behavioral patterns and thought processes.

[0180] "Lifestyle" refers to the patterns of activities and time management that users engage in on a daily basis, and is a concept that includes the behavioral styles and habits based on those patterns.

[0181] A "detailed user profile" refers to a comprehensive collection of data about a user's personal information and background, including attribute information necessary to understand their unique needs and interests.

[0182] "Dialogue within a virtual space" refers to linguistic communication between users in a virtual space using digital technology, and includes exchanges using voice and text.

[0183] "Nonverbal communication" refers to communication that takes place through means other than language, and includes forms of communication such as body language, facial expressions, and tone of voice.

[0184] "Acquiring data in real time" means recording and collecting data at almost the same time as real time, and that the information is reflected instantly without delay.

[0185] "Analyzing" refers to the process of deciphering and classifying collected data as information, and understanding its meaning and characteristics.

[0186] "Customized advice and activities" refers to advice and action suggestions that are specifically designed and tailored based on the individual user's needs and profile.

[0187] In the system that implements this application, three parties—a server, a terminal, and a user—work together to perform their functions. The server collects the user's biometric data and generates a detailed profile that reflects their characteristics and lifestyle. The user enters a virtual space through the terminal and engages in dialogue and nonverbal communication there. The terminal is equipped with a function to acquire voice and movement in real time, and by sending this data to the server, the server analyzes the data and understands the user's emotional state.

[0188] The analysis uses software such as an emotion analysis engine, and the AI ​​performs the analysis in conjunction with non-voice data. By using software such as an "emotion analysis library" and a "profile management module," it is possible to obtain accurate analysis results.

[0189] Based on the analysis results, the server suggests personalized advice and activities to the user. This involves using a generative AI model to select the most suitable activity based on the user's emotional state, using prompts. For example, a text prompt might read, "Analyze the user's emotional state and suggest relaxing dating activities. The user's current profile shows signs of tension."

[0190] In this way, by having the server, terminal, and user cooperate in data processing and supporting communication in a stress-free virtual environment, users can receive advice that is tailored to their individual needs.

[0191] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0192] Step 1:

[0193] The server collects biometric data from the user. The user uses a device to input data such as heart rate, skin temperature, and movement in real time. The server receives this input data and generates a detailed profile that reflects the user's characteristics and lifestyle. This profile digitizes the user's basic features.

[0194] Step 2:

[0195] The user participates in the virtual space through a device. The device acquires nonverbal communication such as the user's voice, facial expressions, and movements in real time. This data is sent to a server. The device uses sensors that detect voice tone and facial expressions to accurately collect data.

[0196] Step 3:

[0197] The server analyzes the acquired data. A sentiment analysis engine is used for data analysis, and a generative AI model determines the user's emotional state. The server analyzes changes in voice tone and facial expressions, and based on the results, identifies the user's current emotional state. The input is non-verbal data, and the output is the identified emotional state.

[0198] Step 4:

[0199] The server generates customized advice and activities based on the user's emotional state. Using a generative AI model, it creates prompts and suggests the most suitable activities for the user based on those prompts. For example, it might generate a prompt such as, "Please suggest a relaxing virtual activity." The input is the emotional state, and the output is a suggestion of a specific activity.

[0200] Step 5:

[0201] The user experiences suggested activities in a virtual space. The device creates a virtual environment based on the user's selected activity, allowing the user to relax and engage in activities in the digital space. The device outputs video and audio appropriate to the activity to support the user's experience.

[0202] Step 6:

[0203] After a digital experience, users send feedback to a server via their device. The server analyzes the received feedback and updates the user's profile in more detail. This feedback allows for more refined advice and suggestions tailored to individual users. The input is the user's feedback, and the output is the updated profile.

[0204] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0205] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0206] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0207] [Second Embodiment]

[0208] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0209] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0210] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0211] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0212] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0213] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0214] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0215] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0216] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0217] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0218] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0219] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0220] This invention is a system that supports relationship building based on user profile data. Specifically, users provide basic information and lifestyle details via their devices, which the server analyzes to generate a detailed profile. This profile is used not only for appropriate matching between users but also for providing support during dates and suggesting activities.

[0221] The server receives conversation and nonverbal communication data from the device during the date. This data is processed in real time and analyzed by AI. Based on the analysis, the server provides the user with appropriate advice and topics through the device. As a result, users can communicate more confidently and smoothly during their dates.

[0222] Furthermore, the server collects feedback after each date and updates the user profile based on that feedback. This process helps users become aware of areas for improvement in future dates and communication, enabling them to sustainably develop their relationships.

[0223] As a concrete example, suppose user A registers information about their values ​​and hobbies. Based on this information, the server suggests user B, who is a good match for user A, and sets up a date. On the day of the date, the device monitors user A's conversation and sends it to the server. When the conversation stalls, the server suggests topics related to shared hobbies to help user A proceed more naturally. After the date, user A provides feedback through the device, and the server reflects this information in their profile, enabling it to make better suggestions for the next date. In this way, the present invention provides an environment in which users can confidently deepen their interpersonal interactions.

[0224] The following describes the processing flow.

[0225] Step 1:

[0226] Users use a device to input data about their basic information, values, hobbies, and lifestyle. Once the information is entered, the device sends it to the server.

[0227] Step 2:

[0228] The server generates a detailed profile based on the received user data. This profile includes the user's characteristics and interests, which helps in matching them with suitable dating partners.

[0229] Step 3:

[0230] The server uses the generated profile to select the most suitable match for the user and notifies the user of a date proposal. The user receives the notification and confirms and accepts the date, time, and activity details.

[0231] Step 4:

[0232] On the day of the date, the device captures the user's conversation in real time. The captured audio and nonverbal communication data is immediately sent to the server.

[0233] Step 5:

[0234] The server uses AI to analyze the received dating data and detect potential conversational pauses or awkward moments that the user might encounter. If necessary, it provides advice and topic suggestions to the user via the device.

[0235] Step 6:

[0236] After the date ends, the device sends a feedback form to the user. The user fills out the feedback and reflects on their date experience.

[0237] Step 7:

[0238] The server collects user feedback and incorporates it into profile updates. This allows for more improved advice and suggestions for future dates.

[0239] (Example 1)

[0240] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0241] In modern society, many people experience difficulties in interpersonal interactions. In particular, finding someone who shares their interests and values, and maintaining smooth communication during interactions, present unique challenges when building new relationships. Furthermore, providing real-time communication support and fostering continuous relationship development based on feedback is not easy. Effective solutions to these challenges are needed.

[0242] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0243] In this invention, the server includes means for collecting user attribute data and generating detailed attribute information based on interests, personality, and lifestyle; means for recording and analyzing conversations and nonverbal communications in real time during interactions; means for proposing customized advice and activities based on the analysis results; and means for collecting feedback from users after interactions and updating attribute information. This enables optimal matching between users, as well as real-time support for smooth interactions and support for the development of continuous relationships.

[0244] A "user" refers to an individual who uses the system and is the entity that provides their attribute data.

[0245] "Attribute data" refers to data that includes information about a user's interests, personality, and lifestyle.

[0246] "Detailed attribute information" refers to information that represents the user's characteristics, generated based on collected attribute data.

[0247] "In conversation" refers to a state in which a user is communicating with another individual.

[0248] "Conversation" refers to the exchange of information, either verbally or in writing, and is a part of the communication that takes place between users.

[0249] "Nonverbal communication" refers to communication that takes place through means other than words, such as gestures and facial expressions.

[0250] "Real-time recording" refers to instantly collecting and saving information while a conversation is in progress.

[0251] "Analyzing" refers to the process of analyzing collected data to extract meaning and patterns.

[0252] "Customized advice" refers to advice optimized for the user's characteristics and circumstances based on analyzed data.

[0253] "Activity suggestions" refer to instructions that present and encourage users to take on activities based on their interests and commonalities.

[0254] This invention provides a system to support relationship building based on user attribute data. The system starts operating when the user sends basic information and lifestyle data collected via a terminal to a server. The server analyzes this data using Python-based data analysis tools and machine learning libraries such as Scikit-learn to generate detailed attribute information.

[0255] The server performs user matching based on the generated attribute information. Here, machine learning algorithms are used to select the most suitable partner based on the user's interests and personality. This process can be accelerated by leveraging common cloud computing platforms.

[0256] The device used by the user during the conversation records the conversation content and nonverbal communication in real time and sends it to the server. The server analyzes this data using natural language processing libraries such as NLTK and SpaCy. Based on the analysis results, the server provides the user with customized advice and suggestions for activities based on shared interests through the device.

[0257] As a concrete example, when user A registers information about their interests and personality, the server suggests user B, who is a good match for user A. On the day of the date, the device monitors user A's conversation and sends data to the server. If the conversation stalls, the server displays a prompt on user A's device such as, "Try talking about music, which is a common hobby you share with user B."

[0258] After the date ends, user A provides feedback to the server via their device. The server stores this feedback in a database and uses it to improve future matching suggestions, thereby continuously supporting the relationship between users.

[0259] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0260] Step 1:

[0261] Users use a terminal to input basic information and lifestyle data. This input data includes name, age, hobbies, values, etc. This information is sent from the terminal to the server. The entered data is stored in the server's database in its original form.

[0262] Step 2:

[0263] The server processes the received user attribute data using a Python-based data analysis tool to generate detailed attribute information. This process extracts feature vectors from the input data and performs clustering or classification using a machine learning library (e.g., Scikit-learn). The output is a profile based on the user's interests and personality. This profile is used for matching in the next step.

[0264] Step 3:

[0265] The server matches users with other registered users based on the generated profile information. The input is the generated profile information, which is used to evaluate compatibility using a machine learning algorithm. The output is a list of the best match. The server analyzes this information, selects the best candidate (e.g., User B), and proposes it to the user via the terminal.

[0266] Step 4:

[0267] On the day of the interaction, the terminal records the user's conversation in real time and sends it to the server. The input consists of voice data and other communication data. The server uses speech recognition technology to convert this data into text and performs analysis using natural language processing libraries (e.g., NLTK, SpaCy). The output provides analysis results regarding the flow and emotions of the conversation. Specifically, it identifies the tone of the conversation and any pauses.

[0268] Step 5:

[0269] The server provides the user with appropriate advice and activity suggestions based on the analysis results. The input is the result of the conversation analysis obtained in the previous step, and the output is expressed as a prompt. This allows the terminal to display customized suggestions, such as "Try talking about cooking, which is a common hobby of yours."

[0270] Step 6:

[0271] Users provide feedback after the interaction ends. The input consists of information regarding their satisfaction with the interaction and areas for improvement. The terminal sends this data to the server, which analyzes the data and updates the profile. The output is the updated user profile information, which improves the accuracy of future matching.

[0272] (Application Example 1)

[0273] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0274] Conventional interpersonal relationship support systems merely collect user profiles, but they are insufficient in providing real-time support during conversations or specific content based on shared interests. Furthermore, a mechanism is needed to smoothly provide common topics and activities in order to deepen mutual understanding between users.

[0275] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0276] In this invention, the server includes means for collecting user attribute data and generating a profile, means for collecting and analyzing conversational and nonverbal communication in real time during a dialogue, and means for presenting advice and activities tailored based on the analysis results. This enables the user to receive appropriate support during the dialogue and to communicate smoothly.

[0277] "User attribute data" refers to distinctive information related to individual users, including elements such as values, personality, and lifestyle.

[0278] A "profile" is a detailed representation of a person based on their attribute data, and is used as digital data for analysis and recommendations.

[0279] "Conversation and non-verbal communication during conversation" refers to various forms of communication including written communication or voice exchange, and visual or physical reactions.

[0280] "Collect and analyze in real time" refers to the process of immediately acquiring data at the moment it is generated and analyzing it immediately to obtain results quickly.

[0281] "Adjusted advice or activities" refers to advice or proposed actions optimized based on the analyzed data, which is specialized according to the needs of the user.

[0282] "Content containing information related to common interests" refers to content related to the concerns shared among users, which is information provided to facilitate conversation and activities.

[0283] "Information contributing to the promotion of conversation" refers to data and proposals that help smooth and deepen the communication between users, and are for supporting communication.

[0284] This system realizes functions for supporting real-time communication between users by utilizing the user's terminal, server, and AI technology.

[0285] The user's terminal collects attribute data from individual users and sends it to the server. This attribute data includes information such as values, personalities, and lifestyles. Thereby, the server generates a detailed profile. The server uses the profile and an AI model for proposing optimal advice and activities for the user. This AI model utilizes generative AI such as OpenAI's GPT-4.

[0286] Furthermore, during the conversation, the user's terminal collects the conversation content and non-verbal communication data in real time and sends them to the server. The server analyzes these data and provides information on common interests and newly interesting information. A natural language processing library (e.g., spaCy) is used for this, and Apache Kafka is used for real-time data analysis.

[0287] As a specific example, when User A and User B have a common hobby, the server provides recommendations for events and movies related to that hobby to the users through their terminals. Also, when a user brings up a new interest, for example, when a topic about tea ceremony comes up, the server can immediately provide relevant video links. In this process, the prompt sentence "The common hobby of User A and User B is Japanese cuisine. Please provide relevant topics and content to propose when the conversation breaks off." is used.

[0288] Thus, according to the present invention, communication between users proceeds more smoothly, and deep conversations based on common interests and new information become possible.

[0289] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0290] Step 1:

[0291] The user's terminal obtains attribute data as input from the user. This includes information about values, personality, and lifestyle. This data is organized within the terminal and prepared for transmission to the connected server. The server receives this data and analyzes the user's attribute data to generate a profile.

[0292] Step 2:

[0293] The server extracts common interests and potential topics based on the generated user profiles. Using a generative AI model, it creates prompts to suggest optimal advice and activities among users. These prompts are input into the AI ​​model, which then outputs suggestions.

[0294] Step 3:

[0295] Once the date begins, the user's device collects conversation content and nonverbal communication in real time. This data is sent to a server, which uses a natural language processing library to analyze the conversation. Based on the input conversation data, common topics and interests are further refined, and information that can be presented to the user is prepared.

[0296] Step 4:

[0297] Based on the analysis, the server suggests tailored advice and activities to the user. For example, it may output information about new movies or events and display it as a notification on the device. Also, if the conversation is interrupted, it will use the AI ​​model again to suggest topics to resume the conversation.

[0298] Step 5:

[0299] After a date, the user submits feedback to the server via their device. This feedback is important data for the system and is used to improve the accuracy of the profile. The server uses this to update the profile and enable more helpful suggestions for the next date.

[0300] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0301] This invention is a system that provides support during dating by taking into account the user's profile data and emotional state. The user inputs basic information and lifestyle data through a terminal, and the server generates a detailed profile based on this information. After profile generation, the server suggests a matched partner to the user and prompts them to prepare for the date.

[0302] During the date, the device is equipped with sensors for voice and non-voice input, which capture conversation and nonverbal communication data in real time. At this time, an emotion engine operates and analyzes the user's emotional state using the captured data. Based on the results of the emotion engine's analysis, the server understands the user's emotional state.

[0303] Based on the analysis of emotional states and profiles, the server suggests personalized advice and activities based on shared interests to the user via the device. This makes it easier for users to communicate in a way that aligns with their emotions, leading to more fulfilling dates.

[0304] For example, if the emotion engine detects that user A is feeling nervous during a date, the server will suggest relaxing activities and present easy-to-talk-about topics through the device. In this way, user A can continue the date in a more relaxed state.

[0305] After a date, users provide feedback using their devices, which is collected by a server. The server updates the user's profile based on the feedback and sentiment engine data, enabling further advice for future dates. This allows users to continuously improve and grow with each date. Through these features, the present invention provides comprehensive support for users to deepen their relationships.

[0306] The following describes the processing flow.

[0307] Step 1:

[0308] The user uses the terminal to input data related to their basic information, values, hobbies, and lifestyle. This information is sent to the server via the terminal.

[0309] Step 2:

[0310] The server generates a detailed profile based on the received user data. The generated profile is used to determine the optimal matching partner for the user.

[0311] Step 3:

[0312] The server proposes an optimal matching partner for the profile and notifies the user of the details of the date. The user checks and accepts according to their schedule and activities content.

[0313] Step 4:

[0314] At the start of the date, the terminal uses the voice sensor and camera to capture conversation and non-verbal communication data in real time. The captured data is sent to the server.

[0315] Step 5:

[0316] The server utilizes the emotion engine to analyze the captured data and detect the user's emotional state. Through this analysis, the server grasps the emotional state of the user, such as whether the user is nervous or relaxed.

[0317] Step 6:

[0318] Based on the analysis results of the emotion engine, the server selects appropriate advice or activities for the user and presents them to the user through the terminal. Thereby, the user can take corresponding actions according to the situation.

[0319] Step 7:

[0320] After the date ends, the device sends a feedback form to the user to collect information about their experience. The user fills out the feedback and sends it to the server via the device.

[0321] Step 8:

[0322] The server integrates feedback data with emotional data from the emotion engine and updates the user profile. This allows for the provision of suggestions for improvement and new advice for the next date.

[0323] (Example 2)

[0324] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0325] In modern society, while the evolution of digital technology has made building relationships online commonplace, deepening mutual understanding can be difficult in face-to-face communication. Especially during initial encounters or early stages of interaction, tension and misunderstandings are more likely to occur, highlighting the need for methods to improve the quality of dates. Furthermore, being able to understand a user's emotional state in real time and provide advice and activities based on that understanding can lead to better communication.

[0326] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0327] In this invention, the server includes means for collecting user characteristic information and generating a detailed profile based on values, personality, and lifestyle; means for acquiring voice and non-voice communication during a date in real time and analyzing the emotional state; and means for suggesting personalized advice and activities based on shared interests based on the analysis results. This enables communication support that is tailored to the user's emotions and interests.

[0328] A "user" refers to anyone who uses the system and provides profile data.

[0329] "Characteristic information" refers to information such as the user's values, personality, and lifestyle, and is used to generate profiles.

[0330] A "profile" refers to a collection of detailed data generated based on a user's characteristics.

[0331] "Voice communication" refers to conversations and exchanges of spoken language between users.

[0332] "Non-verbal communication" refers to interactions between users that do not involve voice, such as gestures and facial expressions.

[0333] "Acquiring information in real time" means obtaining information immediately at the time it is received.

[0334] "Emotional state" refers to the user's psychological state or mood, and is analyzed by the emotion engine.

[0335] "Analysis" refers to the process of processing information based on acquired data to derive meaning or results.

[0336] "Advice" refers to personalized advice and suggestions provided to the user.

[0337] "Activities" refer to specific activities suggested based on user interactions and interests.

[0338] A "generative AI model" refers to an algorithm that uses artificial intelligence to generate information tailored to the user.

[0339] A "prompt statement" refers to a specific instruction or command that causes a generative AI model to operate.

[0340] "Support" refers to assistance or assistance that helps users communicate smoothly.

[0341] To implement this invention, the user first inputs their personal information using a terminal. This terminal can be a smartphone or a personal computer. This information will be used later to create a profile and includes the user's values, personality, and lifestyle.

[0342] The server receives characteristic information sent by the user and generates a detailed profile based on it. This profile generation uses a generative AI model, which optimizes the profile content using prompts. For example, the prompt might read, "Generate a profile based on user A's hobbies."

[0343] During the date, the device uses sensors for voice and non-voice input to capture the user's voice and non-voice communication in real time. The server analyzes this data and activates an emotion engine to understand the user's emotional state. Emotions, including the user's level of tension, are assessed.

[0344] Based on the analysis results, the server suggests advice and activities based on shared interests for the user. This helps support communication during the date. For example, if the user is nervous, the server suggests "relaxing activities" and provides a topic such as "talking about recent travel experiences."

[0345] After a date ends, users provide feedback using their devices. This includes their satisfaction level and areas for improvement. The server collects this feedback appropriately and uses it to update their profile and improve the accuracy of future suggestions. In this way, the system can provide users with a meaningful dating experience.

[0346] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0347] Step 1:

[0348] The user uses a terminal to enter their personal information, such as name, age, hobbies, and areas of interest. This information is sent to the server and used as input data for generating the next profile.

[0349] Step 2:

[0350] The server uses a generative AI model to generate a detailed profile based on the characteristic information received from the user. In this process, the system receives a prompt such as, "Generate a profile based on user A's hobbies." The generative AI model processes this prompt and generates profile data that reflects the user's values ​​and personality. This output forms the basis for matching and providing advice.

[0351] Step 3:

[0352] During the date, the device acquires audio and non-audio data in real time using sensors. The microphone and camera operate, capturing language, facial expressions, and other information. This data is sent to a server as input, where it is analyzed by an emotion engine to determine the user's emotional state. The output is user emotion evaluation data.

[0353] Step 4:

[0354] Based on emotion assessment data and profiles, the server suggests personalized advice and activities for the user. For example, if the analysis indicates that the user is feeling stressed, specific suggestions such as "take a break in a relaxing place" or "talk about recent movies" will be provided. These suggestions are delivered to the user through their device.

[0355] Step 5:

[0356] After the date, the user returns their device to provide feedback. This feedback includes writing down their impressions and feelings about the date. This feedback data is sent to the server and integrated with previous sentiment evaluation data to form the basis for updating the profile. The updated profile contributes to improving and optimizing advice for future dates.

[0357] (Application Example 2)

[0358] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0359] In today's world, improving the quality of interpersonal relationships and communication is crucial. However, face-to-face communication often causes tension and anxiety, and this problem is particularly pronounced in new environments and interactions with new people. Furthermore, providing personalized advice based on each user's different values ​​and characteristics is a challenge. Effective means of addressing these challenges are needed.

[0360] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0361] In this invention, the server includes means for collecting the user's biometric data and generating a detailed user profile based on characteristics and lifestyle; means for acquiring and analyzing dialogue and nonverbal communication within the virtual space in real time; and means for proposing customized advice and activities based on the analysis results. This enables effective communication in a less stressful virtual environment and allows for the proposal of advice and activities optimized for each user.

[0362] "User biometric data" refers to information and signals related to the user's body, specifically including digitizable information such as heart rate, skin temperature, and movement.

[0363] "Characteristics" refer to a user's individuality and tendencies, indicating their unique behavioral patterns and thought processes.

[0364] "Lifestyle" refers to the patterns of activities and time management that users engage in on a daily basis, and is a concept that includes the behavioral styles and habits based on those patterns.

[0365] A "detailed user profile" refers to a comprehensive collection of data about a user's personal information and background, including attribute information necessary to understand their unique needs and interests.

[0366] "Dialogue within a virtual space" refers to linguistic communication between users in a virtual space using digital technology, and includes exchanges using voice and text.

[0367] "Nonverbal communication" refers to communication that takes place through means other than language, and includes forms of communication such as body language, facial expressions, and tone of voice.

[0368] "Acquiring data in real time" means recording and collecting data at almost the same time as real time, and that the information is reflected instantly without delay.

[0369] "Analyzing" refers to the process of deciphering and classifying collected data as information, and understanding its meaning and characteristics.

[0370] "Customized advice and activities" refers to advice and action suggestions that are specifically designed and tailored based on the individual user's needs and profile.

[0371] In the system that implements this application, three parties—a server, a terminal, and a user—work together to perform their functions. The server collects the user's biometric data and generates a detailed profile that reflects their characteristics and lifestyle. The user enters a virtual space through the terminal and engages in dialogue and nonverbal communication there. The terminal is equipped with a function to acquire voice and movement in real time, and by sending this data to the server, the server analyzes the data and understands the user's emotional state.

[0372] The analysis uses software such as an emotion analysis engine, and the AI ​​performs the analysis in conjunction with non-voice data. By using software such as an "emotion analysis library" and a "profile management module," it is possible to obtain accurate analysis results.

[0373] Based on the analysis results, the server suggests personalized advice and activities to the user. This involves using a generative AI model to select the most suitable activity based on the user's emotional state, using prompts. For example, a text prompt might read, "Analyze the user's emotional state and suggest relaxing dating activities. The user's current profile shows signs of tension."

[0374] In this way, by having the server, terminal, and user cooperate in data processing and supporting communication in a stress-free virtual environment, users can receive advice that is tailored to their individual needs.

[0375] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0376] Step 1:

[0377] The server collects biometric data from the user. The user uses a device to input data such as heart rate, skin temperature, and movement in real time. The server receives this input data and generates a detailed profile that reflects the user's characteristics and lifestyle. This profile digitizes the user's basic features.

[0378] Step 2:

[0379] The user participates in the virtual space through a device. The device acquires nonverbal communication such as the user's voice, facial expressions, and movements in real time. This data is sent to a server. The device uses sensors that detect voice tone and facial expressions to accurately collect data.

[0380] Step 3:

[0381] The server analyzes the acquired data. A sentiment analysis engine is used for data analysis, and a generative AI model determines the user's emotional state. The server analyzes changes in voice tone and facial expressions, and based on the results, identifies the user's current emotional state. The input is non-verbal data, and the output is the identified emotional state.

[0382] Step 4:

[0383] The server generates customized advice and activities based on the user's emotional state. Using a generative AI model, it creates prompts and suggests the most suitable activities for the user based on those prompts. For example, it might generate a prompt such as, "Please suggest a relaxing virtual activity." The input is the emotional state, and the output is a suggestion of a specific activity.

[0384] Step 5:

[0385] The user experiences suggested activities in a virtual space. The device creates a virtual environment based on the user's selected activity, allowing the user to relax and engage in activities in the digital space. The device outputs video and audio appropriate to the activity to support the user's experience.

[0386] Step 6:

[0387] After a digital experience, users send feedback to a server via their device. The server analyzes the received feedback and updates the user's profile in more detail. This feedback allows for more refined advice and suggestions tailored to individual users. The input is the user's feedback, and the output is the updated profile.

[0388] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0389] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0390] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0391] [Third Embodiment]

[0392] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0393] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0394] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0395] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0396] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0397] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0398] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0399] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0400] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0401] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0402] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0403] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0404] This invention is a system that supports relationship building based on user profile data. Specifically, users provide basic information and lifestyle details via their devices, which the server analyzes to generate a detailed profile. This profile is used not only for appropriate matching between users but also for providing support during dates and suggesting activities.

[0405] The server receives conversation and nonverbal communication data from the device during the date. This data is processed in real time and analyzed by AI. Based on the analysis, the server provides the user with appropriate advice and topics through the device. As a result, users can communicate more confidently and smoothly during their dates.

[0406] Furthermore, the server collects feedback after each date and updates the user profile based on that feedback. This process helps users become aware of areas for improvement in future dates and communication, enabling them to sustainably develop their relationships.

[0407] As a concrete example, suppose user A registers information about their values ​​and hobbies. Based on this information, the server suggests user B, who is a good match for user A, and sets up a date. On the day of the date, the device monitors user A's conversation and sends it to the server. When the conversation stalls, the server suggests topics related to shared hobbies to help user A proceed more naturally. After the date, user A provides feedback through the device, and the server reflects this information in their profile, enabling it to make better suggestions for the next date. In this way, the present invention provides an environment in which users can confidently deepen their interpersonal interactions.

[0408] The following describes the processing flow.

[0409] Step 1:

[0410] Users use a device to input data about their basic information, values, hobbies, and lifestyle. Once the information is entered, the device sends it to the server.

[0411] Step 2:

[0412] The server generates a detailed profile based on the received user data. This profile includes the user's characteristics and interests, which helps in matching them with suitable dating partners.

[0413] Step 3:

[0414] The server uses the generated profile to select the most suitable match for the user and notifies the user of a date proposal. The user receives the notification and confirms and accepts the date, time, and activity details.

[0415] Step 4:

[0416] On the day of the date, the device captures the user's conversation in real time. The captured audio and nonverbal communication data is immediately sent to the server.

[0417] Step 5:

[0418] The server uses AI to analyze the received dating data and detect potential conversational pauses or awkward moments that the user might encounter. If necessary, it provides advice and topic suggestions to the user via the device.

[0419] Step 6:

[0420] After the date ends, the device sends a feedback form to the user. The user fills out the feedback and reflects on their date experience.

[0421] Step 7:

[0422] The server collects user feedback and incorporates it into profile updates. This allows for more improved advice and suggestions for future dates.

[0423] (Example 1)

[0424] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0425] In modern society, many people experience difficulties in interpersonal interactions. In particular, finding someone who shares their interests and values, and maintaining smooth communication during interactions, present unique challenges when building new relationships. Furthermore, providing real-time communication support and fostering continuous relationship development based on feedback is not easy. Effective solutions to these challenges are needed.

[0426] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0427] In this invention, the server includes means for collecting user attribute data and generating detailed attribute information based on interests, personality, and lifestyle; means for recording and analyzing conversations and nonverbal communications in real time during interactions; means for proposing customized advice and activities based on the analysis results; and means for collecting feedback from users after interactions and updating attribute information. This enables optimal matching between users, as well as real-time support for smooth interactions and support for the development of continuous relationships.

[0428] A "user" refers to an individual who uses the system and is the entity that provides their attribute data.

[0429] "Attribute data" refers to data that includes information about a user's interests, personality, and lifestyle.

[0430] "Detailed attribute information" refers to information that represents the user's characteristics, generated based on collected attribute data.

[0431] "In conversation" refers to a state in which a user is communicating with another individual.

[0432] "Conversation" refers to the exchange of information, either verbally or in writing, and is a part of the communication that takes place between users.

[0433] "Nonverbal communication" refers to communication that takes place through means other than words, such as gestures and facial expressions.

[0434] "Real-time recording" refers to instantly collecting and saving information while a conversation is in progress.

[0435] "Analyzing" refers to the process of analyzing collected data to extract meaning and patterns.

[0436] "Customized advice" refers to advice optimized for the user's characteristics and circumstances based on analyzed data.

[0437] "Activity suggestions" refer to instructions that present and encourage users to take on activities based on their interests and commonalities.

[0438] This invention provides a system to support relationship building based on user attribute data. The system starts operating when the user sends basic information and lifestyle data collected via a terminal to a server. The server analyzes this data using Python-based data analysis tools and machine learning libraries such as Scikit-learn to generate detailed attribute information.

[0439] The server performs user matching based on the generated attribute information. Here, machine learning algorithms are used to select the most suitable partner based on the user's interests and personality. This process can be accelerated by leveraging common cloud computing platforms.

[0440] The device used by the user during the conversation records the conversation content and nonverbal communication in real time and sends it to the server. The server analyzes this data using natural language processing libraries such as NLTK and SpaCy. Based on the analysis results, the server provides the user with customized advice and suggestions for activities based on shared interests through the device.

[0441] As a concrete example, when user A registers information about their interests and personality, the server suggests user B, who is a good match for user A. On the day of the date, the device monitors user A's conversation and sends data to the server. If the conversation stalls, the server displays a prompt on user A's device such as, "Try talking about music, which is a common hobby you share with user B."

[0442] After the date ends, user A provides feedback to the server via their device. The server stores this feedback in a database and uses it to improve future matching suggestions, thereby continuously supporting the relationship between users.

[0443] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0444] Step 1:

[0445] Users use a terminal to input basic information and lifestyle data. This input data includes name, age, hobbies, values, etc. This information is sent from the terminal to the server. The entered data is stored in the server's database in its original form.

[0446] Step 2:

[0447] The server processes the received user attribute data using a Python-based data analysis tool to generate detailed attribute information. This process extracts feature vectors from the input data and performs clustering or classification using a machine learning library (e.g., Scikit-learn). The output is a profile based on the user's interests and personality. This profile is used for matching in the next step.

[0448] Step 3:

[0449] The server matches users with other registered users based on the generated profile information. The input is the generated profile information, which is used to evaluate compatibility using a machine learning algorithm. The output is a list of the best match. The server analyzes this information, selects the best candidate (e.g., User B), and proposes it to the user via the terminal.

[0450] Step 4:

[0451] On the day of the interaction, the terminal records the user's conversation in real time and sends it to the server. The input consists of voice data and other communication data. The server uses speech recognition technology to convert this data into text and performs analysis using natural language processing libraries (e.g., NLTK, SpaCy). The output provides analysis results regarding the flow and emotions of the conversation. Specifically, it identifies the tone of the conversation and any pauses.

[0452] Step 5:

[0453] The server provides the user with appropriate advice and activity suggestions based on the analysis results. The input is the result of the conversation analysis obtained in the previous step, and the output is expressed as a prompt. This allows the terminal to display customized suggestions, such as "Try talking about cooking, which is a common hobby of yours."

[0454] Step 6:

[0455] Users provide feedback after the interaction ends. The input consists of information regarding their satisfaction with the interaction and areas for improvement. The terminal sends this data to the server, which analyzes the data and updates the profile. The output is the updated user profile information, which improves the accuracy of future matching.

[0456] (Application Example 1)

[0457] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0458] Conventional interpersonal relationship support systems merely collect user profiles, but they are insufficient in providing real-time support during conversations or specific content based on shared interests. Furthermore, a mechanism is needed to smoothly provide common topics and activities in order to deepen mutual understanding between users.

[0459] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0460] In this invention, the server includes means for collecting user attribute data and generating a profile, means for collecting and analyzing conversational and nonverbal communication in real time during a dialogue, and means for presenting advice and activities tailored based on the analysis results. This enables the user to receive appropriate support during the dialogue and to communicate smoothly.

[0461] "User attribute data" refers to distinctive information related to individual users, including elements such as values, personality, and lifestyle.

[0462] A "profile" is a detailed representation of a person based on their attribute data, and is used as digital data for analysis and recommendations.

[0463] "Conversation and nonverbal communication in a dialogue" refers to various forms of communication, including written or spoken communication, and visual or physical responses.

[0464] "Collecting and analyzing data in real time" refers to the process of acquiring data instantly the moment it is generated and analyzing it immediately to obtain results quickly.

[0465] "Adjusted advice and actions" refer to advice and suggested actions that are optimized based on analyzed data and are tailored to the user's needs.

[0466] "Content related to common interests" refers to content related to shared interests among users, and is provided to facilitate conversations and activities.

[0467] "Information that contributes to facilitating dialogue" refers to data and suggestions that are useful for smoothing and deepening interactions between users, and are intended to support communication.

[0468] This system utilizes user terminals, servers, and AI technology to provide functions that support real-time communication between users.

[0469] The user's device collects attribute data from individual users and sends it to the server. This attribute data includes information such as values, personality, and lifestyle. The server then generates a detailed profile. Using this profile, the server employs an AI model to suggest optimal advice and activities for the user. This AI model utilizes generative AI such as OpenAI's GPT-4.

[0470] Furthermore, during the conversation, the user's device collects conversation content and nonverbal communication data in real time and sends it to the server. The server analyzes this data and provides information that is of common interest or that might pique new interest. This uses a natural language processing library (e.g., spaCy), and Apache Kafka is used for real-time data analysis.

[0471] As a concrete example, if user A and user B share a common hobby, the server will provide users with recommendations for events and movies related to that hobby via their devices. Furthermore, if a user develops a new interest, for example, if the topic of the tea ceremony comes up, the server can immediately provide relevant video links. In this process, the prompt message used would be, "Users A and B share a common hobby: Japanese cuisine. Please provide relevant topics and content to suggest when the conversation lulls."

[0472] Thus, the present invention facilitates smoother communication between users and enables deeper dialogue based on shared interests and new information.

[0473] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0474] Step 1:

[0475] The user's device receives attribute data from the user as input. This includes information about values, personality, and lifestyle. The device organizes this data and prepares it for transmission to the connected server. The server receives this data, analyzes the user's attribute data, and generates a profile.

[0476] Step 2:

[0477] The server extracts common interests and potential topics based on the generated user profiles. Using a generative AI model, it creates prompts to suggest optimal advice and activities among users. These prompts are input into the AI ​​model, which then outputs suggestions.

[0478] Step 3:

[0479] Once the date begins, the user's device collects conversation content and nonverbal communication in real time. This data is sent to a server, which uses a natural language processing library to analyze the conversation. Based on the input conversation data, common topics and interests are further refined, and information that can be presented to the user is prepared.

[0480] Step 4:

[0481] Based on the analysis, the server suggests tailored advice and activities to the user. For example, it may output information about new movies or events and display it as a notification on the device. Also, if the conversation is interrupted, it will use the AI ​​model again to suggest topics to resume the conversation.

[0482] Step 5:

[0483] After a date, the user submits feedback to the server via their device. This feedback is important data for the system and is used to improve the accuracy of the profile. The server uses this to update the profile and enable more helpful suggestions for the next date.

[0484] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0485] This invention is a system that provides support during dating by taking into account the user's profile data and emotional state. The user inputs basic information and lifestyle data through a terminal, and the server generates a detailed profile based on this information. After profile generation, the server suggests a matched partner to the user and prompts them to prepare for the date.

[0486] During the date, the device is equipped with sensors for voice and non-voice input, which capture conversation and nonverbal communication data in real time. At this time, an emotion engine operates and analyzes the user's emotional state using the captured data. Based on the results of the emotion engine's analysis, the server understands the user's emotional state.

[0487] Based on the analysis of emotional states and profiles, the server suggests personalized advice and activities based on shared interests to the user via the device. This makes it easier for users to communicate in a way that aligns with their emotions, leading to more fulfilling dates.

[0488] For example, if the emotion engine detects that user A is feeling nervous during a date, the server will suggest relaxing activities and present easy-to-talk-about topics through the device. In this way, user A can continue the date in a more relaxed state.

[0489] After a date, users provide feedback using their devices, which is collected by a server. The server updates the user's profile based on the feedback and sentiment engine data, enabling further advice for future dates. This allows users to continuously improve and grow with each date. Through these features, the present invention provides comprehensive support for users to deepen their relationships.

[0490] The following describes the processing flow.

[0491] Step 1:

[0492] Users use their devices to input data about their basic information, values, hobbies, and lifestyle. This information is then transmitted to the server via the device.

[0493] Step 2:

[0494] The server generates a detailed profile based on the received user data. This generated profile is used to determine the best match for the user.

[0495] Step 3:

[0496] The server suggests the most suitable match for the user's profile and notifies the user of the date details. The user reviews the schedule and activities and accepts.

[0497] Step 4:

[0498] At the start of the date, the device uses voice sensors and a camera to capture conversation and nonverbal communication data in real time. The captured data is sent to a server.

[0499] Step 5:

[0500] The server utilizes an emotion engine to analyze captured data and detect the user's emotional state. This analysis allows the server to understand the user's emotional state, such as whether they are stressed or relaxed.

[0501] Step 6:

[0502] Based on the analysis results of the emotion engine, the server selects appropriate advice and activities for the user and presents them to the user through the device. This allows the user to respond appropriately to the situation.

[0503] Step 7:

[0504] After the date ends, the device sends a feedback form to the user to collect information about their experience. The user fills out the feedback and sends it to the server via the device.

[0505] Step 8:

[0506] The server integrates feedback data with emotional data from the emotion engine and updates the user profile. This allows for the provision of suggestions for improvement and new advice for the next date.

[0507] (Example 2)

[0508] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0509] In modern society, while the evolution of digital technology has made building relationships online commonplace, deepening mutual understanding can be difficult in face-to-face communication. Especially during initial encounters or early stages of interaction, tension and misunderstandings are more likely to occur, highlighting the need for methods to improve the quality of dates. Furthermore, being able to understand a user's emotional state in real time and provide advice and activities based on that understanding can lead to better communication.

[0510] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0511] In this invention, the server includes means for collecting user characteristic information and generating a detailed profile based on values, personality, and lifestyle; means for acquiring voice and non-voice communication during a date in real time and analyzing the emotional state; and means for suggesting personalized advice and activities based on shared interests based on the analysis results. This enables communication support that is tailored to the user's emotions and interests.

[0512] A "user" refers to anyone who uses the system and provides profile data.

[0513] "Characteristic information" refers to information such as the user's values, personality, and lifestyle, and is used to generate profiles.

[0514] A "profile" refers to a collection of detailed data generated based on a user's characteristics.

[0515] "Voice communication" refers to conversations and exchanges of spoken language between users.

[0516] "Non-verbal communication" refers to interactions between users that do not involve voice, such as gestures and facial expressions.

[0517] "Acquiring information in real time" means obtaining information immediately at the time it is received.

[0518] "Emotional state" refers to the user's psychological state or mood, and is analyzed by the emotion engine.

[0519] "Analysis" refers to the process of processing information based on acquired data to derive meaning or results.

[0520] "Advice" refers to personalized advice and suggestions provided to the user.

[0521] "Activities" refer to specific activities suggested based on user interactions and interests.

[0522] A "generative AI model" refers to an algorithm that uses artificial intelligence to generate information tailored to the user.

[0523] A "prompt statement" refers to a specific instruction or command that causes a generative AI model to operate.

[0524] "Support" refers to assistance or assistance that helps users communicate smoothly.

[0525] To implement this invention, the user first inputs their personal information using a terminal. This terminal can be a smartphone or a personal computer. This information will be used later to create a profile and includes the user's values, personality, and lifestyle.

[0526] The server receives characteristic information sent by the user and generates a detailed profile based on it. This profile generation uses a generative AI model, which optimizes the profile content using prompts. For example, the prompt might read, "Generate a profile based on user A's hobbies."

[0527] During the date, the device uses sensors for voice and non-voice input to capture the user's voice and non-voice communication in real time. The server analyzes this data and activates an emotion engine to understand the user's emotional state. Emotions, including the user's level of tension, are assessed.

[0528] Based on the analysis results, the server suggests advice and activities based on shared interests for the user. This helps support communication during the date. For example, if the user is nervous, the server suggests "relaxing activities" and provides a topic such as "talking about recent travel experiences."

[0529] After a date ends, users provide feedback using their devices. This includes their satisfaction level and areas for improvement. The server collects this feedback appropriately and uses it to update their profile and improve the accuracy of future suggestions. In this way, the system can provide users with a meaningful dating experience.

[0530] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0531] Step 1:

[0532] The user uses a terminal to enter their personal information, such as name, age, hobbies, and areas of interest. This information is sent to the server and used as input data for generating the next profile.

[0533] Step 2:

[0534] The server uses a generative AI model to generate a detailed profile based on the characteristic information received from the user. In this process, the system receives a prompt such as, "Generate a profile based on user A's hobbies." The generative AI model processes this prompt and generates profile data that reflects the user's values ​​and personality. This output forms the basis for matching and providing advice.

[0535] Step 3:

[0536] During the date, the device acquires audio and non-audio data in real time using sensors. The microphone and camera operate, capturing language, facial expressions, and other information. This data is sent to a server as input, where it is analyzed by an emotion engine to determine the user's emotional state. The output is user emotion evaluation data.

[0537] Step 4:

[0538] Based on emotion assessment data and profiles, the server suggests personalized advice and activities for the user. For example, if the analysis indicates that the user is feeling stressed, specific suggestions such as "take a break in a relaxing place" or "talk about recent movies" will be provided. These suggestions are delivered to the user through their device.

[0539] Step 5:

[0540] After the date, the user returns their device to provide feedback. This feedback includes writing down their impressions and feelings about the date. This feedback data is sent to the server and integrated with previous sentiment evaluation data to form the basis for updating the profile. The updated profile contributes to improving and optimizing advice for future dates.

[0541] (Application Example 2)

[0542] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0543] In today's world, improving the quality of interpersonal relationships and communication is crucial. However, face-to-face communication often causes tension and anxiety, and this problem is particularly pronounced in new environments and interactions with new people. Furthermore, providing personalized advice based on each user's different values ​​and characteristics is a challenge. Effective means of addressing these challenges are needed.

[0544] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0545] In this invention, the server includes means for collecting the user's biometric data and generating a detailed user profile based on characteristics and lifestyle; means for acquiring and analyzing dialogue and nonverbal communication within the virtual space in real time; and means for proposing customized advice and activities based on the analysis results. This enables effective communication in a less stressful virtual environment and allows for the proposal of advice and activities optimized for each user.

[0546] "User biometric data" refers to information and signals related to the user's body, specifically including digitizable information such as heart rate, skin temperature, and movement.

[0547] "Characteristics" refer to a user's individuality and tendencies, indicating their unique behavioral patterns and thought processes.

[0548] "Lifestyle" refers to the patterns of activities and time management that users engage in on a daily basis, and is a concept that includes the behavioral styles and habits based on those patterns.

[0549] A "detailed user profile" refers to a comprehensive collection of data about a user's personal information and background, including attribute information necessary to understand their unique needs and interests.

[0550] "Dialogue within a virtual space" refers to linguistic communication between users in a virtual space using digital technology, and includes exchanges using voice and text.

[0551] "Nonverbal communication" refers to communication that takes place through means other than language, and includes forms of communication such as body language, facial expressions, and tone of voice.

[0552] "Acquiring data in real time" means recording and collecting data at almost the same time as real time, and that the information is reflected instantly without delay.

[0553] "Analyzing" refers to the process of deciphering and classifying collected data as information, and understanding its meaning and characteristics.

[0554] "Customized advice and activities" refers to advice and action suggestions that are specifically designed and tailored based on the individual user's needs and profile.

[0555] In the system that implements this application, three parties—a server, a terminal, and a user—work together to perform their functions. The server collects the user's biometric data and generates a detailed profile that reflects their characteristics and lifestyle. The user enters a virtual space through the terminal and engages in dialogue and nonverbal communication there. The terminal is equipped with a function to acquire voice and movement in real time, and by sending this data to the server, the server analyzes the data and understands the user's emotional state.

[0556] The analysis uses software such as an emotion analysis engine, and the AI ​​performs the analysis in conjunction with non-voice data. By using software such as an "emotion analysis library" and a "profile management module," it is possible to obtain accurate analysis results.

[0557] Based on the analysis results, the server suggests personalized advice and activities to the user. This involves using a generative AI model to select the most suitable activity based on the user's emotional state, using prompts. For example, a text prompt might read, "Analyze the user's emotional state and suggest relaxing dating activities. The user's current profile shows signs of tension."

[0558] In this way, by having the server, terminal, and user cooperate in data processing and supporting communication in a stress-free virtual environment, users can receive advice that is tailored to their individual needs.

[0559] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0560] Step 1:

[0561] The server collects biometric data from the user. The user uses a device to input data such as heart rate, skin temperature, and movement in real time. The server receives this input data and generates a detailed profile that reflects the user's characteristics and lifestyle. This profile digitizes the user's basic features.

[0562] Step 2:

[0563] The user participates in the virtual space through a device. The device acquires nonverbal communication such as the user's voice, facial expressions, and movements in real time. This data is sent to a server. The device uses sensors that detect voice tone and facial expressions to accurately collect data.

[0564] Step 3:

[0565] The server analyzes the acquired data. A sentiment analysis engine is used for data analysis, and a generative AI model determines the user's emotional state. The server analyzes changes in voice tone and facial expressions, and based on the results, identifies the user's current emotional state. The input is non-verbal data, and the output is the identified emotional state.

[0566] Step 4:

[0567] The server generates customized advice and activities based on the user's emotional state. Using a generative AI model, it creates prompts and suggests the most suitable activities for the user based on those prompts. For example, it might generate a prompt such as, "Please suggest a relaxing virtual activity." The input is the emotional state, and the output is a suggestion of a specific activity.

[0568] Step 5:

[0569] The user experiences suggested activities in a virtual space. The device creates a virtual environment based on the user's selected activity, allowing the user to relax and engage in activities in the digital space. The device outputs video and audio appropriate to the activity to support the user's experience.

[0570] Step 6:

[0571] After a digital experience, users send feedback to a server via their device. The server analyzes the received feedback and updates the user's profile in more detail. This feedback allows for more refined advice and suggestions tailored to individual users. The input is the user's feedback, and the output is the updated profile.

[0572] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0573] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0574] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0575] [Fourth Embodiment]

[0576] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0577] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0578] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0579] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0580] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0581] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0582] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0583] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0584] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0585] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0586] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0587] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0588] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0589] This invention is a system that supports relationship building based on user profile data. Specifically, users provide basic information and lifestyle details via their devices, which the server analyzes to generate a detailed profile. This profile is used not only for appropriate matching between users but also for providing support during dates and suggesting activities.

[0590] The server receives conversation and nonverbal communication data from the device during the date. This data is processed in real time and analyzed by AI. Based on the analysis, the server provides the user with appropriate advice and topics through the device. As a result, users can communicate more confidently and smoothly during their dates.

[0591] Furthermore, the server collects feedback after each date and updates the user profile based on that feedback. This process helps users become aware of areas for improvement in future dates and communication, enabling them to sustainably develop their relationships.

[0592] As a concrete example, suppose user A registers information about their values ​​and hobbies. Based on this information, the server suggests user B, who is a good match for user A, and sets up a date. On the day of the date, the device monitors user A's conversation and sends it to the server. When the conversation stalls, the server suggests topics related to shared hobbies to help user A proceed more naturally. After the date, user A provides feedback through the device, and the server reflects this information in their profile, enabling it to make better suggestions for the next date. In this way, the present invention provides an environment in which users can confidently deepen their interpersonal interactions.

[0593] The following describes the processing flow.

[0594] Step 1:

[0595] Users use a device to input data about their basic information, values, hobbies, and lifestyle. Once the information is entered, the device sends it to the server.

[0596] Step 2:

[0597] The server generates a detailed profile based on the received user data. This profile includes the user's characteristics and interests, which helps in matching them with suitable dating partners.

[0598] Step 3:

[0599] The server uses the generated profile to select the most suitable match for the user and notifies the user of a date proposal. The user receives the notification and confirms and accepts the date, time, and activity details.

[0600] Step 4:

[0601] On the day of the date, the device captures the user's conversation in real time. The captured audio and nonverbal communication data is immediately sent to the server.

[0602] Step 5:

[0603] The server uses AI to analyze the received dating data and detect potential conversational pauses or awkward moments that the user might encounter. If necessary, it provides advice and topic suggestions to the user via the device.

[0604] Step 6:

[0605] After the date ends, the device sends a feedback form to the user. The user fills out the feedback and reflects on their date experience.

[0606] Step 7:

[0607] The server collects user feedback and incorporates it into profile updates. This allows for more improved advice and suggestions for future dates.

[0608] (Example 1)

[0609] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0610] In modern society, many people experience difficulties in interpersonal interactions. In particular, finding someone who shares their interests and values, and maintaining smooth communication during interactions, present unique challenges when building new relationships. Furthermore, providing real-time communication support and fostering continuous relationship development based on feedback is not easy. Effective solutions to these challenges are needed.

[0611] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0612] In this invention, the server includes means for collecting user attribute data and generating detailed attribute information based on interests, personality, and lifestyle; means for recording and analyzing conversations and nonverbal communications in real time during interactions; means for proposing customized advice and activities based on the analysis results; and means for collecting feedback from users after interactions and updating attribute information. This enables optimal matching between users, as well as real-time support for smooth interactions and support for the development of continuous relationships.

[0613] A "user" refers to an individual who uses the system and is the entity that provides their attribute data.

[0614] "Attribute data" refers to data that includes information about a user's interests, personality, and lifestyle.

[0615] "Detailed attribute information" refers to information that represents the user's characteristics, generated based on collected attribute data.

[0616] "In conversation" refers to a state in which a user is communicating with another individual.

[0617] "Conversation" refers to the exchange of information, either verbally or in writing, and is a part of the communication that takes place between users.

[0618] "Nonverbal communication" refers to communication that takes place through means other than words, such as gestures and facial expressions.

[0619] "Real-time recording" refers to instantly collecting and saving information while a conversation is in progress.

[0620] "Analyzing" refers to the process of analyzing collected data to extract meaning and patterns.

[0621] "Customized advice" refers to advice optimized for the user's characteristics and circumstances based on analyzed data.

[0622] "Activity suggestions" refer to instructions that present and encourage users to take on activities based on their interests and commonalities.

[0623] This invention provides a system to support relationship building based on user attribute data. The system starts operating when the user sends basic information and lifestyle data collected via a terminal to a server. The server analyzes this data using Python-based data analysis tools and machine learning libraries such as Scikit-learn to generate detailed attribute information.

[0624] The server performs user matching based on the generated attribute information. Here, machine learning algorithms are used to select the most suitable partner based on the user's interests and personality. This process can be accelerated by leveraging common cloud computing platforms.

[0625] The device used by the user during the conversation records the conversation content and nonverbal communication in real time and sends it to the server. The server analyzes this data using natural language processing libraries such as NLTK and SpaCy. Based on the analysis results, the server provides the user with customized advice and suggestions for activities based on shared interests through the device.

[0626] As a concrete example, when user A registers information about their interests and personality, the server suggests user B, who is a good match for user A. On the day of the date, the device monitors user A's conversation and sends data to the server. If the conversation stalls, the server displays a prompt on user A's device such as, "Try talking about music, which is a common hobby you share with user B."

[0627] After the date ends, user A provides feedback to the server via their device. The server stores this feedback in a database and uses it to improve future matching suggestions, thereby continuously supporting the relationship between users.

[0628] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0629] Step 1:

[0630] Users use a terminal to input basic information and lifestyle data. This input data includes name, age, hobbies, values, etc. This information is sent from the terminal to the server. The entered data is stored in the server's database in its original form.

[0631] Step 2:

[0632] The server processes the received user attribute data using a Python-based data analysis tool to generate detailed attribute information. This process extracts feature vectors from the input data and performs clustering or classification using a machine learning library (e.g., Scikit-learn). The output is a profile based on the user's interests and personality. This profile is used for matching in the next step.

[0633] Step 3:

[0634] The server matches users with other registered users based on the generated profile information. The input is the generated profile information, which is used to evaluate compatibility using a machine learning algorithm. The output is a list of the best match. The server analyzes this information, selects the best candidate (e.g., User B), and proposes it to the user via the terminal.

[0635] Step 4:

[0636] On the day of the interaction, the terminal records the user's conversation in real time and sends it to the server. The input consists of voice data and other communication data. The server uses speech recognition technology to convert this data into text and performs analysis using natural language processing libraries (e.g., NLTK, SpaCy). The output provides analysis results regarding the flow and emotions of the conversation. Specifically, it identifies the tone of the conversation and any pauses.

[0637] Step 5:

[0638] The server provides the user with appropriate advice and activity suggestions based on the analysis results. The input is the result of the conversation analysis obtained in the previous step, and the output is expressed as a prompt. This allows the terminal to display customized suggestions, such as "Try talking about cooking, which is a common hobby of yours."

[0639] Step 6:

[0640] Users provide feedback after the interaction ends. The input consists of information regarding their satisfaction with the interaction and areas for improvement. The terminal sends this data to the server, which analyzes the data and updates the profile. The output is the updated user profile information, which improves the accuracy of future matching.

[0641] (Application Example 1)

[0642] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0643] Conventional interpersonal relationship support systems merely collect user profiles, but they are insufficient in providing real-time support during conversations or specific content based on shared interests. Furthermore, a mechanism is needed to smoothly provide common topics and activities in order to deepen mutual understanding between users.

[0644] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0645] In this invention, the server includes means for collecting user attribute data and generating a profile, means for collecting and analyzing conversational and nonverbal communication in real time during a dialogue, and means for presenting advice and activities tailored based on the analysis results. This enables the user to receive appropriate support during the dialogue and to communicate smoothly.

[0646] "User attribute data" refers to distinctive information related to individual users, including elements such as values, personality, and lifestyle.

[0647] A "profile" is a detailed representation of a person based on their attribute data, and is used as digital data for analysis and recommendations.

[0648] "Conversation and nonverbal communication in a dialogue" refers to various forms of communication, including written or spoken communication, and visual or physical responses.

[0649] "Collecting and analyzing data in real time" refers to the process of acquiring data instantly the moment it is generated and analyzing it immediately to obtain results quickly.

[0650] "Adjusted advice and actions" refer to advice and suggested actions that are optimized based on analyzed data and are tailored to the user's needs.

[0651] "Content related to common interests" refers to content related to shared interests among users, and is provided to facilitate conversations and activities.

[0652] "Information that contributes to facilitating dialogue" refers to data and suggestions that are useful for smoothing and deepening interactions between users, and are intended to support communication.

[0653] This system utilizes user terminals, servers, and AI technology to provide functions that support real-time communication between users.

[0654] The user's device collects attribute data from individual users and sends it to the server. This attribute data includes information such as values, personality, and lifestyle. The server then generates a detailed profile. Using this profile, the server employs an AI model to suggest optimal advice and activities for the user. This AI model utilizes generative AI such as OpenAI's GPT-4.

[0655] Furthermore, during the conversation, the user's device collects conversation content and nonverbal communication data in real time and sends it to the server. The server analyzes this data and provides information that is of common interest or that might pique new interest. This uses a natural language processing library (e.g., spaCy), and Apache Kafka is used for real-time data analysis.

[0656] As a concrete example, if user A and user B share a common hobby, the server will provide users with recommendations for events and movies related to that hobby via their devices. Furthermore, if a user develops a new interest, for example, if the topic of the tea ceremony comes up, the server can immediately provide relevant video links. In this process, the prompt message used would be, "Users A and B share a common hobby: Japanese cuisine. Please provide relevant topics and content to suggest when the conversation lulls."

[0657] Thus, the present invention facilitates smoother communication between users and enables deeper dialogue based on shared interests and new information.

[0658] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0659] Step 1:

[0660] The user's device receives attribute data from the user as input. This includes information about values, personality, and lifestyle. The device organizes this data and prepares it for transmission to the connected server. The server receives this data, analyzes the user's attribute data, and generates a profile.

[0661] Step 2:

[0662] The server extracts common interests and potential topics based on the generated user profiles. Using a generative AI model, it creates prompts to suggest optimal advice and activities among users. These prompts are input into the AI ​​model, which then outputs suggestions.

[0663] Step 3:

[0664] Once the date begins, the user's device collects conversation content and nonverbal communication in real time. This data is sent to a server, which uses a natural language processing library to analyze the conversation. Based on the input conversation data, common topics and interests are further refined, and information that can be presented to the user is prepared.

[0665] Step 4:

[0666] Based on the analysis, the server suggests tailored advice and activities to the user. For example, it may output information about new movies or events and display it as a notification on the device. Also, if the conversation is interrupted, it will use the AI ​​model again to suggest topics to resume the conversation.

[0667] Step 5:

[0668] After a date, the user submits feedback to the server via their device. This feedback is important data for the system and is used to improve the accuracy of the profile. The server uses this to update the profile and enable more helpful suggestions for the next date.

[0669] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0670] This invention is a system that provides support during dating by taking into account the user's profile data and emotional state. The user inputs basic information and lifestyle data through a terminal, and the server generates a detailed profile based on this information. After profile generation, the server suggests a matched partner to the user and prompts them to prepare for the date.

[0671] During the date, the device is equipped with sensors for voice and non-voice input, which capture conversation and nonverbal communication data in real time. At this time, an emotion engine operates and analyzes the user's emotional state using the captured data. Based on the results of the emotion engine's analysis, the server understands the user's emotional state.

[0672] Based on the analysis of emotional states and profiles, the server suggests personalized advice and activities based on shared interests to the user via the device. This makes it easier for users to communicate in a way that aligns with their emotions, leading to more fulfilling dates.

[0673] For example, if the emotion engine detects that user A is feeling nervous during a date, the server will suggest relaxing activities and present easy-to-talk-about topics through the device. In this way, user A can continue the date in a more relaxed state.

[0674] After a date, users provide feedback using their devices, which is collected by a server. The server updates the user's profile based on the feedback and sentiment engine data, enabling further advice for future dates. This allows users to continuously improve and grow with each date. Through these features, the present invention provides comprehensive support for users to deepen their relationships.

[0675] The following describes the processing flow.

[0676] Step 1:

[0677] Users use their devices to input data about their basic information, values, hobbies, and lifestyle. This information is then transmitted to the server via the device.

[0678] Step 2:

[0679] The server generates a detailed profile based on the received user data. This generated profile is used to determine the best match for the user.

[0680] Step 3:

[0681] The server suggests the most suitable match for the user's profile and notifies the user of the date details. The user reviews the schedule and activities and accepts.

[0682] Step 4:

[0683] At the start of the date, the device uses voice sensors and a camera to capture conversation and nonverbal communication data in real time. The captured data is sent to a server.

[0684] Step 5:

[0685] The server utilizes an emotion engine to analyze captured data and detect the user's emotional state. This analysis allows the server to understand the user's emotional state, such as whether they are stressed or relaxed.

[0686] Step 6:

[0687] Based on the analysis results of the emotion engine, the server selects appropriate advice and activities for the user and presents them to the user through the device. This allows the user to respond appropriately to the situation.

[0688] Step 7:

[0689] After the date ends, the device sends a feedback form to the user to collect information about their experience. The user fills out the feedback and sends it to the server via the device.

[0690] Step 8:

[0691] The server integrates feedback data with emotional data from the emotion engine and updates the user profile. This allows for the provision of suggestions for improvement and new advice for the next date.

[0692] (Example 2)

[0693] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0694] In modern society, while the evolution of digital technology has made building relationships online commonplace, deepening mutual understanding can be difficult in face-to-face communication. Especially during initial encounters or early stages of interaction, tension and misunderstandings are more likely to occur, highlighting the need for methods to improve the quality of dates. Furthermore, being able to understand a user's emotional state in real time and provide advice and activities based on that understanding can lead to better communication.

[0695] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0696] In this invention, the server includes means for collecting user characteristic information and generating a detailed profile based on values, personality, and lifestyle; means for acquiring voice and non-voice communication during a date in real time and analyzing the emotional state; and means for suggesting personalized advice and activities based on shared interests based on the analysis results. This enables communication support that is tailored to the user's emotions and interests.

[0697] A "user" refers to anyone who uses the system and provides profile data.

[0698] "Characteristic information" refers to information such as the user's values, personality, and lifestyle, and is used to generate profiles.

[0699] A "profile" refers to a collection of detailed data generated based on a user's characteristics.

[0700] "Voice communication" refers to conversations and exchanges of spoken language between users.

[0701] "Non-verbal communication" refers to interactions between users that do not involve voice, such as gestures and facial expressions.

[0702] "Acquiring information in real time" means obtaining information immediately at the time it is received.

[0703] "Emotional state" refers to the user's psychological state or mood, and is analyzed by the emotion engine.

[0704] "Analysis" refers to the process of processing information based on acquired data to derive meaning or results.

[0705] "Advice" refers to personalized advice and suggestions provided to the user.

[0706] "Activities" refer to specific activities suggested based on user interactions and interests.

[0707] A "generative AI model" refers to an algorithm that uses artificial intelligence to generate information tailored to the user.

[0708] A "prompt statement" refers to a specific instruction or command that causes a generative AI model to operate.

[0709] "Support" refers to assistance or assistance that helps users communicate smoothly.

[0710] To implement this invention, the user first inputs their personal information using a terminal. This terminal can be a smartphone or a personal computer. This information will be used later to create a profile and includes the user's values, personality, and lifestyle.

[0711] The server receives characteristic information sent by the user and generates a detailed profile based on it. This profile generation uses a generative AI model, which optimizes the profile content using prompts. For example, the prompt might read, "Generate a profile based on user A's hobbies."

[0712] During the date, the device uses sensors for voice and non-voice input to capture the user's voice and non-voice communication in real time. The server analyzes this data and activates an emotion engine to understand the user's emotional state. Emotions, including the user's level of tension, are assessed.

[0713] Based on the analysis results, the server suggests advice and activities based on shared interests for the user. This helps support communication during the date. For example, if the user is nervous, the server suggests "relaxing activities" and provides a topic such as "talking about recent travel experiences."

[0714] After a date ends, users provide feedback using their devices. This includes their satisfaction level and areas for improvement. The server collects this feedback appropriately and uses it to update their profile and improve the accuracy of future suggestions. In this way, the system can provide users with a meaningful dating experience.

[0715] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0716] Step 1:

[0717] The user uses a terminal to enter their personal information, such as name, age, hobbies, and areas of interest. This information is sent to the server and used as input data for generating the next profile.

[0718] Step 2:

[0719] The server uses a generative AI model to generate a detailed profile based on the characteristic information received from the user. In this process, the system receives a prompt such as, "Generate a profile based on user A's hobbies." The generative AI model processes this prompt and generates profile data that reflects the user's values ​​and personality. This output forms the basis for matching and providing advice.

[0720] Step 3:

[0721] During the date, the device acquires audio and non-audio data in real time using sensors. The microphone and camera operate, capturing language, facial expressions, and other information. This data is sent to a server as input, where it is analyzed by an emotion engine to determine the user's emotional state. The output is user emotion evaluation data.

[0722] Step 4:

[0723] Based on emotion assessment data and profiles, the server suggests personalized advice and activities for the user. For example, if the analysis indicates that the user is feeling stressed, specific suggestions such as "take a break in a relaxing place" or "talk about recent movies" will be provided. These suggestions are delivered to the user through their device.

[0724] Step 5:

[0725] After the date, the user returns their device to provide feedback. This feedback includes writing down their impressions and feelings about the date. This feedback data is sent to the server and integrated with previous sentiment evaluation data to form the basis for updating the profile. The updated profile contributes to improving and optimizing advice for future dates.

[0726] (Application Example 2)

[0727] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0728] In today's world, improving the quality of interpersonal relationships and communication is crucial. However, face-to-face communication often causes tension and anxiety, and this problem is particularly pronounced in new environments and interactions with new people. Furthermore, providing personalized advice based on each user's different values ​​and characteristics is a challenge. Effective means of addressing these challenges are needed.

[0729] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0730] In this invention, the server includes means for collecting the user's biometric data and generating a detailed user profile based on characteristics and lifestyle; means for acquiring and analyzing dialogue and nonverbal communication within the virtual space in real time; and means for proposing customized advice and activities based on the analysis results. This enables effective communication in a less stressful virtual environment and allows for the proposal of advice and activities optimized for each user.

[0731] "User biometric data" refers to information and signals related to the user's body, specifically including digitizable information such as heart rate, skin temperature, and movement.

[0732] "Characteristics" refer to a user's individuality and tendencies, indicating their unique behavioral patterns and thought processes.

[0733] "Lifestyle" refers to the patterns of activities and time management that users engage in on a daily basis, and is a concept that includes the behavioral styles and habits based on those patterns.

[0734] A "detailed user profile" refers to a comprehensive collection of data about a user's personal information and background, including attribute information necessary to understand their unique needs and interests.

[0735] "Dialogue within a virtual space" refers to linguistic communication between users in a virtual space using digital technology, and includes exchanges using voice and text.

[0736] "Nonverbal communication" refers to communication that takes place through means other than language, and includes forms of communication such as body language, facial expressions, and tone of voice.

[0737] "Acquiring data in real time" means recording and collecting data at almost the same time as real time, and that the information is reflected instantly without delay.

[0738] "Analyzing" refers to the process of deciphering and classifying collected data as information, and understanding its meaning and characteristics.

[0739] "Customized advice and activities" refers to advice and action suggestions that are specifically designed and tailored based on the individual user's needs and profile.

[0740] In the system that implements this application, three parties—a server, a terminal, and a user—work together to perform their functions. The server collects the user's biometric data and generates a detailed profile that reflects their characteristics and lifestyle. The user enters a virtual space through the terminal and engages in dialogue and nonverbal communication there. The terminal is equipped with a function to acquire voice and movement in real time, and by sending this data to the server, the server analyzes the data and understands the user's emotional state.

[0741] The analysis uses software such as an emotion analysis engine, and the AI ​​performs the analysis in conjunction with non-voice data. By using software such as an "emotion analysis library" and a "profile management module," it is possible to obtain accurate analysis results.

[0742] Based on the analysis results, the server suggests personalized advice and activities to the user. This involves using a generative AI model to select the most suitable activity based on the user's emotional state, using prompts. For example, a text prompt might read, "Analyze the user's emotional state and suggest relaxing dating activities. The user's current profile shows signs of tension."

[0743] In this way, by having the server, terminal, and user cooperate in data processing and supporting communication in a stress-free virtual environment, users can receive advice that is tailored to their individual needs.

[0744] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0745] Step 1:

[0746] The server collects biometric data from the user. The user uses a device to input data such as heart rate, skin temperature, and movement in real time. The server receives this input data and generates a detailed profile that reflects the user's characteristics and lifestyle. This profile digitizes the user's basic features.

[0747] Step 2:

[0748] The user participates in the virtual space through a device. The device acquires nonverbal communication such as the user's voice, facial expressions, and movements in real time. This data is sent to a server. The device uses sensors that detect voice tone and facial expressions to accurately collect data.

[0749] Step 3:

[0750] The server analyzes the acquired data. A sentiment analysis engine is used for data analysis, and a generative AI model determines the user's emotional state. The server analyzes changes in voice tone and facial expressions, and based on the results, identifies the user's current emotional state. The input is non-verbal data, and the output is the identified emotional state.

[0751] Step 4:

[0752] The server generates customized advice and activities based on the user's emotional state. Using a generative AI model, it creates prompts and suggests the most suitable activities for the user based on those prompts. For example, it might generate a prompt such as, "Please suggest a relaxing virtual activity." The input is the emotional state, and the output is a suggestion of a specific activity.

[0753] Step 5:

[0754] The user experiences suggested activities in a virtual space. The device creates a virtual environment based on the user's selected activity, allowing the user to relax and engage in activities in the digital space. The device outputs video and audio appropriate to the activity to support the user's experience.

[0755] Step 6:

[0756] After a digital experience, users send feedback to a server via their device. The server analyzes the received feedback and updates the user's profile in more detail. This feedback allows for more refined advice and suggestions tailored to individual users. The input is the user's feedback, and the output is the updated profile.

[0757] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0758] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0759] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0760] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0761] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0762] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0763] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0764] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0765] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0766] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0767] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0768] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0769] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0770] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0771] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0772] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0773] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0774] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0775] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0776] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0777] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0778] The following is further disclosed regarding the embodiments described above.

[0779] (Claim 1)

[0780] A means of collecting user profile data and generating a detailed profile based on values, personality, and lifestyle,

[0781] A means of capturing and analyzing conversational and nonverbal communication during a date in real time,

[0782] A means of suggesting personalized advice and activities based on the analysis results,

[0783] A means of collecting feedback from users after a date and updating their profile,

[0784] A system that includes this.

[0785] (Claim 2)

[0786] The system according to claim 1, comprising means for transmitting conversational and nonverbal data captured in real time to a server and providing appropriate support to the user based on the results of AI analysis.

[0787] (Claim 3)

[0788] The system according to claim 1, comprising means for promoting mutual understanding among users by suggesting activities based on common interests.

[0789] "Example 1"

[0790] (Claim 1)

[0791] A means of collecting user attribute data and generating detailed attribute information based on interests, personality, and lifestyle,

[0792] A means for recording and analyzing conversational and nonverbal communication in real time during a dialogue,

[0793] A means of proposing customized advice and activities based on the analysis results,

[0794] A means of collecting feedback from users after the interaction and updating attribute information,

[0795] A system that includes this.

[0796] (Claim 2)

[0797] The system according to claim 1, comprising means for transmitting conversational and nonverbal data recorded in real time to a processing device and providing appropriate support to the user based on the results of analysis by artificial intelligence.

[0798] (Claim 3)

[0799] The system according to claim 1, comprising means for proposing activities based on common interests and promoting mutual understanding among users.

[0800] "Application Example 1"

[0801] (Claim 1)

[0802] A means of collecting user attribute data and generating detailed profiles based on values, personality, and lifestyle,

[0803] A means of collecting and analyzing conversational and nonverbal communication in real time during a dialogue,

[0804] A means of providing advice and activities that are specifically tailored based on the analysis results,

[0805] A means of distributing content that includes information related to specific interests,

[0806] A means of collecting user feedback after the interaction and updating the profile,

[0807] A system that includes this.

[0808] (Claim 2)

[0809] The system according to claim 1, comprising means for transmitting conversational and nonverbal data collected in real time to a server and providing appropriate support to the user based on the results of AI analysis.

[0810] (Claim 3)

[0811] The system according to claim 1, comprising means for promoting mutual understanding among users by proposing activities based on common interests, and means for providing content that includes information that contributes to promoting dialogue.

[0812] "Example 2 of combining an emotion engine"

[0813] (Claim 1)

[0814] A means of collecting user characteristic information and generating detailed profiles based on values, personality, and lifestyle,

[0815] A means of acquiring audio and non-audio communication during a date in real time and analyzing the emotional state,

[0816] A means of suggesting personalized advice and activities based on common interests based on analysis results,

[0817] A means of collecting feedback from users after a date and updating their profile,

[0818] A means for optimizing profile generation and advice suggestions based on prompt text using a generative AI model,

[0819] A system that includes this.

[0820] (Claim 2)

[0821] The system according to claim 1, comprising means for transmitting voice and non-voice data acquired in real time to a processing device and providing appropriate support to the user based on the results of AI analysis.

[0822] (Claim 3)

[0823] The system according to claim 1, comprising means for promoting mutual understanding among users by proposing activities based on common interests.

[0824] "Application example 2 when combining with an emotional engine"

[0825] (Claim 1)

[0826] A means for collecting user biometric data and generating a detailed user profile based on characteristics and lifestyle,

[0827] A means for acquiring and analyzing dialogue and nonverbal communication in a virtual space in real time,

[0828] A means of proposing customized advice and activities based on the analysis results,

[0829] A means of collecting feedback from users after their experience in a virtual space and updating their profile,

[0830] A virtual reality system that includes this.

[0831] (Claim 2)

[0832] The virtual reality system according to claim 1, comprising means for transmitting dialogue and nonverbal information acquired in real time to a network server and providing appropriate support to the user based on the results of analysis by artificial intelligence.

[0833] (Claim 3)

[0834] The virtual reality system according to claim 1, comprising means for promoting empathetic understanding among users by proposing activities based on shared interests. [Explanation of Symbols]

[0835] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting user profile data and generating a detailed profile based on values, personality, and lifestyle, A means of capturing and analyzing conversational and nonverbal communication during a date in real time, A means of suggesting personalized advice and activities based on the analysis results, A means of collecting feedback from users after a date and updating their profile, A system that includes this.

2. The system according to claim 1, comprising means for transmitting conversational and nonverbal data captured in real time to a server and providing appropriate support to the user based on the results of AI analysis.

3. The system according to claim 1, comprising means for promoting mutual understanding among users by suggesting activities based on common interests.

Images