system

Abstract

We provide the system. [Solution] A means for receiving communication data from multiple information devices and analyzing that communication data to identify each individual's interests and emotions, A means for generating multiple activity suggestions tailored to each individual's interests and emotions based on the analysis results, A means for transmitting the generated activity proposals to each information device and receiving responses to the activity proposals, A means of adjusting and deciding on the optimal activities among individuals based on their responses, Means for making reservations and transactions related to the decided activities, A system that includes means to analyze information related to meal planning and automate the selection of the optimal restaurant or delivery procedures.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Due to the busy schedules and indecision of working people, there is a problem that consensus formation is not smoothly carried out when making plans in a group and the decision is not finalized until the end. As a result, stress is felt in communication and a lack of unity occurs due to the non-implementation of the plan.

Means for Solving the Problems

[0005] This invention provides a means for receiving conversation data from multiple communication devices, analyzing that data, and identifying the user's interests and emotions. Based on the analysis results, it generates activity suggestions tailored to each user and sends the suggestions to the user's communication device. By receiving feedback from the user and planning the optimal activity, it supports rapid and effective consensus building. Furthermore, reservations and payments for the suggested activities are completed within the same system, improving user convenience. In addition, by updating the model based on feedback after the activity is completed, the accuracy of future suggestions is improved.

[0006] "Communication devices" refer to electronic devices that allow users to send and receive data via the internet, and include smartphones, tablets, personal computers, etc.

[0007] "Conversation data" refers to information such as text, audio, and video exchanged through the communication methods used by users.

[0008] "Analysis" refers to the process of extracting meaning from conversational data and identifying the user's interests and emotions.

[0009] "User" refers to an individual or group that uses communication devices to share information with other users.

[0010] "Interests" refers to themes or genres that a user is particularly interested in.

[0011] "Emotions" refer to the mood or psychological state determined from the user's conversation data.

[0012] "Activity suggestions" refer to specific activities recommended to the user based on the analysis results.

[0013] "Feedback" refers to the degree of interest and evaluation that users show towards activity suggestions.

[0014] "Reservation" refers to the process of confirming the arrangement of an activity proposal that the user has agreed to.

[0015] "Settlement" refers to the process of completing the payment process for reserved activities.

[0016] "Model" refers to a database and analysis algorithm for recording user behaviors and preferences and optimizing the next activity proposal.

Brief Description of Drawings

[0017] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0021] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disk (e.g., hard disk), or magnetic tape, etc.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention is a system designed to enable busy and indecisive working adults to smoothly manage their schedules and plan collaborative activities. The program processing of this system is described below.

[0039] In this system, users participate in group chats using their own devices and discuss activity plans. All messages sent during the chat are sent from the device to the server and processed in real time.

[0040] The server analyzes the received conversation data using a generating AI model to identify each user's interests and emotions. This analysis is performed using natural language processing technology, generating appropriate suggestions based on the user's past activity history and the current conversation flow.

[0041] Once the analysis is complete, the server generates multiple suggestions based on the user's preferences and sends them to each user's device. At this stage, users can provide feedback on the suggestions they receive. Specifically, they can provide feedback by pressing the "Interested" button for activities they are interested in and the "Pass" button for those they are not interested in.

[0042] Based on the aggregated feedback, the server determines the most suitable activity suggestions among the users. Once the selected activity is confirmed, the server automatically coordinates with the partnered booking system to make the necessary reservations. The server also manages payments as needed, allowing users to complete all payments within the system.

[0043] After the activity ends, users input their impressions and evaluations via their devices and send them to the server. The submitted feedback is used as data to improve the accuracy of future suggestions and is utilized to update the generative AI model.

[0044] As a concrete example, on a weekend, users discuss plans to try a new restaurant via chat. The system suggests the best restaurant based on their preferences and past data, completes the reservation based on the activities chosen by the users, and ensures a competitive experience. In this way, the present invention enables smooth and efficient group planning, reduces the workload on users, and increases their satisfaction.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] A user accesses a group chat using their device and starts a conversation about the activity. The device captures incoming messages and prepares them to be sent to the server in real time.

[0048] Step 2:

[0049] The device sends chat messages entered by the user to the server. In doing so, the entire conversation is sent as data to preserve the context of the conversation.

[0050] Step 3:

[0051] The server applies a generative AI model to analyze the received message data. Specifically, it uses natural language processing techniques to extract keywords and determine the user's interests and current emotions.

[0052] Step 4:

[0053] Based on the analysis results, the server generates multiple activity suggestions tailored to the user's interests and emotions. In doing so, it also references each user's historical data and trend information to select the most suitable activity.

[0054] Step 5:

[0055] The server sends the generated activity proposals to the user's terminal. The sent proposals are displayed in a visually easy-to-understand format.

[0056] Step 6:

[0057] The user reviews the suggestions on their device, selects either "Interested" or "Pass" feedback for each suggestion, and sends this feedback to the server via their device.

[0058] Step 7:

[0059] The server aggregates the feedback submitted by each user. From the aggregated results, it selects the most supported proposal and finalizes the action to be taken.

[0060] Step 8:

[0061] The server coordinates with a partner reservation system for confirmed activities and makes the necessary reservations. At this time, it sends reservation confirmation information to the user's terminal to notify them.

[0062] Step 9:

[0063] The device displays a payment screen along with a reservation confirmation. The user completes the payment process within the app, thus finalizing the reservation.

[0064] Step 10:

[0065] After completing an activity, users enter their impressions and evaluations into their device and send that feedback to the server.

[0066] Step 11:

[0067] The server updates the generated AI model based on the feedback data, using it to improve the accuracy of future activity suggestions.

[0068] (Example 1)

[0069] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0070] In modern society, it is difficult for busy professionals to efficiently and smoothly coordinate schedules and plan collaborative activities. Selecting the most suitable activity for the group, taking into account individual interests and feelings, and handling reservations and payments collectively is time-consuming and stressful. Furthermore, effectively improving future proposals based on past experiences is also a challenge.

[0071] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0072] In this invention, the server includes means for receiving communication data from multiple information terminals and analyzing the communication data to identify the interests and emotions of each user; means for generating multiple activity suggestions tailored to the interests and emotions of each user based on the analysis results; and means for transmitting the generated activity suggestions to each information terminal and receiving responses to the activity suggestions. This makes it possible for even busy working adults to efficiently carry out everything from planning group activities to making reservations and payments in a consistent manner.

[0073] An "information terminal" is an electronic device used by users to send and receive communication data.

[0074] "Communication data" is a general term for messages and information transmitted from an information terminal.

[0075] "Analysis" is the process of identifying users' interests and emotions based on communication data.

[0076] "Interests and feelings" refers to the user's current interests and psychological state.

[0077] An "activity proposal" is a set of suggestions generated by the server based on the user's interests and feelings.

[0078] "Response" refers to feedback that includes opinions and choices expressed by users regarding the proposed activity.

[0079] An "optimal activity" is an activity that is adjusted to maximize the shared benefits for multiple users.

[0080] "Reservation" refers to the act of securing seats or locations necessary for an optimal activity in advance.

[0081] "Payment processing" refers to the process of managing a series of procedures related to the settlement of fees.

[0082] A "generative model" is an artificial intelligence algorithm used to create action plans based on data analysis.

[0083] This system is a digital platform for users to efficiently plan and manage group activities. Specifically, it consists of the following elements:

[0084] Users access the system using their own information devices (e.g., smartphones or personal computers). These devices send messages to the server via electronic communication. Conversations in group chats in which users participate serve as the starting point for this system.

[0085] The server collects communication data received from multiple information terminals and performs data analysis using a generative AI model. Natural language processing technology is used in the analysis to identify each user's interests and emotions. This process includes a database of past activity history and real-time conversation analysis.

[0086] Based on the analysis results, the server generates activity suggestions tailored to each user's preferences. These activity suggestions are sent to each user's information terminal and presented in a specific format. Users can provide feedback on the activity suggestions using the terminal's interface. They can press the "Interested" button for activities they are interested in and the "Pass" button for those they are not.

[0087] Based on the feedback gathered by the server, the optimal activity is determined. At this stage, the server integrates with the relevant reservation system and automatically reserves the necessary facilities and services for the activity. Payment processing is also managed centrally by the server, allowing users to easily complete payments on their devices. In this way, the server reduces the burden on users and provides consistent support from planning to execution.

[0088] After completing an activity, users submit feedback via their devices. This information is collected on a server and used to generate more accurate activity plans in the future. The system continuously improves through ongoing updates to the generating AI model.

[0089] A concrete example is a group of users who want to try a new restaurant on the weekend. The users express their preferences through chat, and the system suggests restaurants based on their past usage data. Reservations and payments are processed efficiently, allowing users to proceed with their plans with peace of mind.

[0090] Example of a prompt:

[0091] "We'd like to try a new restaurant this weekend. Please suggest the best restaurant based on our group's past history and current conversation."

[0092] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0093] Step 1:

[0094] Users join a group chat from their personal devices and enter messages about their planned activities. These messages become input data, which the device formats as text data and sends to the server via the internet. The device encrypts the data during transmission to ensure the security of the communication.

[0095] Step 2:

[0096] The server acquires text data received from information terminals and performs data analysis using a generative AI model. The input text data is analyzed using natural language processing technology to identify each user's interests and emotions. The specific interest and emotion data obtained as a result of the analysis is stored in a database as basic data necessary for generating activity plans.

[0097] Step 3:

[0098] The server generates multiple activity suggestions based on identified interest and sentiment data. This process involves matching past activity history with the user's current browsing data to provide personalized suggestions for each user. The generated activity suggestions are output and sent to each information terminal as a formatted list. The server ensures that the activity suggestions are delivered in a clear and easy-to-understand format.

[0099] Step 4:

[0100] Users send feedback on received activity proposals via their device. Users respond by pressing the "Interested" button for activity proposals they are interested in and the "Pass" button for those they are not interested in. The device collects this feedback data from users and sends it back to the server.

[0101] Step 5:

[0102] Based on the aggregated feedback data, the server begins the process of selecting the optimal activity for the entire group. This optimization process uses an algorithm to select the option that will maximize satisfaction across all users. Once the selected optimal activity is finalized, reservation information corresponding to it is output.

[0103] Step 6:

[0104] Based on the confirmed activity reservation information, the server coordinates with the partner reservation system to make the necessary reservations. It also processes payments through the payment system, completing all the necessary procedures for the activity. This ensures that users can enjoy the activity smoothly without any hassle.

[0105] Step 7:

[0106] After an activity is completed, users input their impressions and evaluations of the activity through their device. This feedback is again aggregated as data and sent to the server. The server incorporates the post-activity feedback data as learning data to improve future activity plans. Through this process, the overall accuracy of the system improves, enabling more satisfying suggestions.

[0107] (Application Example 1)

[0108] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0109] Busy and indecisive working adults face the challenge of efficiently and smoothly planning their meals. Furthermore, manually selecting appropriate restaurants that reflect individual preferences and arranging delivery is time-consuming and labor-intensive, thus creating a need for automation.

[0110] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0111] In this invention, the server includes means for receiving communication data from multiple information devices and analyzing that communication data to identify each individual's interests and emotions; means for generating multiple activity suggestions tailored to each individual's interests and emotions based on the analysis results; and means for analyzing information related to meal planning and automating the selection of the optimal restaurant or delivery procedures. This enables the suggestion of optimal activities based on individual preferences and the automatic execution of reservations and transactions.

[0112] "Information device" is a general term for devices that receive, transmit, and process data by electronic means.

[0113] "Communication data" refers to data related to people's communication exchanged between various information devices.

[0114] "Interest" refers to the things or activities that an individual is particularly interested in.

[0115] "Emotion" refers to the feelings or mental responses an individual experiences in response to a particular situation or piece of information.

[0116] An "activity suggestion" is a selection of activities that are recommended for participation, taking into account an individual's interests and feelings.

[0117] "Response" refers to information that indicates an individual's evaluation or preference for a proposed activity.

[0118] "Meal planning" refers to the planning and arrangements for meals.

[0119] "Restaurant selection" refers to the act of choosing the most suitable restaurant from a list of suggested restaurants based on specific criteria.

[0120] "Delivery procedures" refer to all operations and management steps involved in delivering selected food and beverages to a designated location.

[0121] "Automation" means performing tasks that would normally be done manually under the control of a computer.

[0122] This invention is a system designed to help busy and indecisive working adults efficiently plan their meals. The specific form of this system is described below.

[0123] This system consists of a server and user information devices (terminals). Users participate in group chats using their terminals to discuss meal plans. The terminals send chat data to the server, which analyzes this data using a generating AI model. The analysis utilizes Google Cloud's natural language processing API and OpenAI's GPT model. This identifies each user's interests and emotions, and based on this, meal activity suggestions are generated.

[0124] The server suggests the most suitable restaurants and menus based on the analysis results. It references external restaurant information databases (e.g., online restaurant APIs) to present options that match the user's preferences. Upon receiving user feedback, it automatically handles reservations and delivery procedures. Payments are securely processed using online transaction services such as Stripe.

[0125] For example, if a group of users are discussing in a chat that they "want to eat Italian food," the system will use past data to suggest nearby Italian restaurants, automatically complete reservations for the restaurants chosen by the users, and arrange food delivery if necessary.

[0126] An example of a prompt for a generative AI model is: "Assume the user is planning dinner and recommend the best restaurant based on their past preferences. Generate suggestions that match the appropriate chat content."

[0127] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0128] Step 1:

[0129] The terminal receives communication data from the user and sends it to the server. The input is natural language conversation data from the user, and the output is the data format in which this is sent to the server. The terminal captures the user's chat content in real time and sends this data to the server via a communication protocol.

[0130] Step 2:

[0131] The server analyzes the received interaction data using a generative AI model. The input is natural language conversation data sent from the terminal, and the output is a dataset reflecting each user's interests and emotions. The server utilizes Google Cloud's natural language processing API and OpenAI's GPT model to analyze the tone and keywords of the conversation to infer the user's preferences and emotional state. This analysis process identifies the type of meal and atmosphere the user desires.

[0132] Step 3:

[0133] The server generates activity suggestions based on the analysis results and sends them to the terminal. The input is a dataset reflecting the user's preferences and emotions, and the output is a list of specific activity suggestions provided to each user. The server refers to a pre-linked food and beverage information database to search for and select restaurants and menus suitable for the suggestions. The generated suggestions are presented to the user in an appealing way using prompt text.

[0134] Step 4:

[0135] The terminal collects user selections and responses and feeds them back to the server. Inputs are the user's indication of interest in suggestions or specific selection actions, and outputs are data packets that are sent to the server. The terminal monitors the user's button presses and text inputs and sends this behavioral data to the server for further processing.

[0136] Step 5:

[0137] The server determines the optimal activity based on user feedback and automatically handles booking and payment. Inputs are the user's selected activity suggestions and feedback information, while outputs are confirmed booking information and payment completion confirmation data. The server sends booking requests to selected restaurants and services and processes payments through online transaction services. As a result, users can receive services smoothly.

[0138] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0139] This invention effectively supports the planning of group activities for busy and indecisive users through a system that combines an emotion engine. The system aims to recognize users' interests and emotions through their conversational data and generate activity suggestions based on that information.

[0140] In this system, users first join a group chat using their device. All conversations that take place there are sent to the server in real time via the device. The server applies a sentiment engine in addition to a generative AI module to analyze the received conversation data. This sentiment engine operates as part of natural language processing technology and detects emotions from the user's writing.

[0141] The server uses both emotional information recognized by the emotion engine and interest information based on keywords to create activity suggestions tailored to the user. These suggestions reflect the user's current mood and interests, and the system generates multiple suggestions. The server then sends the generated activity suggestions to the terminal for the user to see.

[0142] Users review the suggestions displayed on their devices and provide feedback on each suggestion. By providing feedback through specific selections such as "Interested" or "Pass," users can reflect their own preferences. Once the feedback is returned to the server, the server aggregates it and selects the most suitable activity.

[0143] Once an activity is decided, the server works in conjunction with the partnered reservation system to take the necessary actions. All the information is then displayed on the device, allowing users to complete reservations and payments within the app, resulting in a highly convenient design.

[0144] After the activity concludes, user feedback is collected again, and the server uses this data to update the generative model, thereby improving the accuracy of future suggestions.

[0145] For example, if a user says in an online chat, "I'm happy because the weather has been nice lately," the emotion engine recognizes this as the emotion of "happiness." Based on this emotion data and past history, the server can suggest activities such as "picnic" or "hiking," and, after receiving user feedback, plan the most suitable activity. In this way, the present invention makes it possible to make emotionally resonant suggestions to the user and facilitate consensus building.

[0146] The following describes the processing flow.

[0147] Step 1:

[0148] Users use their devices to start a group chat and discuss activities.

[0149] Step 2:

[0150] The device sends chat messages from the user to the server in real time. The conversation data is aggregated on the server in text format.

[0151] Step 3:

[0152] The server analyzes the received conversation data using a generating AI model. This analysis incorporates an emotion engine that identifies interests based on keywords in the message and simultaneously analyzes the context to recognize emotions.

[0153] Step 4:

[0154] The server generates activity suggestions that match the user's interests and emotions based on the analysis results. This process takes into account factors such as weather information and geographical conditions, resulting in the creation of multiple suggestions.

[0155] Step 5:

[0156] The server sends the generated activity suggestions to the user's terminal. The suggestions are displayed in a list format, making it easy for the user to select from them.

[0157] Step 6:

[0158] The user reviews the suggestions displayed on their device, selects feedback for each from options such as "Interested" or "Pass," and sends this feedback to the server via their device.

[0159] Step 7:

[0160] The server aggregates the feedback submitted by each user. Based on the aggregated results, it determines the activity that received the most support across the entire user group.

[0161] Step 8:

[0162] The server manages the necessary booking process for the selected activity. It integrates with partnered external systems to retrieve booking information and completes the booking on the system.

[0163] Step 9:

[0164] The device displays a reservation confirmation notification to the user and opens the payment screen if necessary. The user can make payments securely within the app.

[0165] Step 10:

[0166] Users input feedback into their devices after completing an activity, and this data is sent to a server to evaluate the suggestions and experiences provided.

[0167] Step 11:

[0168] The server updates its emotion engine and generative model based on the feedback it receives, improving the accuracy of future suggestions. This feedback is stored as training data for the system.

[0169] (Example 2)

[0170] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0171] The goal is to alleviate the difficulties busy and indecisive users face in planning and implementing group activities, and to enable them to receive optimal activity suggestions that take into account their individual feelings and interests. Furthermore, the aim is to provide a system that streamlines the process from activity selection to booking and payment, facilitating smooth consensus building among users.

[0172] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0173] In this invention, the server includes means for receiving conversation generation data from multiple communication devices and analyzing the generated data to identify each user's intentions and emotions; means for generating multiple activity options that correspond to each user's intentions and psychological state based on the analysis results; and means for transmitting the generated activity options to each communication device and collecting opinions on the activity suggestions. This enables users to quickly and efficiently decide on the optimal activity that aligns with their emotions and interests, and also allows for smooth booking and payment for the decided activity.

[0174] A "communication device" is a terminal used by users to participate in group chats and send and receive data.

[0175] "Conversation generation data" refers to text information transmitted by users through communication devices, and serves as the basis for activity proposals.

[0176] A "user" is a person who receives plans and proposals for group activities, and is also a person who uses the system.

[0177] "Intentions" refer to a user's interests and desires, and represent information that expresses the user's preferences and expectations when choosing activities.

[0178] "Emotions" refer to the user's psychological state and are factors considered when the system suggests activities.

[0179] "Analysis results" refer to information obtained after analyzing conversation generation data, and serve as the basis for activity proposals.

[0180] "Activity options" are action plans proposed to users, reflecting their interests and emotions.

[0181] "Opinions" refer to feedback that users provide regarding activity options, and include specific responses such as "interested" or "pass."

[0182] This invention relates to a system for effectively supporting the planning of group activities involving multiple users. In this system, a server, a terminal, and users work together to generate activity suggestions based on the users' interests and emotions, and determine the optimal activity through feedback.

[0183] First, users join a group chat using a communication device (a terminal). The terminal collects conversation data generated between users and sends it to the server in real time. The conversation data is encrypted, ensuring security and privacy.

[0184] The server receives this conversation generation data and analyzes it using automated generative AI technology. This analysis utilizes natural language processing techniques and an emotion engine for emotion recognition to extract each user's intentions and emotions. Based on the analysis results, the server inputs prompt sentences into the generative AI model and generates activity options. For example, a prompt sentence such as, "The user's current mood is 'happy' and they have shown interest in outdoor activities in the past, so please suggest 'picnic' or 'hiking'," can be used.

[0185] The generated activity options are sent back from the server to the terminal and presented to the user. The user provides specific feedback on the presented suggestions. Feedback such as "Interested" or "Pass" is collected and aggregated again on the server. This feedback data is used to further train the generating AI model based on the user's preferences.

[0186] Once the optimal activity is determined, the server works in conjunction with a partnered reservation management system to handle the necessary reservation and payment procedures. This allows users to complete reservations and payments entirely on their devices. Furthermore, user feedback after the activity is recorded as data to improve future activity proposals.

[0187] In this way, the system aims to be attentive to the user's emotions and interests, and to facilitate consensus building.

[0188] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0189] Step 1:

[0190] A user joins a group chat using their device. The device receives text input from the user in real time and converts the input conversational data into a signal for communication. This signal is sent to the server. The input is text data in natural language and is output to the server as a stream.

[0191] Step 2:

[0192] The server analyzes the conversation-generated data it receives. Natural language processing techniques are used to transform text data into data for syntactic analysis and sentiment recognition. The input is text data sent from the terminal, and the output is analyzed intention and sentiment data. Specifically, the server analyzes word frequencies and extracts keywords.

[0193] Step 3:

[0194] The server generates activity options using generative AI technology. Based on the analysis results, it creates a prompt and inputs it into the generative AI model to generate new activity suggestions. The input consists of analyzed intention and emotion data and a prompt, while the output is multiple activity options. Specifically, the server generates the prompt "Suggest an activity appropriate to the user's current emotion."

[0195] Step 4:

[0196] The server sends the generated activity options to the terminal. Here, the options are visually organized for easier presentation to the user. The input is the generated activity options, and the output is suggested data formatted for display. Specifically, the server converts the data to HTML format or an application-specific format.

[0197] Step 5:

[0198] The user reviews the suggestions via their device and provides feedback. This feedback includes selections such as "Interested" or "Pass." The input is the displayed activity options, and the output is the user's feedback data. The device captures this feedback and sends it back to the server.

[0199] Step 6:

[0200] The server aggregates user feedback and determines the optimal activity. It analyzes the feedback data and selects the most suitable activity, taking into account the opinions of numerous users. The input is the feedback data, and the output is the determined activity plan. Specifically, the server evaluates the feedback using statistical methods.

[0201] Step 7:

[0202] The server integrates with a partnered reservation management system to handle the necessary reservation and payment procedures. Communication with an external system via API takes place here. The input is the decided activity plan, and the output is reservation confirmation information. The server sends this information to the terminal.

[0203] Step 8:

[0204] The user completes the reservation and payment on the terminal. The terminal displays reservation confirmation information and provides a screen for the payment process. The input is reservation confirmation information, and the output is completed transaction information. Specifically, the user enters payment information, and the terminal processes it securely.

[0205] Step 9:

[0206] After the activity ends, the server obtains feedback from the user again. This feedback is used to improve the activity options for future activities. The input is the user's opinion after the activity ends, and the output is update data for the generative model. The server uses this data to train the generative AI model.

[0207] (Application Example 2)

[0208] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0209] In modern families, enriching family time requires planning appropriate events and recreational activities that align with the interests and feelings of each family member. However, since each member has different interests and feelings, selecting the optimal activity is difficult, and decision-making within the family often does not proceed smoothly. Furthermore, efficiently executing these plans and consistently managing related reservations and payments is also a challenge.

[0210] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0211] This invention includes a server that receives conversational information acquired from multiple terminal devices, analyzes the conversational information to recognize the interests and emotions of each user, generates multiple task suggestions tailored to the interests and emotions of each user based on the analysis results, and presents the task suggestions to household appliances in real time to recommend household events and recreational activities. This makes it possible to efficiently plan and implement optimal events and recreational activities while being attentive to the emotions and interests of each member of the household.

[0212] A "terminal device" is a device used by a user to input and receive conversational information.

[0213] "Conversational information" refers to data that includes the content of text and audio exchanges between users, as well as their emotional nuances.

[0214] "Emotion" is an element that indicates the user's psychological state, and it is information extracted from conversational data.

[0215] "Interests" refers to information about topics and activities that a user is particularly interested in.

[0216] "Household machinery and equipment" refers to machines used within the home that have the function of presenting proposed tasks or activities.

[0217] "Proposal suggestions" are recommended plans for events and activities generated based on the user's interests and emotions.

[0218] "Reservation" refers to the procedure of securing the date, time, and location for a decided activity in advance.

[0219] "Payment" refers to the process of completing the necessary financial transaction for a decided activity.

[0220] The system for realizing this invention consists of a home appliance, an internet-connected terminal device, and a server. The server analyzes conversational information received from the terminal device used by the user. The analysis uses a software module that combines natural language processing technology and an emotion recognition engine. Specifically, the server uses "Python" and "TENSORFLOW®" to process conversational data, converts the audio data into text using the "Google Cloud Speech-to-Text API," and analyzes the emotions from that text using the emotion engine.

[0221] The server uses a generative AI model to create task suggestions based on analyzed interest and sentiment information. This generative AI model utilizes technologies such as "OpenAI GPT" to construct optimal event and activity suggestions for the user. The generated suggestions are displayed in real time on home electronic devices, which the user can view and evaluate.

[0222] When a user provides feedback on suggestions displayed on a home appliance, that feedback is sent to a server via the device. The server aggregates this feedback and can use it to plan future activities in order to improve the accuracy of the suggestions. As this process is repeated, events and activities within the home evolve to become more responsive to the user's emotions and interests.

[0223] For example, if a family says "I'd like to go to a pop concert" on a holiday morning, the device transmits this conversation to the server. The server analyzes the conversation to identify the "interest in pop music" and suggests nearby concerts or live streaming options. If the user selects "go to a concert," the server can automatically book and pay for tickets. An example of a prompt to the generative AI model would be, "Please suggest weekend activities for a family who likes pop concerts."

[0224] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0225] Step 1:

[0226] The device records the user's conversation and sends it to the server as audio data. The input is audio data, and the output is an audio file uploaded to the server. The device uses a microphone to capture audio, compresses the data, and efficiently sends it to the server.

[0227] Step 2:

[0228] The server converts the received audio data into text data using the Google Cloud Speech-to-Text API. The input is an audio file, and the output is human-readable text data. The converted text is then used for subsequent sentiment analysis.

[0229] Step 3:

[0230] The server analyzes the converted text data using an emotion engine to identify the user's emotions. The input is text data, and the output is data with emotion labels. The emotion engine uses natural language processing algorithms to determine emotions such as joy, anger, sadness, and happiness.

[0231] Step 4:

[0232] The server generates task suggestions using a generative AI model based on sentiment and text content. The input is sentiment labels and parsed text, and the output is a list of suggested events and activities. The generative AI model uses technologies such as "OpenAI GPT" to create appropriate suggestions.

[0233] Step 5:

[0234] The server transmits and displays the generated task proposals to the home appliance in real time. The input is a list of proposals, and the output is the content of the proposals that the user can view. The user reviews and selects a proposal from the appliance's display screen.

[0235] Step 6:

[0236] The user provides feedback on the suggestions through the interface of the home appliance. The input is the user's selected option, and the output is the feedback data sent back to the server. The selection process can be performed via touch screen or voice recognition.

[0237] Step 7:

[0238] The server stores the collected feedback in a database and analyzes it for the purpose of improving future suggestions. The input is user feedback data, and the output is analyzed information to improve the accuracy of the suggestions. The feedback data is used to adjust the suggestion generation algorithm.

[0239] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0240] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0241] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0242] [Second Embodiment]

[0243] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0244] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0245] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0246] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0247] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0248] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0249] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0250] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0251] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0252] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0253] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0254] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0255] This invention is a system designed to enable busy and indecisive working adults to smoothly manage their schedules and plan collaborative activities. The program processing of this system is described below.

[0256] In this system, users participate in group chats using their own devices and discuss activity plans. All messages sent during the chat are sent from the device to the server and processed in real time.

[0257] The server analyzes the received conversation data using a generating AI model to identify each user's interests and emotions. This analysis is performed using natural language processing technology, generating appropriate suggestions based on the user's past activity history and the current conversation flow.

[0258] Once the analysis is complete, the server generates multiple suggestions based on the user's preferences and sends them to each user's device. At this stage, users can provide feedback on the suggestions they receive. Specifically, they can provide feedback by pressing the "Interested" button for activities they are interested in and the "Pass" button for those they are not interested in.

[0259] Based on the aggregated feedback, the server determines the most suitable activity suggestions among the users. Once the selected activity is confirmed, the server automatically coordinates with the partnered booking system to make the necessary reservations. The server also manages payments as needed, allowing users to complete all payments within the system.

[0260] After the activity ends, users input their impressions and evaluations via their devices and send them to the server. The submitted feedback is used as data to improve the accuracy of future suggestions and is utilized to update the generative AI model.

[0261] As a concrete example, on a weekend, users discuss plans to try a new restaurant via chat. The system suggests the best restaurant based on their preferences and past data, completes the reservation based on the activities chosen by the users, and ensures a competitive experience. In this way, the present invention enables smooth and efficient group planning, reduces the workload on users, and increases their satisfaction.

[0262] The following describes the processing flow.

[0263] Step 1:

[0264] A user accesses a group chat using their device and starts a conversation about the activity. The device captures incoming messages and prepares them to be sent to the server in real time.

[0265] Step 2:

[0266] The device sends chat messages entered by the user to the server. In doing so, the entire conversation is sent as data to preserve the context of the conversation.

[0267] Step 3:

[0268] The server applies a generative AI model to analyze the received message data. Specifically, it uses natural language processing techniques to extract keywords and determine the user's interests and current emotions.

[0269] Step 4:

[0270] Based on the analysis results, the server generates multiple activity suggestions tailored to the user's interests and emotions. In doing so, it also references each user's historical data and trend information to select the most suitable activity.

[0271] Step 5:

[0272] The server sends the generated activity proposals to the user's terminal. The sent proposals are displayed in a visually easy-to-understand format.

[0273] Step 6:

[0274] The user reviews the suggestions on their device, selects either "Interested" or "Pass" feedback for each suggestion, and sends this feedback to the server via their device.

[0275] Step 7:

[0276] The server aggregates the feedback sent from each user. From the aggregation results, the most supported proposal is selected to determine the final activity.

[0277] Step 8:

[0278] The server cooperates with the reservation system for the determined activity and makes the necessary reservations. At this time, the confirmation information of the reservation is sent to the terminal to notify the user.

[0279] Step 9:

[0280] The terminal displays a payment screen along with the reservation confirmation. The user performs the payment procedure within the app, thereby completing the reservation.

[0281] Step 10:

[0282] After the activity ends, the user inputs their impressions and evaluations into the terminal and sends the feedback to the server.

[0283] Step 11:

[0284] The server updates the AI model generated based on the feedback data and uses it to improve the accuracy of activity proposals for subsequent times.

[0285] (Example 1)

[0286] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0287] In modern society, it is difficult for busy working people to efficiently and smoothly schedule adjustments and plan joint activities. Selecting the optimal activity as a group considering individual interests and feelings and performing reservations and payments collectively involves effort and stress. Also, effectively improving subsequent proposals by leveraging past experiences is a difficult problem.

[0288] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0289] In this invention, the server includes means for receiving communication data from multiple information terminals and analyzing the communication data to identify the interests and emotions of each user; means for generating multiple activity suggestions tailored to the interests and emotions of each user based on the analysis results; and means for transmitting the generated activity suggestions to each information terminal and receiving responses to the activity suggestions. This makes it possible for even busy working adults to efficiently carry out everything from planning group activities to making reservations and payments in a consistent manner.

[0290] An "information terminal" is an electronic device used by users to send and receive communication data.

[0291] "Communication data" is a general term for messages and information transmitted from an information terminal.

[0292] "Analysis" is the process of identifying users' interests and emotions based on communication data.

[0293] "Interests and feelings" refers to the user's current interests and psychological state.

[0294] An "activity proposal" is a set of suggestions generated by the server based on the user's interests and feelings.

[0295] "Response" refers to feedback that includes opinions and choices expressed by users regarding the proposed activity.

[0296] An "optimal activity" is an activity that is adjusted to maximize the shared benefits for multiple users.

[0297] "Reservation" refers to the act of securing seats or locations necessary for an optimal activity in advance.

[0298] "Payment processing" refers to the process of managing a series of procedures related to the settlement of fees.

[0299] A "generative model" is an artificial intelligence algorithm used to create action plans based on data analysis.

[0300] This system is a digital platform for users to efficiently plan and manage group activities. Specifically, it consists of the following elements:

[0301] Users access the system using their own information devices (e.g., smartphones or personal computers). These devices send messages to the server via electronic communication. Conversations in group chats in which users participate serve as the starting point for this system.

[0302] The server collects communication data received from multiple information terminals and performs data analysis using a generative AI model. Natural language processing technology is used in the analysis to identify each user's interests and emotions. This process includes a database of past activity history and real-time conversation analysis.

[0303] Based on the analysis results, the server generates activity suggestions tailored to each user's preferences. These activity suggestions are sent to each user's information terminal and presented in a specific format. Users can provide feedback on the activity suggestions using the terminal's interface. They can press the "Interested" button for activities they are interested in and the "Pass" button for those they are not.

[0304] Based on the feedback gathered by the server, the optimal activity is determined. At this stage, the server integrates with the relevant reservation system and automatically reserves the necessary facilities and services for the activity. Payment processing is also managed centrally by the server, allowing users to easily complete payments on their devices. In this way, the server reduces the burden on users and provides consistent support from planning to execution.

[0305] After the activity ends, the user sends feedback through the terminal. This information is aggregated on the server and used to generate more accurate activity plans in the future. The system continues to improve through continuous updates of the generated AI model.

[0306] As a specific example, there is a group of users who want to try a new restaurant on the weekend. The users express their wishes through chat, and the system proposes a restaurant considering past usage data. Reservations and payments are processed efficiently, and the users can implement the plan with confidence.

[0307] Example of a prompt sentence:

[0308] "We want to try a new restaurant this weekend. Please propose the most suitable restaurant based on the group's past history and the current conversation."

[0309] The flow of the specific process in Example 1 will be described using FIG. 11.

[0310] Step 1:

[0311] The user joins the group chat from their information terminal and enters a message about the activity schedule. This message becomes the input data, and the terminal formats it as text data and sends it to the server via the Internet. The terminal encrypts the data during transmission to ensure communication security.

[0312] Step 2:

[0313] The server acquires the text data received from the information terminal and performs data analysis using the generated AI model. The input text data is analyzed by natural language processing technology to identify the interests and emotions of each user. The specific interest and emotion data obtained as the analysis result is stored in the database as the basic data required for generating the activity plan.

[0314] Step 3:

[0315] The server generates multiple activity suggestions based on identified interest and sentiment data. This process involves matching past activity history with the user's current browsing data to provide personalized suggestions for each user. The generated activity suggestions are output and sent to each information terminal as a formatted list. The server ensures that the activity suggestions are delivered in a clear and easy-to-understand format.

[0316] Step 4:

[0317] Users send feedback on received activity proposals via their device. Users respond by pressing the "Interested" button for activity proposals they are interested in and the "Pass" button for those they are not interested in. The device collects this feedback data from users and sends it back to the server.

[0318] Step 5:

[0319] Based on the aggregated feedback data, the server begins the process of selecting the optimal activity for the entire group. This optimization process uses an algorithm to select the option that will maximize satisfaction across all users. Once the selected optimal activity is finalized, reservation information corresponding to it is output.

[0320] Step 6:

[0321] Based on the confirmed activity reservation information, the server coordinates with the partner reservation system to make the necessary reservations. It also processes payments through the payment system, completing all the necessary procedures for the activity. This ensures that users can enjoy the activity smoothly without any hassle.

[0322] Step 7:

[0323] After an activity is completed, users input their impressions and evaluations of the activity through their device. This feedback is again aggregated as data and sent to the server. The server incorporates the post-activity feedback data as learning data to improve future activity plans. Through this process, the overall accuracy of the system improves, enabling more satisfying suggestions.

[0324] (Application Example 1)

[0325] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0326] Busy and indecisive working adults face the challenge of efficiently and smoothly planning their meals. Furthermore, manually selecting appropriate restaurants that reflect individual preferences and arranging delivery is time-consuming and labor-intensive, thus creating a need for automation.

[0327] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0328] In this invention, the server includes means for receiving communication data from multiple information devices and analyzing that communication data to identify each individual's interests and emotions; means for generating multiple activity suggestions tailored to each individual's interests and emotions based on the analysis results; and means for analyzing information related to meal planning and automating the selection of the optimal restaurant or delivery procedures. This enables the suggestion of optimal activities based on individual preferences and the automatic execution of reservations and transactions.

[0329] "Information device" is a general term for devices that receive, transmit, and process data by electronic means.

[0330] "Communication data" refers to data related to people's communication exchanged between various information devices.

[0331] "Interest" refers to the things or activities that an individual is particularly interested in.

[0332] "Emotion" refers to the feelings or mental responses an individual experiences in response to a particular situation or piece of information.

[0333] An "activity suggestion" is a selection of activities that are recommended for participation, taking into account an individual's interests and feelings.

[0334] "Response" refers to information that indicates an individual's evaluation or preference for a proposed activity.

[0335] "Meal planning" refers to the planning and arrangements for meals.

[0336] "Restaurant selection" refers to the act of choosing the most suitable restaurant from a list of suggested restaurants based on specific criteria.

[0337] "Delivery procedures" refer to all operations and management steps involved in delivering selected food and beverages to a designated location.

[0338] "Automation" means performing tasks that would normally be done manually under the control of a computer.

[0339] This invention is a system designed to help busy and indecisive working adults efficiently plan their meals. The specific form of this system is described below.

[0340] This system consists of a server and user information devices (terminals). Users participate in group chats using their terminals to discuss meal plans. The terminals send chat data to the server, which analyzes this data using a generation AI model. The analysis utilizes Google Cloud's natural language processing API and OpenAI's GPT model. This identifies each user's interests and emotions, and based on this, meal activity suggestions are generated.

[0341] The server suggests the most suitable restaurants and menus based on the analysis results. It references external restaurant information databases (e.g., online restaurant APIs) to present options that match the user's preferences. Upon receiving user feedback, it automatically handles reservations and delivery procedures. Payments are securely processed using online transaction services such as Stripe.

[0342] For example, if a group of users are discussing in a chat that they "want to eat Italian food," the system will use past data to suggest nearby Italian restaurants, automatically complete reservations for the restaurants chosen by the users, and arrange food delivery if necessary.

[0343] An example of a prompt for a generative AI model is: "Assume the user is planning dinner and recommend the best restaurant based on their past preferences. Generate suggestions that match the appropriate chat content."

[0344] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0345] Step 1:

[0346] The terminal receives communication data from the user and sends it to the server. The input is natural language conversation data from the user, and the output is the data format in which this is sent to the server. The terminal captures the user's chat content in real time and sends this data to the server via a communication protocol.

[0347] Step 2:

[0348] The server analyzes the received interaction data using a generative AI model. The input is natural language conversation data sent from the terminal, and the output is a dataset reflecting each user's interests and emotions. The server utilizes Google Cloud's natural language processing API and OpenAI's GPT model to analyze the tone and keywords of the conversation to infer the user's preferences and emotional state. This analysis process identifies the type of meal and atmosphere the user desires.

[0349] Step 3:

[0350] The server generates activity suggestions based on the analysis results and sends them to the terminal. The input is a dataset reflecting the user's preferences and emotions, and the output is a list of specific activity suggestions provided to each user. The server refers to a pre-linked food and beverage information database to search for and select restaurants and menus suitable for the suggestions. The generated suggestions are presented to the user in an appealing way using prompt text.

[0351] Step 4:

[0352] The terminal collects user selections and responses and feeds them back to the server. Inputs are the user's indication of interest in suggestions or specific selection actions, and outputs are data packets that are sent to the server. The terminal monitors the user's button presses and text inputs and sends this behavioral data to the server for further processing.

[0353] Step 5:

[0354] The server determines the optimal activity based on user feedback and automatically handles booking and payment. Inputs are the user's selected activity suggestions and feedback information, while outputs are confirmed booking information and payment completion confirmation data. The server sends booking requests to selected restaurants and services and processes payments through online transaction services. As a result, users can receive services smoothly.

[0355] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0356] This invention effectively supports the planning of group activities for busy and indecisive users through a system that combines an emotion engine. The system aims to recognize users' interests and emotions through their conversational data and generate activity suggestions based on that information.

[0357] In this system, users first join a group chat using their device. All conversations that take place there are sent to the server in real time via the device. The server applies a sentiment engine in addition to a generative AI module to analyze the received conversation data. This sentiment engine operates as part of natural language processing technology and detects emotions from the user's writing.

[0358] The server uses both emotional information recognized by the emotion engine and interest information based on keywords to create activity suggestions tailored to the user. These suggestions reflect the user's current mood and interests, and the system generates multiple suggestions. The server then sends the generated activity suggestions to the terminal for the user to see.

[0359] Users review the suggestions displayed on their devices and provide feedback on each suggestion. By providing feedback through specific selections such as "Interested" or "Pass," users can reflect their own preferences. Once the feedback is returned to the server, the server aggregates it and selects the most suitable activity.

[0360] Once an activity is decided, the server works in conjunction with the partnered reservation system to take the necessary actions. All the information is then displayed on the device, allowing users to complete reservations and payments within the app, resulting in a highly convenient design.

[0361] After the activity concludes, user feedback is collected again, and the server uses this data to update the generative model, thereby improving the accuracy of future suggestions.

[0362] For example, if a user says in an online chat, "I'm happy because the weather has been nice lately," the emotion engine recognizes this as the emotion of "happiness." Based on this emotion data and past history, the server can suggest activities such as "picnic" or "hiking," and, after receiving user feedback, plan the most suitable activity. In this way, the present invention makes it possible to make emotionally resonant suggestions to the user and facilitate consensus building.

[0363] The following describes the processing flow.

[0364] Step 1:

[0365] Users use their devices to start a group chat and discuss activities.

[0366] Step 2:

[0367] The device sends chat messages from the user to the server in real time. The conversation data is aggregated on the server in text format.

[0368] Step 3:

[0369] The server analyzes the received conversation data using a generating AI model. This analysis incorporates an emotion engine that identifies interests based on keywords in the message and simultaneously analyzes the context to recognize emotions.

[0370] Step 4:

[0371] The server generates activity suggestions that match the user's interests and emotions based on the analysis results. This process takes into account factors such as weather information and geographical conditions, resulting in the creation of multiple suggestions.

[0372] Step 5:

[0373] The server sends the generated activity suggestions to the user's terminal. The suggestions are displayed in a list format, making it easy for the user to select from them.

[0374] Step 6:

[0375] The user reviews the suggestions displayed on their device, selects feedback for each from options such as "Interested" or "Pass," and sends this feedback to the server via their device.

[0376] Step 7:

[0377] The server aggregates the feedback submitted by each user. Based on the aggregated results, it determines the activity that received the most support across the entire user group.

[0378] Step 8:

[0379] The server manages the necessary booking process for the selected activity. It integrates with partnered external systems to retrieve booking information and completes the booking on the system.

[0380] Step 9:

[0381] The device displays a reservation confirmation notification to the user and opens the payment screen if necessary. The user can make payments securely within the app.

[0382] Step 10:

[0383] Users input feedback into their devices after completing an activity, and this data is sent to a server to evaluate the suggestions and experiences provided.

[0384] Step 11:

[0385] The server updates its emotion engine and generative model based on the feedback it receives, improving the accuracy of future suggestions. This feedback is stored as training data for the system.

[0386] (Example 2)

[0387] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0388] The goal is to alleviate the difficulties busy and indecisive users face in planning and implementing group activities, and to enable them to receive optimal activity suggestions that take into account their individual feelings and interests. Furthermore, the aim is to provide a system that streamlines the process from activity selection to booking and payment, facilitating smooth consensus building among users.

[0389] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0390] In this invention, the server includes means for receiving conversation generation data from multiple communication devices and analyzing the generated data to identify each user's intentions and emotions; means for generating multiple activity options that correspond to each user's intentions and psychological state based on the analysis results; and means for transmitting the generated activity options to each communication device and collecting opinions on the activity suggestions. This enables users to quickly and efficiently decide on the optimal activity that aligns with their emotions and interests, and also allows for smooth booking and payment for the decided activity.

[0391] A "communication device" is a terminal used by users to participate in group chats and send and receive data.

[0392] "Conversation generation data" refers to text information transmitted by users through communication devices, and serves as the basis for activity proposals.

[0393] A "user" is a person who receives plans and proposals for group activities, and is also a person who uses the system.

[0394] "Intentions" refer to a user's interests and desires, and represent information that expresses the user's preferences and expectations when choosing activities.

[0395] "Emotions" refer to the user's psychological state and are factors considered when the system suggests activities.

[0396] "Analysis results" refer to information obtained after analyzing conversation generation data, and serve as the basis for activity proposals.

[0397] "Activity options" are action plans proposed to users, reflecting their interests and emotions.

[0398] "Opinions" refer to feedback that users provide regarding activity options, and include specific responses such as "interested" or "pass."

[0399] This invention relates to a system for effectively supporting the planning of group activities involving multiple users. In this system, a server, a terminal, and users work together to generate activity suggestions based on the users' interests and emotions, and determine the optimal activity through feedback.

[0400] First, users join a group chat using a communication device (a terminal). The terminal collects conversation data generated between users and sends it to the server in real time. The conversation data is encrypted, ensuring security and privacy.

[0401] The server receives this conversation generation data and analyzes it using automated generative AI technology. This analysis utilizes natural language processing techniques and an emotion engine for emotion recognition to extract each user's intentions and emotions. Based on the analysis results, the server inputs prompt sentences into the generative AI model and generates activity options. For example, a prompt sentence such as, "The user's current mood is 'happy' and they have shown interest in outdoor activities in the past, so please suggest 'picnic' or 'hiking'," can be used.

[0402] The generated activity options are sent back from the server to the terminal and presented to the user. The user provides specific feedback on the presented suggestions. Feedback such as "Interested" or "Pass" is collected and aggregated again on the server. This feedback data is used to further train the generating AI model based on the user's preferences.

[0403] Once the optimal activity is determined, the server works in conjunction with a partnered reservation management system to handle the necessary reservation and payment procedures. This allows users to complete reservations and payments entirely on their devices. Furthermore, user feedback after the activity is recorded as data to improve future activity proposals.

[0404] In this way, the system aims to be attentive to the user's emotions and interests, and to facilitate consensus building.

[0405] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0406] Step 1:

[0407] A user joins a group chat using their device. The device receives text input from the user in real time and converts the input conversational data into a signal for communication. This signal is sent to the server. The input is text data in natural language and is output to the server as a stream.

[0408] Step 2:

[0409] The server analyzes the conversation-generated data it receives. Natural language processing techniques are used to transform text data into data for syntactic analysis and sentiment recognition. The input is text data sent from the terminal, and the output is analyzed intention and sentiment data. Specifically, the server analyzes word frequencies and extracts keywords.

[0410] Step 3:

[0411] The server generates activity options using generative AI technology. Based on the analysis results, it creates a prompt and inputs it into the generative AI model to generate new activity suggestions. The input consists of analyzed intention and emotion data and a prompt, while the output is multiple activity options. Specifically, the server generates the prompt "Suggest an activity appropriate to the user's current emotion."

[0412] Step 4:

[0413] The server sends the generated activity options to the terminal. Here, the options are visually organized for easier presentation to the user. The input is the generated activity options, and the output is suggested data formatted for display. Specifically, the server converts the data to HTML format or an application-specific format.

[0414] Step 5:

[0415] The user reviews the suggestions via their device and provides feedback. This feedback includes selections such as "Interested" or "Pass." The input is the displayed activity options, and the output is the user's feedback data. The device captures this feedback and sends it back to the server.

[0416] Step 6:

[0417] The server aggregates user feedback and determines the optimal activity. It analyzes the feedback data and selects the most suitable activity, taking into account the opinions of numerous users. The input is the feedback data, and the output is the determined activity plan. Specifically, the server evaluates the feedback using statistical methods.

[0418] Step 7:

[0419] The server integrates with a partnered reservation management system to handle the necessary reservation and payment procedures. Communication with an external system via API takes place here. The input is the decided activity plan, and the output is reservation confirmation information. The server sends this information to the terminal.

[0420] Step 8:

[0421] The user completes the reservation and payment on the terminal. The terminal displays reservation confirmation information and provides a screen for the payment process. The input is reservation confirmation information, and the output is completed transaction information. Specifically, the user enters payment information, and the terminal processes it securely.

[0422] Step 9:

[0423] After the activity ends, the server obtains feedback from the user again. This feedback is used to improve the activity options for future activities. The input is the user's opinion after the activity ends, and the output is update data for the generative model. The server uses this data to train the generative AI model.

[0424] (Application Example 2)

[0425] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0426] In modern families, enriching family time requires planning appropriate events and recreational activities that align with the interests and feelings of each family member. However, since each member has different interests and feelings, selecting the optimal activity is difficult, and decision-making within the family often does not proceed smoothly. Furthermore, efficiently executing these plans and consistently managing related reservations and payments is also a challenge.

[0427] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0428] This invention includes a server that receives conversational information acquired from multiple terminal devices, analyzes the conversational information to recognize the interests and emotions of each user, generates multiple task suggestions tailored to the interests and emotions of each user based on the analysis results, and presents the task suggestions to household appliances in real time to recommend household events and recreational activities. This makes it possible to efficiently plan and implement optimal events and recreational activities while being attentive to the emotions and interests of each member of the household.

[0429] A "terminal device" is a device used by a user to input and receive conversational information.

[0430] "Conversational information" refers to data that includes the content of text and audio exchanges between users, as well as their emotional nuances.

[0431] "Emotion" is an element that indicates the user's psychological state, and it is information extracted from conversational data.

[0432] "Interests" refers to information about topics and activities that a user is particularly interested in.

[0433] "Household machinery and equipment" refers to machines used within the home that have the function of presenting proposed tasks or activities.

[0434] "Proposal suggestions" are recommended plans for events and activities generated based on the user's interests and emotions.

[0435] "Reservation" refers to the procedure of securing the date, time, and location for a decided activity in advance.

[0436] "Payment" refers to the process of completing the necessary financial transaction for a decided activity.

[0437] The system for realizing this invention consists of a home appliance, an internet-connected terminal device, and a server. The server analyzes conversational information received from the terminal device used by the user. The analysis uses a software module that combines natural language processing technology and an emotion recognition engine. Specifically, the server uses "Python" and "TensorFlow" to process conversational data, converts the audio data into text using the "Google Cloud Speech-to-Text API," and analyzes the emotions from that text using the emotion engine.

[0438] The server uses a generative AI model to create task suggestions based on analyzed interest and sentiment information. This generative AI model utilizes technologies such as "OpenAI GPT" to construct optimal event and activity suggestions for the user. The generated suggestions are displayed in real time on home electronic devices, which the user can view and evaluate.

[0439] When a user provides feedback on suggestions displayed on a home appliance, that feedback is sent to a server via the device. The server aggregates this feedback and can use it to plan future activities in order to improve the accuracy of the suggestions. As this process is repeated, events and activities within the home evolve to become more responsive to the user's emotions and interests.

[0440] For example, if a family says "I'd like to go to a pop concert" on a holiday morning, the device transmits this conversation to the server. The server analyzes the conversation to identify the "interest in pop music" and suggests nearby concerts or live streaming options. If the user selects "go to a concert," the server can automatically book and pay for tickets. An example of a prompt to the generative AI model would be, "Please suggest weekend activities for a family who likes pop concerts."

[0441] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0442] Step 1:

[0443] The device records the user's conversation and sends it to the server as audio data. The input is audio data, and the output is an audio file uploaded to the server. The device uses a microphone to capture audio, compresses the data, and efficiently sends it to the server.

[0444] Step 2:

[0445] The server converts the received audio data into text data using the Google Cloud Speech-to-Text API. The input is an audio file, and the output is human-readable text data. The converted text is then used for subsequent sentiment analysis.

[0446] Step 3:

[0447] The server analyzes the converted text data using an emotion engine to identify the user's emotions. The input is text data, and the output is data with emotion labels. The emotion engine uses natural language processing algorithms to determine emotions such as joy, anger, sadness, and happiness.

[0448] Step 4:

[0449] The server generates task suggestions using a generative AI model based on sentiment and text content. The input is sentiment labels and parsed text, and the output is a list of suggested events and activities. The generative AI model uses technologies such as "OpenAI GPT" to create appropriate suggestions.

[0450] Step 5:

[0451] The server transmits and displays the generated task proposals to the home appliance in real time. The input is a list of proposals, and the output is the content of the proposals that the user can view. The user reviews and selects a proposal from the appliance's display screen.

[0452] Step 6:

[0453] The user provides feedback on the suggestions through the interface of the home appliance. The input is the user's selected option, and the output is the feedback data sent back to the server. The selection process can be performed via touch screen or voice recognition.

[0454] Step 7:

[0455] The server stores the collected feedback in a database and analyzes it for the purpose of improving future suggestions. The input is user feedback data, and the output is analyzed information to improve the accuracy of the suggestions. The feedback data is used to adjust the suggestion generation algorithm.

[0456] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0457] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0458] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0459] [Third Embodiment]

[0460] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0461] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0462] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0463] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0464] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0465] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0466] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0467] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0468] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0469] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0470] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0471] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0472] This invention is a system designed to enable busy and indecisive working adults to smoothly manage their schedules and plan collaborative activities. The program processing of this system is described below.

[0473] In this system, users participate in group chats using their own devices and discuss activity plans. All messages sent during the chat are sent from the device to the server and processed in real time.

[0474] The server analyzes the received conversation data using a generating AI model to identify each user's interests and emotions. This analysis is performed using natural language processing technology, generating appropriate suggestions based on the user's past activity history and the current conversation flow.

[0475] Once the analysis is complete, the server generates multiple suggestions based on the user's preferences and sends them to each user's device. At this stage, users can provide feedback on the suggestions they receive. Specifically, they can provide feedback by pressing the "Interested" button for activities they are interested in and the "Pass" button for those they are not interested in.

[0476] Based on the aggregated feedback, the server determines the most suitable activity suggestions among the users. Once the selected activity is confirmed, the server automatically coordinates with the partnered booking system to make the necessary reservations. The server also manages payments as needed, allowing users to complete all payments within the system.

[0477] After the activity ends, users input their impressions and evaluations via their devices and send them to the server. The submitted feedback is used as data to improve the accuracy of future suggestions and is utilized to update the generative AI model.

[0478] As a concrete example, on a weekend, users discuss plans to try a new restaurant via chat. The system suggests the best restaurant based on their preferences and past data, completes the reservation based on the activities chosen by the users, and ensures a competitive experience. In this way, the present invention enables smooth and efficient group planning, reduces the workload on users, and increases their satisfaction.

[0479] The following describes the processing flow.

[0480] Step 1:

[0481] A user accesses a group chat using their device and starts a conversation about the activity. The device captures incoming messages and prepares them to be sent to the server in real time.

[0482] Step 2:

[0483] The device sends chat messages entered by the user to the server. In doing so, the entire conversation is sent as data to preserve the context of the conversation.

[0484] Step 3:

[0485] The server applies a generative AI model to analyze the received message data. Specifically, it uses natural language processing techniques to extract keywords and determine the user's interests and current emotions.

[0486] Step 4:

[0487] Based on the analysis results, the server generates multiple activity suggestions tailored to the user's interests and emotions. In doing so, it also references each user's historical data and trend information to select the most suitable activity.

[0488] Step 5:

[0489] The server sends the generated activity proposals to the user's terminal. The sent proposals are displayed in a visually easy-to-understand format.

[0490] Step 6:

[0491] The user reviews the suggestions on their device, selects either "Interested" or "Pass" feedback for each suggestion, and sends this feedback to the server via their device.

[0492] Step 7:

[0493] The server aggregates the feedback submitted by each user. From the aggregated results, it selects the most supported proposal and finalizes the action to be taken.

[0494] Step 8:

[0495] The server coordinates with a partner reservation system for confirmed activities and makes the necessary reservations. At this time, it sends reservation confirmation information to the user's terminal to notify them.

[0496] Step 9:

[0497] The device displays a payment screen along with a reservation confirmation. The user completes the payment process within the app, thus finalizing the reservation.

[0498] Step 10:

[0499] After completing an activity, users enter their impressions and evaluations into their device and send that feedback to the server.

[0500] Step 11:

[0501] The server updates the generated AI model based on the feedback data, using it to improve the accuracy of future activity suggestions.

[0502] (Example 1)

[0503] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0504] In modern society, it is difficult for busy professionals to efficiently and smoothly coordinate schedules and plan collaborative activities. Selecting the most suitable activity for the group, taking into account individual interests and feelings, and handling reservations and payments collectively is time-consuming and stressful. Furthermore, effectively improving future proposals based on past experiences is also a challenge.

[0505] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0506] In this invention, the server includes means for receiving communication data from multiple information terminals and analyzing the communication data to identify the interests and emotions of each user; means for generating multiple activity suggestions tailored to the interests and emotions of each user based on the analysis results; and means for transmitting the generated activity suggestions to each information terminal and receiving responses to the activity suggestions. This makes it possible for even busy working adults to efficiently carry out everything from planning group activities to making reservations and payments in a consistent manner.

[0507] An "information terminal" is an electronic device used by users to send and receive communication data.

[0508] "Communication data" is a general term for messages and information transmitted from an information terminal.

[0509] "Analysis" is the process of identifying users' interests and emotions based on communication data.

[0510] "Interests and feelings" refers to the user's current interests and psychological state.

[0511] An "activity proposal" is a set of suggestions generated by the server based on the user's interests and feelings.

[0512] "Response" refers to feedback that includes opinions and choices expressed by users regarding the proposed activity.

[0513] An "optimal activity" is an activity that is adjusted to maximize the shared benefits for multiple users.

[0514] "Reservation" refers to the act of securing seats or locations necessary for an optimal activity in advance.

[0515] "Payment processing" refers to the process of managing a series of procedures related to the settlement of fees.

[0516] A "generative model" is an artificial intelligence algorithm used to create action plans based on data analysis.

[0517] This system is a digital platform for users to efficiently plan and manage group activities. Specifically, it consists of the following elements:

[0518] Users access the system using their own information devices (e.g., smartphones or personal computers). These devices send messages to the server via electronic communication. Conversations in group chats in which users participate serve as the starting point for this system.

[0519] The server collects communication data received from multiple information terminals and performs data analysis using a generative AI model. Natural language processing technology is used in the analysis to identify each user's interests and emotions. This process includes a database of past activity history and real-time conversation analysis.

[0520] Based on the analysis results, the server generates activity suggestions tailored to each user's preferences. These activity suggestions are sent to each user's information terminal and presented in a specific format. Users can provide feedback on the activity suggestions using the terminal's interface. They can press the "Interested" button for activities they are interested in and the "Pass" button for those they are not.

[0521] Based on the feedback gathered by the server, the optimal activity is determined. At this stage, the server integrates with the relevant reservation system and automatically reserves the necessary facilities and services for the activity. Payment processing is also managed centrally by the server, allowing users to easily complete payments on their devices. In this way, the server reduces the burden on users and provides consistent support from planning to execution.

[0522] After completing an activity, users submit feedback via their devices. This information is collected on a server and used to generate more accurate activity plans in the future. The system continuously improves through ongoing updates to the generating AI model.

[0523] A concrete example is a group of users who want to try a new restaurant on the weekend. The users express their preferences through chat, and the system suggests restaurants based on their past usage data. Reservations and payments are processed efficiently, allowing users to proceed with their plans with peace of mind.

[0524] Example of a prompt:

[0525] "We'd like to try a new restaurant this weekend. Please suggest the best restaurant based on our group's past history and current conversation."

[0526] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0527] Step 1:

[0528] Users join a group chat from their personal devices and enter messages about their planned activities. These messages become input data, which the device formats as text data and sends to the server via the internet. The device encrypts the data during transmission to ensure the security of the communication.

[0529] Step 2:

[0530] The server acquires text data received from information terminals and performs data analysis using a generative AI model. The input text data is analyzed using natural language processing technology to identify each user's interests and emotions. The specific interest and emotion data obtained as a result of the analysis is stored in a database as basic data necessary for generating activity plans.

[0531] Step 3:

[0532] The server generates multiple activity suggestions based on identified interest and sentiment data. This process involves matching past activity history with the user's current browsing data to provide personalized suggestions for each user. The generated activity suggestions are output and sent to each information terminal as a formatted list. The server ensures that the activity suggestions are delivered in a clear and easy-to-understand format.

[0533] Step 4:

[0534] Users send feedback on received activity proposals via their device. Users respond by pressing the "Interested" button for activity proposals they are interested in and the "Pass" button for those they are not interested in. The device collects this feedback data from users and sends it back to the server.

[0535] Step 5:

[0536] Based on the aggregated feedback data, the server begins the process of selecting the optimal activity for the entire group. This optimization process uses an algorithm to select the option that will maximize satisfaction across all users. Once the selected optimal activity is finalized, reservation information corresponding to it is output.

[0537] Step 6:

[0538] Based on the confirmed activity reservation information, the server coordinates with the partner reservation system to make the necessary reservations. It also processes payments through the payment system, completing all the necessary procedures for the activity. This ensures that users can enjoy the activity smoothly without any hassle.

[0539] Step 7:

[0540] After an activity is completed, users input their impressions and evaluations of the activity through their device. This feedback is again aggregated as data and sent to the server. The server incorporates the post-activity feedback data as learning data to improve future activity plans. Through this process, the overall accuracy of the system improves, enabling more satisfying suggestions.

[0541] (Application Example 1)

[0542] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0543] Busy and indecisive working adults face the challenge of efficiently and smoothly planning their meals. Furthermore, manually selecting appropriate restaurants that reflect individual preferences and arranging delivery is time-consuming and labor-intensive, thus creating a need for automation.

[0544] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0545] In this invention, the server includes means for receiving communication data from multiple information devices and analyzing that communication data to identify each individual's interests and emotions; means for generating multiple activity suggestions tailored to each individual's interests and emotions based on the analysis results; and means for analyzing information related to meal planning and automating the selection of the optimal restaurant or delivery procedures. This enables the suggestion of optimal activities based on individual preferences and the automatic execution of reservations and transactions.

[0546] "Information device" is a general term for devices that receive, transmit, and process data by electronic means.

[0547] "Communication data" refers to data related to people's communication exchanged between various information devices.

[0548] "Interest" refers to the things or activities that an individual is particularly interested in.

[0549] "Emotion" refers to the feelings or mental responses an individual experiences in response to a particular situation or piece of information.

[0550] An "activity suggestion" is a selection of activities that are recommended for participation, taking into account an individual's interests and feelings.

[0551] "Response" refers to information that indicates an individual's evaluation or preference for a proposed activity.

[0552] "Meal planning" refers to the planning and arrangements for meals.

[0553] "Restaurant selection" refers to the act of choosing the most suitable restaurant from a list of suggested restaurants based on specific criteria.

[0554] "Delivery procedures" refer to all operations and management steps involved in delivering selected food and beverages to a designated location.

[0555] "Automation" means performing tasks that would normally be done manually under the control of a computer.

[0556] This invention is a system designed to help busy and indecisive working adults efficiently plan their meals. The specific form of this system is described below.

[0557] This system consists of a server and user information devices (terminals). Users participate in group chats using their terminals to discuss meal plans. The terminals send chat data to the server, which analyzes this data using a generation AI model. The analysis utilizes Google Cloud's natural language processing API and OpenAI's GPT model. This identifies each user's interests and emotions, and based on this, meal activity suggestions are generated.

[0558] The server suggests the most suitable restaurants and menus based on the analysis results. It references external restaurant information databases (e.g., online restaurant APIs) to present options that match the user's preferences. Upon receiving user feedback, it automatically handles reservations and delivery procedures. Payments are securely processed using online transaction services such as Stripe.

[0559] For example, if a group of users are discussing in a chat that they "want to eat Italian food," the system will use past data to suggest nearby Italian restaurants, automatically complete reservations for the restaurants chosen by the users, and arrange food delivery if necessary.

[0560] An example of a prompt for a generative AI model is: "Assume the user is planning dinner and recommend the best restaurant based on their past preferences. Generate suggestions that match the appropriate chat content."

[0561] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0562] Step 1:

[0563] The terminal receives communication data from the user and sends it to the server. The input is natural language conversation data from the user, and the output is the data format in which this is sent to the server. The terminal captures the user's chat content in real time and sends this data to the server via a communication protocol.

[0564] Step 2:

[0565] The server analyzes the received interaction data using a generative AI model. The input is natural language conversation data sent from the terminal, and the output is a dataset reflecting each user's interests and emotions. The server utilizes Google Cloud's natural language processing API and OpenAI's GPT model to analyze the tone and keywords of the conversation to infer the user's preferences and emotional state. This analysis process identifies the type of meal and atmosphere the user desires.

[0566] Step 3:

[0567] The server generates activity suggestions based on the analysis results and sends them to the terminal. The input is a dataset reflecting the user's preferences and emotions, and the output is a list of specific activity suggestions provided to each user. The server refers to a pre-linked food and beverage information database to search for and select restaurants and menus suitable for the suggestions. The generated suggestions are presented to the user in an appealing way using prompt text.

[0568] Step 4:

[0569] The terminal collects user selections and responses and feeds them back to the server. Inputs are the user's indication of interest in suggestions or specific selection actions, and outputs are data packets that are sent to the server. The terminal monitors the user's button presses and text inputs and sends this behavioral data to the server for further processing.

[0570] Step 5:

[0571] The server determines the optimal activity based on user feedback and automatically handles booking and payment. Inputs are the user's selected activity suggestions and feedback information, while outputs are confirmed booking information and payment completion confirmation data. The server sends booking requests to selected restaurants and services and processes payments through online transaction services. As a result, users can receive services smoothly.

[0572] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0573] This invention effectively supports the planning of group activities for busy and indecisive users through a system that combines an emotion engine. The system aims to recognize users' interests and emotions through their conversational data and generate activity suggestions based on that information.

[0574] In this system, users first join a group chat using their device. All conversations that take place there are sent to the server in real time via the device. The server applies a sentiment engine in addition to a generative AI module to analyze the received conversation data. This sentiment engine operates as part of natural language processing technology and detects emotions from the user's writing.

[0575] The server uses both emotional information recognized by the emotion engine and interest information based on keywords to create activity suggestions tailored to the user. These suggestions reflect the user's current mood and interests, and the system generates multiple suggestions. The server then sends the generated activity suggestions to the terminal for the user to see.

[0576] Users review the suggestions displayed on their devices and provide feedback on each suggestion. By providing feedback through specific selections such as "Interested" or "Pass," users can reflect their own preferences. Once the feedback is returned to the server, the server aggregates it and selects the most suitable activity.

[0577] Once an activity is decided, the server works in conjunction with the partnered reservation system to take the necessary actions. All the information is then displayed on the device, allowing users to complete reservations and payments within the app, resulting in a highly convenient design.

[0578] After the activity concludes, user feedback is collected again, and the server uses this data to update the generative model, thereby improving the accuracy of future suggestions.

[0579] For example, if a user says in an online chat, "I'm happy because the weather has been nice lately," the emotion engine recognizes this as the emotion of "happiness." Based on this emotion data and past history, the server can suggest activities such as "picnic" or "hiking," and, after receiving user feedback, plan the most suitable activity. In this way, the present invention makes it possible to make emotionally resonant suggestions to the user and facilitate consensus building.

[0580] The following describes the processing flow.

[0581] Step 1:

[0582] Users use their devices to start a group chat and discuss activities.

[0583] Step 2:

[0584] The device sends chat messages from the user to the server in real time. The conversation data is aggregated on the server in text format.

[0585] Step 3:

[0586] The server analyzes the received conversation data using a generating AI model. This analysis incorporates an emotion engine that identifies interests based on keywords in the message and simultaneously analyzes the context to recognize emotions.

[0587] Step 4:

[0588] The server generates activity suggestions that match the user's interests and emotions based on the analysis results. This process takes into account factors such as weather information and geographical conditions, resulting in the creation of multiple suggestions.

[0589] Step 5:

[0590] The server sends the generated activity suggestions to the user's terminal. The suggestions are displayed in a list format, making it easy for the user to select from them.

[0591] Step 6:

[0592] The user reviews the suggestions displayed on their device, selects feedback for each from options such as "Interested" or "Pass," and sends this feedback to the server via their device.

[0593] Step 7:

[0594] The server aggregates the feedback submitted by each user. Based on the aggregated results, it determines the activity that received the most support across the entire user group.

[0595] Step 8:

[0596] The server manages the necessary booking process for the selected activity. It integrates with partnered external systems to retrieve booking information and completes the booking on the system.

[0597] Step 9:

[0598] The device displays a reservation confirmation notification to the user and opens the payment screen if necessary. The user can make payments securely within the app.

[0599] Step 10:

[0600] Users input feedback into their devices after completing an activity, and this data is sent to a server to evaluate the suggestions and experiences provided.

[0601] Step 11:

[0602] The server updates its emotion engine and generative model based on the feedback it receives, improving the accuracy of future suggestions. This feedback is stored as training data for the system.

[0603] (Example 2)

[0604] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0605] The goal is to alleviate the difficulties busy and indecisive users face in planning and implementing group activities, and to enable them to receive optimal activity suggestions that take into account their individual feelings and interests. Furthermore, the aim is to provide a system that streamlines the process from activity selection to booking and payment, facilitating smooth consensus building among users.

[0606] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0607] In this invention, the server includes means for receiving conversation generation data from multiple communication devices and analyzing the generated data to identify each user's intentions and emotions; means for generating multiple activity options that correspond to each user's intentions and psychological state based on the analysis results; and means for transmitting the generated activity options to each communication device and collecting opinions on the activity suggestions. This enables users to quickly and efficiently decide on the optimal activity that aligns with their emotions and interests, and also allows for smooth booking and payment for the decided activity.

[0608] A "communication device" is a terminal used by users to participate in group chats and send and receive data.

[0609] "Conversation generation data" refers to text information transmitted by users through communication devices, and serves as the basis for activity proposals.

[0610] A "user" is a person who receives plans and proposals for group activities, and is also a person who uses the system.

[0611] "Intentions" refer to a user's interests and desires, and represent information that expresses the user's preferences and expectations when choosing activities.

[0612] "Emotions" refer to the user's psychological state and are factors considered when the system suggests activities.

[0613] "Analysis results" refer to information obtained after analyzing conversation generation data, and serve as the basis for activity proposals.

[0614] "Activity options" are action plans proposed to users, reflecting their interests and emotions.

[0615] "Opinions" refer to feedback that users provide regarding activity options, and include specific responses such as "interested" or "pass."

[0616] This invention relates to a system for effectively supporting the planning of group activities involving multiple users. In this system, a server, a terminal, and users work together to generate activity suggestions based on the users' interests and emotions, and determine the optimal activity through feedback.

[0617] First, users join a group chat using a communication device (a terminal). The terminal collects conversation data generated between users and sends it to the server in real time. The conversation data is encrypted, ensuring security and privacy.

[0618] The server receives this conversation generation data and analyzes it using automated generative AI technology. This analysis utilizes natural language processing techniques and an emotion engine for emotion recognition to extract each user's intentions and emotions. Based on the analysis results, the server inputs prompt sentences into the generative AI model and generates activity options. For example, a prompt sentence such as, "The user's current mood is 'happy' and they have shown interest in outdoor activities in the past, so please suggest 'picnic' or 'hiking'," can be used.

[0619] The generated activity options are sent back from the server to the terminal and presented to the user. The user provides specific feedback on the presented suggestions. Feedback such as "Interested" or "Pass" is collected and aggregated again on the server. This feedback data is used to further train the generating AI model based on the user's preferences.

[0620] Once the optimal activity is determined, the server works in conjunction with a partnered reservation management system to handle the necessary reservation and payment procedures. This allows users to complete reservations and payments entirely on their devices. Furthermore, user feedback after the activity is recorded as data to improve future activity proposals.

[0621] In this way, the system aims to be attentive to the user's emotions and interests, and to facilitate consensus building.

[0622] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0623] Step 1:

[0624] A user joins a group chat using their device. The device receives text input from the user in real time and converts the input conversational data into a signal for communication. This signal is sent to the server. The input is text data in natural language and is output to the server as a stream.

[0625] Step 2:

[0626] The server analyzes the conversation-generated data it receives. Natural language processing techniques are used to transform text data into data for syntactic analysis and sentiment recognition. The input is text data sent from the terminal, and the output is analyzed intention and sentiment data. Specifically, the server analyzes word frequencies and extracts keywords.

[0627] Step 3:

[0628] The server generates activity options using generative AI technology. Based on the analysis results, it creates a prompt and inputs it into the generative AI model to generate new activity suggestions. The input consists of analyzed intention and emotion data and a prompt, while the output is multiple activity options. Specifically, the server generates the prompt "Suggest an activity appropriate to the user's current emotion."

[0629] Step 4:

[0630] The server sends the generated activity options to the terminal. Here, the options are visually organized for easier presentation to the user. The input is the generated activity options, and the output is suggested data formatted for display. Specifically, the server converts the data to HTML format or an application-specific format.

[0631] Step 5:

[0632] The user reviews the suggestions via their device and provides feedback. This feedback includes selections such as "Interested" or "Pass." The input is the displayed activity options, and the output is the user's feedback data. The device captures this feedback and sends it back to the server.

[0633] Step 6:

[0634] The server aggregates user feedback and determines the optimal activity. It analyzes the feedback data and selects the most suitable activity, taking into account the opinions of numerous users. The input is the feedback data, and the output is the determined activity plan. Specifically, the server evaluates the feedback using statistical methods.

[0635] Step 7:

[0636] The server integrates with a partnered reservation management system to handle the necessary reservation and payment procedures. Communication with an external system via API takes place here. The input is the decided activity plan, and the output is reservation confirmation information. The server sends this information to the terminal.

[0637] Step 8:

[0638] The user completes the reservation and payment on the terminal. The terminal displays reservation confirmation information and provides a screen for the payment process. The input is reservation confirmation information, and the output is completed transaction information. Specifically, the user enters payment information, and the terminal processes it securely.

[0639] Step 9:

[0640] After the activity ends, the server obtains feedback from the user again. This feedback is used to improve the activity options for future activities. The input is the user's opinion after the activity ends, and the output is update data for the generative model. The server uses this data to train the generative AI model.

[0641] (Application Example 2)

[0642] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0643] In modern families, enriching family time requires planning appropriate events and recreational activities that align with the interests and feelings of each family member. However, since each member has different interests and feelings, selecting the optimal activity is difficult, and decision-making within the family often does not proceed smoothly. Furthermore, efficiently executing these plans and consistently managing related reservations and payments is also a challenge.

[0644] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0645] This invention includes a server that receives conversational information acquired from multiple terminal devices, analyzes the conversational information to recognize the interests and emotions of each user, generates multiple task suggestions tailored to the interests and emotions of each user based on the analysis results, and presents the task suggestions to household appliances in real time to recommend household events and recreational activities. This makes it possible to efficiently plan and implement optimal events and recreational activities while being attentive to the emotions and interests of each member of the household.

[0646] A "terminal device" is a device used by a user to input and receive conversational information.

[0647] "Conversational information" refers to data that includes the content of text and audio exchanges between users, as well as their emotional nuances.

[0648] "Emotion" is an element that indicates the user's psychological state, and it is information extracted from conversational data.

[0649] "Interests" refers to information about topics and activities that a user is particularly interested in.

[0650] "Household machinery and equipment" refers to machines used within the home that have the function of presenting proposed tasks or activities.

[0651] "Proposal suggestions" are recommended plans for events and activities generated based on the user's interests and emotions.

[0652] "Reservation" refers to the procedure of securing the date, time, and location for a decided activity in advance.

[0653] "Payment" refers to the process of completing the necessary financial transaction for a decided activity.

[0654] The system for realizing this invention consists of a home appliance, an internet-connected terminal device, and a server. The server analyzes conversational information received from the terminal device used by the user. The analysis uses a software module that combines natural language processing technology and an emotion recognition engine. Specifically, the server uses "Python" and "TensorFlow" to process conversational data, converts the audio data into text using the "Google Cloud Speech-to-Text API," and analyzes the emotions from that text using the emotion engine.

[0655] The server uses a generative AI model to create task suggestions based on analyzed interest and sentiment information. This generative AI model utilizes technologies such as "OpenAI GPT" to construct optimal event and activity suggestions for the user. The generated suggestions are displayed in real time on home electronic devices, which the user can view and evaluate.

[0656] When a user provides feedback on suggestions displayed on a home appliance, that feedback is sent to a server via the device. The server aggregates this feedback and can use it to plan future activities in order to improve the accuracy of the suggestions. As this process is repeated, events and activities within the home evolve to become more responsive to the user's emotions and interests.

[0657] For example, if a family says "I'd like to go to a pop concert" on a holiday morning, the device transmits this conversation to the server. The server analyzes the conversation to identify the "interest in pop music" and suggests nearby concerts or live streaming options. If the user selects "go to a concert," the server can automatically book and pay for tickets. An example of a prompt to the generative AI model would be, "Please suggest weekend activities for a family who likes pop concerts."

[0658] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0659] Step 1:

[0660] The device records the user's conversation and sends it to the server as audio data. The input is audio data, and the output is an audio file uploaded to the server. The device uses a microphone to capture audio, compresses the data, and efficiently sends it to the server.

[0661] Step 2:

[0662] The server converts the received audio data into text data using the Google Cloud Speech-to-Text API. The input is an audio file, and the output is human-readable text data. The converted text is then used for subsequent sentiment analysis.

[0663] Step 3:

[0664] The server analyzes the converted text data using an emotion engine to identify the user's emotions. The input is text data, and the output is data with emotion labels. The emotion engine uses natural language processing algorithms to determine emotions such as joy, anger, sadness, and happiness.

[0665] Step 4:

[0666] The server generates task suggestions using a generative AI model based on sentiment and text content. The input is sentiment labels and parsed text, and the output is a list of suggested events and activities. The generative AI model uses technologies such as "OpenAI GPT" to create appropriate suggestions.

[0667] Step 5:

[0668] The server transmits and displays the generated task proposals to the home appliance in real time. The input is a list of proposals, and the output is the content of the proposals that the user can view. The user reviews and selects a proposal from the appliance's display screen.

[0669] Step 6:

[0670] The user provides feedback on the suggestions through the interface of the home appliance. The input is the user's selected option, and the output is the feedback data sent back to the server. The selection process can be performed via touch screen or voice recognition.

[0671] Step 7:

[0672] The server stores the collected feedback in a database and analyzes it for the purpose of improving future suggestions. The input is user feedback data, and the output is analyzed information to improve the accuracy of the suggestions. The feedback data is used to adjust the suggestion generation algorithm.

[0673] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0674] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0675] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0676] [Fourth Embodiment]

[0677] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0678] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0679] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0680] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0681] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0682] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0683] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0684] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0685] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0686] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0687] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0688] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0689] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0690] This invention is a system designed to enable busy and indecisive working adults to smoothly manage their schedules and plan collaborative activities. The program processing of this system is described below.

[0691] In this system, users participate in group chats using their own devices and discuss activity plans. All messages sent during the chat are sent from the device to the server and processed in real time.

[0692] The server analyzes the received conversation data using a generating AI model to identify each user's interests and emotions. This analysis is performed using natural language processing technology, generating appropriate suggestions based on the user's past activity history and the current conversation flow.

[0693] Once the analysis is complete, the server generates multiple suggestions based on the user's preferences and sends them to each user's device. At this stage, users can provide feedback on the suggestions they receive. Specifically, they can provide feedback by pressing the "Interested" button for activities they are interested in and the "Pass" button for those they are not interested in.

[0694] Based on the aggregated feedback, the server determines the most suitable activity suggestions among the users. Once the selected activity is confirmed, the server automatically coordinates with the partnered booking system to make the necessary reservations. The server also manages payments as needed, allowing users to complete all payments within the system.

[0695] After the activity ends, users input their impressions and evaluations via their devices and send them to the server. The submitted feedback is used as data to improve the accuracy of future suggestions and is utilized to update the generative AI model.

[0696] As a concrete example, on a weekend, users discuss plans to try a new restaurant via chat. The system suggests the best restaurant based on their preferences and past data, completes the reservation based on the activities chosen by the users, and ensures a competitive experience. In this way, the present invention enables smooth and efficient group planning, reduces the workload on users, and increases their satisfaction.

[0697] The following describes the processing flow.

[0698] Step 1:

[0699] A user accesses a group chat using their device and starts a conversation about the activity. The device captures incoming messages and prepares them to be sent to the server in real time.

[0700] Step 2:

[0701] The device sends chat messages entered by the user to the server. In doing so, the entire conversation is sent as data to preserve the context of the conversation.

[0702] Step 3:

[0703] The server applies a generative AI model to analyze the received message data. Specifically, it uses natural language processing techniques to extract keywords and determine the user's interests and current emotions.

[0704] Step 4:

[0705] Based on the analysis results, the server generates multiple activity suggestions tailored to the user's interests and emotions. In doing so, it also references each user's historical data and trend information to select the most suitable activity.

[0706] Step 5:

[0707] The server sends the generated activity proposals to the user's terminal. The sent proposals are displayed in a visually easy-to-understand format.

[0708] Step 6:

[0709] The user reviews the suggestions on their device, selects either "Interested" or "Pass" feedback for each suggestion, and sends this feedback to the server via their device.

[0710] Step 7:

[0711] The server aggregates the feedback submitted by each user. From the aggregated results, it selects the most supported proposal and finalizes the action to be taken.

[0712] Step 8:

[0713] The server coordinates with a partner reservation system for confirmed activities and makes the necessary reservations. At this time, it sends reservation confirmation information to the user's terminal to notify them.

[0714] Step 9:

[0715] The device displays a payment screen along with a reservation confirmation. The user completes the payment process within the app, thus finalizing the reservation.

[0716] Step 10:

[0717] After completing an activity, users enter their impressions and evaluations into their device and send that feedback to the server.

[0718] Step 11:

[0719] The server updates the generated AI model based on the feedback data, using it to improve the accuracy of future activity suggestions.

[0720] (Example 1)

[0721] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0722] In modern society, it is difficult for busy professionals to efficiently and smoothly coordinate schedules and plan collaborative activities. Selecting the most suitable activity for the group, taking into account individual interests and feelings, and handling reservations and payments collectively is time-consuming and stressful. Furthermore, effectively improving future proposals based on past experiences is also a challenge.

[0723] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0724] In this invention, the server includes means for receiving communication data from multiple information terminals and analyzing the communication data to identify the interests and emotions of each user; means for generating multiple activity suggestions tailored to the interests and emotions of each user based on the analysis results; and means for transmitting the generated activity suggestions to each information terminal and receiving responses to the activity suggestions. This makes it possible for even busy working adults to efficiently carry out everything from planning group activities to making reservations and payments in a consistent manner.

[0725] An "information terminal" is an electronic device used by users to send and receive communication data.

[0726] "Communication data" is a general term for messages and information transmitted from an information terminal.

[0727] "Analysis" is the process of identifying users' interests and emotions based on communication data.

[0728] "Interests and feelings" refers to the user's current interests and psychological state.

[0729] An "activity proposal" is a set of suggestions generated by the server based on the user's interests and feelings.

[0730] "Response" refers to feedback that includes opinions and choices expressed by users regarding the proposed activity.

[0731] An "optimal activity" is an activity that is adjusted to maximize the shared benefits for multiple users.

[0732] "Reservation" refers to the act of securing seats or locations necessary for an optimal activity in advance.

[0733] "Payment processing" refers to the process of managing a series of procedures related to the settlement of fees.

[0734] A "generative model" is an artificial intelligence algorithm used to create action plans based on data analysis.

[0735] This system is a digital platform for users to efficiently plan and manage group activities. Specifically, it consists of the following elements:

[0736] Users access the system using their own information devices (e.g., smartphones or personal computers). These devices send messages to the server via electronic communication. Conversations in group chats in which users participate serve as the starting point for this system.

[0737] The server collects communication data received from multiple information terminals and performs data analysis using a generative AI model. Natural language processing technology is used in the analysis to identify each user's interests and emotions. This process includes a database of past activity history and real-time conversation analysis.

[0738] Based on the analysis results, the server generates activity suggestions tailored to each user's preferences. These activity suggestions are sent to each user's information terminal and presented in a specific format. Users can provide feedback on the activity suggestions using the terminal's interface. They can press the "Interested" button for activities they are interested in and the "Pass" button for those they are not.

[0739] Based on the feedback gathered by the server, the optimal activity is determined. At this stage, the server integrates with the relevant reservation system and automatically reserves the necessary facilities and services for the activity. Payment processing is also managed centrally by the server, allowing users to easily complete payments on their devices. In this way, the server reduces the burden on users and provides consistent support from planning to execution.

[0740] After completing an activity, users submit feedback via their devices. This information is collected on a server and used to generate more accurate activity plans in the future. The system continuously improves through ongoing updates to the generating AI model.

[0741] A concrete example is a group of users who want to try a new restaurant on the weekend. The users express their preferences through chat, and the system suggests restaurants based on their past usage data. Reservations and payments are processed efficiently, allowing users to proceed with their plans with peace of mind.

[0742] Example of a prompt:

[0743] "We'd like to try a new restaurant this weekend. Please suggest the best restaurant based on our group's past history and current conversation."

[0744] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0745] Step 1:

[0746] Users join a group chat from their personal devices and enter messages about their planned activities. These messages become input data, which the device formats as text data and sends to the server via the internet. The device encrypts the data during transmission to ensure the security of the communication.

[0747] Step 2:

[0748] The server acquires text data received from information terminals and performs data analysis using a generative AI model. The input text data is analyzed using natural language processing technology to identify each user's interests and emotions. The specific interest and emotion data obtained as a result of the analysis is stored in a database as basic data necessary for generating activity plans.

[0749] Step 3:

[0750] The server generates multiple activity suggestions based on identified interest and sentiment data. This process involves matching past activity history with the user's current browsing data to provide personalized suggestions for each user. The generated activity suggestions are output and sent to each information terminal as a formatted list. The server ensures that the activity suggestions are delivered in a clear and easy-to-understand format.

[0751] Step 4:

[0752] Users send feedback on received activity proposals via their device. Users respond by pressing the "Interested" button for activity proposals they are interested in and the "Pass" button for those they are not interested in. The device collects this feedback data from users and sends it back to the server.

[0753] Step 5:

[0754] Based on the aggregated feedback data, the server begins the process of selecting the optimal activity for the entire group. This optimization process uses an algorithm to select the option that will maximize satisfaction across all users. Once the selected optimal activity is finalized, reservation information corresponding to it is output.

[0755] Step 6:

[0756] Based on the confirmed activity reservation information, the server coordinates with the partner reservation system to make the necessary reservations. It also processes payments through the payment system, completing all the necessary procedures for the activity. This ensures that users can enjoy the activity smoothly without any hassle.

[0757] Step 7:

[0758] After an activity is completed, users input their impressions and evaluations of the activity through their device. This feedback is again aggregated as data and sent to the server. The server incorporates the post-activity feedback data as learning data to improve future activity plans. Through this process, the overall accuracy of the system improves, enabling more satisfying suggestions.

[0759] (Application Example 1)

[0760] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0761] Busy and indecisive working adults face the challenge of efficiently and smoothly planning their meals. Furthermore, manually selecting appropriate restaurants that reflect individual preferences and arranging delivery is time-consuming and labor-intensive, thus creating a need for automation.

[0762] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0763] In this invention, the server includes means for receiving communication data from multiple information devices and analyzing that communication data to identify each individual's interests and emotions; means for generating multiple activity suggestions tailored to each individual's interests and emotions based on the analysis results; and means for analyzing information related to meal planning and automating the selection of the optimal restaurant or delivery procedures. This enables the suggestion of optimal activities based on individual preferences and the automatic execution of reservations and transactions.

[0764] "Information device" is a general term for devices that receive, transmit, and process data by electronic means.

[0765] "Communication data" refers to data related to people's communication exchanged between various information devices.

[0766] "Interest" refers to the things or activities that an individual is particularly interested in.

[0767] "Emotion" refers to the feelings or mental responses an individual experiences in response to a particular situation or piece of information.

[0768] An "activity suggestion" is a selection of activities that are recommended for participation, taking into account an individual's interests and feelings.

[0769] "Response" refers to information that indicates an individual's evaluation or preference for a proposed activity.

[0770] "Meal planning" refers to the planning and arrangements for meals.

[0771] "Restaurant selection" refers to the act of choosing the most suitable restaurant from a list of suggested restaurants based on specific criteria.

[0772] "Delivery procedures" refer to all operations and management steps involved in delivering selected food and beverages to a designated location.

[0773] "Automation" means performing tasks that would normally be done manually under the control of a computer.

[0774] This invention is a system designed to help busy and indecisive working adults efficiently plan their meals. The specific form of this system is described below.

[0775] This system consists of a server and user information devices (terminals). Users participate in group chats using their terminals to discuss meal plans. The terminals send chat data to the server, which analyzes this data using a generation AI model. The analysis utilizes Google Cloud's natural language processing API and OpenAI's GPT model. This identifies each user's interests and emotions, and based on this, meal activity suggestions are generated.

[0776] The server suggests the most suitable restaurants and menus based on the analysis results. It references external restaurant information databases (e.g., online restaurant APIs) to present options that match the user's preferences. Upon receiving user feedback, it automatically handles reservations and delivery procedures. Payments are securely processed using online transaction services such as Stripe.

[0777] For example, if a group of users are discussing in a chat that they "want to eat Italian food," the system will use past data to suggest nearby Italian restaurants, automatically complete reservations for the restaurants chosen by the users, and arrange food delivery if necessary.

[0778] An example of a prompt for a generative AI model is: "Assume the user is planning dinner and recommend the best restaurant based on their past preferences. Generate suggestions that match the appropriate chat content."

[0779] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0780] Step 1:

[0781] The terminal receives communication data from the user and sends it to the server. The input is natural language conversation data from the user, and the output is the data format in which this is sent to the server. The terminal captures the user's chat content in real time and sends this data to the server via a communication protocol.

[0782] Step 2:

[0783] The server analyzes the received interaction data using a generative AI model. The input is natural language conversation data sent from the terminal, and the output is a dataset reflecting each user's interests and emotions. The server utilizes Google Cloud's natural language processing API and OpenAI's GPT model to analyze the tone and keywords of the conversation to infer the user's preferences and emotional state. This analysis process identifies the type of meal and atmosphere the user desires.

[0784] Step 3:

[0785] The server generates activity suggestions based on the analysis results and sends them to the terminal. The input is a dataset reflecting the user's preferences and emotions, and the output is a list of specific activity suggestions provided to each user. The server refers to a pre-linked food and beverage information database to search for and select restaurants and menus suitable for the suggestions. The generated suggestions are presented to the user in an appealing way using prompt text.

[0786] Step 4:

[0787] The terminal collects user selections and responses and feeds them back to the server. Inputs are the user's indication of interest in suggestions or specific selection actions, and outputs are data packets that are sent to the server. The terminal monitors the user's button presses and text inputs and sends this behavioral data to the server for further processing.

[0788] Step 5:

[0789] The server determines the optimal activity based on user feedback and automatically handles booking and payment. Inputs are the user's selected activity suggestions and feedback information, while outputs are confirmed booking information and payment completion confirmation data. The server sends booking requests to selected restaurants and services and processes payments through online transaction services. As a result, users can receive services smoothly.

[0790] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0791] This invention effectively supports the planning of group activities for busy and indecisive users through a system that combines an emotion engine. The system aims to recognize users' interests and emotions through their conversational data and generate activity suggestions based on that information.

[0792] In this system, users first join a group chat using their device. All conversations that take place there are sent to the server in real time via the device. The server applies a sentiment engine in addition to a generative AI module to analyze the received conversation data. This sentiment engine operates as part of natural language processing technology and detects emotions from the user's writing.

[0793] The server uses both emotional information recognized by the emotion engine and interest information based on keywords to create activity suggestions tailored to the user. These suggestions reflect the user's current mood and interests, and the system generates multiple suggestions. The server then sends the generated activity suggestions to the terminal for the user to see.

[0794] Users review the suggestions displayed on their devices and provide feedback on each suggestion. By providing feedback through specific selections such as "Interested" or "Pass," users can reflect their own preferences. Once the feedback is returned to the server, the server aggregates it and selects the most suitable activity.

[0795] Once an activity is decided, the server works in conjunction with the partnered reservation system to take the necessary actions. All the information is then displayed on the device, allowing users to complete reservations and payments within the app, resulting in a highly convenient design.

[0796] After the activity concludes, user feedback is collected again, and the server uses this data to update the generative model, thereby improving the accuracy of future suggestions.

[0797] For example, if a user says in an online chat, "I'm happy because the weather has been nice lately," the emotion engine recognizes this as the emotion of "happiness." Based on this emotion data and past history, the server can suggest activities such as "picnic" or "hiking," and, after receiving user feedback, plan the most suitable activity. In this way, the present invention makes it possible to make emotionally resonant suggestions to the user and facilitate consensus building.

[0798] The following describes the processing flow.

[0799] Step 1:

[0800] Users use their devices to start a group chat and discuss activities.

[0801] Step 2:

[0802] The device sends chat messages from the user to the server in real time. The conversation data is aggregated on the server in text format.

[0803] Step 3:

[0804] The server analyzes the received conversation data using a generating AI model. This analysis incorporates an emotion engine that identifies interests based on keywords in the message and simultaneously analyzes the context to recognize emotions.

[0805] Step 4:

[0806] The server generates activity suggestions that match the user's interests and emotions based on the analysis results. This process takes into account factors such as weather information and geographical conditions, resulting in the creation of multiple suggestions.

[0807] Step 5:

[0808] The server sends the generated activity suggestions to the user's terminal. The suggestions are displayed in a list format, making it easy for the user to select from them.

[0809] Step 6:

[0810] The user reviews the suggestions displayed on their device, selects feedback for each from options such as "Interested" or "Pass," and sends this feedback to the server via their device.

[0811] Step 7:

[0812] The server aggregates the feedback submitted by each user. Based on the aggregated results, it determines the activity that received the most support across the entire user group.

[0813] Step 8:

[0814] The server manages the necessary booking process for the selected activity. It integrates with partnered external systems to retrieve booking information and completes the booking on the system.

[0815] Step 9:

[0816] The device displays a reservation confirmation notification to the user and opens the payment screen if necessary. The user can make payments securely within the app.

[0817] Step 10:

[0818] Users input feedback into their devices after completing an activity, and this data is sent to a server to evaluate the suggestions and experiences provided.

[0819] Step 11:

[0820] The server updates its emotion engine and generative model based on the feedback it receives, improving the accuracy of future suggestions. This feedback is stored as training data for the system.

[0821] (Example 2)

[0822] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0823] The goal is to alleviate the difficulties busy and indecisive users face in planning and implementing group activities, and to enable them to receive optimal activity suggestions that take into account their individual feelings and interests. Furthermore, the aim is to provide a system that streamlines the process from activity selection to booking and payment, facilitating smooth consensus building among users.

[0824] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0825] In this invention, the server includes means for receiving conversation generation data from multiple communication devices and analyzing the generated data to identify each user's intentions and emotions; means for generating multiple activity options that correspond to each user's intentions and psychological state based on the analysis results; and means for transmitting the generated activity options to each communication device and collecting opinions on the activity suggestions. This enables users to quickly and efficiently decide on the optimal activity that aligns with their emotions and interests, and also allows for smooth booking and payment for the decided activity.

[0826] A "communication device" is a terminal used by users to participate in group chats and send and receive data.

[0827] "Conversation generation data" refers to text information transmitted by users through communication devices, and serves as the basis for activity proposals.

[0828] A "user" is a person who receives plans and proposals for group activities, and is also a person who uses the system.

[0829] "Intentions" refer to a user's interests and desires, and represent information that expresses the user's preferences and expectations when choosing activities.

[0830] "Emotions" refer to the user's psychological state and are factors considered when the system suggests activities.

[0831] "Analysis results" refer to information obtained after analyzing conversation generation data, and serve as the basis for activity proposals.

[0832] "Activity options" are action plans proposed to users, reflecting their interests and emotions.

[0833] "Opinions" refer to feedback that users provide regarding activity options, and include specific responses such as "interested" or "pass."

[0834] This invention relates to a system for effectively supporting the planning of group activities involving multiple users. In this system, a server, a terminal, and users work together to generate activity suggestions based on the users' interests and emotions, and determine the optimal activity through feedback.

[0835] First, users join a group chat using a communication device (a terminal). The terminal collects conversation data generated between users and sends it to the server in real time. The conversation data is encrypted, ensuring security and privacy.

[0836] The server receives this conversation generation data and analyzes it using automated generative AI technology. This analysis utilizes natural language processing techniques and an emotion engine for emotion recognition to extract each user's intentions and emotions. Based on the analysis results, the server inputs prompt sentences into the generative AI model and generates activity options. For example, a prompt sentence such as, "The user's current mood is 'happy' and they have shown interest in outdoor activities in the past, so please suggest 'picnic' or 'hiking'," can be used.

[0837] The generated activity options are sent back from the server to the terminal and presented to the user. The user provides specific feedback on the presented suggestions. Feedback such as "Interested" or "Pass" is collected and aggregated again on the server. This feedback data is used to further train the generating AI model based on the user's preferences.

[0838] Once the optimal activity is determined, the server works in conjunction with a partnered reservation management system to handle the necessary reservation and payment procedures. This allows users to complete reservations and payments entirely on their devices. Furthermore, user feedback after the activity is recorded as data to improve future activity proposals.

[0839] In this way, the system aims to be attentive to the user's emotions and interests, and to facilitate consensus building.

[0840] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0841] Step 1:

[0842] A user joins a group chat using their device. The device receives text input from the user in real time and converts the input conversational data into a signal for communication. This signal is sent to the server. The input is text data in natural language and is output to the server as a stream.

[0843] Step 2:

[0844] The server analyzes the conversation-generated data it receives. Natural language processing techniques are used to transform text data into data for syntactic analysis and sentiment recognition. The input is text data sent from the terminal, and the output is analyzed intention and sentiment data. Specifically, the server analyzes word frequencies and extracts keywords.

[0845] Step 3:

[0846] The server generates activity options using generative AI technology. Based on the analysis results, it creates a prompt and inputs it into the generative AI model to generate new activity suggestions. The input consists of analyzed intention and emotion data and a prompt, while the output is multiple activity options. Specifically, the server generates the prompt "Suggest an activity appropriate to the user's current emotion."

[0847] Step 4:

[0848] The server sends the generated activity options to the terminal. Here, the options are visually organized for easier presentation to the user. The input is the generated activity options, and the output is suggested data formatted for display. Specifically, the server converts the data to HTML format or an application-specific format.

[0849] Step 5:

[0850] The user reviews the suggestions via their device and provides feedback. This feedback includes selections such as "Interested" or "Pass." The input is the displayed activity options, and the output is the user's feedback data. The device captures this feedback and sends it back to the server.

[0851] Step 6:

[0852] The server aggregates user feedback and determines the optimal activity. It analyzes the feedback data and selects the most suitable activity, taking into account the opinions of numerous users. The input is the feedback data, and the output is the determined activity plan. Specifically, the server evaluates the feedback using statistical methods.

[0853] Step 7:

[0854] The server integrates with a partnered reservation management system to handle the necessary reservation and payment procedures. Communication with an external system via API takes place here. The input is the decided activity plan, and the output is reservation confirmation information. The server sends this information to the terminal.

[0855] Step 8:

[0856] The user completes the reservation and payment on the terminal. The terminal displays reservation confirmation information and provides a screen for the payment process. The input is reservation confirmation information, and the output is completed transaction information. Specifically, the user enters payment information, and the terminal processes it securely.

[0857] Step 9:

[0858] After the activity ends, the server obtains feedback from the user again. This feedback is used to improve the activity options for future activities. The input is the user's opinion after the activity ends, and the output is update data for the generative model. The server uses this data to train the generative AI model.

[0859] (Application Example 2)

[0860] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0861] In modern families, enriching family time requires planning appropriate events and recreational activities that align with the interests and feelings of each family member. However, since each member has different interests and feelings, selecting the optimal activity is difficult, and decision-making within the family often does not proceed smoothly. Furthermore, efficiently executing these plans and consistently managing related reservations and payments is also a challenge.

[0862] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0863] This invention includes a server that receives conversational information acquired from multiple terminal devices, analyzes the conversational information to recognize the interests and emotions of each user, generates multiple task suggestions tailored to the interests and emotions of each user based on the analysis results, and presents the task suggestions to household appliances in real time to recommend household events and recreational activities. This makes it possible to efficiently plan and implement optimal events and recreational activities while being attentive to the emotions and interests of each member of the household.

[0864] A "terminal device" is a device used by a user to input and receive conversational information.

[0865] "Conversational information" refers to data that includes the content of text and audio exchanges between users, as well as their emotional nuances.

[0866] "Emotion" is an element that indicates the user's psychological state, and it is information extracted from conversational data.

[0867] "Interests" refers to information about topics and activities that a user is particularly interested in.

[0868] "Household machinery and equipment" refers to machines used within the home that have the function of presenting proposed tasks or activities.

[0869] "Proposal suggestions" are recommended plans for events and activities generated based on the user's interests and emotions.

[0870] "Reservation" refers to the procedure of securing the date, time, and location for a decided activity in advance.

[0871] "Payment" refers to the process of completing the necessary financial transaction for a decided activity.

[0872] The system for realizing this invention consists of a home appliance, an internet-connected terminal device, and a server. The server analyzes conversational information received from the terminal device used by the user. The analysis uses a software module that combines natural language processing technology and an emotion recognition engine. Specifically, the server uses "Python" and "TensorFlow" to process conversational data, converts the audio data into text using the "Google Cloud Speech-to-Text API," and analyzes the emotions from that text using the emotion engine.

[0873] The server uses a generative AI model to create task suggestions based on analyzed interest and sentiment information. This generative AI model utilizes technologies such as "OpenAI GPT" to construct optimal event and activity suggestions for the user. The generated suggestions are displayed in real time on home electronic devices, which the user can view and evaluate.

[0874] When a user provides feedback on suggestions displayed on a home appliance, that feedback is sent to a server via the device. The server aggregates this feedback and can use it to plan future activities in order to improve the accuracy of the suggestions. As this process is repeated, events and activities within the home evolve to become more responsive to the user's emotions and interests.

[0875] For example, if a family says "I'd like to go to a pop concert" on a holiday morning, the device transmits this conversation to the server. The server analyzes the conversation to identify the "interest in pop music" and suggests nearby concerts or live streaming options. If the user selects "go to a concert," the server can automatically book and pay for tickets. An example of a prompt to the generative AI model would be, "Please suggest weekend activities for a family who likes pop concerts."

[0876] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0877] Step 1:

[0878] The device records the user's conversation and sends it to the server as audio data. The input is audio data, and the output is an audio file uploaded to the server. The device uses a microphone to capture audio, compresses the data, and efficiently sends it to the server.

[0879] Step 2:

[0880] The server converts the received audio data into text data using the Google Cloud Speech-to-Text API. The input is an audio file, and the output is human-readable text data. The converted text is then used for subsequent sentiment analysis.

[0881] Step 3:

[0882] The server analyzes the converted text data using an emotion engine to identify the user's emotions. The input is text data, and the output is data with emotion labels. The emotion engine uses natural language processing algorithms to determine emotions such as joy, anger, sadness, and happiness.

[0883] Step 4:

[0884] The server generates task suggestions using a generative AI model based on sentiment and text content. The input is sentiment labels and parsed text, and the output is a list of suggested events and activities. The generative AI model uses technologies such as "OpenAI GPT" to create appropriate suggestions.

[0885] Step 5:

[0886] The server transmits and displays the generated task proposals to the home appliance in real time. The input is a list of proposals, and the output is the content of the proposals that the user can view. The user reviews and selects a proposal from the appliance's display screen.

[0887] Step 6:

[0888] The user provides feedback on the suggestions through the interface of the home appliance. The input is the user's selected option, and the output is the feedback data sent back to the server. The selection process can be performed via touch screen or voice recognition.

[0889] Step 7:

[0890] The server stores the collected feedback in a database and analyzes it for the purpose of improving future suggestions. The input is user feedback data, and the output is analyzed information to improve the accuracy of the suggestions. The feedback data is used to adjust the suggestion generation algorithm.

[0891] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0892] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0893] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0894] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0895] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0896] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0897] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0898] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0899] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0900] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0901] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0902] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0903] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0904] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0905] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0906] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0907] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0908] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0909] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0910] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0911] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0912] The following is further disclosed regarding the embodiments described above.

[0913] (Claim 1)

[0914] A means for receiving conversation data from multiple communication devices and analyzing that conversation data to identify the interests and emotions of each user,

[0915] A means for generating multiple activity suggestions tailored to each user's interests and emotions based on the analysis results,

[0916] A means for transmitting the generated activity proposals to each communication device and receiving feedback on the activity proposals,

[0917] A means of coordinating and deciding on the optimal activities among users based on feedback,

[0918] A system that includes means for making reservations and payments for decided activities.

[0919] (Claim 2)

[0920] The system according to claim 1, which records feedback obtained from each user and analyzes it to improve future activity suggestions.

[0921] (Claim 3)

[0922] The system according to claim 1, which obtains feedback from users about the activity after the activity has ended and updates the generative model based on that feedback.

[0923] "Example 1"

[0924] (Claim 1)

[0925] A means of receiving communication data from multiple information terminals, analyzing that communication data, and identifying the interests and emotions of each user,

[0926] A means for generating multiple activity suggestions tailored to each user's interests and emotions based on the analysis results,

[0927] A means for transmitting the generated activity plan to each information terminal and receiving responses to the activity plan,

[0928] A means of coordinating and deciding on the optimal activities among users based on their responses,

[0929] A system that includes means for processing reservations and payments related to the decided activity.

[0930] (Claim 2)

[0931] The system according to claim 1, which records the responses obtained from each user and analyzes them in order to improve future activity plans.

[0932] (Claim 3)

[0933] The system according to claim 1, which obtains user feedback on the activity after the activity has ended and updates the generative model based on that feedback.

[0934] "Application Example 1"

[0935] (Claim 1)

[0936] A means for receiving communication data from multiple information devices and analyzing that communication data to identify each individual's interests and emotions,

[0937] A means for generating multiple activity suggestions tailored to each individual's interests and emotions based on the analysis results,

[0938] A means for transmitting the generated activity proposals to each information device and receiving responses to the activity proposals,

[0939] A means of adjusting and deciding on the optimal activities among individuals based on their responses,

[0940] Means for making reservations and transactions related to the decided activities,

[0941] A system that includes means to analyze information related to meal planning and automate the selection of the optimal restaurant or delivery procedures.

[0942] (Claim 2)

[0943] The system according to claim 1, which records responses obtained from each individual and analyzes them to improve future activity proposals.

[0944] (Claim 3)

[0945] The system according to claim 1, which obtains responses from individuals regarding the activity after the activity has ended and updates the generative model based on those responses.

[0946] "Example 2 of combining an emotion engine"

[0947] (Claim 1)

[0948] A means for receiving conversation generation data from multiple communication devices, analyzing the generated data, and identifying the intentions and emotions of each user,

[0949] A means for generating multiple activity options tailored to each user's intentions and psychological state based on the analysis results,

[0950] A means of transmitting the generated activity options to each communication device and collecting opinions on the activity proposals,

[0951] Based on feedback, a means for users to coordinate and decide on the optimal procedure,

[0952] Means for making reservations and payments for the decided activities,

[0953] A means of detecting each user's emotions using natural language processing technology,

[0954] A system that includes a means of providing activity suggestions in response to prompt sentences using generative AI technology.

[0955] (Claim 2)

[0956] The system according to claim 1, which records opinions obtained from each user and analyzes them in order to improve the activity options for the next time.

[0957] (Claim 3)

[0958] The system according to claim 1, which, after the completion of an activity, obtains feedback from users regarding the activity and updates the generation technology based on that feedback.

[0959] "Application example 2 when combining with an emotional engine"

[0960] (Claim 1)

[0961] A means for receiving conversation information acquired from multiple terminal devices, analyzing that conversation information, and recognizing the interests and emotions of each user,

[0962] A means for generating multiple task suggestions tailored to each user's interests and emotions based on the analysis results,

[0963] A means for transmitting the generated problem proposals to each terminal device and receiving responses to the problem proposals,

[0964] A means for adjusting and deciding on the optimal tasks among users based on their responses,

[0965] Means for making reservations and payments for the decided tasks,

[0966] A system that presents suggested challenges to household appliances in real time and includes means for recommending household events and recreational activities.

[0967] (Claim 2)

[0968] The system according to claim 1, which records responses obtained from each user and analyzes them in order to improve future problem suggestions.

[0969] (Claim 3)

[0970] The system according to claim 1, which collects responses from users regarding the task after the task is completed and updates the generative model based on those responses. [Explanation of symbols]

[0971] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving communication data from multiple information devices and analyzing that communication data to identify each individual's interests and emotions, A means for generating multiple activity suggestions tailored to each individual's interests and emotions based on the analysis results, A means for transmitting the generated activity proposals to each information device and receiving responses to the activity proposals, A means of adjusting and deciding on the optimal activities among individuals based on their responses, Means for making reservations and transactions related to the decided activities, A system that includes means to analyze information related to meal planning and automate the selection of the optimal restaurant or delivery procedures.

2. The system according to claim 1, which records responses obtained from each individual and analyzes them to improve future activity proposals.

3. The system according to claim 1, which obtains responses from individuals regarding the activity after the activity has ended and updates the generative model based on those responses.

Images