System

Abstract

To provide a system.SOLUTION: A system includes: specifying means for specifying information useful for a user from a talk content of the user and a search history in which the user used an internet search tool, contained in various information sources which the user daily uses; notification means for notifying the user of the daily schedule of the user, based on the information specified by inputting specific prompt sentences into a generative AI model, and for notifying the user of the schedule which is information that is not registered in the schedule and which the user may have forgotten; and means for adjusting the timing of the notification and the content of the notification according to an emotion engine which recognizes the emotion of the user and the emotion.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional schedule management tools and reminder services require users to input and manage information themselves, which can be time consuming and leads to forgetting. In addition, there was no system that could comprehensively manage information such as family structure and administrative procedures, and provide schedule management and reminders based on that information. [Means for solving the problem]

[0005] The system of the present invention includes an identification means for identifying information useful to the user from the user's talk content and the user's search history using an Internet search tool, which are included in various information sources that the user uses on a daily basis; a notification means for notifying the user of the user's daily schedule based on the identified information by inputting a specific prompt sentence into a generative AI model, and notifying the user of schedules that are not registered in the schedule and that the user may have forgotten; an emotion engine for recognizing the user's emotions; and a means for adjusting the timing and content of the notification according to the emotion. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. [Figure 19] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system according to the embodiment 2 of the second embodiment in the case where an emotion engine is combined. [Figure 20] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when combined with an emotion engine. [Figure 21] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the embodiment 3 of the form example 3 in the case where an emotion engine is combined. [Figure 22] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention has a means for collecting information from multiple services used by the user (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.). For example, if a user chats on a messenger app saying "I have a meeting tomorrow at 1pm", this information is collected and automatically registered in the calendar. Also, if a user searches for "child's school admission procedures" on an Internet search tool, this information is collected and a reminder for related government procedures is set.

[0029] "Example 2"

[0030] The system of the present invention has a generative AI virtual assistant that learns the user's behavioral patterns based on collected information. For example, if the user learns that they have lunch at a specific restaurant every Friday, the system will remind them of the best time to leave, taking into account the weather and traffic information for that day.

[0031] "Example 3"

[0032] The system of the present invention has a means for automatically generating reminders for the user's life events and administrative procedures based on collected information and notifying the user. For example, when a user shares a change in family structure (e.g., birth, marriage, etc.) via email or messenger app, this information is collected, and a reminder for the related administrative procedures is automatically generated and notified to the user.

[0033] The process flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: A user talks on a messenger app saying, "I have a meeting tomorrow at 1pm."

[0036] Step 2: The system of the present invention collects this information and automatically enters it into your calendar.

[0037] Step 3: The system of the present invention sends a reminder before the meeting time.

[0038] "Example 2"

[0039] Step 1: A user indicates a behavioral pattern of having lunch at a particular restaurant every Friday.

[0040] Step 2: Our system learns this behavior pattern.

[0041] Step 3: The system of the present invention will take into account the weather and traffic conditions for the day and remind you of the best time to leave.

[0042] "Example 3"

[0043] Step 1: The user shares a change in family structure (e.g. birth, marriage, etc.) via email or messenger app.

[0044] Step 2: The system of the present invention collects this information.

[0045] Step 3: The system of the present invention automatically generates a reminder of the relevant administrative procedure and notifies the user.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0048] Modern users use multiple services (Internet search tools, calendars, e-mail, traffic information, weather information, search engines, messaging app communications, etc.), and are required to efficiently manage the information obtained from each service. However, manually collecting this information, analyzing it, and determining the appropriate action is extremely time-consuming. It is also difficult to learn users' behavioral patterns and provide services that meet their needs. Another challenge is automatically generating reminders for life events and government procedures and notifying users of them.

[0049] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for collecting information from multiple services, a means for analyzing the collected information, a means for determining an action based on the analysis result, and a means for executing the determined action. This makes it possible to efficiently manage information obtained from multiple services used by the user, learn the user's behavioral patterns, and provide services according to the needs. In addition, it is possible to automatically generate reminders for life events and administrative procedures and notify the user.

[0050] "Multiple services" refers to multiple information sources that users use on a daily basis, such as internet search tools, calendars, email, traffic information, weather information, search engines, and messaging app communications.

[0051] "Means of collecting information" refers to the function of automatically acquiring information such as user activities and communication content by utilizing the API or data acquisition function of each service.

[0052] "Means of analyzing information" refers to the function of analyzing collected information using natural language processing and data analysis techniques and extracting important information.

[0053] "Means for determining an action" refers to a function that automatically determines the next action to be taken based on the analysis results.

[0054] "Means for executing an action" refers to a function for actually executing a determined action, such as adding an event to a calendar or sending a reminder.

[0055] "Learning user behavior patterns" refers to the function of analyzing a user's past behavior and tendencies based on collected information and predicting future behavior.

[0056] "Providing services that meet the needs" refers to the function of providing optimal information and actions based on the user's behavioral patterns and current situation.

[0057] A "life event" refers to an important event in a user's life (e.g., a birthday, a wedding, a child's admission to school, etc.).

[0058] "Administrative procedures" refers to various official procedures that users must carry out (e.g. obtaining a resident registration card, filing taxes, etc.).

[0059] "Automatically generating reminders" refers to a function that automatically creates necessary notifications and reminders for users based on collected and analyzed information.

[0060] "Notify user" refers to the function of notifying the user of generated reminders and important information via email, calendar notifications, or other means.

[0061] This invention is a system that collects information from multiple services used by a user (Internet search tools, calendars, email, traffic information, weather information, search engines, messaging app communication content, etc.), analyzes this information to learn the user's behavioral patterns, and provides services according to the user's needs.

[0062] Generating a Program

[0063] The server generates a program to collect information from each service used by the user. This program is developed using a programming language such as Python or Java (registered trademark). Specifically, the server acquires data using the API of each service and analyzes it using natural language processing (NLP) technology.

[0064] Program Processing

[0065] The server performs data processing and calculation using the following hardware and software.

[0066] Hardware:

[0067] The server is a computer system equipped with a high-performance processor and sufficient memory, such as an Intel Xeon processor or AMD EPYC processor.

[0068] software:

[0069] The server uses the following software to collect and analyze information:

[0070] API: API of each service (e.g. Google (registered trademark) Calendar API, Microsoft (registered trademark) Outlook API, Gmail API, Google Search Console API, LINE API, WhatsApp API)

[0071] Natural Language Processing (NLP) Tools: Google Cloud Natural Language API, IBM Watson(R) Natural Language Understanding

[0072] Email sending services: Twilio SendGrid, Amazon SES

[0073] Examples

[0074] Example 1: Processing chat content in a messenger app:

[0075] When a user says "I have a meeting tomorrow at 1pm" in a messaging app, the server retrieves the contents of the chat through the messaging app's API. The server then uses the Google Cloud Natural Language API to extract the information "tomorrow at 1pm" and "meeting". The server then automatically registers this information as a "meeting" in the calendar using the Google Calendar API.

[0076] Example 2: Processing Internet search history:

[0077] When a user searches for "child's admission procedure" in an internet search tool, the server retrieves this search history using the Google Search Console API. Then, it analyzes the search query and detects the keyword "child's admission procedure". Based on this information, the server sets up a reminder for the relevant administrative procedure. Specifically, it uses Twilio SendGrid to send a reminder email to the user.

[0078] Examples of prompt statements

[0079] "If a user says 'I have a meeting tomorrow at 1pm' in a messaging app, please create a program that collects this information and automatically registers it in Google Calendar. Also, if a user searches for 'child's school admission procedures' in an internet search tool, please create a program that collects this information and sets a reminder for the related administrative procedures."

[0080] In this way, the server provides a system for analyzing user behavior and automatically collecting and processing necessary information.

[0081] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0082] Step 1:

[0083] The server collects information from each service used by the user. Specifically, it obtains data using the API of each service. For example, it obtains the contents of a user's chat using the API of a messaging app, and obtains search history using the API of an internet search tool. The input is the data obtained from the API of each service, and the output is the collected raw data.

[0084] Step 2:

[0085] The server analyzes the collected information. Specifically, it uses natural language processing (NLP) technology to analyze text data and extract important information. For example, it uses Google Cloud Natural Language API to extract dates and times and event details from chat content in a messaging app. The input is the collected raw data, and the output is analyzed information (e.g., dates and times, event details).

[0086] Step 3:

[0087] The server decides on an action based on the analysis results. Specifically, it determines the next action to be taken based on the extracted information. For example, it decides on the action of registering an event in a calendar based on the extracted date and time and event content. The input is the analyzed information, and the output is the determined action (e.g., registering the event in a calendar).

[0088] Step 4:

[0089] The server executes the determined action. Specifically, it executes the action using the API of each service. For example, it uses the Google Calendar API to automatically register the extracted event in the calendar. It also uses Twilio SendGrid to send a reminder email. The input is the determined action, and the output is the result of the executed action (e.g., the event registered in the calendar, the reminder email sent).

[0090] Step 5:

[0091] The server learns the user's behavioral patterns. Specifically, it analyzes the user's past behavior and tendencies based on the collected and analyzed information, and predicts future behavior. The input is the collected and analyzed information, and the output is the learned behavioral patterns.

[0092] Step 6:

[0093] The server provides services according to the user's needs. Specifically, it provides optimal information and actions to the user based on the learned behavioral patterns. For example, it provides related information based on search queries frequently performed by the user in the past. The input is the learned behavioral patterns, and the output is the provided service (e.g., providing related information).

[0094] In this way, the server analyzes user behavior, automatically collects and processes necessary information, and realizes a system that provides users with the most optimal services.

[0095] (Application example 1)

[0096] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] In modern life, users use many services and applications, and managing each piece of information individually is cumbersome. In addition, schedule management and reminder settings are often done manually, which creates the risk of forgetting important appointments or procedures. Furthermore, reservations for transportation and schedule settings must also be done manually, making efficient time management difficult.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0099] In this invention, the server includes a means for utilizing a generative AI virtual assistant to collect information on multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on city hall and administrative procedures, and information on family composition, and manage daily schedules based on this information, remind life events and administrative procedures, and follow up on forgotten schedules using information not registered in the calendar, a means for automatically setting a schedule for an autonomous vehicle based on the collected information, and a means for automatically reserving travel to a destination based on the schedule for the autonomous vehicle. This allows the user to centrally manage information from multiple services, and efficiently travel using an autonomous vehicle without forgetting important plans or procedures.

[0100] "Multiple services" refers to different information sources such as internet search tools, calendars, email, routes, weather, search, and chat content in messenger apps.

[0101] "City Hall / Administrative Procedures" refers to various administrative procedures and applications that Users must complete.

[0102] "Family information" refers to information about the user's family, such as the names, ages, and relationships of family members.

[0103] "Daily schedule management" refers to organizing and managing a user's daily plans.

[0104] A "life event" refers to an important event in a user's life, such as marriage, childbirth, or moving.

[0105] A "remind" refers to a notification to a user so that the user does not forget a specific schedule or procedure.

[0106] "Information not registered in the calendar" refers to important appointments and events that the user has not entered into the calendar.

[0107] A "generative AI virtual assistant" refers to a virtual assistant that uses artificial intelligence to collect information about the user and manage the schedule and provide reminders.

[0108] An "autonomous vehicle" refers to a vehicle that drives automatically using artificial intelligence and sensor technology.

[0109] "Automatically setting a schedule" refers to automatically determining and managing a user's plans based on collected information.

[0110] "Automatically reserving travel to a destination" refers to reserving an autonomous vehicle based on the user's schedule and arranging travel to the destination.

[0111] As an embodiment of the present invention, the following system is constructed.

[0112] System Overview

[0113] The system collects information from multiple services used by users, manages schedules and sets reminders using a generative AI virtual assistant, and automatically sets schedules for autonomous vehicles based on the collected information, automatically booking trips to destinations.

[0114] Hardware and software used

[0115] Hardware: Smartphones, self-driving cars

[0116] Software: Messenger app, Google Calendar API, Autonomous Vehicle API

[0117] Data processing and calculation

[0118] Getting messages

[0119] The device (smartphone) receives the user's messages from the messenger app. These messages include information about the user's schedule and events.

[0120] Extracting Information

[0121] The server uses natural language processing technology to extract information about schedules and events from the received messages. For example, from a message such as "I have a meeting tomorrow at 1pm," the server extracts the information "meeting," "tomorrow," and "13:00."

[0122] Calendar registration

[0123] The server uses the extracted information to automatically register the event on the user's calendar using the Google Calendar API.

[0124] Scheduling autonomous vehicles

[0125] The server automatically schedules autonomous vehicles using the Autonomous Vehicle API based on event information registered in the calendar. For example, it reserves an autonomous vehicle to travel to the meeting location at the start time of the meeting.

[0126] Examples

[0127] If a user says in a messenger app, "I have a meeting tomorrow at 1 p.m.," the system collects that information, registers the "meeting" on Google Calendar, and even books an autonomous vehicle to head to the meeting location at 1 p.m.

[0128] Examples of prompt statements

[0129] If a user says in a messenger app, "I have a meeting tomorrow at 1pm," create a program that collects this information, automatically registers the "meeting" in Google Calendar, and even books an autonomous vehicle to head to the meeting location at 1pm.

[0130] In this way, a system can be provided that makes users' lives more convenient.

[0131] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0132] Step 1:

[0133] The device (smartphone) retrieves the user's messages from the messenger app. The input is the user's message, and the output is the text data of the retrieved message. The specific operation is to retrieve the latest message using the messenger app's API.

[0134] Step 2:

[0135] The server uses natural language processing technology to extract information about schedules and events from the acquired messages. The input is the text data of the acquired messages, and the output is the extracted schedule or event information (e.g., "Meeting," "Tomorrow," "13:00"). Specifically, the server analyzes the messages using a natural language processing library (e.g., spaCy or NLTK) and extracts keywords and date and time information.

[0136] Step 3:

[0137] The server uses the Google Calendar API based on the extracted information to automatically register events in the user's calendar. The input is the extracted schedule and event information, and the output is the event registered in Google Calendar. Specifically, the server calls the Google Calendar API and registers the extracted information as an event.

[0138] Step 4:

[0139] The server automatically sets the schedule for autonomous vehicles using the Autonomous Vehicle API based on the event information registered in the calendar. The input is the event information registered in Google Calendar, and the output is the reservation information for the autonomous vehicle. Specifically, the server calls the Autonomous Vehicle API and reserves an autonomous vehicle based on the start time and location of the event.

[0140] Step 5:

[0141] The server notifies the user of the reserved autonomous vehicle information. The input is the reservation information for the autonomous vehicle, and the output is a notification message to the user. Specifically, the server uses the API of the messenger app to send the reservation information to the user.

[0142] In this way, a series of processes is realized: extracting plans from the user's messages, registering them on the calendar, and reserving an autonomous vehicle.

[0143] Example 2

[0144] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0145] Conventional virtual assistant systems have difficulty learning the user's behavioral patterns sufficiently and reminding the user at the appropriate time. In addition, they cannot calculate the optimal departure time taking into account weather and traffic information, and therefore cannot effectively support the user's life. For this reason, there is a demand for a system that can learn the user's behavioral patterns, remind the user at the appropriate time, and calculate the optimal departure time taking into account weather and traffic information.

[0146] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0147] In this invention, the server includes a means for collecting information on a plurality of services, information on procedures, and information on family structure, a means for utilizing a virtual assistant of a generative AI that manages daily schedules based on the information, reminds users of life events and procedures, and follows up on forgotten schedules using information not registered in a calendar, a means for collecting user behavior data and storing it in a database, a means for learning user behavior patterns using a generative AI model based on the collected behavior data, a means for predicting reminder timing based on the learned behavior patterns, a means for acquiring weather information and traffic information based on the predicted date and time, a means for calculating an optimal departure time based on the acquired information and generating a reminder message, and a means for notifying the generated reminder message to the user. This makes it possible to learn the user's behavior patterns, perform reminders at appropriate times, and calculate the optimal departure time taking into account weather information and traffic information.

[0148] The "multiple services" refer to various online services that provide information such as internet search tools, calendars, email, routes, weather, search, and chat content in messenger apps.

[0149] "Information on procedures, etc." refers to information on various procedures carried out at city halls and administrative agencies.

[0150] "Family information" refers to information about the user's family members, including, for example, the number of family members and their relationships.

[0151] A "generative AI virtual assistant" is an artificial intelligence system that learns the user's behavioral patterns based on collected information and provides the user with appropriate reminders and services.

[0152] "User behavior data" refers to information such as the date and time and the location where a user performed a particular behavior.

[0153] A "database" is an information management system for storing collected user behavior data.

[0154] A "generative AI model" is an artificial intelligence model that learns user behavior patterns based on collected data.

[0155] The "reminding timing" is the optimal date and time for reminding the user.

[0156] "Weather information" is information about the weather at a specific date and time.

[0157] "Traffic information" refers to information on traffic conditions at a specific date and time.

[0158] The "optimal departure time" is the most appropriate departure time for the user to reach the destination, calculated taking into account weather and traffic information.

[0159] A "remind message" is a message that is notified to a user to encourage the user to take action.

[0160] This invention relates to a generative AI virtual assistant system that learns a user's behavioral patterns and reminds the user at appropriate times. A specific embodiment of this system will be described below.

[0161] System Configuration

[0162] This system is configured using the following hardware and software.

[0163] Hardware: Servers, user devices (smartphones and PCs)

[0164] Software: Generative AI models (e.g., GPT-4®), database management systems (e.g., MySQL®), weather information APIs (e.g., OpenWeatherMap), traffic information APIs (e.g., Google Maps API)

[0165] Data collection and storage

[0166] The server collects user behavior data and stores it in a database. For example, it records information that a user has lunch at a specific restaurant every Friday. This accumulates basic data for understanding the user's behavioral patterns.

[0167] Learning behavioral patterns

[0168] The server inputs the collected behavioral data into a generative AI model to learn the user's behavioral patterns. The generative AI model can perform advanced pattern recognition and prediction, for example by using GPT-4, to understand the tendency to take certain actions on certain days of the week and at certain times of the day.

[0169] Reminder timing prediction

[0170] Based on the learned behavioral patterns, the server predicts the next time the user is likely to take a particular action, for example, to go to a particular restaurant next Friday.

[0171] Get weather and traffic information

[0172] Based on the predicted date and time, the server uses weather information APIs (e.g. OpenWeatherMap) and traffic information APIs (e.g. Google Maps API) to obtain the weather and traffic conditions for that day, allowing it to take into account external factors that affect the user's behavior.

[0173] Calculate the best departure time

[0174] The server calculates the optimal departure time for the user based on the acquired weather and traffic information. For example, if it is a rainy day and traffic is congested, it calculates an earlier departure time than usual.

[0175] Generate and send reminder messages

[0176] The server generates a reminder message to inform the user based on the calculated optimal departure time, for example, "It's raining and traffic is heavy today, so we recommend that you leave 15 minutes earlier than usual."

[0177] The terminal notifies the user of the generated reminder message, for example, by a push notification on a smartphone or by email.

[0178] Examples

[0179] Suppose a user has a behavioral pattern of having lunch at "Restaurant A" every Friday. This information is stored on the server. On Friday morning, the server uses the generative AI model to predict that the user is likely to go to "Restaurant A" that day as well. The server uses the OpenWeatherMap API to get weather information for the day and the Google Maps API to get traffic information. For example, if the weather is rainy and traffic is congested that day, the server generates a reminder message to the user saying, "It's raining and traffic is congested today, so we recommend that you leave 15 minutes earlier than usual."

[0180] Examples of prompt statements

[0181] "Design a system that learns a user's behavioral pattern of having lunch at a particular restaurant every Friday and then reminds them of the best time to leave, taking into account the weather and traffic conditions for that day."

[0182] In this way, the system learns the user's behavioral patterns and supports the user's daily life by reminding them at appropriate times.

[0183] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0184] Step 1:

[0185] Collecting user behavior data

[0186] Input: User behavior information (date, time, location, and content of the action)

[0187] Specific operation: When a user performs a specific action, that information is sent to the server. For example, information that a user checked in to "Restaurant A" at 12:00 on Friday is sent from the smartphone to the server.

[0188] Output: Collected behavioral data is stored in a database.

[0189] Step 2:

[0190] Learning behavioral patterns

[0191] Input: Collected behavioral data

[0192] Specific operation: The server inputs the collected behavioral data into a generative AI model (e.g., GPT-4) to learn the user's behavioral patterns. For example, it learns a pattern that "the user has lunch at 'Restaurant A' every Friday at 12:00."

[0193] Output: A model of the learned behavioral patterns.

[0194] Step 3:

[0195] Reminder timing prediction

[0196] Input: A model of learned behavioral patterns

[0197] Specific operation: Based on the learned behavioral patterns, the server predicts the next time the user is likely to take a certain action. For example, it predicts that the user is likely to go to "Restaurant A" at 12:00 on the next Friday.

[0198] Output: Predicted reminder timing

[0199] Step 4:

[0200] Get weather and traffic information

[0201] Input: Predicted reminder time

[0202] Specific operation: The server obtains the weather and traffic conditions for the day using a weather information API (e.g., OpenWeatherMap) and a traffic information API (e.g., Google Maps API). For example, it obtains information that the weather for next Friday is rainy and the traffic is heavy.

[0203] Output: Obtained weather and traffic information

[0204] Step 5:

[0205] Calculate the best departure time

[0206] Input: Retrieved weather and traffic information

[0207] Specific operation: The server calculates the optimal departure time for the user based on the acquired weather and traffic information. For example, if it is a rainy day and traffic is congested, it calculates that the user should leave 15 minutes earlier than usual.

[0208] Output: Calculated optimal departure time

[0209] Step 6:

[0210] Generate reminder messages

[0211] Input: Calculated optimal departure time

[0212] Specific behavior: The server generates a reminder message to notify the user based on the calculated optimal departure time, for example, "It's raining and traffic is heavy today, so we recommend that you leave 15 minutes earlier than usual."

[0213] Output: The generated reminder message.

[0214] Step 7:

[0215] Reminder message notifications

[0216] Input: The generated reminder message

[0217] Specific operation: The device notifies the user of the generated reminder message, for example, by a smartphone push notification or email.

[0218] Output: Reminder message sent to the user

[0219] In this way, the system learns the user's behavioral patterns and supports the user's daily life by reminding them at appropriate times.

[0220] (Application example 2)

[0221] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0222] Conventional virtual assistant systems were limited to managing users' schedules and providing reminders, and were insufficient in providing personalized services based on users' behavioral patterns and preferences. They also lacked a reminder function that took into account weather and traffic information and provided optimal timing. This resulted in reduced convenience for users and made it difficult to provide efficient lifestyle support.

[0223] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server collects information on a plurality of services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on procedures of public institutions, and information on family composition, and based on this information, it uses a virtual assistant of a generative AI to manage daily schedules, reminds users of procedures of life events and public institutions, and follows up on forgotten schedules with information not registered in the calendar, etc.; the virtual assistant of the generative AI learns the user's behavioral patterns based on the collected information and provides services according to the user's needs; the virtual assistant of the generative AI learns the user's food preferences and order history, and reminds the user to order food at the optimal timing; and the virtual assistant of the generative AI reminds the user to order at the optimal time, taking into account weather information and traffic information. This makes it possible to provide personalized services based on the user's behavioral patterns and preferences, and also realizes a reminder function at the optimal timing taking into account weather information and traffic information.

[0224] "Multiple services" refers to information such as internet search tools, calendars, email, routes, weather, search, and chat content on messenger apps.

[0225] "Procedures at public institutions" refers to various procedures carried out at city halls and government agencies.

[0226] "Family composition information" refers to information about the user's family.

[0227] "Daily schedule management" refers to managing a user's daily plans.

[0228] A "life event" refers to an important occurrence in a user's life.

[0229] A "remind" refers to reminding a user to perform a specific action or schedule.

[0230] A "generative AI virtual assistant" refers to artificial intelligence that learns a user's behavioral patterns based on collected information and provides the user with appropriate services.

[0231] "Behavior patterns" refer to the tendencies of a user's daily behavior.

[0232] "User needs" refers to the services and information required by users.

[0233] "Dining preferences" refers to information about the types of meals and restaurants that a user prefers.

[0234] "Order history" refers to a record of past food orders placed by a user.

[0235] The "optimal timing" refers to the most suitable time for the user.

[0236] "Weather Information" refers to information regarding the weather on a particular day.

[0237] "Traffic Information" refers to information regarding traffic conditions on a particular day.

[0238] "Optimal ordering time" refers to the most suitable time for a user to order a meal.

[0239] As an embodiment of the present invention, the following system is constructed.

[0240] System Configuration

[0241] The system consists of a server, a user device, and a generative AI virtual assistant. The server collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on procedures at public institutions, and information on family composition. The user device is a device such as a smartphone or tablet, on which the generative AI virtual assistant is installed.

[0242] Program Processing

[0243] The server learns the user's behavioral patterns based on the collected information. Specifically, it accumulates data on the user's past actions and choices, and predicts the user's behavioral patterns using a machine learning algorithm (for example, the Linear Regression model of scikit-learn).

[0244] The generative AI virtual assistant learns the user's food preferences and ordering history, and reminds them to order food at the optimal time. It also has a function that takes into account weather and traffic information to remind users of the optimal ordering time. This allows users to order food efficiently.

[0245] Hardware and Software Used

[0246] Hardware: Smartphones, tablets, servers

[0247] Software: Python, scikit-learn, requests

[0248] Examples

[0249] For example, if the system learns that a user has a behavioral pattern of having lunch at a specific restaurant every Friday, it will remind the user to leave earlier than usual if it is expected to rain and have heavy traffic on that day. Also, if the user wants to try a new restaurant, it will suggest a restaurant based on the user's past preferences and ratings.

[0250] Examples of prompt statements

[0251] Build an application that learns the user's behavioral pattern of having lunch at a specific restaurant every Friday and reminds them of the best time to leave, taking into account the weather and traffic information for that day. Use Python and scikit-learn, and get weather and traffic information from an API.

[0252] In this way, it becomes possible to provide personalized services based on the user's behavioral patterns and preferences, and it is also possible to realize a reminder function at the optimal time, taking into account weather and traffic information.

[0253] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0254] Step 1:

[0255] The server collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on procedures at public institutions, and information on family composition. This information is obtained through APIs and stored in a database. The input is information from each service, and the output is an integrated database.

[0256] Step 2:

[0257] The server learns the user's behavioral patterns based on the collected information. Specifically, it trains a machine learning algorithm (for example, scikit-learn's Linear Regression model) using past behavioral data. The input is the past behavioral data, and the output is the learned model.

[0258] Step 3:

[0259] The user device collects the user's food preferences and order history through a generative AI virtual assistant. This information is obtained from the user's input and past order history. The input is the user's food preferences and order history, and the output is personalized food preference data.

[0260] Step 4:

[0261] A generative AI virtual assistant reminds users to order food at the optimal time based on their food preferences and ordering history. Specifically, it calculates the optimal ordering time by taking into account the user's behavioral patterns and the current date and time. The input is the user's behavioral patterns and the current date and time, and the output is a reminder message.

[0262] Step 5:

[0263] The server obtains weather and traffic information and calculates the optimal ordering time based on this information. Specifically, it obtains real-time information through weather and traffic APIs and calculates the optimal time by combining it with the user's behavioral patterns. The input is weather and traffic information, and the output is the optimal ordering time.

[0264] Step 6:

[0265] The user device receives reminder messages from the generative AI virtual assistant and notifies the user. Specifically, it notifies the user of the optimal time to order via push notifications and alerts. The input is the reminder message, and the output is the notification to the user.

[0266] Step 7:

[0267] The user places an order for a meal based on the reminder message. Specifically, the user performs the ordering procedure within the application and the order is completed. The input is the reminder message, and the output is a notification that the order has been completed.

[0268] Example 3

[0269] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0270] In modern society, users receive information from many sources, but it is difficult to effectively manage that information and receive appropriate reminders at the right time. In addition, users often miss information about important events such as changes in family structure or administrative procedures. This puts users at risk of forgetting or delaying important procedures.

[0271] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means. In this invention, the server includes a means for collecting information on a plurality of services, information on procedures of public institutions, and information on family structure, a means for analyzing the collected information and storing it in a database, a means for generating a reminder using a generative AI model based on the stored information, and a means for notifying the user of the generated reminder. This enables the user to receive reminders about important life events and administrative procedures at appropriate times.

[0272] "Multiple services" refers to various information services that users use on a daily basis, such as internet search tools, calendars, email, route information, weather information, search engines, and chat content on messenger apps.

[0273] "Public institution procedures" refers to various procedures and services provided by city halls and administrative agencies, such as submitting a birth notification, applying for child allowances, and submitting a marriage notification.

[0274] "Family information" refers to information about the user's family, such as information about events such as birth, marriage, divorce, and death.

[0275] "Means of collection" refers to the technologies and methods used to obtain necessary information from various services and applications used by users.

[0276] "Means of analysis" refers to techniques and methods for analyzing collected information using natural language processing technology, etc., and extracting meaning from it.

[0277] "Means of storing in a database" refers to a database system for efficiently managing and storing the analyzed information.

[0278] "Generative AI model" refers to an artificial intelligence model, such as a natural language generation model, for generating reminders based on collected and analyzed information.

[0279] "Means for generating reminders" means the technology or methodology for using a generative AI model to create reminders for users.

[0280] "Means of notification" refers to the technology or method for communicating the generated reminder to the user, such as email or a messenger app.

[0281] The present invention relates to a system for automatically generating and notifying a user of reminders regarding life events and administrative procedures. A specific embodiment of this system will be described below.

[0282] The server collects information from multiple services used by the user, including internet search tools, calendars, emails, route information, weather information, search engines, chat content in messenger apps, etc. For example, if a user sends a message on Facebook Messenger saying "Our baby was born!", the device sends this message to the server.

[0283] The server uses natural language processing technology to analyze the collected information. Specifically, it uses the Google Cloud Natural Language API to analyze the content of the messages and extract information about changes in family structure. The analyzed information is then stored in a database.

[0284] Next, the server generates a reminder using a generative AI model based on the stored information. The generative AI model used is GPT-4 from OpenAI (registered trademark). The server sends the following prompt to the generative AI model:

[0285] A user sends a message saying "Our baby is here!" Generate a reminder for the associated administrative process.

[0286] Based on this prompt, the generative AI model generates a reminder such as, "The deadline for submitting the birth notification is XX day. Don't forget to apply for child allowance."

[0287] Finally, the server notifies the user of the generated reminder via email or a messenger app. Specifically, the server uses the Gmail API or Facebook Messenger API to send the reminder.

[0288] This system allows users to receive timely reminders about important life events and administrative procedures. For example, if a user sends a message saying "Our baby was born!", the server collects and analyzes this information, generates a reminder using a generative AI model, and notifies the user so that they do not forget to submit a birth notification or apply for child allowance. The flow of the specific process in the third embodiment is explained with reference to FIG. 15.

[0289] Step 1: Gather information

[0290] A user shares changes in family structure (e.g. birth, marriage, etc.) via email or a messenger app. The device sends this information to the server. Specifically, a user sends a message on Facebook Messenger saying "Our baby is here!" The input is the user's message, and the output is the message data sent to the server.

[0291] Step 2: Analyze the information

[0292] The server analyzes the received message. Natural language processing (NLP) technology is used for the analysis. Specifically, the Google Cloud Natural Language API is used to analyze the content of the message. The input is the message data sent to the server, and the output is the analyzed information on changes to family structure. The server extracts the information that "a baby was born" from the message.

[0293] Step 3: Save your information

[0294] The server saves the analyzed information in a database. The input is the analyzed family structure change information, and the output is the information saved in the database. The server saves the information "a baby was born" in the database.

[0295] Step 4: Generate a reminder

[0296] The server generates reminders using a generative AI model based on the stored information. Specifically, it uses OpenAI's GPT-4. The server sends the following prompt to the generative AI model:

[0297] A user sends a message saying "Our baby is here!" Generate a reminder for the associated administrative process.

[0298] The input is the information stored in the database and a prompt, and the output is the generated reminder. The generative AI model generates a reminder such as, "The deadline for submitting the birth notification is XX day. Don't forget to apply for child allowance."

[0299] Step 5: Reminder notifications

[0300] The server notifies the user's device of the generated reminder. The notification is sent via email or a messenger app. Specifically, the reminder is sent using the Gmail API or Facebook Messenger API. The input is the generated reminder, and the output is the reminder notification sent to the user's device. The server sends a reminder to the user's email address or messenger app, such as "The deadline for submitting the birth notification is XX day. Don't forget to apply for child allowance."

[0301] (Application example 3)

[0302] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0303] Conventional virtual assistant systems can provide users with reminders about life events and administrative procedures, but they lack the functionality to automatically generate security reminders and notify them at the appropriate time, which poses the issue that users may forget important security measures.

[0304] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0305] In this invention, the server includes a means for utilizing a generative AI virtual assistant to collect information on multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on city hall and government procedures, and information on family composition, and manage daily schedules based on this information, remind the user of life events and government procedures, and follow up on forgotten schedules using information not registered in the calendar, a means for generating a security reminder message based on the user's life event information and notifying the user by email, and a means for sending the reminder message one week before the event. This allows the user to receive reminders at the appropriate time without forgetting important security measures.

[0306] "Multiple services" refers to information such as internet search tools, calendars, email, routes, weather, search, and chat content on messenger apps.

[0307] "Information on city hall and administrative procedures, etc." refers to information on various administrative procedures that users need to carry out.

[0308] "Family information" refers to information about the user's family members.

[0309] "Daily schedule management" refers to managing a user's daily plans.

[0310] "Life events" refer to important events in a user's life (e.g. moving, getting married, having a baby, etc.).

[0311] "Administrative procedure reminder" refers to notifying users so that they do not forget to carry out administrative procedures that they need to carry out.

[0312] "Following up on forgotten schedules using information not registered in the calendar" refers to reminding the user of important events that are not registered in the calendar.

[0313] A "generative AI virtual assistant" refers to an artificial intelligence that manages the user's schedule and provides reminders based on collected information.

[0314] "User's life event information" refers to information about important events in the user's life.

[0315] A "security reminder message" refers to a message that notifies a user of security measures that the user should take.

[0316] "Means of notifying by email" refers to a method of sending a reminder message to the user's email address.

[0317] "Method of sending one week before the event" refers to a method of sending a reminder message one week before a life event.

[0318] The system for carrying out the present invention has a function of generating a security-related reminder message based on a user's life event information and notifying the user at an appropriate timing. A specific embodiment of this system will be described below.

[0319] The server collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), city hall and administrative procedures, and family structure information. Based on this information, a generative AI virtual assistant manages the user's daily schedule and reminds them of life events and administrative procedures. It also follows up on forgotten schedules that are not registered in the calendar.

[0320] Based on the user's life event information, a security reminder message is generated. For example, if a user is planning to move, a reminder message is generated one week before the move stating, "Don't forget to update your security settings for the move." This reminder message is sent to the user via email.

[0321] The following hardware and software are used to implement this system. A smartphone is used as the hardware. Python and smtplib (email sending library) are used as the software. The server processes and calculates data based on the collected information, and generates a reminder message.

[0322] For example, if a user is planning to get married, a reminder message will be generated and sent to the user one week before the wedding, saying, "Don't forget to update your emergency contact information for your wedding." In this way, users can receive timely reminders without forgetting to take important security measures.

[0323] Example of a prompt to be input to a generative AI model:

[0324] Please create a program that generates security reminder messages based on the user's life event information and notifies them by email. User life events include moving, getting married, etc. The reminder message should be sent one week before the event.

[0325] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0326] Step 1:

[0327] The server collects information from multiple services (Internet search tools, calendars, email, routes, weather, searches, chat content on messenger apps, etc.), information on city hall and administrative procedures, and information on family structure. This information is entered as data necessary for managing the user's life events and schedule. The server stores this information in a database and makes it available for subsequent processing.

[0328] Step 2:

[0329] The server uses a generative AI virtual assistant based on the collected information to manage the user's daily schedule and remind them of life events and government procedures. Specifically, the server analyzes the collected data and generates the necessary reminders to follow up on forgotten schedules that are not registered in the user's calendar. Natural language processing and machine learning algorithms are used for this analysis.

[0330] Step 3:

[0331] The server generates security-related reminder messages based on the user's life event information. For example, if a user is planning to move, a reminder message is generated one week before the move saying, "Don't forget to update your security settings for the move." This reminder message is automatically created using a generative AI model.

[0332] Step 4:

[0333] The server sends the generated reminder message to the user's email address. The smtplib library is used to send the email. Specifically, the reminder message is set as the body of the email, and the user's email address is specified as the destination. If the email is sent successfully, the server records the result in a log.

[0334] Step 5:

[0335] The user checks the received reminder message and takes necessary security measures. For example, updating security settings when moving or updating emergency contacts when getting married. By taking action based on the reminder message, the user can take important security measures without forgetting them.

[0336] Step 6:

[0337] The server learns from the user's behavioral history to improve the accuracy of future reminder messages. Specifically, it records what actions the user took in response to the reminder message and feeds that data back to the AI ​​model. This allows future reminder messages to be more tailored to the user's needs.

[0338] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0339] "Example 1"

[0340] In one embodiment of the present invention, a system is provided that has an emotion engine that recognizes the emotion of a user. The system recognizes the user's emotional state and adjusts the timing and content of reminders according to the emotional state. For example, if the user feels stressed, the system may reduce the frequency of reminders or suggest activities to help the user relax.

[0341] "Example 2"

[0342] In another embodiment of the present invention, a system is provided in which an emotion engine adjusts the way in which a service is provided according to the emotion state of a user. For example, if the system detects that the user is feeling happy, the system generates a message for sharing the joy and provides it to the user. Also, if the system detects that the user is feeling sad, the system generates a message of comfort and provides it to the user.

[0343] "Example 3"

[0344] Furthermore, as another embodiment of the present invention, a system is provided in which the emotion engine adjusts the timing and content of reminders and the method of providing services according to the user's emotional state. For example, if the user feels angry, the system suggests an activity to calm the anger. Also, if the user feels surprised, the system generates a message to share the surprise and provides it to the user.

[0345] The process flow of each embodiment will be described below.

[0346] "Example 1"

[0347] Step 1: The system recognizes the user's emotional state using an emotion engine.

[0348] Step 2: If the emotion engine recognizes the user's emotional state as "stressed," the system will either reduce the frequency of reminders or suggest activities to help the user relax.

[0349] "Example 2"

[0350] Step 1: The system recognizes the user's emotional state using an emotion engine.

[0351] Step 2: If the emotion engine recognizes the user's emotional state as "happy" or "sad", the system generates a message according to the emotional state and provides it to the user.

[0352] "Example 3"

[0353] Step 1: The system recognizes the user's emotional state using an emotion engine.

[0354] Step 2: If the emotion engine recognizes the user's emotional state as "anger" or "surprise," the system suggests an activity or generates a message corresponding to that emotional state and provides it to the user.

[0355] Example 1

[0356] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0357] Modern users use multiple services (Internet search tools, calendars, email, traffic information, weather information, search engines, messaging apps, etc.) and are required to efficiently manage the information obtained from each service. However, manually organizing this information and managing schedules and reminders is very time-consuming. It is also difficult to provide appropriate reminders according to the user's emotional state. This leads to problems such as users missing important information or not receiving reminders at the right time.

[0358] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for collecting information from multiple services (Internet search tools, calendars, emails, traffic information, weather information, search engines, chat content of messaging apps, etc.), a means for analyzing the collected information and extracting important information, a means for automatically generating schedule management and reminders based on the extracted information and notifying the user, and a means for recognizing the user's emotional state and adjusting the timing and content of the reminder according to the emotional state. This allows the user to efficiently manage information obtained from multiple services and receive reminders at appropriate times.

[0359] "Multiple services" refers to different information sources such as internet search tools, calendars, email, traffic information, weather information, search engines, and chat content in messaging apps.

[0360] "Means of collecting information" refers to the function of obtaining necessary information from multiple services used by a user, using the APIs and data acquisition methods of each service.

[0361] "Means of analyzing information" refers to the function of analyzing collected information using natural language processing technology and data analysis technology and extracting important information.

[0362] "Means for extracting important information" refers to the function of identifying and extracting information that is useful to the user from the analyzed data.

[0363] "Means for automatically managing schedules and generating reminders" refers to a function for automatically managing a user's schedule and generating necessary reminders based on the extracted important information.

[0364] "Means for notifying the user" refers to a function for notifying the user of the generated reminders and schedule information in an appropriate manner.

[0365] "Means for recognizing an emotional state" refers to a function for analyzing the content of a user's messages and behavioral patterns to identify the user's emotional state.

[0366] "Means for adjusting the timing and content of reminders" refers to a function for appropriately changing the timing and content of reminder notifications depending on the recognized emotional state of the user.

[0367] This invention is a system that collects information from multiple services used by the user (Internet search tools, calendars, emails, traffic information, weather information, search engines, chat content of messaging apps, etc.), analyzes this information to extract important information, automatically generates schedule management and reminders, and notifies the user. It also includes a function to recognize the user's emotional state and adjust the timing and content of reminders according to that emotional state.

[0368] Hardware and Software Used

[0369] The server uses the following hardware and software:

[0370] Hardware: A server with a powerful processor, sufficient memory, storage devices, and network interfaces.

[0371] Software: APIs for each service, natural language processing technology (e.g., Google Cloud Natural Language API and IBM Watson Natural Language Understanding), database management system, sentiment analysis engine.

[0372] Data processing and calculation

[0373] The server processes and calculates the data in the following procedure.

[0374] 1. Information Collection:

[0375] The server collects user information using the API of each service. For example, it uses a messaging app's API to obtain chat contents and an internet search tool's API to obtain search history.

[0376] 2. Information analysis:

[0377] The server analyzes the collected information using natural language processing technology, specifically, by using the Google Cloud Natural Language API to extract important information from chat content and search history.

[0378] 3. Automatic registration and reminder generation:

[0379] The server automatically generates schedule management and reminders based on the analyzed information. For example, it registers meetings in the calendar using the Google Calendar API and sets reminders using the API of a reminder app.

[0380] 4. Emotional state recognition and reminder adjustment:

[0381] The server uses an emotion analysis engine to recognize the user's emotional state. If it determines that the user is stressed, it will reduce the frequency of reminders and suggest activities to help them relax. For example, it will use the API of a music streaming service to suggest a relaxing playlist.

[0382] Examples

[0383] Example 1: Automatic conference registration

[0384] A user talks on a messaging app saying, "I have a meeting tomorrow at 1pm."

[0385] The server collects the content of this talk and uses the Google Cloud Natural Language API to extract information such as "There is a meeting tomorrow at 1:00 p.m."

[0386] The server automatically registers the "meeting" in the calendar using the Google Calendar API.

[0387] Example 2: Administrative procedure reminders

[0388] A user searches for "child enrollment" using an internet search tool.

[0389] The server collects this search information and uses the Google Cloud Natural Language API to extract the keyword "child's enrollment procedures."

[0390] The server uses the reminder app's API to set reminders for related government procedures.

[0391] Example 3: Adjusting reminders based on emotions

[0392] A user talks on a messaging app saying, "I've been busy at work lately and I'm feeling stressed."

[0393] The server analyzes the content of the conversation using an emotion engine and determines that the user is feeling stressed.

[0394] The server reduces the frequency of reminders and uses the music streaming service's API to suggest relaxation playlists.

[0395] Examples of prompt statements

[0396] "If a user says in a messaging app, 'I have a meeting tomorrow at 1pm,' generate a program that collects this information, parses it using the Google Cloud Natural Language API, and automatically adds it to the calendar using the Google Calendar API."

[0397] "When a user searches for 'child's admission procedure' using an internet search tool, collect this information, parse it using the Google Cloud Natural Language API, and generate a program that uses the reminder app's API to set a reminder for the relevant administrative procedure."

[0398] "If a user says in a messaging app, 'I've been busy at work lately and I'm stressed,' generate a program that collects this information, analyzes it with an emotion engine to reduce the frequency of reminders, and uses the API of a music streaming service to suggest a relaxing playlist."

[0399] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0400] Step 1: Gather information

[0401] The server collects information from multiple services used by the user (Internet search tools, calendars, email, traffic information, weather information, search engines, chat content of messaging apps, etc.). Specifically, it obtains data using the API of each service. The input is data from the API of each service, and the output is the collected raw data.

[0402] Specific behavior:

[0403] The server calls the messaging app's API and obtains the user's chat content.

[0404] The server obtains the user's search history using the API of the Internet search tool.

[0405] Step 2: Information analysis

[0406] The server analyzes the collected information. It uses natural language processing (NLP) techniques to extract important information from chat content and search history. The input is the collected raw data, and the output is the analyzed important information.

[0407] Specific behavior:

[0408] The server uses the Google Cloud Natural Language API to extract information from the chat content, such as "There is a meeting tomorrow at 1:00 p.m."

[0409] The server extracts the keyword "child's admission procedures" from the search history.

[0410] Step 3: Automatic registration and reminder generation

[0411] The server takes appropriate action based on the parsed information. For example, it uses a calendar API to automatically add meeting information to a calendar. The input is the parsed important information, and the output is automatically generated schedules and reminders.

[0412] Specific behavior:

[0413] The server calls the calendar API and registers "Meeting tomorrow at 1pm" in the calendar.

[0414] The server uses the API of the reminder app to set a reminder for "child's enrollment procedures."

[0415] Step 4: Recognize your emotional state

[0416] The server uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's message content and behavioral patterns to identify emotions. The input is the user's message content and behavioral patterns, and the output is the recognized emotional state.

[0417] Specific behavior:

[0418] The server inputs the chat content from the messaging app into an emotion engine and analyzes the user's emotional state.

[0419] The server uses the Google Cloud Natural Language API to determine whether the user is feeling stressed.

[0420] Step 5: Adjust your reminders

[0421] The server adjusts the timing and content of reminders depending on the user's emotional state. For example, if it determines that the user is stressed, it may reduce the frequency of reminders or suggest activities to help them relax. The input is the perceived emotional state, and the output is adjusted reminders or suggested activities.

[0422] Specific behavior:

[0423] The server calls the reminder app's API and sets the app to reduce the frequency of reminders.

[0424] The server uses the API of a music streaming service to suggest a relaxing playlist to the user.

[0425] (Application example 1)

[0426] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0427] In modern factories, managing workers' schedules and understanding their emotional states are important issues. In particular, it is necessary to respond appropriately when a worker is feeling stressed or has an important meeting. However, with conventional systems, it is difficult to efficiently collect this information and respond appropriately. There is also a demand for systems that can adjust reminders according to the worker's emotional state, improve work efficiency, and improve safety.

[0428] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server collects information on multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on city hall and administrative procedures, and information on family structure, and based on this information, it includes a means using a generative AI virtual assistant that manages daily schedules, reminds users of life events and administrative procedures, and follows up on forgotten schedules using information not registered in the calendar, a means using an emotion engine that recognizes the user's emotional state and adjusts the timing and content of reminders according to the emotional state, and a means for a robot working in a factory to grasp the schedules and emotional states of workers and improve work efficiency and safety. This makes it possible to efficiently manage the schedules and grasp the emotional states of workers, and to respond appropriately.

[0429] "Multiple services" refers to information such as internet search tools, calendars, email, routes, weather, search, and chat content on messenger apps.

[0430] "Information on city hall and administrative procedures, etc." refers to information regarding administrative procedures that users need to carry out.

[0431] "Family composition information" refers to information about the user's family.

[0432] "Daily schedule management" refers to managing a user's daily plans.

[0433] A "life event" refers to an important occurrence in a user's life.

[0434] A "remind" refers to reminding a user to perform a specific action or schedule.

[0435] A "generative AI virtual assistant" refers to a virtual assistant that uses artificial intelligence to collect information about the user and provide appropriate services.

[0436] An "emotion engine" is an engine that recognizes the user's emotional state and responds appropriately according to that state.

[0437] "Robots working in factories" refers to automated machines that perform work in factories.

[0438] "Worker schedule" refers to the schedule of workers working in a factory.

[0439] "Emotional state" refers to the current emotional state of a user or worker.

[0440] "Improving work efficiency" refers to carrying out work more efficiently.

[0441] "Improving safety" refers to improving the working environment and working methods to increase safety.

[0442] As an embodiment of the present invention, a system is constructed in which a robot working in a factory grasps the schedule and emotional state of workers, thereby improving work efficiency and safety. A specific embodiment of the system is shown below.

[0443] System Programming

[0444] The system consists of the following main components:

[0445] 1. Information collection module: Collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content in messenger apps, etc.).

[0446] 2. Emotion recognition engine: Recognizes the user's emotional state and responds appropriately based on that state.

[0447] 3. Schedule Management Module: Based on the collected information, manage the schedules of workers and provide reminders and follow-ups.

[0448] 4. Robot control module: Robots working in factories will understand the schedules and emotional states of workers, improving work efficiency and safety.

[0449] Hardware and Software Used

[0450] Hardware: Factory robots (e.g. ABB's IRB series)

[0451] Software: Python, emotion recognition engine (e.g. Microsoft Azure Emotion API)

[0452] Data processing and calculation

[0453] 1. Information collection: The server analyzes the chat contents of messenger apps and search results from Internet search tools, and registers them in a schedule.

[0454] 2. Emotion Recognition: The server recognizes the emotional state of the worker using an emotion recognition engine.

[0455] 3. Schedule Adjustment: The server adjusts the work load according to the emotional state.

[0456] 4. Remind: The server will remind you based on the current time.

[0457] Examples

[0458] For example, if a worker says "I have a meeting tomorrow at 1pm" on a messenger app, this information is collected and automatically added to the calendar. Also, if a worker searches for "child's school admission procedures" using an internet search tool, this information is collected and a reminder for the related government procedures is set. Furthermore, if the system recognizes that the worker is feeling stressed, it will reduce the frequency of reminders or make suggestions to reduce the workload.

[0459] Examples of prompt statements

[0460] Below are examples of prompts to be input to the generative AI model.

[0461] If a user says "I have a meeting tomorrow at 1pm" on a messenger app, this information is collected and automatically added to the calendar. If a user searches for "child's school admission procedures" on an internet search tool, this information is collected and a reminder for the related government procedures is set. Furthermore, if the system recognizes that the user is feeling stressed, it will reduce the frequency of reminders or suggest activities to help the user relax.

[0462] In this way, a system can be realized in which robots working in a factory can understand the schedules and emotional states of workers, improving work efficiency and safety.

[0463] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0464] Step 1:

[0465] The server collects information from multiple services (Internet search tools, calendars, email, routes, weather, searches, chat content in messenger apps, etc.). Specifically, it obtains data from each service through APIs and collects information about the user's actions and schedules. The input is data from each service, and the output is the storage of the collected information in a database.

[0466] Step 2:

[0467] The server analyzes the collected information and automatically updates the user's schedule. Specifically, it uses natural language processing (NLP) to analyze chat content in messenger apps and search results from internet search tools, and reflects this in the schedule. The input is the collected information, and the output is the updated schedule data.

[0468] Step 3:

[0469] The server uses an emotion recognition engine to recognize the user's emotional state. Specifically, it uses Microsoft Azure's Emotion API to analyze emotions from the user's messages and voice data. The input is the user's message and voice data, and the output is the recognized emotional state.

[0470] Step 4:

[0471] The server adjusts the timing and content of reminders according to the emotional state: for example, if the user is feeling stressed, it reduces the frequency of reminders and suggests activities to relax. The inputs are the recognized emotional state and schedule data, and the output is the adjusted reminder information.

[0472] Step 5:

[0473] The server sends work instructions to the robots working in the factory. Specifically, it adjusts the robots' work based on the schedules and emotional states of the workers. The input is the adjusted reminder information and work instructions, and the output is the transmission of instructions to the robots.

[0474] Step 6:

[0475] The robot performs tasks based on the instructions it receives. Specifically, it performs actions according to the schedule and emotional state of the worker in order to improve work efficiency and safety. The input is the instruction from the server, and the output is the performed task.

[0476] Step 7:

[0477] The server performs reminders based on the current time. Specifically, it notifies the user of events and tasks registered in the schedule. The input is the current time and schedule data, and the output is a reminder notification to the user.

[0478] In this way, a system can be realized in which robots working in a factory can understand the schedules and emotional states of workers, improving work efficiency and safety.

[0479] Example 2

[0480] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0481] Conventional virtual assistant systems have difficulty fully understanding a user's behavioral patterns and emotional state and providing reminders and messages at the right time. In addition, they cannot calculate the optimal departure time taking into account weather and traffic information, or generate messages according to the user's emotions, so they cannot fully meet the needs of users.

[0482] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting information on a plurality of services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on procedures at public institutions, and information on family structure, a means for utilizing a virtual assistant of a generative AI that manages daily schedules based on these information, reminds users of procedures at life events and public institutions, and follows up on forgotten schedules with information not registered in the calendar, a means for collecting user behavior data and sending it to the cloud, a means for analyzing the behavior data stored in the cloud and learning behavior patterns using a generative AI model, a means for acquiring weather information and traffic information using an external API, a means for calculating the optimal departure time based on the acquired information and generating a reminder message, and a means for analyzing the user's emotional state and generating a message according to the emotion. This makes it possible to understand the user's behavior patterns and emotional state and provide reminders and messages at the appropriate time.

[0483] "Multiple services" refers to information such as internet search tools, calendars, email, routes, weather, search, and chat content on messenger apps.

[0484] "Procedures at public institutions" refers to various procedures carried out at city halls and government agencies.

[0485] "Family information" refers to information about the user's family members.

[0486] "Schedule management" refers to organizing and managing a user's daily plans.

[0487] A "life event" refers to an important occurrence in a user's life (e.g., a birthday, a wedding, a move, etc.).

[0488] A "remind" refers to a notification that reminds a user of a specific action or schedule.

[0489] A "generative AI virtual assistant" refers to artificial intelligence that learns the user's behavioral patterns based on collected information and provides appropriate services.

[0490] "Behavioral Data" refers to data regarding a user's daily behavior (e.g., location information, time, activity log, etc.).

[0491] "Cloud" refers to remote servers for storing and managing data over the Internet.

[0492] A "generative AI model" refers to an artificial intelligence model that analyzes a user's behavioral patterns and emotional state to generate appropriate messages and reminders.

[0493] "External API" refers to an application programming interface for connecting with other services or databases.

[0494] "Weather information" refers to information regarding current weather and forecasts.

[0495] "Traffic information" refers to information regarding current traffic conditions and forecasts.

[0496] The "optimal departure time" refers to the most suitable departure time for the user to reach the destination.

[0497] "Emotional state" refers to a user's current emotion (e.g., happy, sad, anger, etc.).

[0498] "Message" means any notification or information sent to a User.

[0499] This invention is a generative AI virtual assistant system that learns the user's behavioral patterns and emotional state and provides reminders and messages at appropriate times. A specific embodiment of this system is described below.

[0500] 1. Program Generation

[0501] The server builds a generative AI virtual assistant to learn the user's behavioral patterns. The virtual assistant includes algorithms to collect and analyze the user's behavioral data. The generative AI model used is, for example, GPT-4.

[0502] 2. Program Processing

[0503] The server uses hardware such as smartphones and wearable devices to collect user behavior data. The collected data is stored in a database on the cloud. The server then analyzes this data and uses generative AI models to learn the user's behavior patterns.

[0504] For example, if a user's behavioral pattern is that they eat lunch at a specific restaurant every Friday, the server will obtain real-time weather and traffic information for that day and calculate the optimal departure time. This information will be sent to the user's smartphone as a reminder.

[0505] In another embodiment, the server uses an emotion engine to analyze the user's emotional state. The emotion engine uses natural language processing (NLP) techniques to analyze the user's voice and text messages. For example, if the user is analyzed to be feeling happy, the server generates a message to share the joy and provides it to the user. Conversely, if the user is analyzed to be feeling sad, the server generates a message of comfort and provides it to the user.

[0506] 3. Examples of concrete examples and prompts

[0507] As a concrete example, consider a case where a user has a behavioral pattern of having lunch at "Restaurant A" every Friday. In this case, the server inputs the following prompt sentence into the generative AI model:

[0508] Example prompt:

[0509] "A user has a behavioral pattern of having lunch at Restaurant A every Friday. Today's weather is sunny and the traffic report shows that it is busier than usual. Calculate the optimal departure time and generate a reminder message."

[0510] The generative AI model calculates the optimal departure time based on this prompt and generates a reminder message like this:

[0511] Example of a reminder message:

[0512] "It's sunny today, but traffic is heavy. We recommend that you leave 10 minutes earlier than usual to get to Restaurant A."

[0513] Using the emotion engine, if the user is parsed as feeling happy, the prompt text would be:

[0514] Example prompt:

[0515] "Your users are excited. Generate a message to share their excitement."

[0516] Based on this prompt, the generative AI model would generate a message like this:

[0517] Example message:

[0518] "That's wonderful! I'm glad I could share that joy with you. I hope you continue to have wonderful times."

[0519] In this way, the server realizes a system that provides appropriate information and messages according to the user's behavioral patterns and emotional state.

[0520] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0521] Step 1: Collect user behavior data

[0522] Users collect daily behavioral data using smartphones and wearable devices. Specifically, location information, time, activity logs, etc. are collected. The input is the behavioral data obtained from the user's device, and the output is the collected behavioral data.

[0523] Step 2: Sending behavioral data to the cloud

[0524] The device periodically transmits the collected behavioral data to the cloud server. The data is encrypted and securely transferred. The input is the behavioral data stored on the device, and the output is the behavioral data stored on the cloud server.

[0525] Step 3: Analyze and learn behavioral patterns

[0526] The server analyzes the behavioral data stored in the cloud. It uses a generative AI model (e.g., GPT-4) to learn the user's behavioral patterns. The input is the behavioral data stored in the cloud, and the output is the learned behavioral patterns. Specifically, it recognizes a pattern that the user has lunch at a specific restaurant every Friday.

[0527] Step 4: Get weather and traffic information

[0528] The server obtains real-time weather and traffic information using external APIs, such as the OpenWeatherMap API or a map service API. The input is the weather and traffic information obtained from the external API, and the output is the obtained weather and traffic information.

[0529] Step 5: Calculate the best departure time

[0530] The server calculates the optimal departure time for the user's behavioral patterns based on the acquired weather and traffic information. The inputs are weather information, traffic information, and the learned behavioral patterns, and the output is the optimal departure time. Specifically, if traffic is congested, the calculation is made to depart earlier than usual.

[0531] Step 6: Generate and send a reminder message

[0532] The server uses the generative AI model to generate a reminder message to send to the user. The generated message is sent as a push notification to the user's smartphone. The input is the optimal departure time, and the output is the reminder message. Specifically, the message generated is "Today is sunny, but traffic is congested. We recommend that you leave 10 minutes earlier than usual to head to Restaurant A."

[0533] Step 7: Analyzing the user's emotional state

[0534] The server analyzes the user's voice and text messages and uses an emotion engine to determine the user's emotional state. The input is the user's voice and text messages, and the output is the analyzed emotional state. Specifically, it uses NLP techniques to analyze whether the user is feeling happy or sad.

[0535] Step 8: Generate and send messages based on emotions

[0536] The server generates an appropriate message depending on the user's emotional state. Using a generative AI model, a message of joy or comfort is generated and sent to the user. The input is the analyzed emotional state, and the output is a message according to the emotion. Specifically, the message generated is "That's wonderful! I'm glad you could share your joy with me. I hope you continue to have a wonderful time."

[0537] (Application example 2)

[0538] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0539] In modern society, users lead busy daily lives and require appropriate services such as schedule management, reminders of life events, and emotional responses. However, conventional systems have difficulty in comprehensively satisfying these needs, and have not realized optimal service provision based on the user's behavioral patterns and emotional states. In particular, in food delivery, there is a demand for reminders of optimal ordering times that take into account the user's past ordering history and behavioral patterns, and for messages that correspond to emotions, but no system exists that can achieve this.

[0540] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0541] In this invention, the server includes a means for utilizing a generative AI virtual assistant to collect information on a plurality of services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on procedures at public institutions, and information on family composition, and to manage daily schedules based on this information, reminders of life events and procedures at public institutions, and follow up on forgotten schedules using information not registered in the calendar, a means for learning the user's behavioral patterns based on the collected information and providing services according to the user's needs, a means for learning the user's past order history and behavioral patterns, and reminding the user of the optimal order time taking into account weather and traffic information, and a means for generating a message according to the user's emotional state and providing it to the user. This makes it possible to provide optimal services based on the user's behavioral patterns and emotional state.

[0542] A "generative AI virtual assistant" is an artificial intelligence system that learns a user's behavioral patterns and emotional state based on collected information, and provides the user with the most optimal services.

[0543] A "behavioral pattern" refers to a user's tendency to repeat a particular behavior at a particular time or on a particular day of the week.

[0544] An "emotional state" refers to the emotion (happiness, sadness, etc.) that a user is feeling at a particular moment.

[0545] A "remind" is a notification or alert sent to a user to encourage them to take a specific action.

[0546] "Weather information" is data about current and future weather conditions.

[0547] "Traffic information" is data about current and future traffic conditions.

[0548] "Order history" is a record of orders a user has placed in the past.

[0549] "Message generation" refers to creating an appropriate message based on the user's emotional state and behavioral patterns.

[0550] "Schedule management" refers to organizing and efficiently managing a user's plans and tasks.

[0551] A "life event" refers to an important event in a user's life (marriage, moving, birthday, etc.).

[0552] "Public institution procedures" refers to administrative procedures and applications that users must carry out.

[0553] The system for implementing this invention utilizes a generative AI virtual assistant that learns the behavioral patterns and emotional state of the user and provides optimal services. A specific embodiment of this system is described below.

[0554] System Configuration

[0555] The system consists of a server, user devices (smartphones, tablets, PCs, etc.), and various APIs (weather information API, traffic information API, etc.). The server runs a generative AI model and learns the user's behavioral patterns and emotional state. The user device receives input from the user and communicates with the server to send and receive information.

[0556] Program Processing

[0557] The server performs the following process.

[0558] 1. Information collection

[0559] In order to learn the user's behavioral patterns and emotional state, the server collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on procedures at public institutions, and information on family composition. This also includes obtaining weather and traffic information using various APIs.

[0560] 2. Learning behavioral patterns

[0561] The server learns user behavior patterns from the collected information, for example recognizing that users tend to order certain dishes on certain days of the week or at certain times of the day.

[0562] 3. Recognizing Emotional States

[0563] The server uses an emotion recognition engine (EmotionRecognizer) to recognize the user's emotional state, thereby understanding whether the user is happy, sad, etc.

[0564] 4. Providing optimal services

[0565] The server provides optimal services to users based on the learned behavioral patterns and the recognized emotional state. For example, if a user has a tendency to order sushi every Friday, and the weather is predicted to be rainy and traffic is heavy on Friday morning, the system will remind the user, "It's raining today and traffic is heavy, so we recommend that you order early." Also, if the system recognizes that the user is happy, it will send a message saying, "Would you like to try our special dessert today?"

[0566] Hardware and Software Used

[0567] Hardware: Servers, smartphones, tablets, PCs

[0568] Software: Python, EmotionRecognizer, Weather API, Traffic API

[0569] Examples

[0570] For example, if a user tends to order sushi every Friday, and the weather is predicted to be rainy and traffic is heavy on Friday morning, the system will remind the user, "It's raining today and traffic is heavy, so we recommend that you order early." Also, if the user is recognized as happy, the system will send a message saying, "Would you like to try our special dessert today?"

[0571] Examples of prompt statements

[0572] Build a food delivery application that learns the user's behavioral patterns based on their past ordering history, and reminds them of the best time to order, taking into account weather and traffic information. Also add the ability to generate messages based on the user's emotional state.

[0573] The flow of the identification process in the application example 2 will be described with reference to FIG.

[0574] Step 1:

[0575] The server receives input from the user's device and collects information on multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on procedures at public institutions, and information on family composition. This also includes obtaining weather and traffic information using various APIs. The input is the user's behavioral history and emotional data, and the output is a dataset of the collected information.

[0576] Step 2:

[0577] The server learns user behavior patterns based on the collected information. Specifically, it analyzes past order history and schedule data to recognize tendencies to repeat certain actions on certain days of the week or at certain times of the day. The input is a dataset of collected information, and the output is a model of the user's behavior patterns.

[0578] Step 3:

[0579] The server uses an emotion recognition engine (EmotionRecognizer) to recognize the user's emotional state. Specifically, it analyzes the chat content and voice data of the messenger app to understand the user's emotions, such as whether they are happy or sad. The input is the user's emotional data, and the output is the recognized emotional state.

[0580] Step 4:

[0581] The server provides optimal services to users based on the learned behavioral patterns and the recognized emotional state. Specifically, if a user has a tendency to order sushi every Friday, and the weather is predicted to be rainy and traffic is heavy on Friday morning, the system will remind the user, "It's raining today and traffic is heavy, so we recommend that you order early." Also, if the system recognizes that the user is happy, it will send a message saying, "Would you like to try a special dessert today?" The input is a model of the behavioral pattern and the emotional state, and the output is a reminder message or a suggestion message.

[0582] Step 5:

[0583] The user device receives the reminder message or suggestion message sent from the server and notifies the user. Specifically, it is displayed as a push notification on the smartphone, an email, or a message on a messenger app. The input is the message from the server, and the output is the notification to the user.

[0584] Step 6:

[0585] The user takes action based on the reminder or suggestion messages he receives, for example, ordering food delivery early or trying a suggested dessert. The input is the notification given to the user, and the output is the user's action.

[0586] Example 3

[0587] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0588] In modern society, users need to manage notifications and reminders from many sources. In particular, information about life events and administrative procedures is important, yet easy to forget. There is also a demand for appropriate services to be provided according to the user's emotional state, but current systems have difficulty doing this automatically. To solve these issues, a system is needed that can collect and analyze user information and provide appropriate reminders and services.

[0589] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means. In this invention, the server includes a means for collecting information on a plurality of services, information on administrative procedures, and information on family structure, a means for analyzing the collected information, a means for generating reminders for life events and administrative procedures based on the analysis results, a means for notifying the user of the generated reminders, a means for analyzing the user's emotional state, and a means for providing services based on the analyzed emotional state. This allows the user to receive appropriate services according to their emotional state without forgetting important life events or administrative procedures.

[0590] "Multiple services" refers to various information sources that users use on a daily basis, such as Internet search tools, calendars, email, route information, weather information, search engines, and chat content on messenger apps.

[0591] "Administrative procedures" refers to all procedures carried out at city halls and other government agencies, such as birth registration, marriage registration, and address change procedures.

[0592] "Family information" refers to information about the user's family, such as information about events such as birth, marriage, divorce, and death.

[0593] "Means of collection" refers to the methods and technologies used to obtain information from users, including, for example, using an API to obtain messages from a messenger app.

[0594] "Means of analysis" refers to methods and techniques for analyzing collected information and understanding its meaning or intent, such as using a natural language processing engine to analyze text.

[0595] "Means of generating reminders" refers to methods or technologies for creating reminders to notify users based on the analysis results, including, for example, using a generative AI model to generate the content of the reminder.

[0596] "Means of notifying" refers to the method or technology for notifying the user of the generated reminder, and includes, for example, sending a notification using a mail server or a messenger app.

[0597] "Means for analyzing emotional state" refers to methods or techniques for analyzing collected information to understand the user's emotions, including, for example, using an emotion engine to analyze the user's emotions.

[0598] The "means for providing a service" refers to a method or technology for providing an appropriate service to a user based on the analyzed emotional state, such as suggesting an activity for relaxation.

[0599] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0600] This invention is a system that automatically generates reminders for a user's life events and administrative procedures and notifies the user. It also has a function to adjust the timing and content of reminders and the method of providing services according to the user's emotional state.

[0601] Hardware and Software Used

[0602] Hardware: Server (e.g. cloud server)

[0603] software:

[0604] Natural language processing engines (e.g. natural language processing APIs)

[0605] Sentiment engines (e.g. sentiment analysis APIs)

[0606] Messenger app API (e.g. Messenger API)

[0607] Mail server (e.g. email sending service)

[0608] Generative AI models (e.g. generative AI models)

[0609] Explanation of program processing

[0610] The server collects messages sent by the user via email or messenger apps. For example, if a user sends a message via messenger app saying "Our baby is born!", the server retrieves this message using the messenger API.

[0611] The server then uses a natural language processing engine to analyze the collected messages, for example extracting the event "birth" from the message "A baby was born."

[0612] The server generates reminders for related administrative procedures based on the analysis results. For example, for the event "birth," it generates a reminder to "submit birth notification." The specific content of this reminder is created using a generative AI model.

[0613] The server notifies the user of the generated reminder, for example by sending the reminder to the user's email address using an email sending service, or by sending a notification to a messenger app using the messenger API.

[0614] In addition, the server uses the emotion engine to analyze the user's emotional state. For example, if a user says "I'm very stressed today" in a messenger app, the server sends this message to the emotion engine, which analyzes the emotion as "anger."

[0615] The server provides appropriate services based on the analyzed emotional state. For example, if the emotion "anger" is analyzed, the server generates a message suggesting an activity to relax (e.g., deep breathing exercises) and notifies the user. This message is also created using a generative AI model.

[0616] Examples

[0617] For example, if a user sends a message on a messenger app saying "Our baby was born!", the server will take this message and use a natural language processing engine to extract the event "birth". It will then use a generative AI model to generate a reminder to "submit birth registration" and notify the user using an email sending service.

[0618] Also, if a user says in the Messenger app, "I'm so stressed today," the server can use the emotion engine to analyze the emotion "anger," use the generative AI model to generate a message suggesting an activity to help the user relax, and notify the user using the Messenger API.

[0619] Examples of prompt statements

[0620] "When a user shares a change in family structure, generate a reminder for the relevant administrative procedures."

[0621] "If you sense that the user is feeling angry, generate a message that suggests an activity to calm the user."

[0622] In this way, the server realizes a system that provides reminders and services according to the user's life events and emotional state based on the collected information. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0623] Program processing flow

[0624] Step 1: Gather information

[0625] The server collects messages sent by the user via email or messenger apps. For example, if a user sends a message via messenger app saying "Our baby is born!", the server retrieves this message using the messenger API.

[0626] Input: Messages sent by users in the Messenger app

[0627] Output: Retrieved message data

[0628] Specific operation: The server calls the Messenger API and retrieves the user's messages.

[0629] Step 2: Analyze the information

[0630] The server uses a natural language processing engine to analyze the collected messages, for example extracting the event "birth" from the message "A baby was born."

[0631] Input: Retrieved message data

[0632] Output: Parsed event information (e.g. "birth")

[0633] Specific operation: The server sends message data to a natural language processing engine to extract event information.

[0634] Step 3: Generate a reminder

[0635] The server generates reminders for related administrative procedures based on the analysis results. For example, for the event "birth," it generates a reminder to "submit birth notification." The specific content of this reminder is created using a generative AI model.

[0636] Input: Parsed event information (e.g. "birth")

[0637] Output: The generated reminder (e.g. "Submit birth registration")

[0638] Specific operation: The server inputs event information into the generative AI model and generates reminder content.

[0639] Step 4: Reminder notifications

[0640] The server notifies the user of the generated reminder. For example, it can send the reminder to the user's email address using an email sending service, or it can send a notification to a messenger app using the messenger API.

[0641] Input: The generated reminder

[0642] Output:Notification to the user

[0643] Specific operation: The server uses an email sending service or messenger API to notify the user of the reminder.

[0644] Step 5: Analyze the emotional state

[0645] The server uses the emotion engine to analyze the user's emotional state. For example, if a user says "I'm very stressed today" in a messenger app, the server sends this message to the emotion engine, which analyzes the emotion as "anger."

[0646] Input: User's message

[0647] Output: Analyzed emotion information (e.g. "anger")

[0648] Specific operation: The server sends a message to the emotion engine and analyzes the emotion information.

[0649] Step 6: Deliver emotional service

[0650] The server provides appropriate services based on the analyzed emotional state. For example, if the emotion "anger" is analyzed, the server generates a message suggesting an activity to relax (e.g., deep breathing exercises) and notifies the user. This message is also created using a generative AI model.

[0651] Input: Analyzed emotion information (e.g. "anger")

[0652] Output: Proposed message

[0653] How it works: The server inputs emotion information into the generative AI model to generate a suggested message, then notifies the user using the Messenger API.

[0654] In this way, the server realizes a system that provides reminders and services based on the collected information according to the user's life events and emotional state.

[0655] (Application example 3)

[0656] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0657] In modern society, users tend to forget to be reminded of life events and administrative procedures in their busy daily lives. In addition, due to a lack of timely notifications based on the user's emotional state and a lack of information on special offers and discounts at physical stores, user convenience is reduced. To solve these problems, a system is needed that automatically provides reminders and services based on the user's life events and emotional state.

[0658] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0659] In this invention, the server includes a means for utilizing a generative AI virtual assistant to collect information on a plurality of services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on city hall and administrative procedures, and information on family composition, and manage daily schedules based on this information, reminders of life events and administrative procedures, and follow up on forgotten schedules using information not registered in the calendar, a means for recognizing the user's emotional state and adjusting the timing and content of reminders according to the emotional state, a means for automatically generating and notifying reminders of products and services related to the user's life events, and a means for providing special offers and discount information at physical stores to motivate the user to visit the stores. This allows the user to receive reminders of life events and administrative procedures at the appropriate time, and by utilizing special offers and discount information at physical stores, convenience can be improved.

[0660] A "generative AI virtual assistant" is an artificial intelligence system that automatically manages the user's schedule and provides reminders based on collected information.

[0661] "Emotional state" refers to the user's current psychological state or emotion, including emotions such as anger, surprise, and joy.

[0662] A "remind" is an action of notifying a user of a specific event or procedure, urging them not to forget.

[0663] A "life event" refers to an important occurrence or milestone in a user's life, such as a birthday, wedding, or birth.

[0664] A "physical store" refers to a commercial facility or store that exists in a physical location, where users visit in person to purchase goods or services.

[0665] "Benefits" refers to additional benefits or services provided to users, such as discounts or gifts.

[0666] "Discount information" refers to information in which the price of a product or service is set lower than usual, providing an economic benefit to the user.

[0667] "Motivation" refers to stimuli or incentives to encourage users to take a certain action, such as encouraging them to visit a physical store by offering special offers or discount information.

[0668] The system for implementing this invention uses multiple hardware and software components to provide reminders and services based on a user's life events and emotional state, including a smartphone, a server, a generative AI model, an emotion recognition engine, and an email sending system.

[0669] The server collects information such as internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, city hall and administrative procedures, and family structure information. Based on this information, a generative AI model is used to manage daily schedules, remind people of life events and administrative procedures, and follow up on forgotten schedules that are not registered in the calendar.

[0670] In addition, the server uses an emotion recognition engine to recognize the user's emotional state and adjusts the timing and content of reminders according to the emotional state. For example, if the user is feeling angry, the server will suggest a relaxing activity, and if the user is feeling surprised, the server will generate a message to share the surprise.

[0671] The server uses a generative AI model to automatically generate and notify users of product and service reminders related to their life events. For example, if a user's birthday is approaching, the server generates a message saying, "Happy birthday! Are you looking for a special gift?" If the user's emotional state is "surprise," the server adds, "We have a surprise for you!"

[0672] In order to provide special offers and discount information at physical stores and motivate users to visit the stores, the server collects special offers and discount information and notifies the user. For example, when a user celebrates their wedding anniversary, the server generates a message saying, "Congratulations on your wedding anniversary! How about making a reservation for a special dinner?" and provides special offer information.

[0673] As an illustration, consider the following prompt:

[0674] "If the user's emotional state is 'surprise', add 'We've got a surprise for you!' to the birthday reminder message."

[0675] In this way, users can receive reminders about life events and administrative procedures at the appropriate time, and can also take advantage of special offers and discount information at physical stores, improving convenience.

[0676] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0677] Step 1:

[0678] The server collects information such as internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, city hall and administrative procedures, and family structure information. This information is stored in a database and used as basic data for understanding the user's life events and schedules.

[0679] Input: Internet search tools, calendar, email, routes, weather, search, chat content in messenger apps, city hall / administrative procedure information, family composition information

[0680] Output: A database of the user's life events and schedules.

[0681] Step 2:

[0682] The server uses a generative AI model based on the collected information to manage daily schedules, remind users of life events and government procedures, and follow up on forgotten schedules using information not registered in the calendar. The generative AI model learns the user's behavioral patterns and generates appropriate reminders.

[0683] Input: A database of the user's life events and schedules

[0684] Output:Reminder message

[0685] Step 3:

[0686] The server recognizes the user's emotional state using an emotion recognition engine that analyzes the user's voice and text data to identify the user's current emotional state.

[0687] Input: User's voice data, text data

[0688] Output: The user's emotional state.

[0689] Step 4:

[0690] The server adjusts the timing and content of reminders according to the user's emotional state: for example, if the user is feeling angry, it suggests a relaxing activity, and if the user is feeling surprised, it generates a message to share the surprise.

[0691] Input: Reminder message, user's emotional state

[0692] Output: Tailored reminder message

[0693] Step 5:

[0694] The server automatically generates and notifies the user of reminders of products and services related to life events. For example, when a user's birthday approaches, it generates a message saying, "Happy birthday! Looking for a special gift?"

[0695] Input: A database of the user's life events and schedules

[0696] Output: Product or service reminder message

[0697] Step 6:

[0698] The server provides special offers and discount information at physical stores to motivate users to visit the stores. For example, when a user is celebrating their wedding anniversary, the server generates a message saying "Congratulations on your wedding anniversary! Would you like to make a reservation for a special dinner?" and provides special offer information.

[0699] Input: A database of user's life events and schedules, special offers and discounts

[0700] Output: Reminder message with special offers and discounts

[0701] Step 7:

[0702] The server sends the generated reminder message to the user's smartphone. By receiving the notification, the user can check the life event and benefit information and take the necessary steps.

[0703] Input: Tailored reminder message

[0704] Output: Notification to the user's smartphone

[0705] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0706] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0707] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0708] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0709] [Second embodiment]

[0710] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0711] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0712] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0713] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0714] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0715] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0716] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0717] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0718] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0719] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0720] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0721] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0722] "Example 1"

[0723] The system of the present invention has a means for collecting information from multiple services used by the user (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.). For example, if a user chats on a messenger app saying, "I have a meeting tomorrow at 1pm," this information is collected and automatically registered in the calendar. Also, if a user searches for "children's school admission procedures" on an Internet search tool, this information is collected and a reminder for related government procedures is set.

[0724] "Example 2"

[0725] The system of the present invention has a generative AI virtual assistant that learns the user's behavioral patterns based on collected information. For example, if the user learns that they have lunch at a specific restaurant every Friday, the system will remind them of the best time to leave, taking into account the weather and traffic information for that day.

[0726] "Example 3"

[0727] The system of the present invention has a means for automatically generating reminders for the user's life events and administrative procedures based on collected information and notifying the user. For example, when a user shares a change in family structure (e.g., birth, marriage, etc.) via email or messenger app, this information is collected, and a reminder for the related administrative procedures is automatically generated and notified to the user.

[0728] The process flow of each embodiment will be described below.

[0729] "Example 1"

[0730] Step 1: A user talks on a messenger app saying, "I have a meeting tomorrow at 1pm."

[0731] Step 2: The system of the present invention collects this information and automatically enters it into your calendar.

[0732] Step 3: The system of the present invention sends a reminder before the meeting time.

[0733] "Example 2"

[0734] Step 1: A user indicates a behavioral pattern of having lunch at a particular restaurant every Friday.

[0735] Step 2: Our system learns this behavior pattern.

[0736] Step 3: The system of the present invention will take into account the weather and traffic conditions for the day and remind you of the best time to leave.

[0737] "Example 3"

[0738] Step 1: The user shares a change in family structure (e.g. birth, marriage, etc.) via email or messenger app.

[0739] Step 2: The system of the present invention collects this information.

[0740] Step 3: The system of the present invention automatically generates a reminder of the relevant administrative procedure and notifies the user.

[0741] Example 1

[0742] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0743] Modern users use multiple services (Internet search tools, calendars, e-mail, traffic information, weather information, search engines, messaging app communications, etc.), and are required to efficiently manage the information obtained from each service. However, manually collecting this information, analyzing it, and determining the appropriate action is extremely time-consuming. It is also difficult to learn users' behavioral patterns and provide services that meet their needs. Another challenge is automatically generating reminders for life events and government procedures and notifying users of them.

[0744] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for collecting information from multiple services, a means for analyzing the collected information, a means for determining an action based on the analysis result, and a means for executing the determined action. This makes it possible to efficiently manage information obtained from multiple services used by the user, learn the user's behavioral patterns, and provide services according to the needs. In addition, it is possible to automatically generate reminders for life events and administrative procedures and notify the user.

[0745] "Multiple services" refers to multiple information sources that users use on a daily basis, such as internet search tools, calendars, email, traffic information, weather information, search engines, and messaging app communications.

[0746] "Means of collecting information" refers to the function of automatically acquiring information such as user activities and communication content by utilizing the API or data acquisition function of each service.

[0747] "Means of analyzing information" refers to the function of analyzing collected information using natural language processing and data analysis techniques and extracting important information.

[0748] "Means for determining an action" refers to a function that automatically determines the next action to be taken based on the analysis results.

[0749] "Means for executing an action" refers to a function for actually executing a determined action, such as adding an event to a calendar or sending a reminder.

[0750] "Learning user behavior patterns" refers to the function of analyzing a user's past behavior and tendencies based on collected information and predicting future behavior.

[0751] "Providing services that meet the needs" refers to the function of providing optimal information and actions based on the user's behavioral patterns and current situation.

[0752] A "life event" refers to an important event in a user's life (e.g., a birthday, a wedding, a child's admission to school, etc.).

[0753] "Administrative procedures" refers to various official procedures that users must carry out (e.g. obtaining a resident registration card, filing taxes, etc.).

[0754] "Automatically generating reminders" refers to a function that automatically creates necessary notifications and reminders for users based on collected and analyzed information.

[0755] "Notify user" refers to the function of notifying the user of generated reminders and important information via email, calendar notifications, or other means.

[0756] This invention is a system that collects information from multiple services used by a user (Internet search tools, calendars, email, traffic information, weather information, search engines, messaging app communication content, etc.), analyzes this information to learn the user's behavioral patterns, and provides services according to the user's needs.

[0757] Generating a Program

[0758] The server generates a program to collect information from each service used by the user. This program is developed using a programming language such as Python or Java. Specifically, the program obtains data using the API of each service and analyzes it using natural language processing (NLP) technology.

[0759] Program Processing

[0760] The server performs data processing and calculation using the following hardware and software.

[0761] Hardware:

[0762] The server is a computer system equipped with a high-performance processor and sufficient memory, such as an Intel Xeon processor or AMD EPYC processor.

[0763] software:

[0764] The server uses the following software to collect and analyze information:

[0765] API: API of each service (e.g. Google Calendar API, Microsoft Outlook API, Gmail API, Google Search Console API, LINE API, WhatsApp API)

[0766] Natural Language Processing (NLP) Tools: Google Cloud Natural Language API, IBM Watson Natural Language Understanding

[0767] Email sending services: Twilio SendGrid, Amazon SES

[0768] Examples

[0769] Example 1: Processing chat content in a messenger app:

[0770] When a user says "I have a meeting tomorrow at 1pm" in a messaging app, the server retrieves the contents of the chat through the messaging app's API. The server then uses the Google Cloud Natural Language API to extract the information "tomorrow at 1pm" and "meeting". The server then automatically registers this information as a "meeting" in the calendar using the Google Calendar API.

[0771] Example 2: Processing Internet search history:

[0772] When a user searches for "child's admission procedure" in an internet search tool, the server retrieves this search history using the Google Search Console API. Then, it analyzes the search query and detects the keyword "child's admission procedure". Based on this information, the server sets up a reminder for the relevant administrative procedure. Specifically, it uses Twilio SendGrid to send a reminder email to the user.

[0773] Examples of prompt statements

[0774] "If a user says 'I have a meeting tomorrow at 1pm' in a messaging app, please create a program that collects this information and automatically registers it in Google Calendar. Also, if a user searches for 'child's school admission procedures' in an internet search tool, please create a program that collects this information and sets a reminder for the related administrative procedures."

[0775] In this way, the server provides a system for analyzing user behavior and automatically collecting and processing necessary information.

[0776] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0777] Step 1:

[0778] The server collects information from each service used by the user. Specifically, it obtains data using the API of each service. For example, it obtains the contents of a user's chat using the API of a messaging app, and obtains search history using the API of an internet search tool. The input is the data obtained from the API of each service, and the output is the collected raw data.

[0779] Step 2:

[0780] The server analyzes the collected information. Specifically, it uses natural language processing (NLP) technology to analyze text data and extract important information. For example, it uses Google Cloud Natural Language API to extract dates and times and event details from chat content in a messaging app. The input is the collected raw data, and the output is analyzed information (e.g., dates and times, event details).

[0781] Step 3:

[0782] The server decides on an action based on the analysis results. Specifically, it determines the next action to be taken based on the extracted information. For example, it decides on the action of registering an event in a calendar based on the extracted date and time and event content. The input is the analyzed information, and the output is the determined action (e.g., registering the event in a calendar).

[0783] Step 4:

[0784] The server executes the determined action. Specifically, it executes the action using the API of each service. For example, it uses the Google Calendar API to automatically register the extracted event in the calendar. It also uses Twilio SendGrid to send a reminder email. The input is the determined action, and the output is the result of the executed action (e.g., the event registered in the calendar, the reminder email sent).

[0785] Step 5:

[0786] The server learns the user's behavioral patterns. Specifically, it analyzes the user's past behavior and tendencies based on the collected and analyzed information, and predicts future behavior. The input is the collected and analyzed information, and the output is the learned behavioral patterns.

[0787] Step 6:

[0788] The server provides services according to the user's needs. Specifically, it provides optimal information and actions to the user based on the learned behavioral patterns. For example, it provides related information based on search queries frequently performed by the user in the past. The input is the learned behavioral patterns, and the output is the provided service (e.g., providing related information).

[0789] In this way, the server analyzes user behavior, automatically collects and processes necessary information, and realizes a system that provides users with the most optimal services.

[0790] (Application example 1)

[0791] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0792] In modern life, users use many services and applications, and managing each piece of information individually is cumbersome. In addition, schedule management and reminder settings are often done manually, which creates the risk of forgetting important appointments or procedures. Furthermore, reservations for transportation and schedule settings must also be done manually, making efficient time management difficult.

[0793] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0794] In this invention, the server includes a means for utilizing a generative AI virtual assistant to collect information on multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on city hall and administrative procedures, and information on family composition, and manage daily schedules based on this information, remind life events and administrative procedures, and follow up on forgotten schedules using information not registered in the calendar, a means for automatically setting a schedule for an autonomous vehicle based on the collected information, and a means for automatically reserving travel to a destination based on the schedule for the autonomous vehicle. This allows the user to centrally manage information from multiple services, and efficiently travel using an autonomous vehicle without forgetting important plans or procedures.

[0795] "Multiple services" refers to different information sources such as internet search tools, calendars, email, routes, weather, search, and chat content in messenger apps.

[0796] "City Hall / Administrative Procedures" refers to various administrative procedures and applications that Users must complete.

[0797] "Family information" refers to information about the user's family, such as the names, ages, and relationships of family members.

[0798] "Daily schedule management" refers to organizing and managing a user's daily plans.

[0799] A "life event" refers to an important event in a user's life, such as marriage, childbirth, or moving.

[0800] A "remind" refers to a notification to a user so that the user does not forget a specific schedule or procedure.

[0801] "Information not registered in the calendar" refers to important appointments and events that the user has not entered into the calendar.

[0802] A "generative AI virtual assistant" refers to a virtual assistant that uses artificial intelligence to collect information about the user and manage the schedule and provide reminders.

[0803] An "autonomous vehicle" refers to a vehicle that drives automatically using artificial intelligence and sensor technology.

[0804] "Automatically setting a schedule" refers to automatically determining and managing a user's plans based on collected information.

[0805] "Automatically reserving travel to a destination" refers to reserving an autonomous vehicle based on the user's schedule and arranging travel to the destination.

[0806] As an embodiment of the present invention, the following system is constructed.

[0807] System Overview

[0808] The system collects information from multiple services used by users, manages schedules and sets reminders using a generative AI virtual assistant, and automatically sets schedules for autonomous vehicles based on the collected information, automatically booking trips to destinations.

[0809] Hardware and software used

[0810] Hardware: Smartphones, self-driving cars

[0811] Software: Messenger app, Google Calendar API, Autonomous Vehicle API

[0812] Data processing and calculation

[0813] Getting messages

[0814] The device (smartphone) receives the user's messages from the messenger app. These messages include information about the user's schedule and events.

[0815] Extracting Information

[0816] The server uses natural language processing technology to extract information about schedules and events from the received messages. For example, from a message such as "I have a meeting tomorrow at 1pm," the server extracts the information "meeting," "tomorrow," and "13:00."

[0817] Calendar registration

[0818] The server uses the extracted information to automatically register the event on the user's calendar using the Google Calendar API.

[0819] Scheduling autonomous vehicles

[0820] The server automatically schedules autonomous vehicles using the Autonomous Vehicle API based on event information registered in the calendar. For example, it reserves an autonomous vehicle to travel to the meeting location at the start time of the meeting.

[0821] Examples

[0822] If a user says in a messenger app, "I have a meeting tomorrow at 1 p.m.," the system collects that information, registers the "meeting" on Google Calendar, and even books an autonomous vehicle to head to the meeting location at 1 p.m.

[0823] Examples of prompt statements

[0824] If a user says in a messenger app, "I have a meeting tomorrow at 1pm," create a program that collects this information, automatically registers the "meeting" in Google Calendar, and even books an autonomous vehicle to head to the meeting location at 1pm.

[0825] In this way, a system can be provided that makes users' lives more convenient.

[0826] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0827] Step 1:

[0828] The device (smartphone) retrieves the user's messages from the messenger app. The input is the user's message, and the output is the text data of the retrieved message. The specific operation is to retrieve the latest message using the messenger app's API.

[0829] Step 2:

[0830] The server uses natural language processing technology to extract information about schedules and events from the acquired messages. The input is the text data of the acquired messages, and the output is the extracted schedule or event information (e.g., "Meeting," "Tomorrow," "13:00"). Specifically, the server analyzes the messages using a natural language processing library (e.g., spaCy or NLTK) and extracts keywords and date and time information.

[0831] Step 3:

[0832] The server uses the Google Calendar API based on the extracted information to automatically register events in the user's calendar. The input is the extracted schedule and event information, and the output is the event registered in Google Calendar. Specifically, the server calls the Google Calendar API and registers the extracted information as an event.

[0833] Step 4:

[0834] The server automatically sets the schedule for autonomous vehicles using the Autonomous Vehicle API based on the event information registered in the calendar. The input is the event information registered in Google Calendar, and the output is the reservation information for the autonomous vehicle. Specifically, the server calls the Autonomous Vehicle API and reserves an autonomous vehicle based on the start time and location of the event.

[0835] Step 5:

[0836] The server notifies the user of the reserved autonomous vehicle information. The input is the reservation information for the autonomous vehicle, and the output is a notification message to the user. Specifically, the server uses the API of the messenger app to send the reservation information to the user.

[0837] In this way, a series of processes is realized: extracting plans from the user's messages, registering them on the calendar, and reserving an autonomous vehicle.

[0838] Example 2

[0839] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0840] Conventional virtual assistant systems have difficulty learning the user's behavioral patterns sufficiently and reminding the user at the appropriate time. In addition, they cannot calculate the optimal departure time taking into account weather and traffic information, and therefore cannot effectively support the user's life. For this reason, there is a demand for a system that can learn the user's behavioral patterns, remind the user at the appropriate time, and calculate the optimal departure time taking into account weather and traffic information.

[0841] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0842] In this invention, the server includes a means for collecting information on a plurality of services, information on procedures, and information on family structure, a means for utilizing a virtual assistant of a generative AI that manages daily schedules based on the information, reminds users of life events and procedures, and follows up on forgotten schedules using information not registered in a calendar, a means for collecting user behavior data and storing it in a database, a means for learning user behavior patterns using a generative AI model based on the collected behavior data, a means for predicting reminder timing based on the learned behavior patterns, a means for acquiring weather information and traffic information based on the predicted date and time, a means for calculating an optimal departure time based on the acquired information and generating a reminder message, and a means for notifying the generated reminder message to the user. This makes it possible to learn the user's behavior patterns, perform reminders at appropriate times, and calculate the optimal departure time taking into account weather information and traffic information.

[0843] The "multiple services" refer to various online services that provide information such as internet search tools, calendars, email, routes, weather, search, and chat content in messenger apps.

[0844] "Information on procedures, etc." refers to information on various procedures carried out at city halls and administrative agencies.

[0845] "Family information" refers to information about the user's family members, including, for example, the number of family members and their relationships.

[0846] A "generative AI virtual assistant" is an artificial intelligence system that learns the user's behavioral patterns based on collected information and provides the user with appropriate reminders and services.

[0847] "User behavior data" refers to information such as the date and time and the location where a user performed a particular behavior.

[0848] A "database" is an information management system for storing collected user behavior data.

[0849] A "generative AI model" is an artificial intelligence model that learns user behavior patterns based on collected data.

[0850] The "reminding timing" is the optimal date and time for reminding the user.

[0851] "Weather information" is information about the weather at a specific date and time.

[0852] "Traffic information" refers to information on traffic conditions at a specific date and time.

[0853] The "optimal departure time" is the most appropriate departure time for the user to reach the destination, calculated taking into account weather and traffic information.

[0854] A "remind message" is a message that is notified to a user to encourage the user to take action.

[0855] This invention relates to a generative AI virtual assistant system that learns a user's behavioral patterns and reminds the user at appropriate times. A specific embodiment of this system will be described below.

[0856] System Configuration

[0857] This system is configured using the following hardware and software.

[0858] Hardware: Servers, user devices (smartphones and PCs)

[0859] Software: Generative AI models (e.g. GPT-4), database management systems (e.g. MySQL), weather APIs (e.g. OpenWeatherMap), traffic APIs (e.g. Google Maps API)

[0860] Data collection and storage

[0861] The server collects user behavior data and stores it in a database. For example, it records information that a user has lunch at a specific restaurant every Friday. This accumulates basic data for understanding the user's behavioral patterns.

[0862] Learning behavioral patterns

[0863] The server inputs the collected behavioral data into a generative AI model to learn the user's behavioral patterns. The generative AI model can perform advanced pattern recognition and prediction, for example by using GPT-4, to understand the tendency to take certain actions on certain days of the week and at certain times of the day.

[0864] Reminder timing prediction

[0865] Based on the learned behavioral patterns, the server predicts the next time the user is likely to take a particular action, for example, to go to a particular restaurant next Friday.

[0866] Get weather and traffic information

[0867] Based on the predicted date and time, the server uses weather information APIs (e.g. OpenWeatherMap) and traffic information APIs (e.g. Google Maps API) to obtain the weather and traffic conditions for that day, allowing it to take into account external factors that affect the user's behavior.

[0868] Calculate the best departure time

[0869] The server calculates the optimal departure time for the user based on the acquired weather and traffic information. For example, if it is a rainy day and traffic is congested, it calculates an earlier departure time than usual.

[0870] Generate and send reminder messages

[0871] The server generates a reminder message to inform the user based on the calculated optimal departure time, for example, "It's raining and traffic is heavy today, so we recommend that you leave 15 minutes earlier than usual."

[0872] The terminal notifies the user of the generated reminder message, for example, by a push notification on a smartphone or by email.

[0873] Examples

[0874] Suppose a user has a behavioral pattern of having lunch at "Restaurant A" every Friday. This information is stored on the server. On Friday morning, the server uses the generative AI model to predict that the user is likely to go to "Restaurant A" that day as well. The server uses the OpenWeatherMap API to get weather information for the day and the Google Maps API to get traffic information. For example, if the weather is rainy and traffic is congested that day, the server generates a reminder message to the user saying, "It's raining and traffic is congested today, so we recommend that you leave 15 minutes earlier than usual."

[0875] Examples of prompt statements

[0876] "Design a system that learns a user's behavioral pattern of having lunch at a particular restaurant every Friday and then reminds them of the best time to leave, taking into account the weather and traffic conditions for that day."

[0877] In this way, the system learns the user's behavioral patterns and supports the user's daily life by reminding them at appropriate times.

[0878] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0879] Step 1:

[0880] Collecting user behavior data

[0881] Input: User behavior information (date, time, location, and content of the action)

[0882] Specific operation: When a user performs a specific action, that information is sent to the server. For example, information that a user checked in to "Restaurant A" at 12:00 on Friday is sent from the smartphone to the server.

[0883] Output: Collected behavioral data is stored in a database.

[0884] Step 2:

[0885] Learning behavioral patterns

[0886] Input: Collected behavioral data

[0887] Specific operation: The server inputs the collected behavioral data into a generative AI model (e.g., GPT-4) to learn the user's behavioral patterns. For example, it learns a pattern that "the user has lunch at 'Restaurant A' every Friday at 12:00."

[0888] Output: A model of the learned behavioral patterns.

[0889] Step 3:

[0890] Reminder timing prediction

[0891] Input: A model of learned behavioral patterns

[0892] Specific operation: Based on the learned behavioral patterns, the server predicts the next time the user is likely to take a certain action. For example, it predicts that the user is likely to go to "Restaurant A" at 12:00 on the next Friday.

[0893] Output: Predicted reminder timing

[0894] Step 4:

[0895] Get weather and traffic information

[0896] Input: Predicted reminder time

[0897] Specific operation: The server obtains the weather and traffic conditions for the day using a weather information API (e.g., OpenWeatherMap) and a traffic information API (e.g., Google Maps API). For example, it obtains information that the weather for next Friday is rainy and the traffic is heavy.

[0898] Output: Obtained weather and traffic information

[0899] Step 5:

[0900] Calculate the best departure time

[0901] Input: Retrieved weather and traffic information

[0902] Specific operation: The server calculates the optimal departure time for the user based on the acquired weather and traffic information. For example, if it is a rainy day and traffic is congested, it calculates that the user should leave 15 minutes earlier than usual.

[0903] Output: Calculated optimal departure time

[0904] Step 6:

[0905] Generate reminder messages

[0906] Input: Calculated optimal departure time

[0907] Specific behavior: The server generates a reminder message to notify the user based on the calculated optimal departure time, for example, "It's raining and traffic is heavy today, so we recommend that you leave 15 minutes earlier than usual."

[0908] Output: The generated reminder message.

[0909] Step 7:

[0910] Reminder message notifications

[0911] Input: The generated reminder message

[0912] Specific operation: The device notifies the user of the generated reminder message, for example, by a smartphone push notification or email.

[0913] Output: Reminder message sent to the user

[0914] In this way, the system learns the user's behavioral patterns and supports the user's daily life by reminding them at appropriate times.

[0915] (Application example 2)

[0916] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0917] Conventional virtual assistant systems were limited to managing users' schedules and providing reminders, and were insufficient in providing personalized services based on users' behavioral patterns and preferences. They also lacked a reminder function that took into account weather and traffic information and provided optimal timing. This resulted in reduced convenience for users and made it difficult to provide efficient lifestyle support.

[0918] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server collects information on a plurality of services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on procedures of public institutions, and information on family composition, and based on this information, it uses a virtual assistant of a generative AI to manage daily schedules, reminds users of procedures of life events and public institutions, and follows up on forgotten schedules with information not registered in the calendar, etc.; the virtual assistant of the generative AI learns the user's behavioral patterns based on the collected information and provides services according to the user's needs; the virtual assistant of the generative AI learns the user's food preferences and order history, and reminds the user to order food at the optimal timing; and the virtual assistant of the generative AI reminds the user to order at the optimal time, taking into account weather information and traffic information. This makes it possible to provide personalized services based on the user's behavioral patterns and preferences, and also realizes a reminder function at the optimal timing taking into account weather information and traffic information.

[0919] "Multiple services" refers to information such as internet search tools, calendars, email, routes, weather, search, and chat content on messenger apps.

[0920] "Procedures at public institutions" refers to various procedures carried out at city halls and government agencies.

[0921] "Family composition information" refers to information about the user's family.

[0922] "Daily schedule management" refers to managing a user's daily plans.

[0923] A "life event" refers to an important occurrence in a user's life.

[0924] A "remind" refers to reminding a user to perform a specific action or schedule.

[0925] A "generative AI virtual assistant" refers to artificial intelligence that learns a user's behavioral patterns based on collected information and provides the user with appropriate services.

[0926] "Behavior patterns" refer to the tendencies of a user's daily behavior.

[0927] "User needs" refers to the services and information required by users.

[0928] "Dining preferences" refers to information about the types of meals and restaurants that a user prefers.

[0929] "Order history" refers to a record of past food orders placed by a user.

[0930] The "optimal timing" refers to the most suitable time for the user.

[0931] "Weather Information" refers to information regarding the weather on a particular day.

[0932] "Traffic Information" refers to information regarding traffic conditions on a particular day.

[0933] "Optimal ordering time" refers to the most suitable time for a user to order a meal.

[0934] As an embodiment of the present invention, the following system is constructed.

[0935] System Configuration

[0936] The system consists of a server, a user device, and a generative AI virtual assistant. The server collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on procedures at public institutions, and information on family composition. The user device is a device such as a smartphone or tablet, on which the generative AI virtual assistant is installed.

[0937] Program Processing

[0938] The server learns the user's behavioral patterns based on the collected information. Specifically, it accumulates data on the user's past actions and choices, and predicts the user's behavioral patterns using a machine learning algorithm (for example, the Linear Regression model of scikit-learn).

[0939] The generative AI virtual assistant learns the user's food preferences and ordering history, and reminds them to order food at the optimal time. It also has a function that takes into account weather and traffic information to remind users of the optimal ordering time. This allows users to order food efficiently.

[0940] Hardware and Software Used

[0941] Hardware: Smartphones, tablets, servers

[0942] Software: Python, scikit-learn, requests

[0943] Examples

[0944] For example, if the system learns that a user has a behavioral pattern of having lunch at a specific restaurant every Friday, it will remind the user to leave earlier than usual if it is expected to rain and have heavy traffic on that day. Also, if the user wants to try a new restaurant, it will suggest a restaurant based on the user's past preferences and ratings.

[0945] Examples of prompt statements

[0946] Build an application that learns the user's behavioral pattern of having lunch at a specific restaurant every Friday and reminds them of the best time to leave, taking into account the weather and traffic information for that day. Use Python and scikit-learn, and get weather and traffic information from an API.

[0947] In this way, it becomes possible to provide personalized services based on the user's behavioral patterns and preferences, and it is also possible to realize a reminder function at the optimal time, taking into account weather and traffic information.

[0948] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0949] Step 1:

[0950] The server collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on procedures at public institutions, and information on family composition. This information is obtained through APIs and stored in a database. The input is information from each service, and the output is an integrated database.

[0951] Step 2:

[0952] The server learns the user's behavioral patterns based on the collected information. Specifically, it trains a machine learning algorithm (for example, scikit-learn's Linear Regression model) using past behavioral data. The input is the past behavioral data, and the output is the learned model.

[0953] Step 3:

[0954] The user device collects the user's food preferences and order history through a generative AI virtual assistant. This information is obtained from the user's input and past order history. The input is the user's food preferences and order history, and the output is personalized food preference data.

[0955] Step 4:

[0956] A generative AI virtual assistant reminds users to order food at the optimal time based on their food preferences and ordering history. Specifically, it calculates the optimal ordering time by taking into account the user's behavioral patterns and the current date and time. The input is the user's behavioral patterns and the current date and time, and the output is a reminder message.

[0957] Step 5:

[0958] The server obtains weather and traffic information and calculates the optimal ordering time based on this information. Specifically, it obtains real-time information through weather and traffic APIs and calculates the optimal time by combining it with the user's behavioral patterns. The input is weather and traffic information, and the output is the optimal ordering time.

[0959] Step 6:

[0960] The user device receives reminder messages from the generative AI virtual assistant and notifies the user. Specifically, it notifies the user of the optimal time to order via push notifications and alerts. The input is the reminder message, and the output is the notification to the user.

[0961] Step 7:

[0962] The user places an order for a meal based on the reminder message. Specifically, the user performs the ordering procedure within the application and the order is completed. The input is the reminder message, and the output is a notification that the order has been completed.

[0963] Example 3

[0964] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0965] In modern society, users receive information from many sources, but it is difficult to effectively manage that information and receive appropriate reminders at the right time. In addition, users often miss information about important events such as changes in family structure or administrative procedures. This puts users at risk of forgetting or delaying important procedures.

[0966] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means. In this invention, the server includes a means for collecting information on a plurality of services, information on procedures of public institutions, and information on family structure, a means for analyzing the collected information and storing it in a database, a means for generating a reminder using a generative AI model based on the stored information, and a means for notifying the user of the generated reminder. This enables the user to receive reminders about important life events and administrative procedures at appropriate times.

[0967] "Multiple services" refers to various information services that users use on a daily basis, such as internet search tools, calendars, email, route information, weather information, search engines, and chat content on messenger apps.

[0968] "Public institution procedures" refers to various procedures and services provided by city halls and administrative agencies, such as submitting a birth notification, applying for child allowances, and submitting a marriage notification.

[0969] "Family information" refers to information about the user's family, such as information about events such as birth, marriage, divorce, and death.

[0970] "Means of collection" refers to the technologies and methods used to obtain necessary information from various services and applications used by users.

[0971] "Means of analysis" refers to techniques and methods for analyzing collected information using natural language processing technology, etc., and extracting meaning from it.

[0972] "Means of storing in a database" refers to a database system for efficiently managing and storing the analyzed information.

[0973] "Generative AI model" refers to an artificial intelligence model, such as a natural language generation model, for generating reminders based on collected and analyzed information.

[0974] "Means for generating reminders" means the technology or methodology for using a generative AI model to create reminders for users.

[0975] "Means of notification" refers to the technology or method for communicating the generated reminder to the user, such as email or a messenger app.

[0976] The present invention relates to a system for automatically generating and notifying a user of reminders regarding life events and administrative procedures. A specific embodiment of this system will be described below.

[0977] The server collects information from multiple services used by the user, including internet search tools, calendars, emails, route information, weather information, search engines, chat content in messenger apps, etc. For example, if a user sends a message on Facebook Messenger saying "Our baby was born!", the device sends this message to the server.

[0978] The server uses natural language processing technology to analyze the collected information. Specifically, it uses the Google Cloud Natural Language API to analyze the content of the messages and extract information about changes in family structure. The analyzed information is then stored in a database.

[0979] Next, the server generates a reminder using a generative AI model based on the stored information. The generative AI model used is OpenAI's GPT-4. The server sends the following prompt to the generative AI model:

[0980] A user sends a message saying "Our baby is here!" Generate a reminder for the associated administrative process.

[0981] Based on this prompt, the generative AI model generates a reminder such as, "The deadline for submitting the birth notification is XX day. Don't forget to apply for child allowance."

[0982] Finally, the server notifies the user of the generated reminder via email or a messenger app. Specifically, the server uses the Gmail API or Facebook Messenger API to send the reminder.

[0983] This system allows users to receive timely reminders about important life events and administrative procedures. For example, if a user sends a message saying "Our baby was born!", the server collects and analyzes this information, generates a reminder using a generative AI model, and notifies the user so that they do not forget to submit a birth notification or apply for child allowance. The flow of the specific process in the third embodiment is explained with reference to FIG. 15.

[0984] Step 1: Gather information

[0985] A user shares changes in family structure (e.g. birth, marriage, etc.) via email or a messenger app. The device sends this information to the server. Specifically, a user sends a message on Facebook Messenger saying "Our baby is here!" The input is the user's message, and the output is the message data sent to the server.

[0986] Step 2: Analyze the information

[0987] The server analyzes the received message. Natural language processing (NLP) technology is used for the analysis. Specifically, the Google Cloud Natural Language API is used to analyze the content of the message. The input is the message data sent to the server, and the output is the analyzed information on changes to family structure. The server extracts the information that "a baby was born" from the message.

[0988] Step 3: Save your information

[0989] The server saves the analyzed information in a database. The input is the analyzed family structure change information, and the output is the information saved in the database. The server saves the information "a baby was born" in the database.

[0990] Step 4: Generate a reminder

[0991] The server generates reminders using a generative AI model based on the stored information. Specifically, it uses OpenAI's GPT-4. The server sends the following prompt to the generative AI model:

[0992] A user sends a message saying "Our baby is here!" Generate a reminder for the associated administrative process.

[0993] The input is the information stored in the database and a prompt, and the output is the generated reminder. The generative AI model generates a reminder such as, "The deadline for submitting the birth notification is XX day. Don't forget to apply for child allowance."

[0994] Step 5: Reminder notifications

[0995] The server notifies the user's device of the generated reminder. The notification is sent via email or a messenger app. Specifically, the reminder is sent using the Gmail API or Facebook Messenger API. The input is the generated reminder, and the output is the reminder notification sent to the user's device. The server sends a reminder to the user's email address or messenger app, such as "The deadline for submitting the birth notification is XX day. Don't forget to apply for child allowance."

[0996] (Application example 3)

[0997] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0998] Conventional virtual assistant systems can provide users with reminders about life events and administrative procedures, but they lack the functionality to automatically generate security reminders and notify them at the appropriate time, which poses the issue that users may forget important security measures.

[0999] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1000] In this invention, the server includes a means for utilizing a generative AI virtual assistant to collect information on multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on city hall and government procedures, and information on family composition, and manage daily schedules based on this information, remind the user of life events and government procedures, and follow up on forgotten schedules using information not registered in the calendar, a means for generating a security reminder message based on the user's life event information and notifying the user by email, and a means for sending the reminder message one week before the event. This allows the user to receive reminders at the appropriate time without forgetting important security measures.

[1001] "Multiple services" refers to information such as internet search tools, calendars, email, routes, weather, search, and chat content on messenger apps.

[1002] "Information on city hall and administrative procedures, etc." refers to information on various administrative procedures that users need to carry out.

[1003] "Family information" refers to information about the user's family members.

[1004] "Daily schedule management" refers to managing a user's daily plans.

[1005] "Life events" refer to important events in a user's life (e.g. moving, getting married, having a baby, etc.).

[1006] "Administrative procedure reminder" refers to notifying users so that they do not forget to carry out administrative procedures that they need to carry out.

[1007] "Following up on forgotten schedules using information not registered in the calendar" refers to reminding the user of important events that are not registered in the calendar.

[1008] A "generative AI virtual assistant" refers to an artificial intelligence that manages the user's schedule and provides reminders based on collected information.

[1009] "User's life event information" refers to information about important events in the user's life.

[1010] A "security reminder message" refers to a message that notifies a user of security measures that the user should take.

[1011] "Means of notifying by email" refers to a method of sending a reminder message to the user's email address.

[1012] "Method of sending one week before the event" refers to a method of sending a reminder message one week before a life event.

[1013] The system for carrying out the present invention has a function of generating a security-related reminder message based on a user's life event information and notifying the user at an appropriate timing. A specific embodiment of this system will be described below.

[1014] The server collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), city hall and administrative procedures, and family structure information. Based on this information, a generative AI virtual assistant manages the user's daily schedule and reminds them of life events and administrative procedures. It also follows up on forgotten schedules that are not registered in the calendar.

[1015] Based on the user's life event information, a security reminder message is generated. For example, if a user is planning to move, a reminder message is generated one week before the move stating, "Don't forget to update your security settings for the move." This reminder message is sent to the user via email.

[1016] The following hardware and software are used to implement this system. A smartphone is used as the hardware. Python and smtplib (email sending library) are used as the software. The server processes and calculates data based on the collected information, and generates a reminder message.

[1017] For example, if a user is planning to get married, a reminder message will be generated and sent to the user one week before the wedding, saying, "Don't forget to update your emergency contact information for your wedding." In this way, users can receive timely reminders without forgetting to take important security measures.

[1018] Example of a prompt to be input to a generative AI model:

[1019] Please create a program that generates security reminder messages based on the user's life event information and notifies them by email. User life events include moving, getting married, etc. The reminder message should be sent one week before the event.

[1020] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1021] Step 1:

[1022] The server collects information from multiple services (Internet search tools, calendars, email, routes, weather, searches, chat content on messenger apps, etc.), information on city hall and administrative procedures, and information on family structure. This information is entered as data necessary for managing the user's life events and schedule. The server stores this information in a database and makes it available for subsequent processing.

[1023] Step 2:

[1024] The server uses a generative AI virtual assistant based on the collected information to manage the user's daily schedule and remind them of life events and government procedures. Specifically, the server analyzes the collected data and generates the necessary reminders to follow up on forgotten schedules that are not registered in the user's calendar. Natural language processing and machine learning algorithms are used for this analysis.

[1025] Step 3:

[1026] The server generates security-related reminder messages based on the user's life event information. For example, if a user is planning to move, a reminder message is generated one week before the move saying, "Don't forget to update your security settings for the move." This reminder message is automatically created using a generative AI model.

[1027] Step 4:

[1028] The server sends the generated reminder message to the user's email address. The smtplib library is used to send the email. Specifically, the reminder message is set as the body of the email, and the user's email address is specified as the destination. If the email is sent successfully, the server records the result in a log.

[1029] Step 5:

[1030] The user checks the received reminder message and takes necessary security measures. For example, updating security settings when moving or updating emergency contacts when getting married. By taking action based on the reminder message, the user can take important security measures without forgetting them.

[1031] Step 6:

[1032] The server learns from the user's behavioral history to improve the accuracy of future reminder messages. Specifically, it records what actions the user took in response to the reminder message and feeds that data back to the AI ​​model. This allows future reminder messages to be more tailored to the user's needs.

[1033] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1034] "Example 1"

[1035] In one embodiment of the present invention, a system is provided that has an emotion engine that recognizes the emotion of a user. The system recognizes the user's emotional state and adjusts the timing and content of reminders according to the emotional state. For example, if the user feels stressed, the system may reduce the frequency of reminders or suggest activities to help the user relax.

[1036] "Example 2"

[1037] In another embodiment of the present invention, a system is provided in which an emotion engine adjusts the way in which a service is provided according to the emotion state of a user. For example, if the system detects that the user is feeling happy, the system generates a message for sharing the joy and provides it to the user. Also, if the system detects that the user is feeling sad, the system generates a message of comfort and provides it to the user.

[1038] "Example 3"

[1039] Furthermore, as another embodiment of the present invention, a system is provided in which the emotion engine adjusts the timing and content of reminders and the method of providing services according to the user's emotional state. For example, if the user feels angry, the system suggests an activity to calm the anger. Also, if the user feels surprised, the system generates a message to share the surprise and provides it to the user.

[1040] The process flow of each embodiment will be described below.

[1041] "Example 1"

[1042] Step 1: The system recognizes the user's emotional state using an emotion engine.

[1043] Step 2: If the emotion engine recognizes the user's emotional state as "stressed," the system will either reduce the frequency of reminders or suggest activities to help the user relax.

[1044] "Example 2"

[1045] Step 1: The system recognizes the user's emotional state using an emotion engine.

[1046] Step 2: If the emotion engine recognizes the user's emotional state as "happy" or "sad", the system generates a message according to the emotional state and provides it to the user.

[1047] "Example 3"

[1048] Step 1: The system recognizes the user's emotional state using an emotion engine.

[1049] Step 2: If the emotion engine recognizes the user's emotional state as "anger" or "surprise," the system suggests an activity or generates a message corresponding to that emotional state and provides it to the user.

[1050] Example 1

[1051] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1052] Modern users use multiple services (Internet search tools, calendars, email, traffic information, weather information, search engines, messaging apps, etc.) and are required to efficiently manage the information obtained from each service. However, manually organizing this information and managing schedules and reminders is very time-consuming. It is also difficult to provide appropriate reminders according to the user's emotional state. This leads to problems such as users missing important information or not receiving reminders at the right time.

[1053] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for collecting information from multiple services (Internet search tools, calendars, emails, traffic information, weather information, search engines, chat content of messaging apps, etc.), a means for analyzing the collected information and extracting important information, a means for automatically generating schedule management and reminders based on the extracted information and notifying the user, and a means for recognizing the user's emotional state and adjusting the timing and content of the reminder according to the emotional state. This allows the user to efficiently manage information obtained from multiple services and receive reminders at appropriate times.

[1054] "Multiple services" refers to different information sources such as internet search tools, calendars, email, traffic information, weather information, search engines, and chat content in messaging apps.

[1055] "Means of collecting information" refers to the function of obtaining necessary information from multiple services used by a user, using the APIs and data acquisition methods of each service.

[1056] "Means of analyzing information" refers to the function of analyzing collected information using natural language processing technology and data analysis technology and extracting important information.

[1057] "Means for extracting important information" refers to the function of identifying and extracting information that is useful to the user from the analyzed data.

[1058] "Means for automatically managing schedules and generating reminders" refers to a function for automatically managing a user's schedule and generating necessary reminders based on the extracted important information.

[1059] "Means for notifying the user" refers to a function for notifying the user of the generated reminders and schedule information in an appropriate manner.

[1060] "Means for recognizing an emotional state" refers to a function for analyzing the content of a user's messages and behavioral patterns to identify the user's emotional state.

[1061] "Means for adjusting the timing and content of reminders" refers to a function for appropriately changing the timing and content of reminder notifications depending on the recognized emotional state of the user.

[1062] This invention is a system that collects information from multiple services used by the user (Internet search tools, calendars, emails, traffic information, weather information, search engines, chat content of messaging apps, etc.), analyzes this information to extract important information, automatically generates schedule management and reminders, and notifies the user. It also includes a function to recognize the user's emotional state and adjust the timing and content of reminders according to that emotional state.

[1063] Hardware and Software Used

[1064] The server uses the following hardware and software:

[1065] Hardware: A server with a powerful processor, sufficient memory, storage devices, and network interfaces.

[1066] Software: APIs for each service, natural language processing technology (e.g., Google Cloud Natural Language API and IBM Watson Natural Language Understanding), database management system, sentiment analysis engine.

[1067] Data processing and calculation

[1068] The server processes and calculates the data in the following procedure.

[1069] 1. Information Collection:

[1070] The server collects user information using the API of each service. For example, it uses a messaging app's API to obtain chat contents and an internet search tool's API to obtain search history.

[1071] 2. Information analysis:

[1072] The server analyzes the collected information using natural language processing technology, specifically, by using the Google Cloud Natural Language API to extract important information from chat content and search history.

[1073] 3. Automatic registration and reminder generation:

[1074] The server automatically generates schedule management and reminders based on the analyzed information. For example, it registers meetings in the calendar using the Google Calendar API and sets reminders using the API of a reminder app.

[1075] 4. Emotional state recognition and reminder adjustment:

[1076] The server uses an emotion analysis engine to recognize the user's emotional state. If it determines that the user is stressed, it will reduce the frequency of reminders and suggest activities to help them relax. For example, it will use the API of a music streaming service to suggest a relaxing playlist.

[1077] Examples

[1078] Example 1: Automatic conference registration

[1079] A user talks on a messaging app saying, "I have a meeting tomorrow at 1pm."

[1080] The server collects the content of this talk and uses the Google Cloud Natural Language API to extract information such as "There is a meeting tomorrow at 1:00 p.m."

[1081] The server automatically registers the "meeting" in the calendar using the Google Calendar API.

[1082] Example 2: Administrative procedure reminders

[1083] A user searches for "child enrollment" using an internet search tool.

[1084] The server collects this search information and uses the Google Cloud Natural Language API to extract the keyword "child's enrollment procedures."

[1085] The server uses the reminder app's API to set reminders for related government procedures.

[1086] Example 3: Adjusting reminders based on emotions

[1087] A user talks on a messaging app saying, "I've been busy at work lately and I'm feeling stressed."

[1088] The server analyzes the content of the conversation using an emotion engine and determines that the user is feeling stressed.

[1089] The server reduces the frequency of reminders and uses the music streaming service's API to suggest relaxation playlists.

[1090] Examples of prompt statements

[1091] "If a user says in a messaging app, 'I have a meeting tomorrow at 1pm,' generate a program that collects this information, parses it using the Google Cloud Natural Language API, and automatically adds it to the calendar using the Google Calendar API."

[1092] "When a user searches for 'child's admission procedure' using an internet search tool, collect this information, parse it using the Google Cloud Natural Language API, and generate a program that uses the reminder app's API to set a reminder for the relevant administrative procedure."

[1093] "If a user says in a messaging app, 'I've been busy at work lately and I'm stressed,' generate a program that collects this information, analyzes it with an emotion engine to reduce the frequency of reminders, and uses the API of a music streaming service to suggest a relaxing playlist."

[1094] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1095] Step 1: Gather information

[1096] The server collects information from multiple services used by the user (Internet search tools, calendars, email, traffic information, weather information, search engines, chat content of messaging apps, etc.). Specifically, it obtains data using the API of each service. The input is data from the API of each service, and the output is the collected raw data.

[1097] Specific behavior:

[1098] The server calls the messaging app's API and obtains the user's chat content.

[1099] The server obtains the user's search history using the API of the Internet search tool.

[1100] Step 2: Information analysis

[1101] The server analyzes the collected information. It uses natural language processing (NLP) techniques to extract important information from chat content and search history. The input is the collected raw data, and the output is the analyzed important information.

[1102] Specific behavior:

[1103] The server uses the Google Cloud Natural Language API to extract information from the chat content, such as "There is a meeting tomorrow at 1:00 p.m."

[1104] The server extracts the keyword "child's admission procedures" from the search history.

[1105] Step 3: Automatic registration and reminder generation

[1106] The server takes appropriate action based on the parsed information. For example, it uses a calendar API to automatically add meeting information to a calendar. The input is the parsed important information, and the output is automatically generated schedules and reminders.

[1107] Specific behavior:

[1108] The server calls the calendar API and registers "Meeting tomorrow at 1pm" in the calendar.

[1109] The server uses the API of the reminder app to set a reminder for "child's enrollment procedures."

[1110] Step 4: Recognize your emotional state

[1111] The server uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's message content and behavioral patterns to identify emotions. The input is the user's message content and behavioral patterns, and the output is the recognized emotional state.

[1112] Specific behavior:

[1113] The server inputs the chat content from the messaging app into an emotion engine and analyzes the user's emotional state.

[1114] The server uses the Google Cloud Natural Language API to determine whether the user is feeling stressed.

[1115] Step 5: Adjust your reminders

[1116] The server adjusts the timing and content of reminders depending on the user's emotional state. For example, if it determines that the user is stressed, it may reduce the frequency of reminders or suggest activities to help them relax. The input is the perceived emotional state, and the output is adjusted reminders or suggested activities.

[1117] Specific behavior:

[1118] The server calls the reminder app's API and sets the app to reduce the frequency of reminders.

[1119] The server uses the API of a music streaming service to suggest a relaxing playlist to the user.

[1120] (Application example 1)

[1121] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1122] In modern factories, managing workers' schedules and understanding their emotional states are important issues. In particular, it is necessary to respond appropriately when a worker is feeling stressed or has an important meeting. However, with conventional systems, it is difficult to efficiently collect this information and respond appropriately. There is also a demand for systems that can adjust reminders according to the worker's emotional state, improve work efficiency, and improve safety.

[1123] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server collects information on multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on city hall and administrative procedures, and information on family structure, and based on this information, it includes a means using a generative AI virtual assistant that manages daily schedules, reminds users of life events and administrative procedures, and follows up on forgotten schedules using information not registered in the calendar, a means using an emotion engine that recognizes the user's emotional state and adjusts the timing and content of reminders according to the emotional state, and a means for a robot working in a factory to grasp the schedules and emotional states of workers and improve work efficiency and safety. This makes it possible to efficiently manage the schedules and grasp the emotional states of workers, and to respond appropriately.

[1124] "Multiple services" refers to information such as internet search tools, calendars, email, routes, weather, search, and chat content on messenger apps.

[1125] "Information on city hall and administrative procedures, etc." refers to information regarding administrative procedures that users need to carry out.

[1126] "Family composition information" refers to information about the user's family.

[1127] "Daily schedule management" refers to managing a user's daily plans.

[1128] A "life event" refers to an important occurrence in a user's life.

[1129] A "remind" refers to reminding a user to perform a specific action or schedule.

[1130] A "generative AI virtual assistant" refers to a virtual assistant that uses artificial intelligence to collect information about the user and provide appropriate services.

[1131] An "emotion engine" is an engine that recognizes the user's emotional state and responds appropriately according to that state.

[1132] "Robots working in factories" refers to automated machines that perform work in factories.

[1133] "Worker schedule" refers to the schedule of workers working in a factory.

[1134] "Emotional state" refers to the current emotional state of a user or worker.

[1135] "Improving work efficiency" refers to carrying out work more efficiently.

[1136] "Improving safety" refers to improving the working environment and working methods to increase safety.

[1137] As an embodiment of the present invention, a system is constructed in which a robot working in a factory grasps the schedule and emotional state of workers, thereby improving work efficiency and safety. A specific embodiment of the system is shown below.

[1138] System Programming

[1139] The system consists of the following main components:

[1140] 1. Information collection module: Collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content in messenger apps, etc.).

[1141] 2. Emotion recognition engine: Recognizes the user's emotional state and responds appropriately based on that state.

[1142] 3. Schedule Management Module: Based on the collected information, manage the schedules of workers and provide reminders and follow-ups.

[1143] 4. Robot control module: Robots working in factories will understand the schedules and emotional states of workers, improving work efficiency and safety.

[1144] Hardware and Software Used

[1145] Hardware: Factory robots (e.g. ABB's IRB series)

[1146] Software: Python, emotion recognition engine (e.g. Microsoft Azure's Emotion API)

[1147] Data processing and calculation

[1148] 1. Information collection: The server analyzes the chat contents of messenger apps and search results from Internet search tools, and registers them in a schedule.

[1149] 2. Emotion Recognition: The server recognizes the emotional state of the worker using an emotion recognition engine.

[1150] 3. Schedule Adjustment: The server adjusts the work load according to the emotional state.

[1151] 4. Remind: The server will remind you based on the current time.

[1152] Examples

[1153] For example, if a worker says "I have a meeting tomorrow at 1pm" on a messenger app, this information is collected and automatically added to the calendar. Also, if a worker searches for "child's school admission procedures" using an internet search tool, this information is collected and a reminder for the related government procedures is set. Furthermore, if the system recognizes that the worker is feeling stressed, it will reduce the frequency of reminders or make suggestions to reduce the workload.

[1154] Examples of prompt statements

[1155] Below are examples of prompts to be input to the generative AI model.

[1156] If a user says "I have a meeting tomorrow at 1pm" on a messenger app, this information is collected and automatically added to the calendar. If a user searches for "child's school admission procedures" on an internet search tool, this information is collected and a reminder for the related government procedures is set. Furthermore, if the system recognizes that the user is feeling stressed, it will reduce the frequency of reminders or suggest activities to help the user relax.

[1157] In this way, a system can be realized in which robots working in a factory can understand the schedules and emotional states of workers, improving work efficiency and safety.

[1158] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1159] Step 1:

[1160] The server collects information from multiple services (Internet search tools, calendars, email, routes, weather, searches, chat content in messenger apps, etc.). Specifically, it obtains data from each service through APIs and collects information about the user's actions and schedules. The input is data from each service, and the output is the storage of the collected information in a database.

[1161] Step 2:

[1162] The server analyzes the collected information and automatically updates the user's schedule. Specifically, it uses natural language processing (NLP) to analyze chat content in messenger apps and search results from internet search tools, and reflects this in the schedule. The input is the collected information, and the output is the updated schedule data.

[1163] Step 3:

[1164] The server uses an emotion recognition engine to recognize the user's emotional state. Specifically, it uses Microsoft Azure's Emotion API to analyze emotions from the user's messages and voice data. The input is the user's message and voice data, and the output is the recognized emotional state.

[1165] Step 4:

[1166] The server adjusts the timing and content of reminders according to the emotional state: for example, if the user is feeling stressed, it reduces the frequency of reminders and suggests activities to relax. The inputs are the recognized emotional state and schedule data, and the output is the adjusted reminder information.

[1167] Step 5:

[1168] The server sends work instructions to the robots working in the factory. Specifically, it adjusts the robots' work based on the schedules and emotional states of the workers. The input is the adjusted reminder information and work instructions, and the output is the transmission of instructions to the robots.

[1169] Step 6:

[1170] The robot performs tasks based on the instructions it receives. Specifically, it performs actions according to the schedule and emotional state of the worker in order to improve work efficiency and safety. The input is the instruction from the server, and the output is the performed task.

[1171] Step 7:

[1172] The server performs reminders based on the current time. Specifically, it notifies the user of events and tasks registered in the schedule. The input is the current time and schedule data, and the output is a reminder notification to the user.

[1173] In this way, a system can be realized in which robots working in a factory can understand the schedules and emotional states of workers, improving work efficiency and safety.

[1174] Example 2

[1175] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1176] Conventional virtual assistant systems have difficulty fully understanding a user's behavioral patterns and emotional state and providing reminders and messages at the right time. In addition, they cannot calculate the optimal departure time taking into account weather and traffic information, or generate messages according to the user's emotions, so they cannot fully meet the needs of users.

[1177] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting information on a plurality of services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on procedures at public institutions, and information on family structure, a means for utilizing a virtual assistant of a generative AI that manages daily schedules based on these information, reminds users of procedures at life events and public institutions, and follows up on forgotten schedules with information not registered in the calendar, a means for collecting user behavior data and sending it to the cloud, a means for analyzing the behavior data stored in the cloud and learning behavior patterns using a generative AI model, a means for acquiring weather information and traffic information using an external API, a means for calculating the optimal departure time based on the acquired information and generating a reminder message, and a means for analyzing the user's emotional state and generating a message according to the emotion. This makes it possible to understand the user's behavior patterns and emotional state and provide reminders and messages at the appropriate time.

[1178] "Multiple services" refers to information such as internet search tools, calendars, email, routes, weather, search, and chat content on messenger apps.

[1179] "Procedures at public institutions" refers to various procedures carried out at city halls and government agencies.

[1180] "Family information" refers to information about the user's family members.

[1181] "Schedule management" refers to organizing and managing a user's daily plans.

[1182] A "life event" refers to an important occurrence in a user's life (e.g., a birthday, a wedding, a move, etc.).

[1183] A "remind" refers to a notification that reminds a user of a specific action or schedule.

[1184] A "generative AI virtual assistant" refers to artificial intelligence that learns the user's behavioral patterns based on collected information and provides appropriate services.

[1185] "Behavioral Data" refers to data regarding a user's daily behavior (e.g., location information, time, activity log, etc.).

[1186] "Cloud" refers to remote servers for storing and managing data over the Internet.

[1187] A "generative AI model" refers to an artificial intelligence model that analyzes a user's behavioral patterns and emotional state to generate appropriate messages and reminders.

[1188] "External API" refers to an application programming interface for connecting with other services or databases.

[1189] "Weather information" refers to information regarding current weather and forecasts.

[1190] "Traffic information" refers to information regarding current traffic conditions and forecasts.

[1191] The "optimal departure time" refers to the most suitable departure time for the user to reach the destination.

[1192] "Emotional state" refers to a user's current emotion (e.g., happy, sad, anger, etc.).

[1193] "Message" means any notification or information sent to a User.

[1194] This invention is a generative AI virtual assistant system that learns the user's behavioral patterns and emotional state and provides reminders and messages at appropriate times. A specific embodiment of this system is described below.

[1195] 1. Program Generation

[1196] The server builds a generative AI virtual assistant to learn the user's behavioral patterns. The virtual assistant includes algorithms to collect and analyze the user's behavioral data. The generative AI model used is, for example, GPT-4.

[1197] 2. Program Processing

[1198] The server uses hardware such as smartphones and wearable devices to collect user behavior data. The collected data is stored in a database on the cloud. The server then analyzes this data and uses generative AI models to learn the user's behavior patterns.

[1199] For example, if a user's behavioral pattern is that they eat lunch at a specific restaurant every Friday, the server will obtain real-time weather and traffic information for that day and calculate the optimal departure time. This information will be sent to the user's smartphone as a reminder.

[1200] In another embodiment, the server uses an emotion engine to analyze the user's emotional state. The emotion engine uses natural language processing (NLP) techniques to analyze the user's voice and text messages. For example, if the user is analyzed to be feeling happy, the server generates a message to share the joy and provides it to the user. Conversely, if the user is analyzed to be feeling sad, the server generates a message of comfort and provides it to the user.

[1201] 3. Examples of concrete examples and prompts

[1202] As a concrete example, consider a case where a user has a behavioral pattern of having lunch at "Restaurant A" every Friday. In this case, the server inputs the following prompt sentence into the generative AI model:

[1203] Example prompt:

[1204] "A user has a behavioral pattern of having lunch at Restaurant A every Friday. Today's weather is sunny and the traffic report shows that it is busier than usual. Calculate the optimal departure time and generate a reminder message."

[1205] The generative AI model calculates the optimal departure time based on this prompt and generates a reminder message like this:

[1206] Example of a reminder message:

[1207] "It's sunny today, but traffic is heavy. We recommend that you leave 10 minutes earlier than usual to get to Restaurant A."

[1208] Using the emotion engine, if the user is parsed as feeling happy, the prompt text would be:

[1209] Example prompt:

[1210] "Your users are excited. Generate a message to share their excitement."

[1211] Based on this prompt, the generative AI model would generate a message like this:

[1212] Example message:

[1213] "That's wonderful! I'm glad I could share that joy with you. I hope you continue to have wonderful times."

[1214] In this way, the server realizes a system that provides appropriate information and messages according to the user's behavioral patterns and emotional state.

[1215] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1216] Step 1: Collect user behavior data

[1217] Users collect daily behavioral data using smartphones and wearable devices. Specifically, location information, time, activity logs, etc. are collected. The input is the behavioral data obtained from the user's device, and the output is the collected behavioral data.

[1218] Step 2: Sending behavioral data to the cloud

[1219] The device periodically transmits the collected behavioral data to the cloud server. The data is encrypted and securely transferred. The input is the behavioral data stored on the device, and the output is the behavioral data stored on the cloud server.

[1220] Step 3: Analyze and learn behavioral patterns

[1221] The server analyzes the behavioral data stored in the cloud. It uses a generative AI model (e.g., GPT-4) to learn the user's behavioral patterns. The input is the behavioral data stored in the cloud, and the output is the learned behavioral patterns. Specifically, it recognizes a pattern that the user has lunch at a specific restaurant every Friday.

[1222] Step 4: Get weather and traffic information

[1223] The server obtains real-time weather and traffic information using external APIs, such as the OpenWeatherMap API or a map service API. The input is the weather and traffic information obtained from the external API, and the output is the obtained weather and traffic information.

[1224] Step 5: Calculate the best departure time

[1225] The server calculates the optimal departure time for the user's behavioral patterns based on the acquired weather and traffic information. The inputs are weather information, traffic information, and the learned behavioral patterns, and the output is the optimal departure time. Specifically, if traffic is congested, the calculation is made to depart earlier than usual.

[1226] Step 6: Generate and send a reminder message

[1227] The server uses the generative AI model to generate a reminder message to send to the user. The generated message is sent as a push notification to the user's smartphone. The input is the optimal departure time, and the output is the reminder message. Specifically, the message generated is "Today is sunny, but traffic is congested. We recommend that you leave 10 minutes earlier than usual to head to Restaurant A."

[1228] Step 7: Analyzing the user's emotional state

[1229] The server analyzes the user's voice and text messages and uses an emotion engine to determine the user's emotional state. The input is the user's voice and text messages, and the output is the analyzed emotional state. Specifically, it uses NLP techniques to analyze whether the user is feeling happy or sad.

[1230] Step 8: Generate and send messages based on emotions

[1231] The server generates an appropriate message depending on the user's emotional state. Using a generative AI model, a message of joy or comfort is generated and sent to the user. The input is the analyzed emotional state, and the output is a message according to the emotion. Specifically, the message generated is "That's wonderful! I'm glad you could share your joy with me. I hope you continue to have a wonderful time."

[1232] (Application example 2)

[1233] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1234] In modern society, users lead busy daily lives and require appropriate services such as schedule management, reminders of life events, and emotional responses. However, conventional systems have difficulty in comprehensively satisfying these needs, and have not realized optimal service provision based on the user's behavioral patterns and emotional states. In particular, in food delivery, there is a demand for reminders of optimal ordering times that take into account the user's past ordering history and behavioral patterns, and for messages that correspond to emotions, but no system exists that can achieve this.

[1235] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1236] In this invention, the server includes a means for utilizing a generative AI virtual assistant to collect information on a plurality of services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on procedures at public institutions, and information on family composition, and to manage daily schedules based on this information, reminders of life events and procedures at public institutions, and follow up on forgotten schedules using information not registered in the calendar, a means for learning the user's behavioral patterns based on the collected information and providing services according to the user's needs, a means for learning the user's past order history and behavioral patterns, and reminding the user of the optimal order time taking into account weather and traffic information, and a means for generating a message according to the user's emotional state and providing it to the user. This makes it possible to provide optimal services based on the user's behavioral patterns and emotional state.

[1237] A "generative AI virtual assistant" is an artificial intelligence system that learns a user's behavioral patterns and emotional state based on collected information, and provides the user with the most optimal services.

[1238] A "behavioral pattern" refers to a user's tendency to repeat a particular behavior at a particular time or on a particular day of the week.

[1239] An "emotional state" refers to the emotion (happiness, sadness, etc.) that a user is feeling at a particular moment.

[1240] A "remind" is a notification or alert sent to a user to encourage them to take a specific action.

[1241] "Weather information" is data about current and future weather conditions.

[1242] "Traffic information" is data about current and future traffic conditions.

[1243] "Order history" is a record of orders a user has placed in the past.

[1244] "Message generation" refers to creating an appropriate message based on the user's emotional state and behavioral patterns.

[1245] "Schedule management" refers to organizing and efficiently managing a user's plans and tasks.

[1246] A "life event" refers to an important event in a user's life (marriage, moving, birthday, etc.).

[1247] "Public institution procedures" refers to administrative procedures and applications that users must carry out.

[1248] The system for implementing this invention utilizes a generative AI virtual assistant that learns the behavioral patterns and emotional state of the user and provides optimal services. A specific embodiment of this system is described below.

[1249] System Configuration

[1250] The system consists of a server, user devices (smartphones, tablets, PCs, etc.), and various APIs (weather information API, traffic information API, etc.). The server runs a generative AI model and learns the user's behavioral patterns and emotional state. The user device receives input from the user and communicates with the server to send and receive information.

[1251] Program Processing

[1252] The server performs the following process.

[1253] 1. Information collection

[1254] In order to learn the user's behavioral patterns and emotional state, the server collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on procedures at public institutions, and information on family composition. This also includes obtaining weather and traffic information using various APIs.

[1255] 2. Learning behavioral patterns

[1256] The server learns user behavior patterns from the collected information, for example recognizing that users tend to order certain dishes on certain days of the week or at certain times of the day.

[1257] 3. Recognizing Emotional States

[1258] The server uses an emotion recognition engine (EmotionRecognizer) to recognize the user's emotional state, thereby understanding whether the user is happy, sad, etc.

[1259] 4. Providing optimal services

[1260] The server provides optimal services to users based on the learned behavioral patterns and the recognized emotional state. For example, if a user has a tendency to order sushi every Friday, and the weather is predicted to be rainy and traffic is heavy on Friday morning, the system will remind the user, "It's raining today and traffic is heavy, so we recommend that you order early." Also, if the system recognizes that the user is happy, it will send a message saying, "Would you like to try our special dessert today?"

[1261] Hardware and Software Used

[1262] Hardware: Servers, smartphones, tablets, PCs

[1263] Software: Python, EmotionRecognizer, Weather API, Traffic API

[1264] Examples

[1265] For example, if a user tends to order sushi every Friday, and the weather is predicted to be rainy and traffic is heavy on Friday morning, the system will remind the user, "It's raining today and traffic is heavy, so we recommend that you order early." Also, if the user is recognized as happy, the system will send a message saying, "Would you like to try our special dessert today?"

[1266] Examples of prompt statements

[1267] Build a food delivery application that learns the user's behavioral patterns based on their past ordering history, and reminds them of the best time to order, taking into account weather and traffic information. Also add the ability to generate messages based on the user's emotional state.

[1268] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1269] Step 1:

[1270] The server receives input from the user's device and collects information on multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on procedures at public institutions, and information on family composition. This also includes obtaining weather and traffic information using various APIs. The input is the user's behavioral history and emotional data, and the output is a dataset of the collected information.

[1271] Step 2:

[1272] The server learns user behavior patterns based on the collected information. Specifically, it analyzes past order history and schedule data to recognize tendencies to repeat certain actions on certain days of the week or at certain times of the day. The input is a dataset of collected information, and the output is a model of the user's behavior patterns.

[1273] Step 3:

[1274] The server uses an emotion recognition engine (EmotionRecognizer) to recognize the user's emotional state. Specifically, it analyzes the chat content and voice data of the messenger app to understand the user's emotions, such as whether they are happy or sad. The input is the user's emotional data, and the output is the recognized emotional state.

[1275] Step 4:

[1276] The server provides optimal services to users based on the learned behavioral patterns and the recognized emotional state. Specifically, if a user has a tendency to order sushi every Friday, and the weather is predicted to be rainy and traffic is heavy on Friday morning, the system will remind the user, "It's raining today and traffic is heavy, so we recommend that you order early." Also, if the system recognizes that the user is happy, it will send a message saying, "Would you like to try a special dessert today?" The input is a model of the behavioral pattern and the emotional state, and the output is a reminder message or a suggestion message.

[1277] Step 5:

[1278] The user device receives the reminder message or suggestion message sent from the server and notifies the user. Specifically, it is displayed as a push notification on the smartphone, an email, or a message on a messenger app. The input is the message from the server, and the output is the notification to the user.

[1279] Step 6:

[1280] The user takes action based on the reminder or suggestion messages he receives, for example, ordering food delivery early or trying a suggested dessert. The input is the notification to the user, and the output is the user's action.

[1281] Example 3

[1282] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1283] In modern society, users need to manage notifications and reminders from many sources. In particular, information about life events and administrative procedures is important, yet easy to forget. There is also a demand for appropriate services to be provided according to the user's emotional state, but current systems have difficulty doing this automatically. To solve these issues, a system is needed that can collect and analyze user information and provide appropriate reminders and services.

[1284] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means. In this invention, the server includes a means for collecting information on a plurality of services, information on administrative procedures, and information on family structure, a means for analyzing the collected information, a means for generating reminders for life events and administrative procedures based on the analysis results, a means for notifying the user of the generated reminders, a means for analyzing the user's emotional state, and a means for providing services based on the analyzed emotional state. This allows the user to receive appropriate services according to their emotional state without forgetting important life events or administrative procedures.

[1285] "Multiple services" refers to various information sources that users use on a daily basis, such as Internet search tools, calendars, email, route information, weather information, search engines, and chat content on messenger apps.

[1286] "Administrative procedures" refers to all procedures carried out at city halls and other government agencies, such as birth registration, marriage registration, and address change procedures.

[1287] "Family information" refers to information about the user's family, such as information about events such as birth, marriage, divorce, and death.

[1288] "Means of collection" refers to the methods and technologies used to obtain information from users, including, for example, using an API to obtain messages from a messenger app.

[1289] "Means of analysis" refers to methods and techniques for analyzing collected information and understanding its meaning or intent, such as using a natural language processing engine to analyze text.

[1290] "Means of generating reminders" refers to methods or technologies for creating reminders to notify users based on the analysis results, including, for example, using a generative AI model to generate the content of the reminder.

[1291] "Means of notifying" refers to the method or technology for notifying the user of the generated reminder, and includes, for example, sending a notification using a mail server or a messenger app.

[1292] "Means for analyzing emotional state" refers to methods or techniques for analyzing collected information to understand the user's emotions, including, for example, using an emotion engine to analyze the user's emotions.

[1293] The "means for providing a service" refers to a method or technology for providing an appropriate service to a user based on the analyzed emotional state, such as suggesting an activity for relaxation.

[1294] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1295] This invention is a system that automatically generates reminders for a user's life events and administrative procedures and notifies the user. It also has a function to adjust the timing and content of reminders and the method of providing services according to the user's emotional state.

[1296] Hardware and Software Used

[1297] Hardware: Server (e.g. cloud server)

[1298] software:

[1299] Natural language processing engines (e.g. natural language processing APIs)

[1300] Sentiment engines (e.g. sentiment analysis APIs)

[1301] Messenger app API (e.g. Messenger API)

[1302] Mail server (e.g. email sending service)

[1303] Generative AI models (e.g. generative AI models)

[1304] Explanation of program processing

[1305] The server collects messages sent by the user via email or messenger apps. For example, if a user sends a message via messenger app saying "Our baby is born!", the server retrieves this message using the messenger API.

[1306] The server then uses a natural language processing engine to analyze the collected messages, for example extracting the event "birth" from the message "A baby was born."

[1307] The server generates reminders for related administrative procedures based on the analysis results. For example, for the event "birth," it generates a reminder to "submit birth notification." The specific content of this reminder is created using a generative AI model.

[1308] The server notifies the user of the generated reminder, for example by sending the reminder to the user's email address using an email sending service, or by sending a notification to a messenger app using the messenger API.

[1309] In addition, the server uses the emotion engine to analyze the user's emotional state. For example, if a user says "I'm very stressed today" in a messenger app, the server sends this message to the emotion engine, which analyzes the emotion as "anger."

[1310] The server provides appropriate services based on the analyzed emotional state. For example, if the emotion "anger" is analyzed, the server generates a message suggesting an activity to relax (e.g., deep breathing exercises) and notifies the user. This message is also created using a generative AI model.

[1311] Examples

[1312] For example, if a user sends a message on a messenger app saying "Our baby was born!", the server will take this message and use a natural language processing engine to extract the event "birth". It will then use a generative AI model to generate a reminder to "submit birth registration" and notify the user using an email sending service.

[1313] Also, if a user says in the Messenger app, "I'm so stressed today," the server can use the emotion engine to analyze the emotion "anger," use the generative AI model to generate a message suggesting an activity to help the user relax, and notify the user using the Messenger API.

[1314] Examples of prompt statements

[1315] "When a user shares a change in family structure, generate a reminder for the relevant administrative procedures."

[1316] "If you sense that the user is feeling angry, generate a message that suggests an activity to calm the user."

[1317] In this way, the server realizes a system that provides reminders and services according to the user's life events and emotional state based on the collected information. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1318] Program processing flow

[1319] Step 1: Gather information

[1320] The server collects messages sent by the user via email or messenger apps. For example, if a user sends a message via messenger app saying "Our baby is born!", the server retrieves this message using the messenger API.

[1321] Input: Messages sent by users in the Messenger app

[1322] Output: Retrieved message data

[1323] Specific operation: The server calls the Messenger API and retrieves the user's messages.

[1324] Step 2: Analyze the information

[1325] The server uses a natural language processing engine to analyze the collected messages, for example extracting the event "birth" from the message "A baby was born."

[1326] Input: Retrieved message data

[1327] Output: Parsed event information (e.g. "birth")

[1328] Specific operation: The server sends message data to a natural language processing engine to extract event information.

[1329] Step 3: Generate a reminder

[1330] The server generates reminders for related administrative procedures based on the analysis results. For example, for the event "birth," it generates a reminder to "submit birth notification." The specific content of this reminder is created using a generative AI model.

[1331] Input: Parsed event information (e.g. "birth")

[1332] Output: The generated reminder (e.g. "Submit birth registration")

[1333] Specific operation: The server inputs event information into the generative AI model and generates reminder content.

[1334] Step 4: Reminder notifications

[1335] The server notifies the user of the generated reminder. For example, it can send the reminder to the user's email address using an email sending service, or it can send a notification to a messenger app using the messenger API.

[1336] Input: The generated reminder

[1337] Output:Notification to the user

[1338] Specific operation: The server uses an email sending service or messenger API to notify the user of the reminder.

[1339] Step 5: Analyze the emotional state

[1340] The server uses the emotion engine to analyze the user's emotional state. For example, if a user says "I'm very stressed today" in a messenger app, the server sends this message to the emotion engine, which analyzes the emotion as "anger."

[1341] Input: User's message

[1342] Output: Analyzed emotion information (e.g. "anger")

[1343] Specific operation: The server sends a message to the emotion engine and analyzes the emotion information.

[1344] Step 6: Deliver emotional service

[1345] The server provides appropriate services based on the analyzed emotional state. For example, if the emotion "anger" is analyzed, the server generates a message suggesting an activity to relax (e.g., deep breathing exercises) and notifies the user. This message is also created using a generative AI model.

[1346] Input: Analyzed emotion information (e.g. "anger")

[1347] Output: Proposed message

[1348] How it works: The server inputs emotion information into the generative AI model to generate a suggested message, then notifies the user using the Messenger API.

[1349] In this way, the server realizes a system that provides reminders and services based on the collected information according to the user's life events and emotional state.

[1350] (Application example 3)

[1351] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1352] In modern society, users tend to forget to be reminded of life events and administrative procedures in their busy daily lives. In addition, due to a lack of timely notifications based on the user's emotional state and a lack of information on special offers and discounts at physical stores, user convenience is reduced. To solve these problems, a system is needed that automatically provides reminders and services based on the user's life events and emotional state.

[1353] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1354] In this invention, the server includes a means for utilizing a generative AI virtual assistant to collect information on a plurality of services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on city hall and administrative procedures, and information on family composition, and manage daily schedules based on this information, reminders of life events and administrative procedures, and follow up on forgotten schedules using information not registered in the calendar, a means for recognizing the user's emotional state and adjusting the timing and content of reminders according to the emotional state, a means for automatically generating and notifying reminders of products and services related to the user's life events, and a means for providing special offers and discount information at physical stores to motivate the user to visit the stores. This allows the user to receive reminders of life events and administrative procedures at the appropriate time, and by utilizing special offers and discount information at physical stores, convenience can be improved.

[1355] A "generative AI virtual assistant" is an artificial intelligence system that automatically manages the user's schedule and provides reminders based on collected information.

[1356] "Emotional state" refers to the user's current psychological state or emotion, including emotions such as anger, surprise, and joy.

[1357] A "remind" is an action of notifying a user of a specific event or procedure, urging them not to forget.

[1358] A "life event" refers to an important occurrence or milestone in a user's life, such as a birthday, wedding, or birth.

[1359] A "physical store" refers to a commercial facility or store that exists in a physical location, where users visit in person to purchase goods or services.

[1360] "Benefits" refers to additional benefits or services provided to users, such as discounts or gifts.

[1361] "Discount information" refers to information in which the price of a product or service is set lower than usual, providing an economic benefit to the user.

[1362] "Motivation" refers to stimuli or incentives to encourage users to take a certain action, such as encouraging them to visit a physical store by offering special offers or discount information.

[1363] The system for implementing this invention uses multiple hardware and software components to provide reminders and services based on a user's life events and emotional state, including a smartphone, a server, a generative AI model, an emotion recognition engine, and an email sending system.

[1364] The server collects information such as internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, city hall and administrative procedures, and family structure information. Based on this information, a generative AI model is used to manage daily schedules, remind people of life events and administrative procedures, and follow up on forgotten schedules that are not registered in the calendar.

[1365] In addition, the server uses an emotion recognition engine to recognize the user's emotional state and adjusts the timing and content of reminders according to the emotional state. For example, if the user is feeling angry, the server will suggest a relaxing activity, and if the user is feeling surprised, the server will generate a message to share the surprise.

[1366] The server uses a generative AI model to automatically generate and notify users of product and service reminders related to their life events. For example, if a user's birthday is approaching, the server generates a message saying, "Happy birthday! Are you looking for a special gift?" If the user's emotional state is "surprise," the server adds, "We have a surprise for you!"

[1367] In order to provide special offers and discount information at physical stores and motivate users to visit the stores, the server collects special offers and discount information and notifies the user. For example, when a user celebrates their wedding anniversary, the server generates a message saying, "Congratulations on your wedding anniversary! How about making a reservation for a special dinner?" and provides special offer information.

[1368] As an illustration, consider the following prompt:

[1369] "If the user's emotional state is 'surprise', add 'We've got a surprise for you!' to the birthday reminder message."

[1370] In this way, users can receive reminders about life events and administrative procedures at the appropriate time, and can also take advantage of special offers and discount information at physical stores, improving convenience.

[1371] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1372] Step 1:

[1373] The server collects information such as internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, city hall and administrative procedures, and family structure information. This information is stored in a database and used as basic data for understanding the user's life events and schedules.

[1374] Input: Internet search tools, calendar, email, routes, weather, search, chat content in messenger apps, city hall / administrative procedure information, family composition information

[1375] Output: A database of the user's life events and schedules.

[1376] Step 2:

[1377] The server uses a generative AI model based on the collected information to manage daily schedules, remind users of life events and government procedures, and follow up on forgotten schedules using information not registered in the calendar. The generative AI model learns the user's behavioral patterns and generates appropriate reminders.

[1378] Input: A database of the user's life events and schedules

[1379] Output:Reminder message

[1380] Step 3:

[1381] The server recognizes the user's emotional state using an emotion recognition engine that analyzes the user's voice and text data to identify the user's current emotional state.

[1382] Input: User's voice data, text data

[1383] Output: The user's emotional state.

[1384] Step 4:

[1385] The server adjusts the timing and content of reminders according to the user's emotional state: for example, if the user is feeling angry, it suggests a relaxing activity, and if the user is feeling surprised, it generates a message to share the surprise.

[1386] Input: Reminder message, user's emotional state

[1387] Output: Tailored reminder message

[1388] Step 5:

[1389] The server automatically generates and notifies the user of reminders of products and services related to life events. For example, when a user's birthday approaches, it generates a message saying, "Happy birthday! Looking for a special gift?"

[1390] Input: A database of the user's life events and schedules

[1391] Output: Product or service reminder message

[1392] Step 6:

[1393] The server provides special offers and discount information at physical stores to motivate users to visit the stores. For example, when a user is celebrating their wedding anniversary, the server generates a message saying "Congratulations on your wedding anniversary! Would you like to make a reservation for a special dinner?" and provides special offer information.

[1394] Input: A database of user's life events and schedules, special offers and discounts

[1395] Output: Reminder message with special offers and discounts

[1396] Step 7:

[1397] The server sends the generated reminder message to the user's smartphone. By receiving the notification, the user can check the life event and benefit information and take the necessary steps.

[1398] Input: Tailored reminder message

[1399] Output: Notification to the user's smartphone

[1400] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1401] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1402] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1403] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1404] [Third embodiment]

[1405] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1406] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1407] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1408] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1409] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1410] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1411] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1412] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1413] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1414] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1415] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1416] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1417] "Example 1"

[1418] The system of the present invention has a means for collecting information from multiple services used by the user (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.). For example, if a user chats on a messenger app saying "I have a meeting tomorrow at 1pm", this information is collected and automatically registered in the calendar. Also, if a user searches for "child's school admission procedures" on an Internet search tool, this information is collected and a reminder for related government procedures is set.

[1419] "Example 2"

[1420] The system of the present invention has a generative AI virtual assistant that learns the user's behavioral patterns based on collected information. For example, if the user learns that they have lunch at a specific restaurant every Friday, the system will remind them of the best time to leave, taking into account the weather and traffic information for that day.

[1421] "Example 3"

[1422] The system of the present invention has a means for automatically generating reminders for the user's life events and administrative procedures based on collected information and notifying the user. For example, when a user shares a change in family structure (e.g., birth, marriage, etc.) via email or messenger app, this information is collected, and a reminder for the related administrative procedures is automatically generated and notified to the user.

[1423] The process flow of each embodiment will be described below.

[1424] "Example 1"

[1425] Step 1: A user talks on a messenger app saying, "I have a meeting tomorrow at 1pm."

[1426] Step 2: The system of the present invention collects this information and automatically enters it into your calendar.

[1427] Step 3: The system of the present invention sends a reminder before the meeting time.

[1428] "Example 2"

[1429] Step 1: The user's behavioral pattern is that they eat lunch at a particular restaurant every Friday.

[1430] Indicates the

[1431] Step 2: Our system learns this behavior pattern.

[1432] Step 3: The system of the present invention will take into account the weather and traffic conditions for the day and remind you of the best time to leave.

[1433] "Example 3"

[1434] Step 1: The user shares a change in family structure (e.g. birth, marriage, etc.) via email or messenger app.

[1435] Step 2: The system of the present invention collects this information.

[1436] Step 3: The system of the present invention automatically generates a reminder of the relevant administrative procedure and notifies the user.

[1437] Example 1

[1438] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1439] Modern users use multiple services (Internet search tools, calendars, e-mail, traffic information, weather information, search engines, messaging app communications, etc.), and are required to efficiently manage the information obtained from each service. However, manually collecting this information, analyzing it, and determining the appropriate action is extremely time-consuming. It is also difficult to learn users' behavioral patterns and provide services that meet their needs. Another challenge is automatically generating reminders for life events and government procedures and notifying users of them.

[1440] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for collecting information from multiple services, a means for analyzing the collected information, a means for determining an action based on the analysis result, and a means for executing the determined action. This makes it possible to efficiently manage information obtained from multiple services used by the user, learn the user's behavioral patterns, and provide services according to the needs. In addition, it is possible to automatically generate reminders for life events and administrative procedures and notify the user.

[1441] "Multiple services" refers to the multiple information provision methods that users use on a daily basis, such as Internet search tools, calendars, email, traffic information, weather information, search engines, and messaging app communications.

[1442] "Means of collecting information" refers to the function of automatically acquiring information such as user activities and communication content by utilizing the API or data acquisition function of each service.

[1443] "Means of analyzing information" refers to the function of analyzing collected information using natural language processing and data analysis techniques and extracting important information.

[1444] "Means for determining an action" refers to a function that automatically determines the next action to be taken based on the analysis results.

[1445] "Means for executing an action" refers to a function for actually executing a determined action, such as adding an event to a calendar or sending a reminder.

[1446] "Learning user behavior patterns" refers to the function of analyzing a user's past behavior and tendencies based on collected information and predicting future behavior.

[1447] "Providing services that meet the needs" refers to the function of providing optimal information and actions based on the user's behavioral patterns and current situation.

[1448] A "life event" refers to an important event in a user's life (e.g., a birthday, a wedding, a child's admission to school, etc.).

[1449] "Administrative procedures" refers to various official procedures that users must carry out (e.g. obtaining a resident registration card, filing taxes, etc.).

[1450] "Automatically generating reminders" refers to a function that automatically creates necessary notifications and reminders for users based on collected and analyzed information.

[1451] "Notify user" refers to the function of notifying the user of generated reminders and important information via email, calendar notifications, or other means.

[1452] This invention is a system that collects information from multiple services used by a user (Internet search tools, calendars, email, traffic information, weather information, search engines, messaging app communication content, etc.), analyzes this information to learn the user's behavioral patterns, and provides services according to the user's needs.

[1453] Generating a Program

[1454] The server generates a program to collect information from each service used by the user. This program is developed using a programming language such as Python or Java. Specifically, the program obtains data using the API of each service and analyzes it using natural language processing (NLP) technology.

[1455] Program Processing

[1456] The server performs data processing and calculation using the following hardware and software.

[1457] Hardware:

[1458] The server is a computer system equipped with a high-performance processor and sufficient memory, such as an Intel Xeon processor or AMD EPYC processor.

[1459] software:

[1460] The server uses the following software to collect and analyze information:

[1461] API: API of each service (e.g. Google Calendar API, Microsoft Outlook API, Gmail API, Google Search Console API, LINE API, WhatsApp API)

[1462] Natural Language Processing (NLP) Tools: Google Cloud Natural Language API, IBM Watson Natural Language Understanding

[1463] Email sending services: Twilio SendGrid, Amazon SES

[1464] Examples

[1465] Example 1: Processing chat content in a messenger app:

[1466] When a user says "I have a meeting tomorrow at 1pm" in a messaging app, the server retrieves the contents of the chat through the messaging app's API. The server then uses the Google Cloud Natural Language API to extract the information "tomorrow at 1pm" and "meeting". The server then automatically registers this information as a "meeting" in the calendar using the Google Calendar API.

[1467] Example 2: Processing Internet search history:

[1468] When a user searches for "child's admission procedure" in an internet search tool, the server retrieves this search history using the Google Search Console API. Then, it analyzes the search query and detects the keyword "child's admission procedure". Based on this information, the server sets up a reminder for the relevant administrative procedure. Specifically, it uses Twilio SendGrid to send a reminder email to the user.

[1469] Examples of prompt statements

[1470] "If a user says 'I have a meeting tomorrow at 1pm' in a messaging app, please create a program that collects this information and automatically registers it in Google Calendar. Also, if a user searches for 'child's school admission procedures' in an internet search tool, please create a program that collects this information and sets a reminder for the related administrative procedures."

[1471] In this way, the server provides a system for analyzing user behavior and automatically collecting and processing necessary information.

[1472] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1473] Step 1:

[1474] The server collects information from each service used by the user. Specifically, it obtains data using the API of each service. For example, it obtains the contents of a user's chat using the API of a messaging app, and obtains search history using the API of an internet search tool. The input is the data obtained from the API of each service, and the output is the collected raw data.

[1475] Step 2:

[1476] The server analyzes the collected information. Specifically, it uses natural language processing (NLP) technology to analyze text data and extract important information. For example, it uses Google Cloud Natural Language API to extract dates and times and event details from chat content in a messaging app. The input is the collected raw data, and the output is analyzed information (e.g., dates and times, event details).

[1477] Step 3:

[1478] The server decides on an action based on the analysis results. Specifically, it determines the next action to be taken based on the extracted information. For example, it decides on the action of registering an event in a calendar based on the extracted date and time and event content. The input is the analyzed information, and the output is the determined action (e.g., registering the event in a calendar).

[1479] Step 4:

[1480] The server executes the determined action. Specifically, it executes the action using the API of each service. For example, it uses the Google Calendar API to automatically register the extracted event in the calendar. It also uses Twilio SendGrid to send a reminder email. The input is the determined action, and the output is the result of the executed action (e.g., the event registered in the calendar, the reminder email sent).

[1481] Step 5:

[1482] The server learns the user's behavioral patterns. Specifically, it analyzes the user's past behavior and tendencies based on the collected and analyzed information, and predicts future behavior. The input is the collected and analyzed information, and the output is the learned behavioral patterns.

[1483] Step 6:

[1484] The server provides services according to the user's needs. Specifically, it provides optimal information and actions to the user based on the learned behavioral patterns. For example, it provides related information based on search queries frequently performed by the user in the past. The input is the learned behavioral patterns, and the output is the provided service (e.g., providing related information).

[1485] In this way, the server analyzes user behavior, automatically collects and processes necessary information, and realizes a system that provides users with the most optimal services.

[1486] (Application example 1)

[1487] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1488] In modern life, users use many services and applications, and managing each piece of information individually is cumbersome. In addition, schedule management and reminder settings are often done manually, which creates the risk of forgetting important appointments or procedures. Furthermore, reservations for transportation and schedule settings must also be done manually, making efficient time management difficult.

[1489] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1490] In this invention, the server includes a means for utilizing a generative AI virtual assistant to collect information on multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on city hall and administrative procedures, and information on family composition, and manage daily schedules based on this information, remind life events and administrative procedures, and follow up on forgotten schedules using information not registered in the calendar, a means for automatically setting a schedule for an autonomous vehicle based on the collected information, and a means for automatically reserving travel to a destination based on the schedule for the autonomous vehicle. This allows the user to centrally manage information from multiple services, and efficiently travel using an autonomous vehicle without forgetting important plans or procedures.

[1491] "Multiple services" refers to different information sources such as internet search tools, calendars, email, routes, weather, search, and chat content in messenger apps.

[1492] "City Hall / Administrative Procedures" refers to various administrative procedures and applications that Users must complete.

[1493] "Family information" refers to information about the user's family, such as the names, ages, and relationships of family members.

[1494] "Daily schedule management" refers to organizing and managing a user's daily plans.

[1495] A "life event" refers to an important event in a user's life, such as marriage, childbirth, or moving.

[1496] A "remind" refers to a notification to a user so that the user does not forget a specific schedule or procedure.

[1497] "Information not registered in the calendar" refers to important appointments and events that the user has not entered into the calendar.

[1498] A "generative AI virtual assistant" refers to a virtual assistant that uses artificial intelligence to collect information about the user and manage the schedule and provide reminders.

[1499] An "autonomous vehicle" refers to a vehicle that drives automatically using artificial intelligence and sensor technology.

[1500] "Automatically setting a schedule" refers to automatically determining and managing a user's plans based on collected information.

[1501] "Automatically reserving travel to a destination" refers to reserving an autonomous vehicle based on the user's schedule and arranging travel to the destination.

[1502] As an embodiment of the present invention, the following system is constructed.

[1503] System Overview

[1504] The system collects information from multiple services used by users, manages schedules and sets reminders using a generative AI virtual assistant, and automatically sets schedules for autonomous vehicles based on the collected information, automatically booking trips to destinations.

[1505] Hardware and software used

[1506] Hardware: Smartphones, self-driving cars

[1507] Software: Messenger app, Google Calendar API, Autonomous Vehicle API

[1508] Data processing and calculation

[1509] Getting messages

[1510] The device (smartphone) receives the user's messages from the messenger app. These messages include information about the user's schedule and events.

[1511] Extracting Information

[1512] The server uses natural language processing technology to extract information about schedules and events from the received messages. For example, from a message such as "I have a meeting tomorrow at 1pm," the server extracts the information "meeting," "tomorrow," and "13:00."

[1513] Calendar registration

[1514] The server uses the extracted information to automatically register the event on the user's calendar using the Google Calendar API.

[1515] Scheduling autonomous vehicles

[1516] The server automatically schedules autonomous vehicles using the Autonomous Vehicle API based on event information registered in the calendar. For example, it reserves an autonomous vehicle to travel to the meeting location at the start time of the meeting.

[1517] Examples

[1518] If a user says in a messenger app, "I have a meeting tomorrow at 1 p.m.," the system collects that information, registers the "meeting" on Google Calendar, and even books an autonomous vehicle to head to the meeting location at 1 p.m.

[1519] Examples of prompt statements

[1520] If a user says in a messenger app, "I have a meeting tomorrow at 1pm," create a program that collects this information, automatically registers the "meeting" in Google Calendar, and even books an autonomous vehicle to head to the meeting location at 1pm.

[1521] In this way, a system can be provided that makes users' lives more convenient.

[1522] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1523] Step 1:

[1524] The device (smartphone) retrieves the user's messages from the messenger app. The input is the user's message, and the output is the text data of the retrieved message. The specific operation is to retrieve the latest message using the messenger app's API.

[1525] Step 2:

[1526] The server uses natural language processing technology to extract information about schedules and events from the acquired messages. The input is the text data of the acquired messages, and the output is the extracted schedule or event information (e.g., "Meeting," "Tomorrow," "13:00"). Specifically, the server analyzes the messages using a natural language processing library (e.g., spaCy or NLTK) and extracts keywords and date and time information.

[1527] Step 3:

[1528] The server uses the Google Calendar API based on the extracted information to automatically register events in the user's calendar. The input is the extracted schedule and event information, and the output is the event registered in Google Calendar. Specifically, the server calls the Google Calendar API and registers the extracted information as an event.

[1529] Step 4:

[1530] The server automatically sets the schedule for autonomous vehicles using the Autonomous Vehicle API based on the event information registered in the calendar. The input is the event information registered in Google Calendar, and the output is the reservation information for the autonomous vehicle. Specifically, the server calls the Autonomous Vehicle API and reserves an autonomous vehicle based on the start time and location of the event.

[1531] Step 5:

[1532] The server notifies the user of the reserved autonomous vehicle information. The input is the reservation information for the autonomous vehicle, and the output is a notification message to the user. Specifically, the server uses the API of the messenger app to send the reservation information to the user.

[1533] In this way, a series of processes is realized: extracting plans from the user's messages, registering them on the calendar, and reserving an autonomous vehicle.

[1534] Example 2

[1535] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1536] Conventional virtual assistant systems have difficulty learning the user's behavioral patterns sufficiently and reminding the user at the appropriate time. In addition, they cannot calculate the optimal departure time taking into account weather and traffic information, and therefore cannot effectively support the user's life. For this reason, there is a demand for a system that can learn the user's behavioral patterns, remind the user at the appropriate time, and calculate the optimal departure time taking into account weather and traffic information.

[1537] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1538] In this invention, the server includes a means for collecting information on a plurality of services, information on procedures, and information on family structure, a means for utilizing a virtual assistant of a generative AI that manages daily schedules based on the information, reminds users of life events and procedures, and follows up on forgotten schedules using information not registered in a calendar, a means for collecting user behavior data and storing it in a database, a means for learning user behavior patterns using a generative AI model based on the collected behavior data, a means for predicting reminder timing based on the learned behavior patterns, a means for acquiring weather information and traffic information based on the predicted date and time, a means for calculating an optimal departure time based on the acquired information and generating a reminder message, and a means for notifying the generated reminder message to the user. This makes it possible to learn the user's behavior patterns, perform reminders at appropriate times, and calculate the optimal departure time taking into account weather information and traffic information.

[1539] The "multiple services" refer to various online services that provide information such as internet search tools, calendars, email, routes, weather, search, and chat content in messenger apps.

[1540] "Information on procedures, etc." refers to information on various procedures carried out at city halls and administrative agencies.

[1541] "Family information" refers to information about the user's family members, including, for example, the number of family members and their relationships.

[1542] A "generative AI virtual assistant" is an artificial intelligence system that learns the user's behavioral patterns based on collected information and provides the user with appropriate reminders and services.

[1543] "User behavior data" refers to information such as the date and time and the location where a user performed a particular behavior.

[1544] A "database" is an information management system for storing collected user behavior data.

[1545] A "generative AI model" is an artificial intelligence model that learns user behavior patterns based on collected data.

[1546] The "reminding timing" is the optimal date and time for reminding the user.

[1547] "Weather information" is information about the weather at a specific date and time.

[1548] "Traffic information" refers to information on traffic conditions at a specific date and time.

[1549] The "optimal departure time" is the most appropriate departure time for the user to reach the destination, calculated taking into account weather and traffic information.

[1550] A "remind message" is a message that is notified to a user to encourage the user to take action.

[1551] This invention relates to a generative AI virtual assistant system that learns a user's behavioral patterns and reminds the user at appropriate times. A specific embodiment of this system will be described below.

[1552] System Configuration

[1553] This system is configured using the following hardware and software.

[1554] Hardware: Servers, user devices (smartphones and PCs)

[1555] Software: Generative AI models (e.g. GPT-4), database management systems (e.g. MySQL), weather APIs (e.g. OpenWeatherMap), traffic APIs (e.g. Google Maps API)

[1556] Data collection and storage

[1557] The server collects user behavior data and stores it in a database. For example, it records information that a user has lunch at a specific restaurant every Friday. This accumulates basic data for understanding the user's behavioral patterns.

[1558] Learning behavioral patterns

[1559] The server inputs the collected behavioral data into a generative AI model to learn the user's behavioral patterns. The generative AI model can perform advanced pattern recognition and prediction, for example by using GPT-4, to understand the tendency to take certain actions on certain days of the week and at certain times of the day.

[1560] Reminder timing prediction

[1561] Based on the learned behavioral patterns, the server predicts the next time the user is likely to take a particular action, for example, to go to a particular restaurant next Friday.

[1562] Get weather and traffic information

[1563] Based on the predicted date and time, the server uses weather information APIs (e.g. OpenWeatherMap) and traffic information APIs (e.g. Google Maps API) to obtain the weather and traffic conditions for that day, allowing it to take into account external factors that affect the user's behavior.

[1564] Calculate the best departure time

[1565] The server calculates the optimal departure time for the user based on the acquired weather and traffic information. For example, if it is a rainy day and traffic is congested, it calculates an earlier departure time than usual.

[1566] Generate and send reminder messages

[1567] The server generates a reminder message to inform the user based on the calculated optimal departure time, for example, "It's raining and traffic is heavy today, so we recommend that you leave 15 minutes earlier than usual."

[1568] The terminal notifies the user of the generated reminder message, for example, by a push notification on a smartphone or by email.

[1569] Examples

[1570] Suppose a user has a behavioral pattern of having lunch at "Restaurant A" every Friday. This information is stored on the server. On Friday morning, the server uses the generative AI model to predict that the user is likely to go to "Restaurant A" that day as well. The server uses the OpenWeatherMap API to get weather information for the day and the Google Maps API to get traffic information. For example, if the weather is rainy and traffic is congested that day, the server generates a reminder message to the user saying, "It's raining and traffic is congested today, so we recommend that you leave 15 minutes earlier than usual."

[1571] Examples of prompt statements

[1572] "Design a system that learns a user's behavioral pattern of having lunch at a particular restaurant every Friday and then reminds them of the best time to leave, taking into account the weather and traffic conditions for that day."

[1573] In this way, the system learns the user's behavioral patterns and supports the user's daily life by reminding them at appropriate times.

[1574] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1575] Step 1:

[1576] Collecting user behavior data

[1577] Input: User behavior information (date, time, location, and content of the action)

[1578] Specific operation: When a user performs a specific action, that information is sent to the server. For example, information that a user checked in to "Restaurant A" at 12:00 on Friday is sent from the smartphone to the server.

[1579] Output: Collected behavioral data is stored in a database.

[1580] Step 2:

[1581] Learning behavioral patterns

[1582] Input: Collected behavioral data

[1583] Specific operation: The server inputs the collected behavioral data into a generative AI model (e.g., GPT-4) to learn the user's behavioral patterns. For example, it learns a pattern that "the user has lunch at 'Restaurant A' every Friday at 12:00."

[1584] Output: A model of the learned behavioral patterns.

[1585] Step 3:

[1586] Reminder timing prediction

[1587] Input: A model of learned behavioral patterns

[1588] Specific operation: Based on the learned behavioral patterns, the server predicts the next time the user is likely to take a certain action. For example, it predicts that the user is likely to go to "Restaurant A" at 12:00 on the next Friday.

[1589] Output: Predicted reminder timing

[1590] Step 4:

[1591] Get weather and traffic information

[1592] Input: Predicted reminder time

[1593] Specific operation: The server obtains the weather and traffic conditions for the day using a weather information API (e.g., OpenWeatherMap) and a traffic information API (e.g., Google Maps API). For example, it obtains information that the weather for next Friday is rainy and the traffic is heavy.

[1594] Output: Obtained weather and traffic information

[1595] Step 5:

[1596] Calculate the best departure time

[1597] Input: Retrieved weather and traffic information

[1598] Specific operation: The server calculates the optimal departure time for the user based on the acquired weather and traffic information. For example, if it is a rainy day and traffic is congested, it calculates that the user should leave 15 minutes earlier than usual.

[1599] Output: Calculated optimal departure time

[1600] Step 6:

[1601] Generate reminder messages

[1602] Input: Calculated optimal departure time

[1603] Specific behavior: The server generates a reminder message to notify the user based on the calculated optimal departure time, for example, "It's raining and traffic is heavy today, so we recommend that you leave 15 minutes earlier than usual."

[1604] Output: The generated reminder message.

[1605] Step 7:

[1606] Reminder message notifications

[1607] Input: The generated reminder message

[1608] Specific operation: The device notifies the user of the generated reminder message, for example, by a smartphone push notification or email.

[1609] Output: Reminder message sent to the user

[1610] In this way, the system learns the user's behavioral patterns and supports the user's daily life by reminding them at appropriate times.

[1611] (Application example 2)

[1612] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1613] Conventional virtual assistant systems were limited to managing users' schedules and providing reminders, and were insufficient in providing personalized services based on users' behavioral patterns and preferences. They also lacked a reminder function that took into account weather and traffic information and provided optimal timing. This resulted in reduced convenience for users and made it difficult to provide efficient lifestyle support.

[1614] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server collects information on a plurality of services (Internet search tools, calendars, emails, routes, weather, searches, chat contents of messenger apps, etc.), information on procedures of public institutions, and information on family composition, and based on this information, it uses a virtual assistant of a generative AI to manage daily schedules, reminds users of procedures of life events and public institutions, and follows up on forgotten schedules with information not registered in the calendar, etc.; the virtual assistant of the generative AI learns the user's behavioral patterns based on the collected information and provides services according to the user's needs; the virtual assistant of the generative AI learns the user's food preferences and order history, and reminds the user to order food at the optimal timing; and the virtual assistant of the generative AI reminds the user to order at the optimal time, taking into account weather information and traffic information. This makes it possible to provide personalized services based on the user's behavioral patterns and preferences, and also realizes a reminder function at the optimal timing taking into account weather information and traffic information.

[1615] "Multiple services" refers to information such as internet search tools, calendars, email, routes, weather, search, and chat content on messenger apps.

[1616] "Procedures at public institutions" refers to various procedures carried out at city halls and government agencies.

[1617] "Family composition information" refers to information about the user's family.

[1618] "Daily schedule management" refers to managing a user's daily plans.

[1619] A "life event" refers to an important occurrence in a user's life.

[1620] A "remind" refers to reminding a user to perform a specific action or schedule.

[1621] A "generative AI virtual assistant" refers to artificial intelligence that learns a user's behavioral patterns based on collected information and provides the user with appropriate services.

[1622] "Behavior patterns" refer to the tendencies of a user's daily behavior.

[1623] "User needs" refers to the services and information required by users.

[1624] "Dining preferences" refers to information about the types of meals and restaurants that a user prefers.

[1625] "Order history" refers to a record of past food orders placed by a user.

[1626] The "optimal timing" refers to the most suitable time for the user.

[1627] "Weather Information" refers to information regarding the weather on a particular day.

[1628] "Traffic Information" refers to information regarding traffic conditions on a particular day.

[1629] "Optimal ordering time" refers to the most suitable time for a user to order a meal.

[1630] As an embodiment of the present invention, the following system is constructed.

[1631] System Configuration

[1632] The system consists of a server, a user device, and a generative AI virtual assistant. The server collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content on messenger apps, etc.), information on procedures at public institutions, and information on family composition. The user device is a device such as a smartphone or tablet, on which the generative AI virtual assistant is installed.

[1633] Program Processing

[1634] The server learns the user's behavioral patterns based on the collected information. Specifically, it accumulates data on the user's past actions and choices, and predicts the user's behavioral patterns using a machine learning algorithm (for example, the Linear Regression model of scikit-learn).

[1635] The generative AI virtual assistant learns the user's food preferences and ordering history, and reminds them to order food at the optimal time. It also has a function that takes into account weather and traffic information to remind users of the optimal ordering time. This allows users to order food efficiently.

[1636] Hardware and Software Used

[1637] Hardware: Smartphones, tablets, servers

[1638] Software: Python, scikit-learn, requests

[1639] Examples

[1640] For example, if the system learns that a user has a behavioral pattern of having lunch at a specific restaurant every Friday, it will remind the user to leave earlier than usual if it is expected to rain and have heavy traffic on that day. Also, if the user wants to try a new restaurant, it will suggest a restaurant based on the user's past preferences and ratings.

[1641] Examples of prompt statements

[1642] Build an application that learns the user's behavioral pattern of having lunch at a specific restaurant every Friday and reminds them of the best time to leave, taking into account the weather and traffic information for that day. Use Python and scikit-learn, and get weather and traffic information from an API.

[1643] In this way, it becomes possible to provide personalized services based on the user's behavioral patterns and preferences, and it is also possible to realize a reminder function at the optimal time, taking into account weather and traffic information.

[1644] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1645] Step 1:

[1646] The server collects information from multiple services (Internet search tools, calendars, emails, routes, weather, searches, chat content...

Claims

1. A means for identifying information useful to the user from the contents of the user's chat and a search history of the user using an Internet search tool, the contents being included in various information sources that the user uses on a daily basis; A notification means for inputting a specific prompt sentence into a generative AI model, and notifying the user of the user's daily schedule based on the specified information, and notifying the user of schedules that may be forgotten by the user that are not registered in the schedule; an emotion engine that recognizes the emotion of the user; means for adjusting the timing and content of the notification in response to the emotion; A system including:

2. A means for learning the behavioral patterns of the user using a generative AI model by collecting behavioral data including the date and time and the location where the user performed a specific behavior; means for calculating the most appropriate departure time for the user to reach the destination based on weather information and traffic information, generating a message encouraging the user to depart at the calculated time, and providing the message to the user; The system of claim 1 , comprising:

3. A means for generating reminders for products and services related to the user's life events included in the information source and notifying the user of the generated reminders; A means for providing in-store special offers and discount information when notifying the user of the reminder; The system of claim 1 , comprising:

Images