System

Abstract

A system is provided.SOLUTION: A system comprising: means for monitoring schedule information; means for obtaining pre-input contact information from a database; means for analyzing past conversation history and generating messages using natural language processing; means for sending generated messages using communication means; means for receiving and analyzing reply messages; means for accessing a reservation site and making reservations; means for updating and notifying new schedule information; and means for automatic conversation generation and sending follow-up conversations.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, schedule management is becoming more complex every day, both in business and personal settings. In particular, the cumbersome process of coordinating locations and times when scheduling meetings and events requires efficient communication. Conventional methods require users to manually contact and coordinate schedules, which poses significant challenges in terms of time and effort. Therefore, there is a need for a system that automates this process and improves the efficiency of schedule management. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides a system comprising: means for monitoring schedule information; means for retrieving pre-input contact information from a database; means for analyzing past conversation history and generating a message using natural language processing; and means for transmitting the generated message using a communication means.

[0006] Furthermore, the system includes means for receiving and analyzing the reply message, means for accessing the reservation site and making the reservation, means for updating and notifying new schedule information, and means for automatically generating conversations and sending follow-up conversations.

[0007] This significantly reduces the amount of manual communication work that users must do in schedule management, allowing for efficient schedule adjustment. Furthermore, necessary reservations are also made automatically, further reducing the user's workload.

[0008] "Schedule information" is information that includes details of a user's plans and events, such as time, location, and participants.

[0009] "Contact information" refers to information about the other person's means of contact, such as their email address, phone number, or social media account, which has been entered in advance.

[0010] A "database" is an information storage system for storing and managing schedule information, contact information, past conversation history, and the like.

[0011] "Conversation history" refers to data relating to past messages sent and received between a user and a partner.

[0012] "Natural language processing" is a technology that uses computers to analyze and understand human language.

[0013] "Communication means" refers to a means for sending and receiving data using the Internet or a mobile communication network.

[0014] A "reservation site" is a website or application for making online reservations for stores, conference rooms, etc.

[0015] "Automatic conversation generation" is a process in which a computer automatically generates natural conversational sentences based on predefined rules and past conversation history.

[0016] A "follow-up conversation" is an additional conversation that takes place in conjunction with scheduling or confirming a reservation, and is intended to facilitate communication with the other party.

[0017] "Schedule monitoring" is the process of periodically scanning the contents of the schedule database to detect pending entries.

[0018] "Message generation" is the process of automatically creating a communication based on information stored in a database.

[0019] "Analysis" is the process of analyzing received data and past history to understand its meaning and intent.

[0020] "Notification" is the process of notifying users and their contacts of schedule updates and reservation completion information. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] The present invention is a system that streamlines users' schedule management and automates the acquisition of necessary information and communication. This system has the functions of monitoring schedule information, generating and sending messages based on contact information, and analyzing replies. It also has the functions of making necessary reservations, updating schedules, and notifying users.

[0043] System configuration

[0044] This system consists of the following main components:

[0045] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications.

[0046] Terminal: A device operated by a user that communicates with the server and sends and receives messages.

[0047] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[0048] Program processing

[0049] Monitoring schedule information

[0050] The server periodically scans the schedule database to detect pending schedule entries, specifically entries for which the date, time, and location of the schedule have not been set.

[0051] Obtaining information about the other person and creating a message

[0052] The server retrieves the contact information of the person in question from a database based on the pending schedule entry, analyzes past conversation history, and uses natural language processing to generate an appropriate message, such as, "Could you please suggest a location and time for our next lunch?"

[0053] Sending automated messages

[0054] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[0055] Parsing reply messages

[0056] When the other person sends a reply message, the device receives it and sends it to the server, which analyzes the received message and extracts the reply content (location, time, etc.).

[0057] Make the necessary reservations

[0058] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[0059] New schedule updates and notifications

[0060] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[0061] Optional conversation generation

[0062] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history and new schedule information. For example, when a reservation is confirmed, the server will send a message to the other party saying, "Your reservation is complete. I look forward to seeing you."

[0063] Specific examples

[0064] A user schedules a business meeting, but the date, time, and location are not yet determined.

[0065] The server detects pending schedules and obtains contact information for the client.

[0066] The server refers to past communications and generates a message saying, "We need to decide on a meeting location and time. Can you make a suggestion?" and sends it to the client via the terminal.

[0067] The client replies, "How about a conference room in the ABC building at 2pm?"

[0068] The terminal receives this reply and sends it to the server.

[0069] The server parses the reply and confirms the proposed location and time.

[0070] The server accesses the conference room reservation site and makes the reservation at 2 p.m.

[0071] The server updates the confirmed schedule and notifies the user and client.

[0072] Since the user has enabled automatic conversation generation, a message saying "Reservation completed. We look forward to seeing you on the day" is sent to the client.

[0073] In this way, this system significantly reduces the user's workload by automating schedule management and communication, enabling efficient schedule adjustments.

[0074] The processing flow will be explained below.

[0075] Step 1:

[0076] The server periodically scans the schedule book database and detects unconfirmed schedules for which the date, time, and location have not been set.

[0077] Step 2:

[0078] Based on the detected pending schedule entry, the server retrieves the previously inputted contact information of the other person from the database.

[0079] Step 3:

[0080] The server analyzes past conversation history and uses natural language processing to generate an appropriate message, such as, "Can you suggest a location and time for our next lunch?"

[0081] Step 4:

[0082] The server converts the generated message into email or LINE format and sends it to the recipient via the device.

[0083] Step 5:

[0084] When the other person sends a reply message, the device receives the reply and sends the contents to the server.

[0085] Step 6:

[0086] The server analyzes the received message and extracts the location and time information of the reply.

[0087] Step 7:

[0088] Based on the extracted location information, the server accesses the reservation website of the relevant store or conference room and makes the necessary reservation. To make a reservation, the user inputs the date and time, number of people, special request information, etc.

[0089] Step 8:

[0090] The server acquires the reservation confirmation information and reflects it in the schedule book.

[0091] Step 9:

[0092] The server updates the schedule information to the schedule book database and notifies the user and the other party of the updated information via the terminal.

[0093] Step 10:

[0094] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history and new schedule information, such as "Your reservation is complete. I look forward to it."

[0095] Step 11:

[0096] The follow-up conversation generated by the server is converted into email or LINE format and sent to the other party via the device.

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] Conventional schedule management systems require users to manually adjust schedules, obtain necessary information, and communicate, which is time-consuming and labor-intensive and inefficient. In addition, the inability to analyze reply messages or automatically execute reservations increases the amount of work required and places a heavy burden on users.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and generating messages using natural language processing, means for sending the generated messages using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and making a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations, means for generating generated messages using a natural language processing model, means for identifying entries for which the date, time, and location of the schedule have not been set, means for analyzing received messages and extracting reply content, and means for continuously monitoring schedule information. This enables automation of schedule management and communication.

[0102] "Schedule information" refers to information such as date, time, location, and content that allows a user to manage their schedule.

[0103] "Contact information" refers to information in a database that stores contact information for users and their contacts, including telephone numbers and email addresses.

[0104] A "database" refers to a system for efficiently storing, managing, and searching various types of information.

[0105] "Conversation history" refers to the history of messages and conversations that a user has had with other people in the past.

[0106] "Natural language processing" refers to the technology that enables computers to understand, analyze, and generate human language.

[0107] "Message generation" refers to automatically creating appropriate sentences using natural language processing.

[0108] "Means of communication" refers to methods and technologies for sending and receiving information, including the Internet and SMS.

[0109] A "reply message" refers to a message received as a response from the other party.

[0110] "Analysis" refers to the process of understanding the content of a received message and extracting the necessary information.

[0111] "Reservation Site" refers to a website or application that allows you to make reservations for locations, dates, and times.

[0112] "Making a reservation" refers to accessing a reservation site, entering the necessary information, and confirming the reservation.

[0113] "Updating schedule information" refers to replacing existing schedule information with new information.

[0114] "Notification" refers to notifying the user of important information.

[0115] "Automatic conversation generation" refers to the system automatically generating new conversation messages.

[0116] A "follow-up conversation" refers to any additional interaction that occurs after a particular event.

[0117] A "natural language processing model" refers to an algorithm or system trained to perform natural language processing.

[0118] An "unspecified entry" refers to an item of schedule information for which the date, time, and location have not yet been determined.

[0119] "Reply content extraction" refers to extracting important information from received messages.

[0120] "Continuous monitoring" refers to the process of checking schedule information at regular intervals.

[0121] This invention is a system that improves the efficiency of users' schedule management and automates the acquisition of necessary information and communication. This system is composed of three main elements: a server, a terminal, and a user, and these elements work in conjunction with each other.

[0122] server

[0123] The server acts as the central control unit of the system and uses the following hardware and software:

[0124] Hardware: High-performance processors, memory, and storage devices

[0125] Software: A database (e.g., SQL Server) for managing schedule information, libraries for running natural language processing models (e.g., BERT, GPT-3), and a web server (e.g., Apache, Nginx).

[0126] The server monitors schedule information, retrieves contact information, generates and sends messages, analyzes reply messages, executes reservations, updates and notifies new schedule information, and automatically generates conversations.

[0127] Terminal

[0128] A terminal is a device operated by a user. The terminal uses the following hardware and software:

[0129] Hardware: Smartphones, tablets, computers

[0130] Software: Email clients, SMS applications, push notification services

[0131] The terminal communicates with the server to send and receive the generated messages, and also sends schedule updates and reservation completion notifications to the user.

[0132] User

[0133] Users are the main users of the system. They input schedule information, receive schedule updates and notifications, and can enable automatic conversation generation and follow-up messaging on request.

[0134] Specific examples

[0135] As an example, consider a situation where a user schedules a business meeting, but the date, time and location are still pending.

[0136] 1. A user schedules a business meeting, but the date, time, and location are not yet determined.

[0137] 2. The server periodically scans the scheduler database to detect any pending schedule entries.

[0138] 3. The server retrieves the contact information from the contact database and uses a natural language processing model to generate a message saying, "We need to decide on a meeting location and time. Can you suggest one?"

[0139] 4. The server generates the message and sends it to the other party via the device.

[0140] 5. The other person replies, "How about meeting in the conference room in the ABC building at 2 p.m.?" The device receives this reply and sends it to the server.

[0141] 6. The server parses the reply message and determines the proposed location and time.

[0142] 7. The server accesses the reservation site and reserves the conference room for 2 p.m.

[0143] 8. The server updates the confirmed schedule information in the schedule book database and notifies the user and the other party via the terminal.

[0144] 9. If the user has enabled automatic conversation generation, the server will generate an additional message saying, "Reservation completed. We look forward to seeing you on the day." and send it to the other party via the device.

[0145] Prompt Sentence Examples

[0146] "What should I do if I need to schedule a business meeting but the date, time, and location are not yet decided?"

[0147] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0148] Step 1:

[0149] The server periodically scans the planner database.

[0150] Input: Schedule database

[0151] Action: Run a SQL query to identify schedule entries that have no date, time, or location set. For example, use a query like "SELECT FROM schedules WHERE date IS NULL OR location IS NULL;"

[0152] Output: A list of pending schedule entries

[0153] Step 2:

[0154] The server retrieves the contact information of the other person from the user database based on the pending schedule entry.

[0155] Input: Pending schedule entry, user database

[0156] Action: Search and retrieve the relevant contact information. For example, run a query like "SELECT contact_info FROM user_data WHERE user_id = ?"

[0157] Output: Recipient's contact information

[0158] Step 3:

[0159] The server analyzes past conversation history and generates messages using natural language processing.

[0160] Input: Contact information of the other person, conversation history database

[0161] Processing: Using natural language processing models (e.g., BERT, GPT-3) to analyze past conversation history and generate appropriate messages based on the context, such as "Can you suggest a location and time for our next lunch?"

[0162] Output: The generated message

[0163] Step 4:

[0164] The server generates the message and sends it to the other party via the device.

[0165] Input: Generated message, contact information of the recipient

[0166] Process: Send a message using an SMS or email API. For example, use the Twilio API and execute the function "send_message(to=contact_number, body=generated_message)".

[0167] Output: Message sending status

[0168] Step 5:

[0169] The terminal receives the reply message and sends it to the server.

[0170] Input: Reply message

[0171] Processing: Use the message reception API to transfer the message received on the device to the server.

[0172] Output: Forwarding the received message to the server

[0173] Step 6:

[0174] The server analyzes the received message and extracts the reply content.

[0175] Input: Received message

[0176] Processing: Using natural language processing models and regular expressions, we extract important information from the reply messages, such as date, time, and location.

[0177] Output: Extracted reply content

[0178] Step 7:

[0179] The server accesses the reservation site and makes the reservation.

[0180] Input: Extracted reply content (date, time, location), reservation site URL

[0181] Processing: Using a browser automation tool such as Selenium, access the reservation site, enter the required information, and confirm the reservation. For example, execute "browser.find_element_by_name('date').send_keys('14:00')".

[0182] Output: Reservation confirmation information

[0183] Step 8:

[0184] The server updates the schedule information in the schedule book database, and the terminal sends a notification to the user.

[0185] Input: Reservation confirmation information, schedule database

[0186] Processing: Executes a SQL query to update schedule information and notify users via push notification or email. For example, execute a query like "UPDATE schedules SET date = '2023-11-05 14:00', location = 'ABC Building' WHERE id = ?"

[0187] Output: Updated schedule information, notification sending status

[0188] Step 9:

[0189] The server will auto-generate the conversation and send a follow-up conversation to the other person via their device (if the user has enabled auto-generated conversation).

[0190] Input: Updated schedule information, past conversation history

[0191] Processing: Using a natural language processing model, a follow-up message is generated and sent via email or SMS, such as "Your reservation is complete. We look forward to seeing you on the day."

[0192] Output: Follow message sending status

[0193] (Application example 1)

[0194] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0195] When on-site workers manually manage schedules and communicate, they are unable to respond in a timely manner, resulting in reduced efficiency. Adjusting schedules and procuring supplies is also time-consuming and often results in errors. This reduces production efficiency and hinders the smooth progress of work.

[0196] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0197] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-entered contact information from a database, means for analyzing past conversation history and generating messages using natural language processing, means for sending the generated messages using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and executing a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations, means for displaying schedule information and instructions in real time on a user's device, means for analyzing reply messages and automatically procuring necessary items and equipment, and means for notifying new instructions and changes in real time. This allows each worker to efficiently manage their schedule and communicate as needed in a timely manner. Furthermore, the automatic procurement of necessary items and equipment and real-time notification of new instructions and changes improve production efficiency and smooth business operations.

[0198] The "means for monitoring schedule information" refers to a means by which the system periodically scans the schedule book database to monitor schedule events related to the user.

[0199] The "means for obtaining contact information from a database" refers to a means for reading pre-entered contact information from a database.

[0200] "Means for analyzing past conversation history and generating messages using natural language processing" refers to means for analyzing past conversation data and automatically generating appropriate messages using natural language processing technology.

[0201] The "means for transmitting the generated message using a communication means" refers to a means for transmitting the generated message to another device or system via a communication means.

[0202] The "means for receiving and analyzing a reply message" is a means for analyzing a received reply message.

[0203] "Means for accessing a reservation site and making a reservation" refers to means for accessing an online reservation system and automatically making the necessary reservations.

[0204] The "means for updating and notifying new schedule information" is a means for updating schedule information to the latest information and notifying the user of the contents.

[0205] The "means for automatically generating a conversation and sending a follow-up conversation" is a means for the system to automatically generate conversation content and send the subsequent dialogue to the user.

[0206] The "means for displaying schedule information and instructions in real time on a user's device" refers to a means for displaying schedule information and necessary instructions in real time on a device held by a user.

[0207] The "means for analyzing the reply message and automatically procuring the necessary goods and equipment" is a means for analyzing the contents of the reply message and automatically procuring the necessary goods and equipment based on the results of the analysis.

[0208] The "means for notifying new instructions or changes in real time" refers to a means for notifying the user in real time when new instructions or changes are made.

[0209] This invention is a system that realizes efficient schedule management and automated communication for factory workers. This system uses smart glasses to display information in real time and automatically procure necessary goods and equipment.

[0210] System configuration

[0211] This system consists of the following main components:

[0212] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating and notifying schedules, generating automatic conversations, and automatically procuring goods and equipment.

[0213] Terminal: A device such as smart glasses operated by the user that communicates with the server and displays schedule information and instructions in real time.

[0214] User: A worker who uses the system, receives schedule information, and performs tasks according to instructions.

[0215] Program processing

[0216] Monitoring schedule information

[0217] The server periodically scans the schedule database to detect pending schedule entries.

[0218] Get contact information

[0219] The server retrieves relevant contact information from a database based on the pending schedule entry.

[0220] Message Creation and Transmission

[0221] The server analyzes past conversation history and generates appropriate messages using natural language processing technology. The generated messages are then sent via the terminal using communication means.

[0222] Parsing reply messages

[0223] When a reply message is received, the server analyzes it and extracts the necessary information based on the reply. For example, if a message is received saying, "Please check if there are any problems with the maintenance of equipment A at 1:00 PM," the server analyzes the reply and performs maintenance on equipment A.

[0224] Making a reservation

[0225] The server accesses the reservation site as needed and automatically makes the necessary reservations, entering information such as the date and time, number of people, and special requests to confirm the reservation.

[0226] Schedule Updates and Notifications

[0227] The server updates new schedule information and notifies the user through the terminal, and the user can receive schedule information and instructions in real time through the smart glasses.

[0228] Automated Procurement

[0229] The server analyzes the response message and includes means for automatically procuring the necessary goods or equipment, for example, if a specific part is needed, it automatically orders that part.

[0230] Real-time notifications

[0231] The server notifies the user's smart glasses in real time of any new instructions or changes.

[0232] Hardware and Software Used

[0233] Smart glasses: Using devices such as Google Glass and Vuzix.

[0234] API endpoint: Build a server that manages schedule data and message sending.

[0235] Python: Implement the program that communicates with the server and processes the data.

[0236] Specific examples

[0237] 1. When a user puts on the smart glasses, the system displays today's schedule.

[0238] Example: "8:00 AM - Maintenance on Equipment A"

[0239] Example: "10:00 AM - Quality Inspection Meeting"

[0240] 2. If there is no scheduled maintenance, the system will automatically generate a message.

[0241] Example prompt: "Please check if there are any problems with the maintenance on equipment A at 1 PM."

[0242] 3. The reply is then analyzed and the necessary reservations and supplies are automatically procured.

[0243] For example, if you reply "Confirmed at 1pm", the reservation is confirmed.

[0244] In this way, this system can improve the efficiency of worker schedule management and communication, thereby improving productivity.

[0245] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0246] Step 1:

[0247] The server monitors schedule information. It periodically scans the schedule book database as input to detect pending schedule entries. It generates a list of pending schedule entries as output and passes it to the next step.

[0248] Step 2:

[0249] The server retrieves contact information from the database. It uses the list of pending schedule entries as input and reads the associated contact information from the database. As output, it generates a list of schedule entries with the contact information, which it passes to the next step.

[0250] Step 3:

[0251] The server analyzes past conversation history and generates messages using natural language processing. The input is a list of schedule entries containing contact information and past conversation history. Data processing involves analyzing the conversation history and generating appropriate messages using a generative AI model. The output is a list of the generated messages, which are passed to the next step.

[0252] Step 4:

[0253] The server sends the generated message through the terminal. It uses the generated message and the recipient's contact information as input. Specifically, it sends the message using a communication protocol. As output, it records the result of the message transmission and passes it to the next step.

[0254] Step 5:

[0255] The terminal receives the reply message and sends it to the server. It uses the received message as input and sends it to the server via the endpoint. As output, it creates an object of the received message and passes it to the server.

[0256] Step 6:

[0257] The server analyzes the reply message, using the object of the received message as input. For data processing, it uses natural language processing technology to analyze the reply content and extract information necessary for scheduling and procuring goods. As output, it lists the extracted information and passes it on to the next step.

[0258] Step 7:

[0259] The server accesses the reservation site as needed and makes the reservation. The extracted reservation information is used as input. Specifically, it uses the reservation site's API to input the necessary information and confirm the reservation. As output, it generates reservation confirmation information and passes it to the next step.

[0260] Step 8:

[0261] The server updates the new schedule information and notifies the user via the terminal. The updated schedule information and the device reservation confirmation information are used as input. As output, the updated schedule data is saved in the schedule book database and a notification message is sent to the terminal.

[0262] Step 9:

[0263] The terminal displays real-time schedule information and instructions on the user's device. It uses notification messages as input and displays them in real-time on the user's smart glasses. As output, the displayed schedule and instructions are confirmed by the user.

[0264] Step 10:

[0265] The server analyzes the reply message and automatically procures the necessary goods and equipment. The analyzed data is used as input. Specifically, it automatically places the necessary orders using the goods procurement site's API. As output, it generates order confirmation information and passes it to the next step.

[0266] Step 11:

[0267] The server notifies new instructions and changes in real time. It uses new instructions and change information as input. As output, it generates notification messages in real time and sends them to the terminal.

[0268] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0269] The present invention is a system that streamlines user schedule management, automates the acquisition of necessary information, and automates communication. It also has the ability to recognize the user's emotions and dynamically change messages and follow-up conversations based on those emotions.

[0270] System configuration

[0271] This system consists of the following main components:

[0272] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[0273] Terminal: A device operated by the user that communicates with the server, sends and receives messages, and collects emotional data.

[0274] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[0275] Emotion Engine Functions

[0276] The emotion engine has the following functions:

[0277] It recognizes the user's emotional state by analyzing their voice, text messages, and facial expression data.

[0278] Dynamically change message content and follow-up conversations based on perceived emotional state.

[0279] Program processing

[0280] Monitoring schedule information and collecting emotion data

[0281] The server periodically scans the schedule database to detect unconfirmed schedules for which the date, time, and location have not yet been set, and the device collects facial expression data from the user via voice input, text messages, or the camera, and sends this data to the server.

[0282] Obtaining information about the other person and creating a message

[0283] The server retrieves the contact information of the person previously entered from a database based on the pending schedule entry. Next, it analyzes past conversation history and the user's emotional data and generates an appropriate message using natural language processing. For example, if it determines that the user is feeling stressed, it generates a message such as, "Could you please suggest a location and time for our next lunch?"

[0284] Sending automated messages

[0285] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[0286] Parsing reply messages

[0287] When the other person sends a reply message, the device receives the reply and sends the contents to the server, which analyzes the received message and extracts the location and time information of the reply.

[0288] Make the necessary reservations

[0289] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[0290] New schedule updates and notifications

[0291] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[0292] Optional conversation generation

[0293] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history, new schedule information, and emotional data. For example, if the user is feeling stressed, the server will generate a message like, "Relax and have fun," rather than, "Your reservation is complete, and I'm looking forward to it."

[0294] Specific examples

[0295] A user schedules a business meeting, but the date, time, and location are not yet determined.

[0296] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[0297] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[0298] The client replies, "How about a conference room in the ABC building at 2pm?"

[0299] The terminal receives this reply and sends it to the server.

[0300] The server parses the reply and confirms the proposed location and time.

[0301] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[0302] The server updates the confirmed schedule and notifies the user and client.

[0303] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[0304] In this way, this system significantly reduces the user's workload by automating schedule management and communication, and provides responses that take the user's feelings into consideration.

[0305] The processing flow will be explained below.

[0306] Step 1:

[0307] The server periodically scans the schedule book database and detects unconfirmed schedules for which the date, time, and location have not been set.

[0308] Step 2:

[0309] Based on the detected pending schedule entry, the server retrieves the previously inputted contact information of the other party from the database.

[0310] Step 3:

[0311] The device collects facial expression data using the user's voice input, text messages, or camera, and transmits it to a server in real time or periodically.

[0312] Step 4:

[0313] The server analyzes voice, text messages, and facial expression data to recognize the user's emotional state, for example, detecting stress levels from voice tone and smiles or discomfort from facial expression data.

[0314] Step 5:

[0315] The server uses natural language processing to generate an appropriate message based on the user's past conversation history and emotional state. For example, if the user is feeling stressed, it uses calmer language and generates a message like, "Could you please suggest a place and time for our next lunch?"

[0316] Step 6:

[0317] The server converts the generated message into email or LINE format and sends it to the recipient via the device.

[0318] Step 7:

[0319] When the other person sends a reply message, the device receives the reply and sends the contents to the server.

[0320] Step 8:

[0321] The server analyzes the received message and extracts the location and time information of the reply.

[0322] Step 9:

[0323] Based on the extracted location information, the server accesses the reservation website of the target store or conference room and makes the necessary reservation, entering the date and time, number of people, special request information, etc.

[0324] Step 10:

[0325] The server acquires the reservation confirmation information and reflects it in the schedule book.

[0326] Step 11:

[0327] The server updates the schedule information to the schedule book database and notifies the user and the other party of the updated information via the terminal.

[0328] Step 12:

[0329] If the user has enabled automatic conversation generation, the server generates follow-up conversations based on past conversation history, new schedule information, and emotional data. For example, if the user is relaxed, the server generates a message like, "Your reservation is complete. I'm looking forward to it." However, if the user is stressed, the server generates a message like, "Relax and have fun."

[0330] Step 13:

[0331] The follow-up conversation generated by the server is converted into email or LINE format and sent to the other party via the device.

[0332] Example 2

[0333] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0334] Conventional schedule management systems require users to manually manage their schedules, requiring a great deal of time and effort for tasks such as obtaining contact information, sending messages, and making reservations. Furthermore, the system does not take into account the user's emotions, which can lead to a decline in the quality of communication. To solve these problems, a system is needed that can streamline user schedule management, automate the acquisition of necessary information and communication, and dynamically respond based on emotions.

[0335] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0336] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and emotion data and generating a message using natural language processing technology, means for sending the generated message using a communication device, means for receiving and analyzing reply messages, means for accessing a reservation service and making a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations based on the user's emotions, and means for analyzing the user's voice input, text messages, and facial expression data to recognize the user's emotional state and changing responses based on this. This allows users to improve the efficiency of schedule management and automate communication, and also enables dynamic responses according to emotions.

[0337] "Schedule information" is information about plans and events entered by the user, including the date, time, location, and content.

[0338] The "monitoring means" is a means for continuously or periodically checking schedule information and performing processing based on specific conditions.

[0339] "Contact information" is information about a person or organization that is previously input by the user, including name, phone number, email address, etc.

[0340] A "database" is a system for managing, searching, and accessing data efficiently and quickly.

[0341] "Conversation history" is a record of messages and communications sent and received in the past.

[0342] "Emotion data" is data that indicates the user's emotional state. It is obtained by analyzing voice, text, facial expressions, etc.

[0343] "Natural language processing technology" is a technology that enables computers to understand, generate, and process human language.

[0344] "Message generation" is the process of creating an appropriate text message based on the input data.

[0345] A "communications device" is a hardware or software device for transmitting and receiving data or information.

[0346] A "reply message" is a reply message sent by the other party.

[0347] "Reservation Service" means a service that allows you to manage and make reservations online.

[0348] The "means for making a reservation" refers to the means for accessing the reservation service, inputting the information required for the reservation, and finalizing the reservation.

[0349] "Means for updating schedule information" refers to means for adding or modifying new or changed information in the schedule database.

[0350] "Means for notifying" refers to means for notifying users of updates and other important information.

[0351] "Automatic conversation generation" is a process that uses an algorithm to automatically generate appropriate conversation content.

[0352] A "follow-up conversation" is additional communication that occurs following a particular event or situation.

[0353] "Voice input" is a means of capturing the user's spoken voice as digital data.

[0354] "Facial expression data" is data obtained by analyzing the facial expression of the user.

[0355] An "emotional state" is a state that indicates how a user is feeling at a particular point in time.

[0356] The present invention is a system that streamlines user schedule management, automates the acquisition of necessary information, and automates communication. It also has the ability to recognize the user's emotions and dynamically change messages and follow-up conversations based on those emotions.

[0357] System configuration

[0358] The system consists of the following major components:

[0359] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[0360] Terminal: A device operated by the user that communicates with the server, sends and receives messages, and collects emotional data.

[0361] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[0362] Emotion Engine Functions

[0363] The emotion engine has the following functions:

[0364] It recognizes the user's emotional state by analyzing their voice, text messages, and facial expression data.

[0365] Dynamically change message content and follow-up conversations based on perceived emotional state.

[0366] Program processing

[0367] The server periodically scans the schedule database to detect unconfirmed schedules with unspecified dates, times, and locations. The device collects facial expression data from the user using voice input, text messages, and the camera, and sends it to the server.

[0368] The server retrieves pre-entered contact information from a database based on pending schedule entries, analyzes past conversation history and user emotion data, and generates an appropriate message using natural language processing technology.

[0369] For example, if it is determined that the user is feeling stressed, the system generates a message saying, "Could you please suggest a place and time for our next lunch?" This message is then sent to the other party via the device.

[0370] When a reply is received, the device receives it and sends it to the server. The server parses the reply and extracts the suggested location and time information. If necessary, the server accesses the reservation site and inputs the date, time, number of people, and any special requests to confirm the reservation.

[0371] Once the reservation is complete, the server updates the schedule database with the new schedule information, and the terminal sends the updated information and a notification that the reservation is complete to the user.

[0372] If the user has enabled automatic conversation generation, the server will generate follow-up conversations based on past conversation history, new schedule information, and emotional data. For example, if the user is feeling stressed, the server will send a message to the client saying, "Relax and have fun."

[0373] Specific examples

[0374] A user schedules a business meeting, but the date, time, and location are not yet determined.

[0375] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[0376] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[0377] The client replies, "How about a conference room in the ABC building at 2pm?"

[0378] The terminal receives this reply and sends it to the server.

[0379] The server parses the reply and confirms the proposed location and time.

[0380] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[0381] The server updates the confirmed schedule and notifies the user and client.

[0382] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[0383] Prompt Sentence Examples

[0384] "Analyze the user's voice input, text messages, and facial expression data to recognize the user's emotional state. Then, generate an appropriate message based on the recognized emotion. For example, if the user is stressed, generate a message saying, 'Relax and have fun.'"

[0385] The above is an embodiment of the present invention. This system improves the efficiency of the user's schedule management and communication, and is characterized by its ability to dynamically respond to emotions.

[0386] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0387] Step 1:

[0388] The server periodically scans the schedule database to detect unconfirmed schedules with unset dates, times, and locations. The input is the schedule database, and the output is a list of unconfirmed schedule entries. Data processing includes analyzing the schedule entries and selecting unconfirmed entries.

[0389] Step 2:

[0390] The device collects the user's voice input, text messages, and facial expression data using a camera, and sends it to a server. The input is the user's voice, text, and image data, and the output is emotional data sent to the server. Specifically, when the user operating the device says, "Tell me what's on my schedule for today," the device performs voice recognition and image analysis to quantify the user's emotional state.

[0391] Step 3:

[0392] The server retrieves previously entered contact information from a database based on the pending schedule entry. The input is the pending schedule entry and the database, and the output is the retrieved contact information. Data processing includes searching for appropriate contact information using keywords related to the schedule entry.

[0393] Step 4:

[0394] The server analyzes past conversation history and the user's emotional data and generates an appropriate message using a generative AI model. The input is the past conversation history and emotional data, and the output is the generated message. Specifically, if the emotional data obtained indicates that the user is feeling stressed, the server inputs a prompt to the AI ​​model saying, "Generate an appropriate message in a situation where the user is feeling stressed," and obtains the generated message.

[0395] Step 5:

[0396] The generated message is sent to the other party via the device. The input is the generated message, and the output is the message sent to the other party. Specifically, the device uses a message sending application to send the message to the other party.

[0397] Step 6:

[0398] When the other party sends a reply message, the device receives the reply and sends it to the server. The input is the reply message from the other party, and the output is the reply data sent to the server. The device uploads the reply to the server via the Internet.

[0399] Step 7:

[0400] The server analyzes the reply content and extracts the suggested location and time information. The input is the reply message, and the output is the extracted location and time information. Data processing involves using text analysis methods to extract specific information from the reply data.

[0401] Step 8:

[0402] The server accesses the booking site and confirms the booking by entering the date, time, number of people, and any special requests. The input is the extracted location and time information, and the output is the booking confirmation information. The server uses an automated script to enter the information into the booking site.

[0403] Step 9:

[0404] Once the reservation is completed, the server updates the schedule information to the schedule database, and the terminal sends a notification to the user. The input is the reservation confirmation information, and the output is the updated schedule information and a notification message. The terminal uses the notification system to send the message to the user.

[0405] Step 10:

[0406] If the user has enabled automatic conversation generation, the server generates a follow-up conversation based on the past conversation history, new schedule information, and emotional data. The input is the past conversation history, new schedule information, and emotional data, and the output is the generated follow-up conversation. Specifically, it generates a message saying, "Relax and have fun," and sends it to the client via the terminal.

[0407] (Application example 2)

[0408] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0409] Autonomous vehicles are required to not only efficiently manage users' schedules and automate necessary information acquisition and communication, but also to recognize the user's emotional state and dynamically change responses based on that. Conventional systems manage schedules without taking the user's emotional state into consideration, resulting in a poor user experience. In addition, navigation efficiency is lacking because destinations based on the schedule must be set manually.

[0410] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and generating a message using natural language processing, means for sending the generated message using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and making a reservation, means for updating and notifying new schedule information, means for automatically generating a conversation and sending a follow-up conversation, means for recognizing the user's emotional state using emotion recognition means and dynamically changing the message content and follow-up conversation based on the emotion, and means for cooperating with the vehicle's navigation system and setting a destination based on the user's schedule. This makes it possible to improve schedule management that takes the user's emotional state into consideration and the efficiency of navigation in autonomous vehicles.

[0411] "Schedule information" is time-related information such as the user's schedule, appointments, and events.

[0412] "Monitoring means" refers to devices or software that have the function of checking schedule information in real time or periodically and detecting abnormalities or updates.

[0413] "Contact information" is data about a person's means of contact, such as phone number, email address, or address.

[0414] A "database" is an organized collection of data for efficiently storing, retrieving, and managing information.

[0415] "Conversation history" is a record of past conversations and communications.

[0416] "Natural language processing" is the technology that enables computers to understand, generate, and respond to human language.

[0417] "Communication means" refers to the interface or infrastructure for sending and receiving data and messages.

[0418] A "reply message" is the content of the response received from the other party.

[0419] "Means of analysis" are devices or software that have the ability to examine data and extract specific patterns or information.

[0420] A "reservation site" is a website that allows online reservations for specific services (restaurants, hotels, conference rooms, etc.).

[0421] The "means for making a reservation" refers to a device or software that has the function of accessing a reservation site, inputting the necessary information, and finalizing the reservation.

[0422] The "means for updating schedule information" refers to a device or software that has the function of updating the schedule database with new information or changes.

[0423] "Notification means" refers to devices or software that have the functionality to notify users or relevant persons of updated information or reminders.

[0424] "Automatic conversation generation" is a technology that uses algorithms and AI to automatically create dialogue with users.

[0425] A "follow-up conversation" is any additional communication that follows an initial interaction or event.

[0426] "Emotion recognition means" is a technology that analyzes voice, text, facial expressions, etc. to determine the user's emotional state.

[0427] A "navigation system" is a device or software that calculates and provides directions to a destination.

[0428] The "means for setting a destination" refers to a device or software that has the function of determining a destination based on the user's schedule information and inputting it into the navigation system.

[0429] This system streamlines user schedule management and automates necessary information acquisition and communication. It also recognizes the user's emotions and dynamically changes messages and follow-up conversations based on those emotions. This system, especially when applied to autonomous vehicles, improves the in-car user experience.

[0430] System configuration

[0431] The system consists of the following main components:

[0432] 1. Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules, and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[0433] 2. Terminal: A device operated by the user that communicates with the server, sends and receives messages, collects emotional data, and works with the navigation system of the autonomous vehicle.

[0434] 3. User: The entity that uses the system, inputs schedule information, and receives updated schedules and notifications.

[0435] Program processing

[0436] Monitoring schedule information and collecting emotion data

[0437] The server periodically scans the schedule database to detect unconfirmed schedules for which the date, time, and location have not yet been set. The device also collects facial expression data from the user via voice input, text messages, or the camera, and sends this data to the server.

[0438] Obtaining information about the other person and creating a message

[0439] The server retrieves the contact information of the person previously entered from a database based on the pending schedule entry. Next, it analyzes past conversation history and the user's emotional data and generates an appropriate message using natural language processing. For example, if it determines that the user is feeling stressed, it generates a message such as, "Could you please suggest a location and time for our next lunch?"

[0440] Sending automated messages

[0441] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[0442] Parsing reply messages

[0443] When the other person sends a reply message, the device receives the reply and sends the contents to the server, which analyzes the received message and extracts the location and time information of the reply.

[0444] Make the necessary reservations

[0445] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[0446] New schedule updates and notifications

[0447] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[0448] Linking with navigation systems

[0449] It works in conjunction with the navigation system of the autonomous vehicle to set the destination based on the user's schedule. To do this, schedule data is input into the navigation system so that the vehicle automatically heads to the destination.

[0450] Specific examples

[0451] A user schedules a business meeting, but the date, time, and location are not yet determined.

[0452] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[0453] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[0454] The client replies, "How about a conference room in the ABC building at 2pm?"

[0455] The terminal receives this reply and sends it to the server.

[0456] The server parses the reply and confirms the proposed location and time.

[0457] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[0458] The server updates the confirmed schedule and notifies the user and client.

[0459] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[0460] Prompt Sentence Examples

[0461] "Generate a message to ask the user to suggest a location and time for an unscheduled event."

[0462] "Generate a message to help the user relax if they are currently feeling stressed."

[0463] As described above, this system significantly reduces the user's workload through the automation of schedule management and communication, and can respond in a way that takes the user's feelings into consideration. Furthermore, by linking with the navigation system of an autonomous vehicle, it can improve the efficiency and comfort of travel.

[0464] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0465] Step 1:

[0466] The server periodically scans the schedule book database. It takes all entries in the schedule book database as input and detects unconfirmed schedules (those for which the date, time, and location have not been set). It generates a list of unconfirmed schedules as output. This data processing or data calculation checks the fields of the schedule entries and lists the entries with unconfirmed fields.

[0467] Step 2:

[0468] The server retrieves the contact information of the other party that was previously input based on the pending schedule from the database. It uses the list of pending schedules as input and accesses the contact information database of the other party. It generates a list containing the contact information of the other party as output. In this process, it searches the database for the contact information of the other party that corresponds to each schedule entry and compiles it into a list.

[0469] Step 3:

[0470] The server collects facial expression data from the user via voice input, text messages, or a camera, and analyzes it with an emotion engine. The inputs are voice, text, and facial expression data from the user. The output is an analyzed emotional state (e.g., stress, joy). This process uses a generative AI model to analyze the emotional data and performs calculations to identify the user's emotional state.

[0471] Step 4:

[0472] The server uses natural language processing to generate an appropriate message based on the pending schedule, the other person's contact information, and the analyzed emotional state. Details of the pending schedule, contact information, and emotional state are used as input. The generated message is obtained as output. This process utilizes natural language processing with a generative AI model, and generates messages while also referencing past conversation history.

[0473] Step 5:

[0474] The server sends the generated message to the other party using a communication method. It uses the generated message and the other party's contact information as input. It obtains the sending completion status of the message sent to the other party as output. This process uses the messaging service API to perform the specific operations to send the message.

[0475] Step 6:

[0476] The device receives the message that the other party has replied to and sends it to the server. The reply message from the other party is used as input. The reply message sent to the server is obtained as output. This process uses the device's message receiving function to transfer the received message to the server via the network.

[0477] Step 7:

[0478] The server analyzes the reply message and extracts the location and time information of the reply. It uses the reply message as input and obtains the extracted location and time information as output. This process uses natural language processing technology to analyze the message content and performs specific calculations to extract the necessary information.

[0479] Step 8:

[0480] The server accesses the reservation site based on the extracted information and automatically executes the reservation. The extracted location and time information is used as input. The reservation completion status and reservation confirmation information are obtained as output. This process automatically inputs information using the reservation site's API and confirms the reservation.

[0481] Step 9:

[0482] The server updates the schedule book database with the new schedule information and sends a notification. It uses the reservation confirmation information and the new schedule information as input. It obtains the updated schedule and the notification sending completion status as output. This process performs the specific operations of updating the database and sending the notification using the notification service API.

[0483] Step 10:

[0484] The server works with the autonomous vehicle's navigation system to set a destination based on the user's schedule. It uses new schedule information as input and reflects the set destination information in the navigation system as output. This process uses the navigation system's API to perform specific operations to set the destination.

[0485] Step 11:

[0486] If the user has enabled automatic conversation generation, the server generates a follow-up conversation based on the user's emotional data and schedule information and sends it to the other party. The server uses the user's emotional data and schedule information as input. The generated follow-up conversation is obtained as output. This process uses a generative AI model to generate a follow-up conversation and sends it using the messaging service API.

[0487] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0488] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0489] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0490] [Second embodiment]

[0491] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0492] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0493] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0494] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0495] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0496] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0497] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0498] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0499] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0500] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0501] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0502] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0503] The present invention is a system that streamlines users' schedule management and automates the acquisition of necessary information and communication. This system has the functions of monitoring schedule information, generating and sending messages based on contact information, and analyzing replies. It also has the functions of making necessary reservations, updating schedules, and notifying users.

[0504] System configuration

[0505] This system consists of the following main components:

[0506] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications.

[0507] Terminal: A device operated by a user that communicates with the server and sends and receives messages.

[0508] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[0509] Program processing

[0510] Monitoring schedule information

[0511] The server periodically scans the schedule database to detect pending schedule entries, specifically entries for which the date, time, and location of the schedule have not been set.

[0512] Obtaining information about the other person and creating a message

[0513] The server retrieves the contact information of the person in question from a database based on the pending schedule entry, analyzes past conversation history, and uses natural language processing to generate an appropriate message, such as, "Could you please suggest a location and time for our next lunch?"

[0514] Sending automated messages

[0515] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[0516] Parsing reply messages

[0517] When the other person sends a reply message, the device receives it and sends it to the server, which analyzes the received message and extracts the reply content (location, time, etc.).

[0518] Make the necessary reservations

[0519] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[0520] New schedule updates and notifications

[0521] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[0522] Optional conversation generation

[0523] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history and new schedule information. For example, when a reservation is confirmed, the server will send a message to the other party saying, "Your reservation is complete. I look forward to seeing you."

[0524] Specific examples

[0525] A user schedules a business meeting, but the date, time, and location are not yet determined.

[0526] The server detects pending schedules and obtains contact information for the client.

[0527] The server refers to past communications and generates a message saying, "We need to decide on a meeting location and time. Can you make a suggestion?" and sends it to the client via the terminal.

[0528] The client replies, "How about a conference room in the ABC building at 2pm?"

[0529] The terminal receives this reply and sends it to the server.

[0530] The server parses the reply and confirms the proposed location and time.

[0531] The server accesses the conference room reservation site and makes the reservation at 2 p.m.

[0532] The server updates the confirmed schedule and notifies the user and client.

[0533] Since the user has enabled automatic conversation generation, a message saying "Reservation completed. We look forward to seeing you on the day" is sent to the client.

[0534] In this way, this system significantly reduces the user's workload by automating schedule management and communication, enabling efficient schedule adjustments.

[0535] The processing flow will be explained below.

[0536] Step 1:

[0537] The server periodically scans the schedule book database and detects unconfirmed schedules for which the date, time, and location have not been set.

[0538] Step 2:

[0539] Based on the detected pending schedule entry, the server retrieves the previously inputted contact information of the other person from the database.

[0540] Step 3:

[0541] The server analyzes past conversation history and uses natural language processing to generate an appropriate message, such as, "Can you suggest a location and time for our next lunch?"

[0542] Step 4:

[0543] The server converts the generated message into email or LINE format and sends it to the recipient via the device.

[0544] Step 5:

[0545] When the other person sends a reply message, the device receives the reply and sends the contents to the server.

[0546] Step 6:

[0547] The server analyzes the received message and extracts the location and time information of the reply.

[0548] Step 7:

[0549] Based on the extracted location information, the server accesses the reservation website of the relevant store or conference room and makes the necessary reservation. To make a reservation, the user inputs the date and time, number of people, special request information, etc.

[0550] Step 8:

[0551] The server acquires the reservation confirmation information and reflects it in the schedule book.

[0552] Step 9:

[0553] The server updates the schedule information to the schedule book database and notifies the user and the other party of the updated information via the terminal.

[0554] Step 10:

[0555] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history and new schedule information, such as "Your reservation is complete. I look forward to it."

[0556] Step 11:

[0557] The follow-up conversation generated by the server is converted into email or LINE format and sent to the other party via the device.

[0558] Example 1

[0559] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0560] Conventional schedule management systems require users to manually adjust schedules, obtain necessary information, and communicate, which is time-consuming and labor-intensive and inefficient. In addition, the inability to analyze reply messages or automatically execute reservations increases the amount of work required and places a heavy burden on users.

[0561] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0562] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and generating messages using natural language processing, means for sending the generated messages using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and making a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations, means for generating generated messages using a natural language processing model, means for identifying entries for which the date, time, and location of the schedule have not been set, means for analyzing received messages and extracting reply content, and means for continuously monitoring schedule information. This enables automation of schedule management and communication.

[0563] "Schedule information" refers to information such as date, time, location, and content that allows a user to manage their schedule.

[0564] "Contact information" refers to information in a database that stores contact information for users and their contacts, including telephone numbers and email addresses.

[0565] A "database" refers to a system for efficiently storing, managing, and searching various types of information.

[0566] "Conversation history" refers to the history of messages and conversations that a user has had with other people in the past.

[0567] "Natural language processing" refers to the technology that enables computers to understand, analyze, and generate human language.

[0568] "Message generation" refers to automatically creating appropriate sentences using natural language processing.

[0569] "Means of communication" refers to methods and technologies for sending and receiving information, including the Internet and SMS.

[0570] A "reply message" refers to a message received as a response from the other party.

[0571] "Analysis" refers to the process of understanding the content of a received message and extracting the necessary information.

[0572] "Reservation Site" refers to a website or application that allows you to make reservations for locations, dates, and times.

[0573] "Making a reservation" refers to accessing a reservation site, entering the necessary information, and confirming the reservation.

[0574] "Updating schedule information" refers to replacing existing schedule information with new information.

[0575] "Notification" refers to notifying the user of important information.

[0576] "Automatic conversation generation" refers to the system automatically generating new conversation messages.

[0577] A "follow-up conversation" refers to any additional interaction that occurs after a particular event.

[0578] A "natural language processing model" refers to an algorithm or system trained to perform natural language processing.

[0579] An "unspecified entry" refers to an item of schedule information for which the date, time, and location have not yet been determined.

[0580] "Reply content extraction" refers to extracting important information from received messages.

[0581] "Continuous monitoring" refers to the process of checking schedule information at regular intervals.

[0582] This invention is a system that improves the efficiency of users' schedule management and automates the acquisition of necessary information and communication. This system is composed of three main elements: a server, a terminal, and a user, and these elements work in conjunction with each other.

[0583] server

[0584] The server acts as the central control unit of the system and uses the following hardware and software:

[0585] Hardware: High-performance processors, memory, and storage devices

[0586] Software: A database (e.g., SQL Server) for managing schedule information, libraries for running natural language processing models (e.g., BERT, GPT-3), and a web server (e.g., Apache, Nginx).

[0587] The server monitors schedule information, retrieves contact information, generates and sends messages, analyzes reply messages, executes reservations, updates and notifies new schedule information, and automatically generates conversations.

[0588] Terminal

[0589] A terminal is a device operated by a user. The terminal uses the following hardware and software:

[0590] Hardware: Smartphones, tablets, computers

[0591] Software: Email clients, SMS applications, push notification services

[0592] The terminal communicates with the server to send and receive the generated messages, and also sends schedule updates and reservation completion notifications to the user.

[0593] User

[0594] Users are the main users of the system. They input schedule information, receive schedule updates and notifications, and can enable automatic conversation generation and follow-up messaging on request.

[0595] Specific examples

[0596] As an example, consider a situation where a user schedules a business meeting, but the date, time and location are still pending.

[0597] 1. A user schedules a business meeting, but the date, time, and location are not yet determined.

[0598] 2. The server periodically scans the scheduler database to detect any pending schedule entries.

[0599] 3. The server retrieves the contact information from the contact database and uses a natural language processing model to generate a message saying, "We need to decide on a meeting location and time. Can you suggest one?"

[0600] 4. The server generates the message and sends it to the other party via the device.

[0601] 5. The other person replies, "How about meeting in the conference room in the ABC building at 2 p.m.?" The device receives this reply and sends it to the server.

[0602] 6. The server parses the reply message and determines the proposed location and time.

[0603] 7. The server accesses the reservation site and reserves the conference room for 2 p.m.

[0604] 8. The server updates the confirmed schedule information in the schedule book database and notifies the user and the other party via the terminal.

[0605] 9. If the user has enabled automatic conversation generation, the server will generate an additional message saying, "Reservation completed. We look forward to seeing you on the day." and send it to the other party via the device.

[0606] Prompt Sentence Examples

[0607] "What should I do if I need to schedule a business meeting but the date, time, and location are not yet decided?"

[0608] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0609] Step 1:

[0610] The server periodically scans the planner database.

[0611] Input: Schedule database

[0612] Action: Run a SQL query to identify schedule entries that have no date, time, or location set. For example, use a query like "SELECT FROM schedules WHERE date IS NULL OR location IS NULL;"

[0613] Output: A list of pending schedule entries

[0614] Step 2:

[0615] The server retrieves the contact information of the other person from the user database based on the pending schedule entry.

[0616] Input: Pending schedule entry, user database

[0617] Action: Search and retrieve the relevant contact information. For example, run a query like "SELECT contact_info FROM user_data WHERE user_id = ?"

[0618] Output: Recipient's contact information

[0619] Step 3:

[0620] The server analyzes past conversation history and generates messages using natural language processing.

[0621] Input: Contact information of the other person, conversation history database

[0622] Processing: Using natural language processing models (e.g., BERT, GPT-3) to analyze past conversation history and generate appropriate messages based on the context, such as "Can you suggest a location and time for our next lunch?"

[0623] Output: The generated message

[0624] Step 4:

[0625] The server generates the message and sends it to the other party via the device.

[0626] Input: Generated message, contact information of the recipient

[0627] Process: Send a message using an SMS or email API. For example, use the Twilio API and execute the function "send_message(to=contact_number, body=generated_message)".

[0628] Output: Message sending status

[0629] Step 5:

[0630] The terminal receives the reply message and sends it to the server.

[0631] Input: Reply message

[0632] Processing: Use the message reception API to transfer the message received on the device to the server.

[0633] Output: Forwarding the received message to the server

[0634] Step 6:

[0635] The server analyzes the received message and extracts the reply content.

[0636] Input: Received message

[0637] Processing: Using natural language processing models and regular expressions, we extract important information from the reply messages, such as date, time, and location.

[0638] Output: Extracted reply content

[0639] Step 7:

[0640] The server accesses the reservation site and makes the reservation.

[0641] Input: Extracted reply content (date, time, location), reservation site URL

[0642] Processing: Using a browser automation tool such as Selenium, access the reservation site, enter the required information, and confirm the reservation. For example, execute "browser.find_element_by_name('date').send_keys('14:00')".

[0643] Output: Reservation confirmation information

[0644] Step 8:

[0645] The server updates the schedule information in the schedule book database, and the terminal sends a notification to the user.

[0646] Input: Reservation confirmation information, schedule database

[0647] Processing: Executes a SQL query to update schedule information and notify users via push notification or email. For example, execute a query like "UPDATE schedules SET date = '2023-11-05 14:00', location = 'ABC Building' WHERE id = ?"

[0648] Output: Updated schedule information, notification sending status

[0649] Step 9:

[0650] The server will auto-generate the conversation and send a follow-up conversation to the other person via their device (if the user has enabled auto-generated conversation).

[0651] Input: Updated schedule information, past conversation history

[0652] Processing: Using a natural language processing model, a follow-up message is generated and sent via email or SMS, such as "Your reservation is complete. We look forward to seeing you on the day."

[0653] Output: Follow message sending status

[0654] (Application example 1)

[0655] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0656] When on-site workers manually manage schedules and communicate, they are unable to respond in a timely manner, resulting in reduced efficiency. Adjusting schedules and procuring supplies is also time-consuming and often results in errors. This reduces production efficiency and hinders the smooth progress of work.

[0657] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0658] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-entered contact information from a database, means for analyzing past conversation history and generating messages using natural language processing, means for sending the generated messages using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and executing a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations, means for displaying schedule information and instructions in real time on a user's device, means for analyzing reply messages and automatically procuring necessary items and equipment, and means for notifying new instructions and changes in real time. This allows each worker to efficiently manage their schedule and communicate as needed in a timely manner. Furthermore, the automatic procurement of necessary items and equipment and real-time notification of new instructions and changes improve production efficiency and smooth business operations.

[0659] The "means for monitoring schedule information" refers to a means by which the system periodically scans the schedule book database to monitor schedule events related to the user.

[0660] The "means for obtaining contact information from a database" refers to a means for reading pre-entered contact information from a database.

[0661] "Means for analyzing past conversation history and generating messages using natural language processing" refers to means for analyzing past conversation data and automatically generating appropriate messages using natural language processing technology.

[0662] The "means for transmitting the generated message using a communication means" refers to a means for transmitting the generated message to another device or system via a communication means.

[0663] The "means for receiving and analyzing a reply message" is a means for analyzing a received reply message.

[0664] "Means for accessing a reservation site and making a reservation" refers to means for accessing an online reservation system and automatically making the necessary reservations.

[0665] The "means for updating and notifying new schedule information" is a means for updating schedule information to the latest information and notifying the user of the contents.

[0666] The "means for automatically generating a conversation and sending a follow-up conversation" is a means for the system to automatically generate conversation content and send the subsequent dialogue to the user.

[0667] The "means for displaying schedule information and instructions in real time on a user's device" refers to a means for displaying schedule information and necessary instructions in real time on a device held by a user.

[0668] The "means for analyzing the reply message and automatically procuring the necessary goods and equipment" is a means for analyzing the contents of the reply message and automatically procuring the necessary goods and equipment based on the results of the analysis.

[0669] The "means for notifying new instructions or changes in real time" refers to a means for notifying the user in real time when new instructions or changes are made.

[0670] This invention is a system that realizes efficient schedule management and automated communication for factory workers. This system uses smart glasses to display information in real time and automatically procure necessary goods and equipment.

[0671] System configuration

[0672] This system consists of the following main components:

[0673] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating and notifying schedules, generating automatic conversations, and automatically procuring goods and equipment.

[0674] Terminal: A device such as smart glasses operated by the user that communicates with the server and displays schedule information and instructions in real time.

[0675] User: A worker who uses the system, receives schedule information, and performs tasks according to instructions.

[0676] Program processing

[0677] Monitoring schedule information

[0678] The server periodically scans the schedule database to detect pending schedule entries.

[0679] Get contact information

[0680] The server retrieves relevant contact information from a database based on the pending schedule entry.

[0681] Message Creation and Transmission

[0682] The server analyzes past conversation history and generates appropriate messages using natural language processing technology. The generated messages are then sent via the terminal using communication means.

[0683] Parsing reply messages

[0684] When a reply message is received, the server analyzes it and extracts the necessary information based on the reply. For example, if a message is received saying, "Please check if there are any problems with the maintenance of equipment A at 1:00 PM," the server analyzes the reply and performs maintenance on equipment A.

[0685] Making a reservation

[0686] The server accesses the reservation site as needed and automatically makes the necessary reservations, entering information such as the date and time, number of people, and special requests to confirm the reservation.

[0687] Schedule Updates and Notifications

[0688] The server updates new schedule information and notifies the user through the terminal, and the user can receive schedule information and instructions in real time through the smart glasses.

[0689] Automated Procurement

[0690] The server analyzes the response message and includes means for automatically procuring the necessary goods or equipment, for example, if a specific part is needed, it automatically orders that part.

[0691] Real-time notifications

[0692] The server notifies the user's smart glasses in real time of any new instructions or changes.

[0693] Hardware and Software Used

[0694] Smart glasses: Using devices such as Google Glass and Vuzix.

[0695] API endpoint: Build a server that manages schedule data and message sending.

[0696] Python: Implement the program that communicates with the server and processes the data.

[0697] Specific examples

[0698] 1. When a user puts on the smart glasses, the system displays today's schedule.

[0699] Example: "8:00 AM - Maintenance on Equipment A"

[0700] Example: "10:00 AM - Quality Inspection Meeting"

[0701] 2. If there is no scheduled maintenance, the system will automatically generate a message.

[0702] Example prompt: "Please check if there are any problems with the maintenance on equipment A at 1 PM."

[0703] 3. The reply is then analyzed and the necessary reservations and supplies are automatically procured.

[0704] For example, if you reply "Confirmed at 1pm", the reservation is confirmed.

[0705] In this way, this system can improve the efficiency of worker schedule management and communication, thereby improving productivity.

[0706] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0707] Step 1:

[0708] The server monitors schedule information. It periodically scans the schedule book database as input to detect pending schedule entries. It generates a list of pending schedule entries as output and passes it to the next step.

[0709] Step 2:

[0710] The server retrieves contact information from the database. It uses the list of pending schedule entries as input and reads the associated contact information from the database. As output, it generates a list of schedule entries with the contact information, which it passes to the next step.

[0711] Step 3:

[0712] The server analyzes past conversation history and generates messages using natural language processing. The input is a list of schedule entries containing contact information and past conversation history. Data processing involves analyzing the conversation history and generating appropriate messages using a generative AI model. The output is a list of the generated messages, which are passed to the next step.

[0713] Step 4:

[0714] The server sends the generated message through the terminal. It uses the generated message and the recipient's contact information as input. Specifically, it sends the message using a communication protocol. As output, it records the result of the message transmission and passes it to the next step.

[0715] Step 5:

[0716] The terminal receives the reply message and sends it to the server. It uses the received message as input and sends it to the server via the endpoint. As output, it creates an object of the received message and passes it to the server.

[0717] Step 6:

[0718] The server analyzes the reply message, using the object of the received message as input. For data processing, it uses natural language processing technology to analyze the reply content and extract information necessary for scheduling and procuring goods. As output, it lists the extracted information and passes it on to the next step.

[0719] Step 7:

[0720] The server accesses the reservation site as needed and makes the reservation. The extracted reservation information is used as input. Specifically, it uses the reservation site's API to input the necessary information and confirm the reservation. As output, it generates reservation confirmation information and passes it to the next step.

[0721] Step 8:

[0722] The server updates the new schedule information and notifies the user via the terminal. The updated schedule information and the device reservation confirmation information are used as input. As output, the updated schedule data is saved in the schedule book database and a notification message is sent to the terminal.

[0723] Step 9:

[0724] The terminal displays real-time schedule information and instructions on the user's device. It uses notification messages as input and displays them in real-time on the user's smart glasses. As output, the displayed schedule and instructions are confirmed by the user.

[0725] Step 10:

[0726] The server analyzes the reply message and automatically procures the necessary goods and equipment. The analyzed data is used as input. Specifically, it automatically places the necessary orders using the goods procurement site's API. As output, it generates order confirmation information and passes it to the next step.

[0727] Step 11:

[0728] The server notifies new instructions and changes in real time. It uses new instructions and change information as input. As output, it generates notification messages in real time and sends them to the terminal.

[0729] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0730] The present invention is a system that streamlines user schedule management, automates the acquisition of necessary information, and automates communication. It also has the ability to recognize the user's emotions and dynamically change messages and follow-up conversations based on those emotions.

[0731] System configuration

[0732] This system consists of the following main components:

[0733] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[0734] Terminal: A device operated by the user that communicates with the server, sends and receives messages, and collects emotional data.

[0735] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[0736] Emotion Engine Functions

[0737] The emotion engine has the following functions:

[0738] It recognizes the user's emotional state by analyzing their voice, text messages, and facial expression data.

[0739] Dynamically change message content and follow-up conversations based on perceived emotional state.

[0740] Program processing

[0741] Monitoring schedule information and collecting emotion data

[0742] The server periodically scans the schedule database to detect unconfirmed schedules for which the date, time, and location have not yet been set, and the device collects facial expression data from the user via voice input, text messages, or the camera, and sends this data to the server.

[0743] Obtaining information about the other person and creating a message

[0744] The server retrieves the contact information of the person previously entered from a database based on the pending schedule entry. Next, it analyzes past conversation history and the user's emotional data and generates an appropriate message using natural language processing. For example, if it determines that the user is feeling stressed, it generates a message such as, "Could you please suggest a location and time for our next lunch?"

[0745] Sending automated messages

[0746] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[0747] Parsing reply messages

[0748] When the other person sends a reply message, the device receives the reply and sends the contents to the server, which analyzes the received message and extracts the location and time information of the reply.

[0749] Make the necessary reservations

[0750] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[0751] New schedule updates and notifications

[0752] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[0753] Optional conversation generation

[0754] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history, new schedule information, and emotional data. For example, if the user is feeling stressed, the server will generate a message like, "Relax and have fun," rather than, "Your reservation is complete, and I'm looking forward to it."

[0755] Specific examples

[0756] A user schedules a business meeting, but the date, time, and location are not yet determined.

[0757] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[0758] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[0759] The client replies, "How about a conference room in the ABC building at 2pm?"

[0760] The terminal receives this reply and sends it to the server.

[0761] The server parses the reply and confirms the proposed location and time.

[0762] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[0763] The server updates the confirmed schedule and notifies the user and client.

[0764] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[0765] In this way, this system significantly reduces the user's workload by automating schedule management and communication, and provides responses that take the user's feelings into consideration.

[0766] The processing flow will be explained below.

[0767] Step 1:

[0768] The server periodically scans the schedule book database and detects unconfirmed schedules for which the date, time, and location have not been set.

[0769] Step 2:

[0770] Based on the detected pending schedule entry, the server retrieves the previously inputted contact information of the other party from the database.

[0771] Step 3:

[0772] The device collects facial expression data using the user's voice input, text messages, or camera, and transmits it to a server in real time or periodically.

[0773] Step 4:

[0774] The server analyzes voice, text messages, and facial expression data to recognize the user's emotional state, for example, detecting stress levels from voice tone and smiles or discomfort from facial expression data.

[0775] Step 5:

[0776] The server uses natural language processing to generate an appropriate message based on the user's past conversation history and emotional state. For example, if the user is feeling stressed, it uses calmer language and generates a message like, "Could you please suggest a place and time for our next lunch?"

[0777] Step 6:

[0778] The server converts the generated message into email or LINE format and sends it to the recipient via the device.

[0779] Step 7:

[0780] When the other person sends a reply message, the device receives the reply and sends the contents to the server.

[0781] Step 8:

[0782] The server analyzes the received message and extracts the location and time information of the reply.

[0783] Step 9:

[0784] Based on the extracted location information, the server accesses the reservation website of the target store or conference room and makes the necessary reservation, entering the date and time, number of people, special request information, etc.

[0785] Step 10:

[0786] The server acquires the reservation confirmation information and reflects it in the schedule book.

[0787] Step 11:

[0788] The server updates the schedule information to the schedule book database and notifies the user and the other party of the updated information via the terminal.

[0789] Step 12:

[0790] If the user has enabled automatic conversation generation, the server generates follow-up conversations based on past conversation history, new schedule information, and emotional data. For example, if the user is relaxed, the server generates a message like, "Your reservation is complete. I'm looking forward to it." However, if the user is stressed, the server generates a message like, "Relax and have fun."

[0791] Step 13:

[0792] The follow-up conversation generated by the server is converted into email or LINE format and sent to the other party via the device.

[0793] Example 2

[0794] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0795] Conventional schedule management systems require users to manually manage their schedules, requiring a great deal of time and effort for tasks such as obtaining contact information, sending messages, and making reservations. Furthermore, the system does not take into account the user's emotions, which can lead to a decline in the quality of communication. To solve these problems, a system is needed that can streamline user schedule management, automate the acquisition of necessary information and communication, and dynamically respond based on emotions.

[0796] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0797] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and emotion data and generating a message using natural language processing technology, means for sending the generated message using a communication device, means for receiving and analyzing reply messages, means for accessing a reservation service and making a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations based on the user's emotions, and means for analyzing the user's voice input, text messages, and facial expression data to recognize the user's emotional state and changing responses based on this. This allows users to improve the efficiency of schedule management and automate communication, and also enables dynamic responses according to emotions.

[0798] "Schedule information" is information about plans and events entered by the user, including the date, time, location, and content.

[0799] The "monitoring means" is a means for continuously or periodically checking schedule information and performing processing based on specific conditions.

[0800] "Contact information" is information about a person or organization that is previously input by the user, including name, phone number, email address, etc.

[0801] A "database" is a system for managing, searching, and accessing data efficiently and quickly.

[0802] "Conversation history" is a record of messages and communications sent and received in the past.

[0803] "Emotion data" is data that indicates the user's emotional state. It is obtained by analyzing voice, text, facial expressions, etc.

[0804] "Natural language processing technology" is a technology that enables computers to understand, generate, and process human language.

[0805] "Message generation" is the process of creating an appropriate text message based on the input data.

[0806] A "communications device" is a hardware or software device for transmitting and receiving data or information.

[0807] A "reply message" is a reply message sent by the other party.

[0808] "Reservation Service" means a service that allows you to manage and make reservations online.

[0809] The "means for making a reservation" refers to the means for accessing the reservation service, inputting the information required for the reservation, and finalizing the reservation.

[0810] "Means for updating schedule information" refers to means for adding or modifying new or changed information in the schedule database.

[0811] "Means for notifying" refers to means for notifying users of updates and other important information.

[0812] "Automatic conversation generation" is a process that uses an algorithm to automatically generate appropriate conversation content.

[0813] A "follow-up conversation" is additional communication that occurs following a particular event or situation.

[0814] "Voice input" is a means of capturing the user's spoken voice as digital data.

[0815] "Facial expression data" is data obtained by analyzing the facial expression of the user.

[0816] An "emotional state" is a state that indicates how a user is feeling at a particular point in time.

[0817] The present invention is a system that streamlines user schedule management, automates the acquisition of necessary information, and automates communication. It also has the ability to recognize the user's emotions and dynamically change messages and follow-up conversations based on those emotions.

[0818] System configuration

[0819] The system consists of the following major components:

[0820] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[0821] Terminal: A device operated by the user that communicates with the server, sends and receives messages, and collects emotional data.

[0822] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[0823] Emotion Engine Functions

[0824] The emotion engine has the following functions:

[0825] It recognizes the user's emotional state by analyzing their voice, text messages, and facial expression data.

[0826] Dynamically change message content and follow-up conversations based on perceived emotional state.

[0827] Program processing

[0828] The server periodically scans the schedule database to detect unconfirmed schedules with unspecified dates, times, and locations. The device collects facial expression data from the user using voice input, text messages, and the camera, and sends it to the server.

[0829] The server retrieves pre-entered contact information from a database based on pending schedule entries, analyzes past conversation history and user emotion data, and generates an appropriate message using natural language processing technology.

[0830] For example, if it is determined that the user is feeling stressed, the system generates a message saying, "Could you please suggest a place and time for our next lunch?" This message is then sent to the other party via the device.

[0831] When a reply is received, the device receives it and sends it to the server. The server parses the reply and extracts the suggested location and time information. If necessary, the server accesses the reservation site and inputs the date, time, number of people, and any special requests to confirm the reservation.

[0832] Once the reservation is complete, the server updates the schedule database with the new schedule information, and the terminal sends the updated information and a notification that the reservation is complete to the user.

[0833] If the user has enabled automatic conversation generation, the server will generate follow-up conversations based on past conversation history, new schedule information, and emotional data. For example, if the user is feeling stressed, the server will send a message to the client saying, "Relax and have fun."

[0834] Specific examples

[0835] A user schedules a business meeting, but the date, time, and location are not yet determined.

[0836] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[0837] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[0838] The client replies, "How about a conference room in the ABC building at 2pm?"

[0839] The terminal receives this reply and sends it to the server.

[0840] The server parses the reply and confirms the proposed location and time.

[0841] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[0842] The server updates the confirmed schedule and notifies the user and client.

[0843] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[0844] Prompt Sentence Examples

[0845] "Analyze the user's voice input, text messages, and facial expression data to recognize the user's emotional state. Then, generate an appropriate message based on the recognized emotion. For example, if the user is stressed, generate a message saying, 'Relax and have fun.'"

[0846] The above is an embodiment of the present invention. This system improves the efficiency of the user's schedule management and communication, and is characterized by its ability to dynamically respond to emotions.

[0847] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0848] Step 1:

[0849] The server periodically scans the schedule database to detect unconfirmed schedules with unset dates, times, and locations. The input is the schedule database, and the output is a list of unconfirmed schedule entries. Data processing includes analyzing the schedule entries and selecting unconfirmed entries.

[0850] Step 2:

[0851] The device collects the user's voice input, text messages, and facial expression data using a camera, and sends it to a server. The input is the user's voice, text, and image data, and the output is emotional data sent to the server. Specifically, when the user operating the device says, "Tell me what's on my schedule for today," the device performs voice recognition and image analysis to quantify the user's emotional state.

[0852] Step 3:

[0853] The server retrieves previously entered contact information from a database based on the pending schedule entry. The input is the pending schedule entry and the database, and the output is the retrieved contact information. Data processing includes searching for appropriate contact information using keywords related to the schedule entry.

[0854] Step 4:

[0855] The server analyzes past conversation history and the user's emotional data and generates an appropriate message using a generative AI model. The input is the past conversation history and emotional data, and the output is the generated message. Specifically, if the emotional data obtained indicates that the user is feeling stressed, the server inputs a prompt to the AI ​​model saying, "Generate an appropriate message in a situation where the user is feeling stressed," and obtains the generated message.

[0856] Step 5:

[0857] The generated message is sent to the other party via the device. The input is the generated message, and the output is the message sent to the other party. Specifically, the device uses a message sending application to send the message to the other party.

[0858] Step 6:

[0859] When the other party sends a reply message, the device receives the reply and sends it to the server. The input is the reply message from the other party, and the output is the reply data sent to the server. The device uploads the reply to the server via the Internet.

[0860] Step 7:

[0861] The server analyzes the reply content and extracts the suggested location and time information. The input is the reply message, and the output is the extracted location and time information. Data processing involves using text analysis methods to extract specific information from the reply data.

[0862] Step 8:

[0863] The server accesses the booking site and confirms the booking by entering the date, time, number of people, and any special requests. The input is the extracted location and time information, and the output is the booking confirmation information. The server uses an automated script to enter the information into the booking site.

[0864] Step 9:

[0865] Once the reservation is completed, the server updates the schedule information to the schedule database, and the terminal sends a notification to the user. The input is the reservation confirmation information, and the output is the updated schedule information and a notification message. The terminal uses the notification system to send the message to the user.

[0866] Step 10:

[0867] If the user has enabled automatic conversation generation, the server generates a follow-up conversation based on the past conversation history, new schedule information, and emotional data. The input is the past conversation history, new schedule information, and emotional data, and the output is the generated follow-up conversation. Specifically, it generates a message saying, "Relax and have fun," and sends it to the client via the terminal.

[0868] (Application example 2)

[0869] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0870] Autonomous vehicles are required to not only efficiently manage users' schedules and automate necessary information acquisition and communication, but also to recognize the user's emotional state and dynamically change responses based on that. Conventional systems manage schedules without taking the user's emotional state into consideration, resulting in a poor user experience. In addition, navigation efficiency is lacking because destinations based on the schedule must be set manually.

[0871] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and generating a message using natural language processing, means for sending the generated message using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and making a reservation, means for updating and notifying new schedule information, means for automatically generating a conversation and sending a follow-up conversation, means for recognizing the user's emotional state using emotion recognition means and dynamically changing the message content and follow-up conversation based on the emotion, and means for cooperating with the vehicle's navigation system and setting a destination based on the user's schedule. This makes it possible to improve schedule management that takes the user's emotional state into consideration and the efficiency of navigation in autonomous vehicles.

[0872] "Schedule information" is time-related information such as the user's schedule, appointments, and events.

[0873] "Monitoring means" refers to devices or software that have the function of checking schedule information in real time or periodically and detecting abnormalities or updates.

[0874] "Contact information" is data about a person's means of contact, such as phone number, email address, or address.

[0875] A "database" is an organized collection of data for efficiently storing, retrieving, and managing information.

[0876] "Conversation history" is a record of past conversations and communications.

[0877] "Natural language processing" is the technology that enables computers to understand, generate, and respond to human language.

[0878] "Communication means" refers to the interface or infrastructure for sending and receiving data and messages.

[0879] A "reply message" is the content of the response received from the other party.

[0880] "Means of analysis" are devices or software that have the ability to examine data and extract specific patterns or information.

[0881] A "reservation site" is a website that allows online reservations for specific services (restaurants, hotels, conference rooms, etc.).

[0882] The "means for making a reservation" refers to a device or software that has the function of accessing a reservation site, inputting the necessary information, and finalizing the reservation.

[0883] The "means for updating schedule information" refers to a device or software that has the function of updating the schedule database with new information or changes.

[0884] "Notification means" refers to devices or software that have the functionality to notify users or relevant persons of updated information or reminders.

[0885] "Automatic conversation generation" is a technology that uses algorithms and AI to automatically create dialogue with users.

[0886] A "follow-up conversation" is any additional communication that follows an initial interaction or event.

[0887] "Emotion recognition means" is a technology that analyzes voice, text, facial expressions, etc. to determine the user's emotional state.

[0888] A "navigation system" is a device or software that calculates and provides directions to a destination.

[0889] The "means for setting a destination" refers to a device or software that has the function of determining a destination based on the user's schedule information and inputting it into the navigation system.

[0890] This system streamlines user schedule management and automates necessary information acquisition and communication. It also recognizes the user's emotions and dynamically changes messages and follow-up conversations based on those emotions. This system, especially when applied to autonomous vehicles, improves the in-car user experience.

[0891] System configuration

[0892] The system consists of the following main components:

[0893] 1. Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules, and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[0894] 2. Terminal: A device operated by the user that communicates with the server, sends and receives messages, collects emotional data, and works with the navigation system of the autonomous vehicle.

[0895] 3. User: The entity that uses the system, inputs schedule information, and receives updated schedules and notifications.

[0896] Program processing

[0897] Monitoring schedule information and collecting emotion data

[0898] The server periodically scans the schedule database to detect unconfirmed schedules for which the date, time, and location have not yet been set. The device also collects facial expression data from the user via voice input, text messages, or the camera, and sends this data to the server.

[0899] Obtaining information about the other person and creating a message

[0900] The server retrieves the contact information of the person previously entered from a database based on the pending schedule entry. Next, it analyzes past conversation history and the user's emotional data and generates an appropriate message using natural language processing. For example, if it determines that the user is feeling stressed, it generates a message such as, "Could you please suggest a location and time for our next lunch?"

[0901] Sending automated messages

[0902] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[0903] Parsing reply messages

[0904] When the other person sends a reply message, the device receives the reply and sends the contents to the server, which analyzes the received message and extracts the location and time information of the reply.

[0905] Make the necessary reservations

[0906] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[0907] New schedule updates and notifications

[0908] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[0909] Linking with navigation systems

[0910] It works in conjunction with the navigation system of the autonomous vehicle to set the destination based on the user's schedule. To do this, schedule data is input into the navigation system so that the vehicle automatically heads to the destination.

[0911] Specific examples

[0912] A user schedules a business meeting, but the date, time, and location are not yet determined.

[0913] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[0914] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[0915] The client replies, "How about a conference room in the ABC building at 2pm?"

[0916] The terminal receives this reply and sends it to the server.

[0917] The server parses the reply and confirms the proposed location and time.

[0918] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[0919] The server updates the confirmed schedule and notifies the user and client.

[0920] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[0921] Prompt Sentence Examples

[0922] "Generate a message to ask the user to suggest a location and time for an unscheduled event."

[0923] "Generate a message to help the user relax if they are currently feeling stressed."

[0924] As described above, this system significantly reduces the user's workload through the automation of schedule management and communication, and can respond in a way that takes the user's feelings into consideration. Furthermore, by linking with the navigation system of an autonomous vehicle, it can improve the efficiency and comfort of travel.

[0925] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0926] Step 1:

[0927] The server periodically scans the schedule book database. It takes all entries in the schedule book database as input and detects unconfirmed schedules (those for which the date, time, and location have not been set). It generates a list of unconfirmed schedules as output. This data processing or data calculation checks the fields of the schedule entries and lists the entries with unconfirmed fields.

[0928] Step 2:

[0929] The server retrieves the contact information of the other party that was previously input based on the pending schedule from the database. It uses the list of pending schedules as input and accesses the contact information database of the other party. It generates a list containing the contact information of the other party as output. In this process, it searches the database for the contact information of the other party that corresponds to each schedule entry and compiles it into a list.

[0930] Step 3:

[0931] The server collects facial expression data from the user via voice input, text messages, or a camera, and analyzes it with an emotion engine. The inputs are voice, text, and facial expression data from the user. The output is an analyzed emotional state (e.g., stress, joy). This process uses a generative AI model to analyze the emotional data and performs calculations to identify the user's emotional state.

[0932] Step 4:

[0933] The server uses natural language processing to generate an appropriate message based on the pending schedule, the other person's contact information, and the analyzed emotional state. Details of the pending schedule, contact information, and emotional state are used as input. The generated message is obtained as output. This process utilizes natural language processing with a generative AI model, and generates messages while also referencing past conversation history.

[0934] Step 5:

[0935] The server sends the generated message to the other party using a communication method. It uses the generated message and the other party's contact information as input. It obtains the sending completion status of the message sent to the other party as output. This process uses the messaging service API to perform the specific operations to send the message.

[0936] Step 6:

[0937] The device receives the message that the other party has replied to and sends it to the server. The reply message from the other party is used as input. The reply message sent to the server is obtained as output. This process uses the device's message receiving function to transfer the received message to the server via the network.

[0938] Step 7:

[0939] The server analyzes the reply message and extracts the location and time information of the reply. It uses the reply message as input and obtains the extracted location and time information as output. This process uses natural language processing technology to analyze the message content and performs specific calculations to extract the necessary information.

[0940] Step 8:

[0941] The server accesses the reservation site based on the extracted information and automatically executes the reservation. The extracted location and time information is used as input. The reservation completion status and reservation confirmation information are obtained as output. This process automatically inputs information using the reservation site's API and confirms the reservation.

[0942] Step 9:

[0943] The server updates the schedule book database with the new schedule information and sends a notification. It uses the reservation confirmation information and the new schedule information as input. It obtains the updated schedule and the notification sending completion status as output. This process performs the specific operations of updating the database and sending the notification using the notification service API.

[0944] Step 10:

[0945] The server works with the autonomous vehicle's navigation system to set a destination based on the user's schedule. It uses new schedule information as input and reflects the set destination information in the navigation system as output. This process uses the navigation system's API to perform specific operations to set the destination.

[0946] Step 11:

[0947] If the user has enabled automatic conversation generation, the server generates a follow-up conversation based on the user's emotional data and schedule information and sends it to the other party. The server uses the user's emotional data and schedule information as input. The generated follow-up conversation is obtained as output. This process uses a generative AI model to generate a follow-up conversation and sends it using the messaging service API.

[0948] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0949] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0950] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0951] [Third embodiment]

[0952] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0953] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0954] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0955] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0956] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0957] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0958] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0959] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0960] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0961] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0962] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0963] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0964] The present invention is a system that streamlines users' schedule management and automates the acquisition of necessary information and communication. This system has the functions of monitoring schedule information, generating and sending messages based on contact information, and analyzing replies. It also has the functions of making necessary reservations, updating schedules, and notifying users.

[0965] System configuration

[0966] This system consists of the following main components:

[0967] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications.

[0968] Terminal: A device operated by a user that communicates with the server and sends and receives messages.

[0969] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[0970] Program processing

[0971] Monitoring schedule information

[0972] The server periodically scans the schedule database to detect pending schedule entries, specifically entries for which the date, time, and location of the schedule have not been set.

[0973] Obtaining information about the other person and creating a message

[0974] The server retrieves the contact information of the person in question from a database based on the pending schedule entry, analyzes past conversation history, and uses natural language processing to generate an appropriate message, such as, "Could you please suggest a location and time for our next lunch?"

[0975] Sending automated messages

[0976] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[0977] Parsing reply messages

[0978] When the other person sends a reply message, the device receives it and sends it to the server, which analyzes the received message and extracts the reply content (location, time, etc.).

[0979] Make the necessary reservations

[0980] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[0981] New schedule updates and notifications

[0982] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[0983] Optional conversation generation

[0984] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history and new schedule information. For example, when a reservation is confirmed, the server will send a message to the other party saying, "Your reservation is complete. I look forward to seeing you."

[0985] Specific examples

[0986] A user schedules a business meeting, but the date, time, and location are not yet determined.

[0987] The server detects pending schedules and obtains contact information for the client.

[0988] The server refers to past communications and generates a message saying, "We need to decide on a meeting location and time. Can you make a suggestion?" and sends it to the client via the terminal.

[0989] The client replies, "How about a conference room in the ABC building at 2pm?"

[0990] The terminal receives this reply and sends it to the server.

[0991] The server parses the reply and confirms the proposed location and time.

[0992] The server accesses the conference room reservation site and makes the reservation at 2 p.m.

[0993] The server updates the confirmed schedule and notifies the user and client.

[0994] Since the user has enabled automatic conversation generation, a message saying "Reservation completed. We look forward to seeing you on the day" is sent to the client.

[0995] In this way, this system significantly reduces the user's workload by automating schedule management and communication, enabling efficient schedule adjustments.

[0996] The processing flow will be explained below.

[0997] Step 1:

[0998] The server periodically scans the schedule book database and detects unconfirmed schedules for which the date, time, and location have not been set.

[0999] Step 2:

[1000] Based on the detected pending schedule entry, the server retrieves the previously inputted contact information of the other person from the database.

[1001] Step 3:

[1002] The server analyzes past conversation history and uses natural language processing to generate an appropriate message, such as, "Can you suggest a location and time for our next lunch?"

[1003] Step 4:

[1004] The server converts the generated message into email or LINE format and sends it to the recipient via the device.

[1005] Step 5:

[1006] When the other person sends a reply message, the device receives the reply and sends the contents to the server.

[1007] Step 6:

[1008] The server analyzes the received message and extracts the location and time information of the reply.

[1009] Step 7:

[1010] Based on the extracted location information, the server accesses the reservation website of the relevant store or conference room and makes the necessary reservation. To make a reservation, the user inputs the date and time, number of people, special request information, etc.

[1011] Step 8:

[1012] The server acquires the reservation confirmation information and reflects it in the schedule book.

[1013] Step 9:

[1014] The server updates the schedule information to the schedule book database and notifies the user and the other party of the updated information via the terminal.

[1015] Step 10:

[1016] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history and new schedule information, such as "Your reservation is complete. I look forward to it."

[1017] Step 11:

[1018] The follow-up conversation generated by the server is converted into email or LINE format and sent to the other party via the device.

[1019] Example 1

[1020] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1021] Conventional schedule management systems require users to manually adjust schedules, obtain necessary information, and communicate, which is time-consuming and labor-intensive and inefficient. In addition, the inability to analyze reply messages or automatically execute reservations increases the amount of work required and places a heavy burden on users.

[1022] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1023] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and generating messages using natural language processing, means for sending the generated messages using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and making a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations, means for generating generated messages using a natural language processing model, means for identifying entries for which the date, time, and location of the schedule have not been set, means for analyzing received messages and extracting reply content, and means for continuously monitoring schedule information. This enables automation of schedule management and communication.

[1024] "Schedule information" refers to information such as date, time, location, and content that allows a user to manage their schedule.

[1025] "Contact information" refers to information in a database that stores contact information for users and their contacts, including telephone numbers and email addresses.

[1026] A "database" refers to a system for efficiently storing, managing, and searching various types of information.

[1027] "Conversation history" refers to the history of messages and conversations that a user has had with other people in the past.

[1028] "Natural language processing" refers to the technology that enables computers to understand, analyze, and generate human language.

[1029] "Message generation" refers to automatically creating appropriate sentences using natural language processing.

[1030] "Means of communication" refers to methods and technologies for sending and receiving information, including the Internet and SMS.

[1031] A "reply message" refers to a message received as a response from the other party.

[1032] "Analysis" refers to the process of understanding the content of a received message and extracting the necessary information.

[1033] "Reservation Site" refers to a website or application that allows you to make reservations for locations, dates, and times.

[1034] "Making a reservation" refers to accessing a reservation site, entering the necessary information, and confirming the reservation.

[1035] "Updating schedule information" refers to replacing existing schedule information with new information.

[1036] "Notification" refers to notifying the user of important information.

[1037] "Automatic conversation generation" refers to the system automatically generating new conversation messages.

[1038] A "follow-up conversation" refers to any additional interaction that occurs after a particular event.

[1039] A "natural language processing model" refers to an algorithm or system trained to perform natural language processing.

[1040] An "unspecified entry" refers to an item of schedule information for which the date, time, and location have not yet been determined.

[1041] "Reply content extraction" refers to extracting important information from received messages.

[1042] "Continuous monitoring" refers to the process of checking schedule information at regular intervals.

[1043] This invention is a system that improves the efficiency of users' schedule management and automates the acquisition of necessary information and communication. This system is composed of three main elements: a server, a terminal, and a user, and these elements work in conjunction with each other.

[1044] server

[1045] The server acts as the central control unit of the system and uses the following hardware and software:

[1046] Hardware: High-performance processors, memory, and storage devices

[1047] Software: A database (e.g., SQL Server) for managing schedule information, libraries for running natural language processing models (e.g., BERT, GPT-3), and a web server (e.g., Apache, Nginx).

[1048] The server monitors schedule information, retrieves contact information, generates and sends messages, analyzes reply messages, executes reservations, updates and notifies new schedule information, and automatically generates conversations.

[1049] Terminal

[1050] A terminal is a device operated by a user. The terminal uses the following hardware and software:

[1051] Hardware: Smartphones, tablets, computers

[1052] Software: Email clients, SMS applications, push notification services

[1053] The terminal communicates with the server to send and receive the generated messages, and also sends schedule updates and reservation completion notifications to the user.

[1054] User

[1055] Users are the main users of the system. They input schedule information, receive schedule updates and notifications, and can enable automatic conversation generation and follow-up messaging on request.

[1056] Specific examples

[1057] As an example, consider a situation where a user schedules a business meeting, but the date, time and location are still pending.

[1058] 1. A user schedules a business meeting, but the date, time, and location are not yet determined.

[1059] 2. The server periodically scans the scheduler database to detect any pending schedule entries.

[1060] 3. The server retrieves the contact information from the contact database and uses a natural language processing model to generate a message saying, "We need to decide on a meeting location and time. Can you suggest one?"

[1061] 4. The server generates the message and sends it to the other party via the device.

[1062] 5. The other person replies, "How about meeting in the conference room in the ABC building at 2 p.m.?" The device receives this reply and sends it to the server.

[1063] 6. The server parses the reply message and determines the proposed location and time.

[1064] 7. The server accesses the reservation site and reserves the conference room for 2 p.m.

[1065] 8. The server updates the confirmed schedule information in the schedule book database and notifies the user and the other party via the terminal.

[1066] 9. If the user has enabled automatic conversation generation, the server will generate an additional message saying, "Reservation completed. We look forward to seeing you on the day." and send it to the other party via the device.

[1067] Prompt Sentence Examples

[1068] "What should I do if I need to schedule a business meeting but the date, time, and location are not yet decided?"

[1069] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1070] Step 1:

[1071] The server periodically scans the planner database.

[1072] Input: Schedule database

[1073] Action: Run a SQL query to identify schedule entries that have no date, time, or location set. For example, use a query like "SELECT FROM schedules WHERE date IS NULL OR location IS NULL;"

[1074] Output: A list of pending schedule entries

[1075] Step 2:

[1076] The server retrieves the contact information of the other person from the user database based on the pending schedule entry.

[1077] Input: Pending schedule entry, user database

[1078] Action: Search and retrieve the relevant contact information. For example, run a query like "SELECT contact_info FROM user_data WHERE user_id = ?"

[1079] Output: Recipient's contact information

[1080] Step 3:

[1081] The server analyzes past conversation history and generates messages using natural language processing.

[1082] Input: Contact information of the other person, conversation history database

[1083] Processing: Using natural language processing models (e.g., BERT, GPT-3) to analyze past conversation history and generate appropriate messages based on the context, such as "Can you suggest a location and time for our next lunch?"

[1084] Output: The generated message

[1085] Step 4:

[1086] The server generates the message and sends it to the other party via the device.

[1087] Input: Generated message, contact information of the recipient

[1088] Process: Send a message using an SMS or email API. For example, use the Twilio API and execute the function "send_message(to=contact_number, body=generated_message)".

[1089] Output: Message sending status

[1090] Step 5:

[1091] The terminal receives the reply message and sends it to the server.

[1092] Input: Reply message

[1093] Processing: Use the message reception API to transfer the message received on the device to the server.

[1094] Output: Forwarding the received message to the server

[1095] Step 6:

[1096] The server analyzes the received message and extracts the reply content.

[1097] Input: Received message

[1098] Processing: Using natural language processing models and regular expressions, we extract important information from the reply messages, such as date, time, and location.

[1099] Output: Extracted reply content

[1100] Step 7:

[1101] The server accesses the reservation site and makes the reservation.

[1102] Input: Extracted reply content (date, time, location), reservation site URL

[1103] Processing: Using a browser automation tool such as Selenium, access the reservation site, enter the required information, and confirm the reservation. For example, execute "browser.find_element_by_name('date').send_keys('14:00')".

[1104] Output: Reservation confirmation information

[1105] Step 8:

[1106] The server updates the schedule information in the schedule book database, and the terminal sends a notification to the user.

[1107] Input: Reservation confirmation information, schedule database

[1108] Processing: Executes a SQL query to update schedule information and notify users via push notification or email. For example, execute a query like "UPDATE schedules SET date = '2023-11-05 14:00', location = 'ABC Building' WHERE id = ?"

[1109] Output: Updated schedule information, notification sending status

[1110] Step 9:

[1111] The server will auto-generate the conversation and send a follow-up conversation to the other person via their device (if the user has enabled auto-generated conversation).

[1112] Input: Updated schedule information, past conversation history

[1113] Processing: Using a natural language processing model, a follow-up message is generated and sent via email or SMS, such as "Your reservation is complete. We look forward to seeing you on the day."

[1114] Output: Follow message sending status

[1115] (Application example 1)

[1116] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1117] When on-site workers manually manage schedules and communicate, they are unable to respond in a timely manner, resulting in reduced efficiency. Adjusting schedules and procuring supplies is also time-consuming and often results in errors. This reduces production efficiency and hinders the smooth progress of work.

[1118] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1119] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-entered contact information from a database, means for analyzing past conversation history and generating messages using natural language processing, means for sending the generated messages using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and executing a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations, means for displaying schedule information and instructions in real time on a user's device, means for analyzing reply messages and automatically procuring necessary items and equipment, and means for notifying new instructions and changes in real time. This allows each worker to efficiently manage their schedule and communicate as needed in a timely manner. Furthermore, the automatic procurement of necessary items and equipment and real-time notification of new instructions and changes improve production efficiency and smooth business operations.

[1120] The "means for monitoring schedule information" refers to a means by which the system periodically scans the schedule book database to monitor schedule events related to the user.

[1121] The "means for obtaining contact information from a database" refers to a means for reading pre-entered contact information from a database.

[1122] "Means for analyzing past conversation history and generating messages using natural language processing" refers to means for analyzing past conversation data and automatically generating appropriate messages using natural language processing technology.

[1123] The "means for transmitting the generated message using a communication means" refers to a means for transmitting the generated message to another device or system via a communication means.

[1124] The "means for receiving and analyzing a reply message" is a means for analyzing a received reply message.

[1125] "Means for accessing a reservation site and making a reservation" refers to means for accessing an online reservation system and automatically making the necessary reservations.

[1126] The "means for updating and notifying new schedule information" is a means for updating schedule information to the latest information and notifying the user of the contents.

[1127] The "means for automatically generating a conversation and sending a follow-up conversation" is a means for the system to automatically generate conversation content and send the subsequent dialogue to the user.

[1128] The "means for displaying schedule information and instructions in real time on a user's device" refers to a means for displaying schedule information and necessary instructions in real time on a device held by a user.

[1129] The "means for analyzing the reply message and automatically procuring the necessary goods and equipment" is a means for analyzing the contents of the reply message and automatically procuring the necessary goods and equipment based on the results of the analysis.

[1130] The "means for notifying new instructions or changes in real time" refers to a means for notifying the user in real time when new instructions or changes are made.

[1131] This invention is a system that realizes efficient schedule management and automated communication for factory workers. This system uses smart glasses to display information in real time and automatically procure necessary goods and equipment.

[1132] System configuration

[1133] This system consists of the following main components:

[1134] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating and notifying schedules, generating automatic conversations, and automatically procuring goods and equipment.

[1135] Terminal: A device such as smart glasses operated by the user that communicates with the server and displays schedule information and instructions in real time.

[1136] User: A worker who uses the system, receives schedule information, and performs tasks according to instructions.

[1137] Program processing

[1138] Monitoring schedule information

[1139] The server periodically scans the schedule database to detect pending schedule entries.

[1140] Get contact information

[1141] The server retrieves relevant contact information from a database based on the pending schedule entry.

[1142] Message Creation and Transmission

[1143] The server analyzes past conversation history and generates appropriate messages using natural language processing technology. The generated messages are then sent via the terminal using communication means.

[1144] Parsing reply messages

[1145] When a reply message is received, the server analyzes it and extracts the necessary information based on the reply. For example, if a message is received saying, "Please check if there are any problems with the maintenance of equipment A at 1:00 PM," the server analyzes the reply and performs maintenance on equipment A.

[1146] Making a reservation

[1147] The server accesses the reservation site as needed and automatically makes the necessary reservations, entering information such as the date and time, number of people, and special requests to confirm the reservation.

[1148] Schedule Updates and Notifications

[1149] The server updates new schedule information and notifies the user through the terminal, and the user can receive schedule information and instructions in real time through the smart glasses.

[1150] Automated Procurement

[1151] The server analyzes the response message and includes means for automatically procuring the necessary goods or equipment, for example, if a specific part is needed, it automatically orders that part.

[1152] Real-time notifications

[1153] The server notifies the user's smart glasses in real time of any new instructions or changes.

[1154] Hardware and Software Used

[1155] Smart glasses: Using devices such as Google Glass and Vuzix.

[1156] API endpoint: Build a server that manages schedule data and message sending.

[1157] Python: Implement the program that communicates with the server and processes the data.

[1158] Specific examples

[1159] 1. When a user puts on the smart glasses, the system displays today's schedule.

[1160] Example: "8:00 AM - Maintenance on Equipment A"

[1161] Example: "10:00 AM - Quality Inspection Meeting"

[1162] 2. If there is no scheduled maintenance, the system will automatically generate a message.

[1163] Example prompt: "Please check if there are any problems with the maintenance on equipment A at 1 PM."

[1164] 3. The reply is then analyzed and the necessary reservations and supplies are automatically procured.

[1165] For example, if you reply "Confirmed at 1pm", the reservation is confirmed.

[1166] In this way, this system can improve the efficiency of worker schedule management and communication, thereby improving productivity.

[1167] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1168] Step 1:

[1169] The server monitors schedule information. It periodically scans the schedule book database as input to detect pending schedule entries. It generates a list of pending schedule entries as output and passes it to the next step.

[1170] Step 2:

[1171] The server retrieves contact information from the database. It uses the list of pending schedule entries as input and reads the associated contact information from the database. As output, it generates a list of schedule entries with the contact information, which it passes to the next step.

[1172] Step 3:

[1173] The server analyzes past conversation history and generates messages using natural language processing. The input is a list of schedule entries containing contact information and past conversation history. Data processing involves analyzing the conversation history and generating appropriate messages using a generative AI model. The output is a list of the generated messages, which are passed to the next step.

[1174] Step 4:

[1175] The server sends the generated message through the terminal. It uses the generated message and the recipient's contact information as input. Specifically, it sends the message using a communication protocol. As output, it records the result of the message transmission and passes it to the next step.

[1176] Step 5:

[1177] The terminal receives the reply message and sends it to the server. It uses the received message as input and sends it to the server via the endpoint. As output, it creates an object of the received message and passes it to the server.

[1178] Step 6:

[1179] The server analyzes the reply message, using the object of the received message as input. For data processing, it uses natural language processing technology to analyze the reply content and extract information necessary for scheduling and procuring goods. As output, it lists the extracted information and passes it on to the next step.

[1180] Step 7:

[1181] The server accesses the reservation site as needed and makes the reservation. The extracted reservation information is used as input. Specifically, it uses the reservation site's API to input the necessary information and confirm the reservation. As output, it generates reservation confirmation information and passes it to the next step.

[1182] Step 8:

[1183] The server updates the new schedule information and notifies the user via the terminal. The updated schedule information and the device reservation confirmation information are used as input. As output, the updated schedule data is saved in the schedule book database and a notification message is sent to the terminal.

[1184] Step 9:

[1185] The terminal displays real-time schedule information and instructions on the user's device. It uses notification messages as input and displays them in real-time on the user's smart glasses. As output, the displayed schedule and instructions are confirmed by the user.

[1186] Step 10:

[1187] The server analyzes the reply message and automatically procures the necessary goods and equipment. The analyzed data is used as input. Specifically, it automatically places the necessary orders using the goods procurement site's API. As output, it generates order confirmation information and passes it to the next step.

[1188] Step 11:

[1189] The server notifies new instructions and changes in real time. It uses new instructions and change information as input. As output, it generates notification messages in real time and sends them to the terminal.

[1190] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1191] The present invention is a system that streamlines user schedule management, automates the acquisition of necessary information, and automates communication. It also has the ability to recognize the user's emotions and dynamically change messages and follow-up conversations based on those emotions.

[1192] System configuration

[1193] This system consists of the following main components:

[1194] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[1195] Terminal: A device operated by the user that communicates with the server, sends and receives messages, and collects emotional data.

[1196] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[1197] Emotion Engine Functions

[1198] The emotion engine has the following functions:

[1199] It recognizes the user's emotional state by analyzing their voice, text messages, and facial expression data.

[1200] Dynamically change message content and follow-up conversations based on perceived emotional state.

[1201] Program processing

[1202] Monitoring schedule information and collecting emotion data

[1203] The server periodically scans the schedule database to detect unconfirmed schedules for which the date, time, and location have not yet been set, and the device collects facial expression data from the user via voice input, text messages, or the camera, and sends this data to the server.

[1204] Obtaining information about the other person and creating a message

[1205] The server retrieves the contact information of the person previously entered from a database based on the pending schedule entry. Next, it analyzes past conversation history and the user's emotional data and generates an appropriate message using natural language processing. For example, if it determines that the user is feeling stressed, it generates a message such as, "Could you please suggest a location and time for our next lunch?"

[1206] Sending automated messages

[1207] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[1208] Parsing reply messages

[1209] When the other person sends a reply message, the device receives the reply and sends the contents to the server, which analyzes the received message and extracts the location and time information of the reply.

[1210] Make the necessary reservations

[1211] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[1212] New schedule updates and notifications

[1213] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[1214] Optional conversation generation

[1215] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history, new schedule information, and emotional data. For example, if the user is feeling stressed, the server will generate a message like, "Relax and have fun," rather than, "Your reservation is complete, and I'm looking forward to it."

[1216] Specific examples

[1217] A user schedules a business meeting, but the date, time, and location are not yet determined.

[1218] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[1219] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[1220] The client replies, "How about a conference room in the ABC building at 2pm?"

[1221] The terminal receives this reply and sends it to the server.

[1222] The server parses the reply and confirms the proposed location and time.

[1223] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[1224] The server updates the confirmed schedule and notifies the user and client.

[1225] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[1226] In this way, this system significantly reduces the user's workload by automating schedule management and communication, and provides responses that take the user's feelings into consideration.

[1227] The processing flow will be explained below.

[1228] Step 1:

[1229] The server periodically scans the schedule book database and detects unconfirmed schedules for which the date, time, and location have not been set.

[1230] Step 2:

[1231] Based on the detected pending schedule entry, the server retrieves the previously inputted contact information of the other party from the database.

[1232] Step 3:

[1233] The device collects facial expression data using the user's voice input, text messages, or camera, and transmits it to a server in real time or periodically.

[1234] Step 4:

[1235] The server analyzes voice, text messages, and facial expression data to recognize the user's emotional state, for example, detecting stress levels from voice tone and smiles or discomfort from facial expression data.

[1236] Step 5:

[1237] The server uses natural language processing to generate an appropriate message based on the user's past conversation history and emotional state. For example, if the user is feeling stressed, it uses calmer language and generates a message like, "Could you please suggest a place and time for our next lunch?"

[1238] Step 6:

[1239] The server converts the generated message into email or LINE format and sends it to the recipient via the device.

[1240] Step 7:

[1241] When the other person sends a reply message, the device receives the reply and sends the contents to the server.

[1242] Step 8:

[1243] The server analyzes the received message and extracts the location and time information of the reply.

[1244] Step 9:

[1245] Based on the extracted location information, the server accesses the reservation website of the target store or conference room and makes the necessary reservation, entering the date and time, number of people, special request information, etc.

[1246] Step 10:

[1247] The server acquires the reservation confirmation information and reflects it in the schedule book.

[1248] Step 11:

[1249] The server updates the schedule information to the schedule book database and notifies the user and the other party of the updated information via the terminal.

[1250] Step 12:

[1251] If the user has enabled automatic conversation generation, the server generates follow-up conversations based on past conversation history, new schedule information, and emotional data. For example, if the user is relaxed, the server generates a message like, "Your reservation is complete. I'm looking forward to it." However, if the user is stressed, the server generates a message like, "Relax and have fun."

[1252] Step 13:

[1253] The follow-up conversation generated by the server is converted into email or LINE format and sent to the other party via the device.

[1254] Example 2

[1255] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1256] Conventional schedule management systems require users to manually manage their schedules, requiring a great deal of time and effort for tasks such as obtaining contact information, sending messages, and making reservations. Furthermore, the system does not take into account the user's emotions, which can lead to a decline in the quality of communication. To solve these problems, a system is needed that can streamline user schedule management, automate the acquisition of necessary information and communication, and dynamically respond based on emotions.

[1257] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1258] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and emotion data and generating a message using natural language processing technology, means for sending the generated message using a communication device, means for receiving and analyzing reply messages, means for accessing a reservation service and making a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations based on the user's emotions, and means for analyzing the user's voice input, text messages, and facial expression data to recognize the user's emotional state and changing responses based on this. This allows users to improve the efficiency of schedule management and automate communication, and also enables dynamic responses according to emotions.

[1259] "Schedule information" is information about plans and events entered by the user, including the date, time, location, and content.

[1260] The "monitoring means" is a means for continuously or periodically checking schedule information and performing processing based on specific conditions.

[1261] "Contact information" is information about a person or organization that is previously input by the user, including name, phone number, email address, etc.

[1262] A "database" is a system for managing, searching, and accessing data efficiently and quickly.

[1263] "Conversation history" is a record of messages and communications sent and received in the past.

[1264] "Emotion data" is data that indicates the user's emotional state. It is obtained by analyzing voice, text, facial expressions, etc.

[1265] "Natural language processing technology" is a technology that enables computers to understand, generate, and process human language.

[1266] "Message generation" is the process of creating an appropriate text message based on the input data.

[1267] A "communications device" is a hardware or software device for transmitting and receiving data or information.

[1268] A "reply message" is a reply message sent by the other party.

[1269] "Reservation Service" means a service that allows you to manage and make reservations online.

[1270] The "means for making a reservation" refers to the means for accessing the reservation service, inputting the information required for the reservation, and finalizing the reservation.

[1271] "Means for updating schedule information" refers to means for adding or modifying new or changed information in the schedule database.

[1272] "Means for notifying" refers to means for notifying users of updates and other important information.

[1273] "Automatic conversation generation" is a process that uses an algorithm to automatically generate appropriate conversation content.

[1274] A "follow-up conversation" is additional communication that occurs following a particular event or situation.

[1275] "Voice input" is a means of capturing the user's spoken voice as digital data.

[1276] "Facial expression data" is data obtained by analyzing the facial expression of the user.

[1277] An "emotional state" is a state that indicates how a user is feeling at a particular point in time.

[1278] The present invention is a system that streamlines user schedule management, automates the acquisition of necessary information, and automates communication. It also has the ability to recognize the user's emotions and dynamically change messages and follow-up conversations based on those emotions.

[1279] System configuration

[1280] The system consists of the following major components:

[1281] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[1282] Terminal: A device operated by the user that communicates with the server, sends and receives messages, and collects emotional data.

[1283] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[1284] Emotion Engine Functions

[1285] The emotion engine has the following functions:

[1286] It recognizes the user's emotional state by analyzing their voice, text messages, and facial expression data.

[1287] Dynamically change message content and follow-up conversations based on perceived emotional state.

[1288] Program processing

[1289] The server periodically scans the schedule database to detect unconfirmed schedules with unspecified dates, times, and locations. The device collects facial expression data from the user using voice input, text messages, and the camera, and sends it to the server.

[1290] The server retrieves pre-entered contact information from a database based on pending schedule entries, analyzes past conversation history and user emotion data, and generates an appropriate message using natural language processing technology.

[1291] For example, if it is determined that the user is feeling stressed, the system generates a message saying, "Could you please suggest a place and time for our next lunch?" This message is then sent to the other party via the device.

[1292] When a reply is received, the device receives it and sends it to the server. The server parses the reply and extracts the suggested location and time information. If necessary, the server accesses the reservation site and inputs the date, time, number of people, and any special requests to confirm the reservation.

[1293] Once the reservation is complete, the server updates the schedule database with the new schedule information, and the terminal sends the updated information and a notification that the reservation is complete to the user.

[1294] If the user has enabled automatic conversation generation, the server will generate follow-up conversations based on past conversation history, new schedule information, and emotional data. For example, if the user is feeling stressed, the server will send a message to the client saying, "Relax and have fun."

[1295] Specific examples

[1296] A user schedules a business meeting, but the date, time, and location are not yet determined.

[1297] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[1298] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[1299] The client replies, "How about a conference room in the ABC building at 2pm?"

[1300] The terminal receives this reply and sends it to the server.

[1301] The server parses the reply and confirms the proposed location and time.

[1302] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[1303] The server updates the confirmed schedule and notifies the user and client.

[1304] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[1305] Prompt Sentence Examples

[1306] "Analyze the user's voice input, text messages, and facial expression data to recognize the user's emotional state. Then, generate an appropriate message based on the recognized emotion. For example, if the user is stressed, generate a message saying, 'Relax and have fun.'"

[1307] The above is an embodiment of the present invention. This system improves the efficiency of the user's schedule management and communication, and is characterized by its ability to dynamically respond to emotions.

[1308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1309] Step 1:

[1310] The server periodically scans the schedule database to detect unconfirmed schedules with unset dates, times, and locations. The input is the schedule database, and the output is a list of unconfirmed schedule entries. Data processing includes analyzing the schedule entries and selecting unconfirmed entries.

[1311] Step 2:

[1312] The device collects the user's voice input, text messages, and facial expression data using a camera, and sends it to a server. The input is the user's voice, text, and image data, and the output is emotional data sent to the server. Specifically, when the user operating the device says, "Tell me what's on my schedule for today," the device performs voice recognition and image analysis to quantify the user's emotional state.

[1313] Step 3:

[1314] The server retrieves previously entered contact information from a database based on the pending schedule entry. The input is the pending schedule entry and the database, and the output is the retrieved contact information. Data processing includes searching for appropriate contact information using keywords related to the schedule entry.

[1315] Step 4:

[1316] The server analyzes past conversation history and the user's emotional data and generates an appropriate message using a generative AI model. The input is the past conversation history and emotional data, and the output is the generated message. Specifically, if the emotional data obtained indicates that the user is feeling stressed, the server inputs a prompt to the AI ​​model saying, "Generate an appropriate message in a situation where the user is feeling stressed," and obtains the generated message.

[1317] Step 5:

[1318] The generated message is sent to the other party via the device. The input is the generated message, and the output is the message sent to the other party. Specifically, the device uses a message sending application to send the message to the other party.

[1319] Step 6:

[1320] When the other party sends a reply message, the device receives the reply and sends it to the server. The input is the reply message from the other party, and the output is the reply data sent to the server. The device uploads the reply to the server via the Internet.

[1321] Step 7:

[1322] The server analyzes the reply content and extracts the suggested location and time information. The input is the reply message, and the output is the extracted location and time information. Data processing involves using text analysis methods to extract specific information from the reply data.

[1323] Step 8:

[1324] The server accesses the booking site and confirms the booking by entering the date, time, number of people, and any special requests. The input is the extracted location and time information, and the output is the booking confirmation information. The server uses an automated script to enter the information into the booking site.

[1325] Step 9:

[1326] Once the reservation is completed, the server updates the schedule information to the schedule database, and the terminal sends a notification to the user. The input is the reservation confirmation information, and the output is the updated schedule information and a notification message. The terminal uses the notification system to send the message to the user.

[1327] Step 10:

[1328] If the user has enabled automatic conversation generation, the server generates a follow-up conversation based on the past conversation history, new schedule information, and emotional data. The input is the past conversation history, new schedule information, and emotional data, and the output is the generated follow-up conversation. Specifically, it generates a message saying, "Relax and have fun," and sends it to the client via the terminal.

[1329] (Application example 2)

[1330] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1331] Autonomous vehicles are required to not only efficiently manage users' schedules and automate necessary information acquisition and communication, but also to recognize the user's emotional state and dynamically change responses based on that. Conventional systems manage schedules without taking the user's emotional state into consideration, resulting in a poor user experience. In addition, navigation efficiency is lacking because destinations based on the schedule must be set manually.

[1332] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and generating a message using natural language processing, means for sending the generated message using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and making a reservation, means for updating and notifying new schedule information, means for automatically generating a conversation and sending a follow-up conversation, means for recognizing the user's emotional state using emotion recognition means and dynamically changing the message content and follow-up conversation based on the emotion, and means for cooperating with the vehicle's navigation system and setting a destination based on the user's schedule. This makes it possible to improve schedule management that takes the user's emotional state into consideration and the efficiency of navigation in autonomous vehicles.

[1333] "Schedule information" is time-related information such as the user's schedule, appointments, and events.

[1334] "Monitoring means" refers to devices or software that have the function of checking schedule information in real time or periodically and detecting abnormalities or updates.

[1335] "Contact information" is data about a person's means of contact, such as phone number, email address, or address.

[1336] A "database" is an organized collection of data for efficiently storing, retrieving, and managing information.

[1337] "Conversation history" is a record of past conversations and communications.

[1338] "Natural language processing" is the technology that enables computers to understand, generate, and respond to human language.

[1339] "Communication means" refers to the interface or infrastructure for sending and receiving data and messages.

[1340] A "reply message" is the content of the response received from the other party.

[1341] "Means of analysis" are devices or software that have the ability to examine data and extract specific patterns or information.

[1342] A "reservation site" is a website that allows online reservations for specific services (restaurants, hotels, conference rooms, etc.).

[1343] The "means for making a reservation" refers to a device or software that has the function of accessing a reservation site, inputting the necessary information, and finalizing the reservation.

[1344] The "means for updating schedule information" refers to a device or software that has the function of updating the schedule database with new information or changes.

[1345] "Notification means" refers to devices or software that have the functionality to notify users or relevant persons of updated information or reminders.

[1346] "Automatic conversation generation" is a technology that uses algorithms and AI to automatically create dialogue with users.

[1347] A "follow-up conversation" is any additional communication that follows an initial interaction or event.

[1348] "Emotion recognition means" is a technology that analyzes voice, text, facial expressions, etc. to determine the user's emotional state.

[1349] A "navigation system" is a device or software that calculates and provides directions to a destination.

[1350] The "means for setting a destination" refers to a device or software that has the function of determining a destination based on the user's schedule information and inputting it into the navigation system.

[1351] This system streamlines user schedule management and automates necessary information acquisition and communication. It also recognizes the user's emotions and dynamically changes messages and follow-up conversations based on those emotions. This system, especially when applied to autonomous vehicles, improves the in-car user experience.

[1352] System configuration

[1353] The system consists of the following main components:

[1354] 1. Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules, and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[1355] 2. Terminal: A device operated by the user that communicates with the server, sends and receives messages, collects emotional data, and works with the navigation system of the autonomous vehicle.

[1356] 3. User: The entity that uses the system, inputs schedule information, and receives updated schedules and notifications.

[1357] Program processing

[1358] Monitoring schedule information and collecting emotion data

[1359] The server periodically scans the schedule database to detect unconfirmed schedules for which the date, time, and location have not yet been set. The device also collects facial expression data from the user via voice input, text messages, or the camera, and sends this data to the server.

[1360] Obtaining information about the other person and creating a message

[1361] The server retrieves the contact information of the person previously entered from a database based on the pending schedule entry. Next, it analyzes past conversation history and the user's emotional data and generates an appropriate message using natural language processing. For example, if it determines that the user is feeling stressed, it generates a message such as, "Could you please suggest a location and time for our next lunch?"

[1362] Sending automated messages

[1363] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[1364] Parsing reply messages

[1365] When the other person sends a reply message, the device receives the reply and sends the contents to the server, which analyzes the received message and extracts the location and time information of the reply.

[1366] Make the necessary reservations

[1367] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[1368] New schedule updates and notifications

[1369] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[1370] Linking with navigation systems

[1371] It works in conjunction with the navigation system of the autonomous vehicle to set the destination based on the user's schedule. To do this, schedule data is input into the navigation system so that the vehicle automatically heads to the destination.

[1372] Specific examples

[1373] A user schedules a business meeting, but the date, time, and location are not yet determined.

[1374] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[1375] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[1376] The client replies, "How about a conference room in the ABC building at 2pm?"

[1377] The terminal receives this reply and sends it to the server.

[1378] The server parses the reply and confirms the proposed location and time.

[1379] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[1380] The server updates the confirmed schedule and notifies the user and client.

[1381] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[1382] Prompt Sentence Examples

[1383] "Generate a message to ask the user to suggest a location and time for an unscheduled event."

[1384] "Generate a message to help the user relax if they are currently feeling stressed."

[1385] As described above, this system significantly reduces the user's workload through the automation of schedule management and communication, and can respond in a way that takes the user's feelings into consideration. Furthermore, by linking with the navigation system of an autonomous vehicle, it can improve the efficiency and comfort of travel.

[1386] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1387] Step 1:

[1388] The server periodically scans the schedule book database. It takes all entries in the schedule book database as input and detects unconfirmed schedules (those for which the date, time, and location have not been set). It generates a list of unconfirmed schedules as output. This data processing or data calculation checks the fields of the schedule entries and lists the entries with unconfirmed fields.

[1389] Step 2:

[1390] The server retrieves the contact information of the other party that was previously input based on the pending schedule from the database. It uses the list of pending schedules as input and accesses the contact information database of the other party. It generates a list containing the contact information of the other party as output. In this process, it searches the database for the contact information of the other party that corresponds to each schedule entry and compiles it into a list.

[1391] Step 3:

[1392] The server collects facial expression data from the user via voice input, text messages, or a camera, and analyzes it with an emotion engine. The inputs are voice, text, and facial expression data from the user. The output is an analyzed emotional state (e.g., stress, joy). This process uses a generative AI model to analyze the emotional data and performs calculations to identify the user's emotional state.

[1393] Step 4:

[1394] The server uses natural language processing to generate an appropriate message based on the pending schedule, the other person's contact information, and the analyzed emotional state. Details of the pending schedule, contact information, and emotional state are used as input. The generated message is obtained as output. This process utilizes natural language processing with a generative AI model, and generates messages while also referencing past conversation history.

[1395] Step 5:

[1396] The server sends the generated message to the other party using a communication method. It uses the generated message and the other party's contact information as input. It obtains the sending completion status of the message sent to the other party as output. This process uses the messaging service API to perform the specific operations to send the message.

[1397] Step 6:

[1398] The device receives the message that the other party has replied to and sends it to the server. The reply message from the other party is used as input. The reply message sent to the server is obtained as output. This process uses the device's message receiving function to transfer the received message to the server via the network.

[1399] Step 7:

[1400] The server analyzes the reply message and extracts the location and time information of the reply. It uses the reply message as input and obtains the extracted location and time information as output. This process uses natural language processing technology to analyze the message content and performs specific calculations to extract the necessary information.

[1401] Step 8:

[1402] The server accesses the reservation site based on the extracted information and automatically executes the reservation. The extracted location and time information is used as input. The reservation completion status and reservation confirmation information are obtained as output. This process automatically inputs information using the reservation site's API and confirms the reservation.

[1403] Step 9:

[1404] The server updates the schedule book database with the new schedule information and sends a notification. It uses the reservation confirmation information and the new schedule information as input. It obtains the updated schedule and the notification sending completion status as output. This process performs the specific operations of updating the database and sending the notification using the notification service API.

[1405] Step 10:

[1406] The server works with the autonomous vehicle's navigation system to set a destination based on the user's schedule. It uses new schedule information as input and reflects the set destination information in the navigation system as output. This process uses the navigation system's API to perform specific operations to set the destination.

[1407] Step 11:

[1408] If the user has enabled automatic conversation generation, the server generates a follow-up conversation based on the user's emotional data and schedule information and sends it to the other party. The server uses the user's emotional data and schedule information as input. The generated follow-up conversation is obtained as output. This process uses a generative AI model to generate a follow-up conversation and sends it using the messaging service API.

[1409] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1410] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1411] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1412] [Fourth embodiment]

[1413] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1414] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1415] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1416] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1417] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1418] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1419] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1420] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1421] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1422] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1423] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1424] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1425] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1426] The present invention is a system that streamlines users' schedule management and automates the acquisition of necessary information and communication. This system has the functions of monitoring schedule information, generating and sending messages based on contact information, and analyzing replies. It also has the functions of making necessary reservations, updating schedules, and notifying users.

[1427] System configuration

[1428] This system consists of the following main components:

[1429] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications.

[1430] Terminal: A device operated by a user that communicates with the server and sends and receives messages.

[1431] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[1432] Program processing

[1433] Monitoring schedule information

[1434] The server periodically scans the schedule database to detect pending schedule entries, specifically entries for which the date, time, and location of the schedule have not been set.

[1435] Obtaining information about the other person and creating a message

[1436] The server retrieves the contact information of the person in question from a database based on the pending schedule entry, analyzes past conversation history, and uses natural language processing to generate an appropriate message, such as, "Could you please suggest a location and time for our next lunch?"

[1437] Sending automated messages

[1438] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[1439] Parsing reply messages

[1440] When the other person sends a reply message, the device receives it and sends it to the server, which analyzes the received message and extracts the reply content (location, time, etc.).

[1441] Make the necessary reservations

[1442] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[1443] New schedule updates and notifications

[1444] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[1445] Optional conversation generation

[1446] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history and new schedule information. For example, when a reservation is confirmed, the server will send a message to the other party saying, "Your reservation is complete. I look forward to seeing you."

[1447] Specific examples

[1448] A user schedules a business meeting, but the date, time, and location are not yet determined.

[1449] The server detects pending schedules and obtains contact information for the client.

[1450] The server refers to past communications and generates a message saying, "We need to decide on a meeting location and time. Can you make a suggestion?" and sends it to the client via the terminal.

[1451] The client replies, "How about a conference room in the ABC building at 2pm?"

[1452] The terminal receives this reply and sends it to the server.

[1453] The server parses the reply and confirms the proposed location and time.

[1454] The server accesses the conference room reservation site and makes the reservation at 2 p.m.

[1455] The server updates the confirmed schedule and notifies the user and client.

[1456] Since the user has enabled automatic conversation generation, a message saying "Reservation completed. We look forward to seeing you on the day" is sent to the client.

[1457] In this way, this system significantly reduces the user's workload by automating schedule management and communication, enabling efficient schedule adjustments.

[1458] The processing flow will be explained below.

[1459] Step 1:

[1460] The server periodically scans the schedule book database and detects unconfirmed schedules for which the date, time, and location have not been set.

[1461] Step 2:

[1462] Based on the detected pending schedule entry, the server retrieves the previously inputted contact information of the other person from the database.

[1463] Step 3:

[1464] The server analyzes past conversation history and uses natural language processing to generate an appropriate message, such as, "Can you suggest a location and time for our next lunch?"

[1465] Step 4:

[1466] The server converts the generated message into email or LINE format and sends it to the recipient via the device.

[1467] Step 5:

[1468] When the other person sends a reply message, the device receives the reply and sends the contents to the server.

[1469] Step 6:

[1470] The server analyzes the received message and extracts the location and time information of the reply.

[1471] Step 7:

[1472] Based on the extracted location information, the server accesses the reservation website of the relevant store or conference room and makes the necessary reservation. To make a reservation, the user inputs the date and time, number of people, special request information, etc.

[1473] Step 8:

[1474] The server acquires the reservation confirmation information and reflects it in the schedule book.

[1475] Step 9:

[1476] The server updates the schedule information to the schedule book database and notifies the user and the other party of the updated information via the terminal.

[1477] Step 10:

[1478] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history and new schedule information, such as "Your reservation is complete. I look forward to it."

[1479] Step 11:

[1480] The follow-up conversation generated by the server is converted into email or LINE format and sent to the other party via the device.

[1481] Example 1

[1482] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1483] Conventional schedule management systems require users to manually adjust schedules, obtain necessary information, and communicate, which is time-consuming and labor-intensive and inefficient. In addition, the inability to analyze reply messages or automatically execute reservations increases the amount of work required and places a heavy burden on users.

[1484] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1485] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and generating messages using natural language processing, means for sending the generated messages using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and making a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations, means for generating generated messages using a natural language processing model, means for identifying entries for which the date, time, and location of the schedule have not been set, means for analyzing received messages and extracting reply content, and means for continuously monitoring schedule information. This enables automation of schedule management and communication.

[1486] "Schedule information" refers to information such as date, time, location, and content that allows a user to manage their schedule.

[1487] "Contact information" refers to information in a database that stores contact information for users and their contacts, including telephone numbers and email addresses.

[1488] A "database" refers to a system for efficiently storing, managing, and searching various types of information.

[1489] "Conversation history" refers to the history of messages and conversations that a user has had with other people in the past.

[1490] "Natural language processing" refers to the technology that enables computers to understand, analyze, and generate human language.

[1491] "Message generation" refers to automatically creating appropriate sentences using natural language processing.

[1492] "Means of communication" refers to methods and technologies for sending and receiving information, including the Internet and SMS.

[1493] A "reply message" refers to a message received as a response from the other party.

[1494] "Analysis" refers to the process of understanding the content of a received message and extracting the necessary information.

[1495] "Reservation Site" refers to a website or application that allows you to make reservations for locations, dates, and times.

[1496] "Making a reservation" refers to accessing a reservation site, entering the necessary information, and confirming the reservation.

[1497] "Updating schedule information" refers to replacing existing schedule information with new information.

[1498] "Notification" refers to notifying the user of important information.

[1499] "Automatic conversation generation" refers to the system automatically generating new conversation messages.

[1500] A "follow-up conversation" refers to any additional interaction that occurs after a particular event.

[1501] A "natural language processing model" refers to an algorithm or system trained to perform natural language processing.

[1502] An "unspecified entry" refers to an item of schedule information for which the date, time, and location have not yet been determined.

[1503] "Reply content extraction" refers to extracting important information from received messages.

[1504] "Continuous monitoring" refers to the process of checking schedule information at regular intervals.

[1505] This invention is a system that improves the efficiency of users' schedule management and automates the acquisition of necessary information and communication. This system is composed of three main elements: a server, a terminal, and a user, and these elements work in conjunction with each other.

[1506] server

[1507] The server acts as the central control unit of the system and uses the following hardware and software:

[1508] Hardware: High-performance processors, memory, and storage devices

[1509] Software: A database (e.g., SQL Server) for managing schedule information, libraries for running natural language processing models (e.g., BERT, GPT-3), and a web server (e.g., Apache, Nginx).

[1510] The server monitors schedule information, retrieves contact information, generates and sends messages, analyzes reply messages, executes reservations, updates and notifies new schedule information, and automatically generates conversations.

[1511] Terminal

[1512] A terminal is a device operated by a user. The terminal uses the following hardware and software:

[1513] Hardware: Smartphones, tablets, computers

[1514] Software: Email clients, SMS applications, push notification services

[1515] The terminal communicates with the server to send and receive the generated messages, and also sends schedule updates and reservation completion notifications to the user.

[1516] User

[1517] Users are the main users of the system. They input schedule information, receive schedule updates and notifications, and can enable automatic conversation generation and follow-up messaging on request.

[1518] Specific examples

[1519] As an example, consider a situation where a user schedules a business meeting, but the date, time and location are still pending.

[1520] 1. A user schedules a business meeting, but the date, time, and location are not yet determined.

[1521] 2. The server periodically scans the scheduler database to detect any pending schedule entries.

[1522] 3. The server retrieves the contact information from the contact database and uses a natural language processing model to generate a message saying, "We need to decide on a meeting location and time. Can you suggest one?"

[1523] 4. The server generates the message and sends it to the other party via the device.

[1524] 5. The other person replies, "How about meeting in the conference room in the ABC building at 2 p.m.?" The device receives this reply and sends it to the server.

[1525] 6. The server parses the reply message and determines the proposed location and time.

[1526] 7. The server accesses the reservation site and reserves the conference room for 2 p.m.

[1527] 8. The server updates the confirmed schedule information in the schedule book database and notifies the user and the other party via the terminal.

[1528] 9. If the user has enabled automatic conversation generation, the server will generate an additional message saying, "Reservation completed. We look forward to seeing you on the day." and send it to the other party via the device.

[1529] Prompt Sentence Examples

[1530] "What should I do if I need to schedule a business meeting but the date, time, and location are not yet decided?"

[1531] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1532] Step 1:

[1533] The server periodically scans the planner database.

[1534] Input: Schedule database

[1535] Action: Run a SQL query to identify schedule entries that have no date, time, or location set. For example, use a query like "SELECT FROM schedules WHERE date IS NULL OR location IS NULL;"

[1536] Output: A list of pending schedule entries

[1537] Step 2:

[1538] The server retrieves the contact information of the other person from the user database based on the pending schedule entry.

[1539] Input: Pending schedule entry, user database

[1540] Action: Search and retrieve the relevant contact information. For example, run a query like "SELECT contact_info FROM user_data WHERE user_id = ?"

[1541] Output: Recipient's contact information

[1542] Step 3:

[1543] The server analyzes past conversation history and generates messages using natural language processing.

[1544] Input: Contact information of the other person, conversation history database

[1545] Processing: Using natural language processing models (e.g., BERT, GPT-3) to analyze past conversation history and generate appropriate messages based on the context, such as "Can you suggest a location and time for our next lunch?"

[1546] Output: The generated message

[1547] Step 4:

[1548] The server generates the message and sends it to the other party via the device.

[1549] Input: Generated message, contact information of the recipient

[1550] Process: Send a message using an SMS or email API. For example, use the Twilio API and execute the function "send_message(to=contact_number, body=generated_message)".

[1551] Output: Message sending status

[1552] Step 5:

[1553] The terminal receives the reply message and sends it to the server.

[1554] Input: Reply message

[1555] Processing: Use the message reception API to transfer the message received on the device to the server.

[1556] Output: Forwarding the received message to the server

[1557] Step 6:

[1558] The server analyzes the received message and extracts the reply content.

[1559] Input: Received message

[1560] Processing: Using natural language processing models and regular expressions, we extract important information from the reply messages, such as date, time, and location.

[1561] Output: Extracted reply content

[1562] Step 7:

[1563] The server accesses the reservation site and makes the reservation.

[1564] Input: Extracted reply content (date, time, location), reservation site URL

[1565] Processing: Using a browser automation tool such as Selenium, access the reservation site, enter the required information, and confirm the reservation. For example, execute "browser.find_element_by_name('date').send_keys('14:00')".

[1566] Output: Reservation confirmation information

[1567] Step 8:

[1568] The server updates the schedule information in the schedule book database, and the terminal sends a notification to the user.

[1569] Input: Reservation confirmation information, schedule database

[1570] Processing: Executes a SQL query to update schedule information and notify users via push notification or email. For example, execute a query like "UPDATE schedules SET date = '2023-11-05 14:00', location = 'ABC Building' WHERE id = ?"

[1571] Output: Updated schedule information, notification sending status

[1572] Step 9:

[1573] The server will auto-generate the conversation and send a follow-up conversation to the other person via their device (if the user has enabled auto-generated conversation).

[1574] Input: Updated schedule information, past conversation history

[1575] Processing: Using a natural language processing model, a follow-up message is generated and sent via email or SMS, such as "Your reservation is complete. We look forward to seeing you on the day."

[1576] Output: Follow message sending status

[1577] (Application example 1)

[1578] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1579] When on-site workers manually manage schedules and communicate, they are unable to respond in a timely manner, resulting in reduced efficiency. Adjusting schedules and procuring supplies is also time-consuming and often results in errors. This reduces production efficiency and hinders the smooth progress of work.

[1580] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1581] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-entered contact information from a database, means for analyzing past conversation history and generating messages using natural language processing, means for sending the generated messages using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and executing a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations, means for displaying schedule information and instructions in real time on a user's device, means for analyzing reply messages and automatically procuring necessary items and equipment, and means for notifying new instructions and changes in real time. This allows each worker to efficiently manage their schedule and communicate as needed in a timely manner. Furthermore, the automatic procurement of necessary items and equipment and real-time notification of new instructions and changes improve production efficiency and smooth business operations.

[1582] The "means for monitoring schedule information" refers to a means by which the system periodically scans the schedule book database to monitor schedule events related to the user.

[1583] The "means for obtaining contact information from a database" refers to a means for reading pre-entered contact information from a database.

[1584] "Means for analyzing past conversation history and generating messages using natural language processing" refers to means for analyzing past conversation data and automatically generating appropriate messages using natural language processing technology.

[1585] The "means for transmitting the generated message using a communication means" refers to a means for transmitting the generated message to another device or system via a communication means.

[1586] The "means for receiving and analyzing a reply message" is a means for analyzing a received reply message.

[1587] "Means for accessing a reservation site and making a reservation" refers to means for accessing an online reservation system and automatically making the necessary reservations.

[1588] The "means for updating and notifying new schedule information" is a means for updating schedule information to the latest information and notifying the user of the contents.

[1589] The "means for automatically generating a conversation and sending a follow-up conversation" is a means for the system to automatically generate conversation content and send the subsequent dialogue to the user.

[1590] The "means for displaying schedule information and instructions in real time on a user's device" refers to a means for displaying schedule information and necessary instructions in real time on a device held by a user.

[1591] The "means for analyzing the reply message and automatically procuring the necessary goods and equipment" is a means for analyzing the contents of the reply message and automatically procuring the necessary goods and equipment based on the results of the analysis.

[1592] The "means for notifying new instructions or changes in real time" refers to a means for notifying the user in real time when new instructions or changes are made.

[1593] This invention is a system that realizes efficient schedule management and automated communication for factory workers. This system uses smart glasses to display information in real time and automatically procure necessary goods and equipment.

[1594] System configuration

[1595] This system consists of the following main components:

[1596] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating and notifying schedules, generating automatic conversations, and automatically procuring goods and equipment.

[1597] Terminal: A device such as smart glasses operated by the user that communicates with the server and displays schedule information and instructions in real time.

[1598] User: A worker who uses the system, receives schedule information, and performs tasks according to instructions.

[1599] Program processing

[1600] Monitoring schedule information

[1601] The server periodically scans the schedule database to detect pending schedule entries.

[1602] Get contact information

[1603] The server retrieves relevant contact information from a database based on the pending schedule entry.

[1604] Message Creation and Transmission

[1605] The server analyzes past conversation history and generates appropriate messages using natural language processing technology. The generated messages are then sent via the terminal using communication means.

[1606] Parsing reply messages

[1607] When a reply message is received, the server analyzes it and extracts the necessary information based on the reply. For example, if a message is received saying, "Please check if there are any problems with the maintenance of equipment A at 1:00 PM," the server analyzes the reply and performs maintenance on equipment A.

[1608] Making a reservation

[1609] The server accesses the reservation site as needed and automatically makes the necessary reservations, entering information such as the date and time, number of people, and special requests to confirm the reservation.

[1610] Schedule Updates and Notifications

[1611] The server updates new schedule information and notifies the user through the terminal, and the user can receive schedule information and instructions in real time through the smart glasses.

[1612] Automated Procurement

[1613] The server analyzes the response message and includes means for automatically procuring the necessary goods or equipment, for example, if a specific part is needed, it automatically orders that part.

[1614] Real-time notifications

[1615] The server notifies the user's smart glasses in real time of any new instructions or changes.

[1616] Hardware and Software Used

[1617] Smart glasses: Using devices such as Google Glass and Vuzix.

[1618] API endpoint: Build a server that manages schedule data and message sending.

[1619] Python: Implement the program that communicates with the server and processes the data.

[1620] Specific examples

[1621] 1. When a user puts on the smart glasses, the system displays today's schedule.

[1622] Example: "8:00 AM - Maintenance on Equipment A"

[1623] Example: "10:00 AM - Quality Inspection Meeting"

[1624] 2. If there is no scheduled maintenance, the system will automatically generate a message.

[1625] Example prompt: "Please check if there are any problems with the maintenance on equipment A at 1 PM."

[1626] 3. The reply is then analyzed and the necessary reservations and supplies are automatically procured.

[1627] For example, if you reply "Confirmed at 1pm", the reservation is confirmed.

[1628] In this way, this system can improve the efficiency of worker schedule management and communication, thereby improving productivity.

[1629] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1630] Step 1:

[1631] The server monitors schedule information. It periodically scans the schedule book database as input to detect pending schedule entries. It generates a list of pending schedule entries as output and passes it to the next step.

[1632] Step 2:

[1633] The server retrieves contact information from the database. It uses the list of pending schedule entries as input and reads the associated contact information from the database. As output, it generates a list of schedule entries with the contact information, which it passes to the next step.

[1634] Step 3:

[1635] The server analyzes past conversation history and generates messages using natural language processing. The input is a list of schedule entries containing contact information and past conversation history. Data processing involves analyzing the conversation history and generating appropriate messages using a generative AI model. The output is a list of the generated messages, which are passed to the next step.

[1636] Step 4:

[1637] The server sends the generated message through the terminal. It uses the generated message and the recipient's contact information as input. Specifically, it sends the message using a communication protocol. As output, it records the result of the message transmission and passes it to the next step.

[1638] Step 5:

[1639] The terminal receives the reply message and sends it to the server. It uses the received message as input and sends it to the server via the endpoint. As output, it creates an object of the received message and passes it to the server.

[1640] Step 6:

[1641] The server analyzes the reply message, using the object of the received message as input. For data processing, it uses natural language processing technology to analyze the reply content and extract information necessary for scheduling and procuring goods. As output, it lists the extracted information and passes it on to the next step.

[1642] Step 7:

[1643] The server accesses the reservation site as needed and makes the reservation. The extracted reservation information is used as input. Specifically, it uses the reservation site's API to input the necessary information and confirm the reservation. As output, it generates reservation confirmation information and passes it to the next step.

[1644] Step 8:

[1645] The server updates the new schedule information and notifies the user via the terminal. The updated schedule information and the device reservation confirmation information are used as input. As output, the updated schedule data is saved in the schedule book database and a notification message is sent to the terminal.

[1646] Step 9:

[1647] The terminal displays real-time schedule information and instructions on the user's device. It uses notification messages as input and displays them in real-time on the user's smart glasses. As output, the displayed schedule and instructions are confirmed by the user.

[1648] Step 10:

[1649] The server analyzes the reply message and automatically procures the necessary goods and equipment. The analyzed data is used as input. Specifically, it automatically places the necessary orders using the goods procurement site's API. As output, it generates order confirmation information and passes it to the next step.

[1650] Step 11:

[1651] The server notifies new instructions and changes in real time. It uses new instructions and change information as input. As output, it generates notification messages in real time and sends them to the terminal.

[1652] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1653] The present invention is a system that streamlines user schedule management, automates the acquisition of necessary information, and automates communication. It also has the ability to recognize the user's emotions and dynamically change messages and follow-up conversations based on those emotions.

[1654] System configuration

[1655] This system consists of the following main components:

[1656] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[1657] Terminal: A device operated by the user that communicates with the server, sends and receives messages, and collects emotional data.

[1658] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[1659] Emotion Engine Functions

[1660] The emotion engine has the following functions:

[1661] It recognizes the user's emotional state by analyzing their voice, text messages, and facial expression data.

[1662] Dynamically change message content and follow-up conversations based on perceived emotional state.

[1663] Program processing

[1664] Monitoring schedule information and collecting emotion data

[1665] The server periodically scans the schedule database to detect unconfirmed schedules for which the date, time, and location have not yet been set, and the device collects facial expression data from the user via voice input, text messages, or the camera, and sends this data to the server.

[1666] Obtaining information about the other person and creating a message

[1667] The server retrieves the contact information of the person previously entered from a database based on the pending schedule entry. Next, it analyzes past conversation history and the user's emotional data and generates an appropriate message using natural language processing. For example, if it determines that the user is feeling stressed, it generates a message such as, "Could you please suggest a location and time for our next lunch?"

[1668] Sending automated messages

[1669] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[1670] Parsing reply messages

[1671] When the other person sends a reply message, the device receives the reply and sends the contents to the server, which analyzes the received message and extracts the location and time information of the reply.

[1672] Make the necessary reservations

[1673] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[1674] New schedule updates and notifications

[1675] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[1676] Optional conversation generation

[1677] If the user has enabled automatic conversation generation, the server will generate a follow-up conversation based on past conversation history, new schedule information, and emotional data. For example, if the user is feeling stressed, the server will generate a message like, "Relax and have fun," rather than, "Your reservation is complete, and I'm looking forward to it."

[1678] Specific examples

[1679] A user schedules a business meeting, but the date, time, and location are not yet determined.

[1680] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[1681] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[1682] The client replies, "How about a conference room in the ABC building at 2pm?"

[1683] The terminal receives this reply and sends it to the server.

[1684] The server parses the reply and confirms the proposed location and time.

[1685] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[1686] The server updates the confirmed schedule and notifies the user and client.

[1687] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[1688] In this way, this system significantly reduces the user's workload by automating schedule management and communication, and provides responses that take the user's feelings into consideration.

[1689] The processing flow will be explained below.

[1690] Step 1:

[1691] The server periodically scans the schedule book database and detects unconfirmed schedules for which the date, time, and location have not been set.

[1692] Step 2:

[1693] Based on the detected pending schedule entry, the server retrieves the previously inputted contact information of the other party from the database.

[1694] Step 3:

[1695] The device collects facial expression data using the user's voice input, text messages, or camera, and transmits it to a server in real time or periodically.

[1696] Step 4:

[1697] The server analyzes voice, text messages, and facial expression data to recognize the user's emotional state, for example, detecting stress levels from voice tone and smiles or discomfort from facial expression data.

[1698] Step 5:

[1699] The server uses natural language processing to generate an appropriate message based on the user's past conversation history and emotional state. For example, if the user is feeling stressed, it uses calmer language and generates a message like, "Could you please suggest a place and time for our next lunch?"

[1700] Step 6:

[1701] The server converts the generated message into email or LINE format and sends it to the recipient via the device.

[1702] Step 7:

[1703] When the other person sends a reply message, the device receives the reply and sends the contents to the server.

[1704] Step 8:

[1705] The server analyzes the received message and extracts the location and time information of the reply.

[1706] Step 9:

[1707] Based on the extracted location information, the server accesses the reservation website of the target store or conference room and makes the necessary reservation, entering the date and time, number of people, special request information, etc.

[1708] Step 10:

[1709] The server acquires the reservation confirmation information and reflects it in the schedule book.

[1710] Step 11:

[1711] The server updates the schedule information to the schedule book database and notifies the user and the other party of the updated information via the terminal.

[1712] Step 12:

[1713] If the user has enabled automatic conversation generation, the server generates follow-up conversations based on past conversation history, new schedule information, and emotional data. For example, if the user is relaxed, the server generates a message like, "Your reservation is complete. I'm looking forward to it." However, if the user is stressed, the server generates a message like, "Relax and have fun."

[1714] Step 13:

[1715] The follow-up conversation generated by the server is converted into email or LINE format and sent to the other party via the device.

[1716] Example 2

[1717] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1718] Conventional schedule management systems require users to manually manage their schedules, requiring a great deal of time and effort for tasks such as obtaining contact information, sending messages, and making reservations. Furthermore, the system does not take into account the user's emotions, which can lead to a decline in the quality of communication. To solve these problems, a system is needed that can streamline user schedule management, automate the acquisition of necessary information and communication, and dynamically respond based on emotions.

[1719] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1720] In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and emotion data and generating a message using natural language processing technology, means for sending the generated message using a communication device, means for receiving and analyzing reply messages, means for accessing a reservation service and making a reservation, means for updating and notifying new schedule information, means for automatically generating conversations and sending follow-up conversations based on the user's emotions, and means for analyzing the user's voice input, text messages, and facial expression data to recognize the user's emotional state and changing responses based on this. This allows users to improve the efficiency of schedule management and automate communication, and also enables dynamic responses according to emotions.

[1721] "Schedule information" is information about plans and events entered by the user, including the date, time, location, and content.

[1722] The "monitoring means" is a means for continuously or periodically checking schedule information and performing processing based on specific conditions.

[1723] "Contact information" is information about a person or organization that is previously input by the user, including name, phone number, email address, etc.

[1724] A "database" is a system for managing, searching, and accessing data efficiently and quickly.

[1725] "Conversation history" is a record of messages and communications sent and received in the past.

[1726] "Emotion data" is data that indicates the user's emotional state. It is obtained by analyzing voice, text, facial expressions, etc.

[1727] "Natural language processing technology" is a technology that enables computers to understand, generate, and process human language.

[1728] "Message generation" is the process of creating an appropriate text message based on the input data.

[1729] A "communications device" is a hardware or software device for transmitting and receiving data or information.

[1730] A "reply message" is a reply message sent by the other party.

[1731] "Reservation Service" means a service that allows you to manage and make reservations online.

[1732] The "means for making a reservation" refers to the means for accessing the reservation service, inputting the information required for the reservation, and finalizing the reservation.

[1733] "Means for updating schedule information" refers to means for adding or modifying new or changed information in the schedule database.

[1734] "Means for notifying" refers to means for notifying users of updates and other important information.

[1735] "Automatic conversation generation" is a process that uses an algorithm to automatically generate appropriate conversation content.

[1736] A "follow-up conversation" is additional communication that occurs following a particular event or situation.

[1737] "Voice input" is a means of capturing the user's spoken voice as digital data.

[1738] "Facial expression data" is data obtained by analyzing the facial expression of the user.

[1739] An "emotional state" is a state that indicates how a user is feeling at a particular point in time.

[1740] The present invention is a system that streamlines user schedule management, automates the acquisition of necessary information, and automates communication. It also has the ability to recognize the user's emotions and dynamically change messages and follow-up conversations based on those emotions.

[1741] System configuration

[1742] The system consists of the following major components:

[1743] Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[1744] Terminal: A device operated by the user that communicates with the server, sends and receives messages, and collects emotional data.

[1745] User: The entity that uses the system, inputs schedule information, and receives schedule updates and notifications.

[1746] Emotion Engine Functions

[1747] The emotion engine has the following functions:

[1748] It recognizes the user's emotional state by analyzing their voice, text messages, and facial expression data.

[1749] Dynamically change message content and follow-up conversations based on perceived emotional state.

[1750] Program processing

[1751] The server periodically scans the schedule database to detect unconfirmed schedules with unspecified dates, times, and locations. The device collects facial expression data from the user using voice input, text messages, and the camera, and sends it to the server.

[1752] The server retrieves pre-entered contact information from a database based on pending schedule entries, analyzes past conversation history and user emotion data, and generates an appropriate message using natural language processing technology.

[1753] For example, if it is determined that the user is feeling stressed, the system generates a message saying, "Could you please suggest a place and time for our next lunch?" This message is then sent to the other party via the device.

[1754] When a reply is received, the device receives it and sends it to the server. The server parses the reply and extracts the suggested location and time information. If necessary, the server accesses the reservation site and inputs the date, time, number of people, and any special requests to confirm the reservation.

[1755] Once the reservation is complete, the server updates the schedule database with the new schedule information, and the terminal sends the updated information and a notification that the reservation is complete to the user.

[1756] If the user has enabled automatic conversation generation, the server will generate follow-up conversations based on past conversation history, new schedule information, and emotional data. For example, if the user is feeling stressed, the server will send a message to the client saying, "Relax and have fun."

[1757] Specific examples

[1758] A user schedules a business meeting, but the date, time, and location are not yet determined.

[1759] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[1760] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[1761] The client replies, "How about a conference room in the ABC building at 2pm?"

[1762] The terminal receives this reply and sends it to the server.

[1763] The server parses the reply and confirms the proposed location and time.

[1764] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[1765] The server updates the confirmed schedule and notifies the user and client.

[1766] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[1767] Prompt Sentence Examples

[1768] "Analyze the user's voice input, text messages, and facial expression data to recognize the user's emotional state. Then, generate an appropriate message based on the recognized emotion. For example, if the user is stressed, generate a message saying, 'Relax and have fun.'"

[1769] The above is an embodiment of the present invention. This system improves the efficiency of the user's schedule management and communication, and is characterized by its ability to dynamically respond to emotions.

[1770] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1771] Step 1:

[1772] The server periodically scans the schedule database to detect unconfirmed schedules with unset dates, times, and locations. The input is the schedule database, and the output is a list of unconfirmed schedule entries. Data processing includes analyzing the schedule entries and selecting unconfirmed entries.

[1773] Step 2:

[1774] The device collects the user's voice input, text messages, and facial expression data using a camera, and sends it to a server. The input is the user's voice, text, and image data, and the output is emotional data sent to the server. Specifically, when the user operating the device says, "Tell me what's on my schedule for today," the device performs voice recognition and image analysis to quantify the user's emotional state.

[1775] Step 3:

[1776] The server retrieves previously entered contact information from a database based on the pending schedule entry. The input is the pending schedule entry and the database, and the output is the retrieved contact information. Data processing includes searching for appropriate contact information using keywords related to the schedule entry.

[1777] Step 4:

[1778] The server analyzes past conversation history and the user's emotional data and generates an appropriate message using a generative AI model. The input is the past conversation history and emotional data, and the output is the generated message. Specifically, if the emotional data obtained indicates that the user is feeling stressed, the server inputs a prompt to the AI ​​model saying, "Generate an appropriate message in a situation where the user is feeling stressed," and obtains the generated message.

[1779] Step 5:

[1780] The generated message is sent to the other party via the device. The input is the generated message, and the output is the message sent to the other party. Specifically, the device uses a message sending application to send the message to the other party.

[1781] Step 6:

[1782] When the other party sends a reply message, the device receives the reply and sends it to the server. The input is the reply message from the other party, and the output is the reply data sent to the server. The device uploads the reply to the server via the Internet.

[1783] Step 7:

[1784] The server analyzes the reply content and extracts the suggested location and time information. The input is the reply message, and the output is the extracted location and time information. Data processing involves using text analysis methods to extract specific information from the reply data.

[1785] Step 8:

[1786] The server accesses the booking site and confirms the booking by entering the date, time, number of people, and any special requests. The input is the extracted location and time information, and the output is the booking confirmation information. The server uses an automated script to enter the information into the booking site.

[1787] Step 9:

[1788] Once the reservation is completed, the server updates the schedule information to the schedule database, and the terminal sends a notification to the user. The input is the reservation confirmation information, and the output is the updated schedule information and a notification message. The terminal uses the notification system to send the message to the user.

[1789] Step 10:

[1790] If the user has enabled automatic conversation generation, the server generates a follow-up conversation based on the past conversation history, new schedule information, and emotional data. The input is the past conversation history, new schedule information, and emotional data, and the output is the generated follow-up conversation. Specifically, it generates a message saying, "Relax and have fun," and sends it to the client via the terminal.

[1791] (Application example 2)

[1792] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1793] Autonomous vehicles are required to not only efficiently manage users' schedules and automate necessary information acquisition and communication, but also to recognize the user's emotional state and dynamically change responses based on that. Conventional systems manage schedules without taking the user's emotional state into consideration, resulting in a poor user experience. In addition, navigation efficiency is lacking because destinations based on the schedule must be set manually.

[1794] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring schedule information, means for retrieving pre-input contact information from a database, means for analyzing past conversation history and generating a message using natural language processing, means for sending the generated message using communication means, means for receiving and analyzing reply messages, means for accessing a reservation site and making a reservation, means for updating and notifying new schedule information, means for automatically generating a conversation and sending a follow-up conversation, means for recognizing the user's emotional state using emotion recognition means and dynamically changing the message content and follow-up conversation based on the emotion, and means for cooperating with the vehicle's navigation system and setting a destination based on the user's schedule. This makes it possible to improve schedule management that takes the user's emotional state into consideration and the efficiency of navigation in autonomous vehicles.

[1795] "Schedule information" is time-related information such as the user's schedule, appointments, and events.

[1796] "Monitoring means" refers to devices or software that have the function of checking schedule information in real time or periodically and detecting abnormalities or updates.

[1797] "Contact information" is data about a person's means of contact, such as phone number, email address, or address.

[1798] A "database" is an organized collection of data for efficiently storing, retrieving, and managing information.

[1799] "Conversation history" is a record of past conversations and communications.

[1800] "Natural language processing" is the technology that enables computers to understand, generate, and respond to human language.

[1801] "Communication means" refers to the interface or infrastructure for sending and receiving data and messages.

[1802] A "reply message" is the content of the response received from the other party.

[1803] "Means of analysis" are devices or software that have the ability to examine data and extract specific patterns or information.

[1804] A "reservation site" is a website that allows online reservations for specific services (restaurants, hotels, conference rooms, etc.).

[1805] The "means for making a reservation" refers to a device or software that has the function of accessing a reservation site, inputting the necessary information, and finalizing the reservation.

[1806] The "means for updating schedule information" refers to a device or software that has the function of updating the schedule database with new information or changes.

[1807] "Notification means" refers to devices or software that have the functionality to notify users or relevant persons of updated information or reminders.

[1808] "Automatic conversation generation" is a technology that uses algorithms and AI to automatically create dialogue with users.

[1809] A "follow-up conversation" is any additional communication that follows an initial interaction or event.

[1810] "Emotion recognition means" is a technology that analyzes voice, text, facial expressions, etc. to determine the user's emotional state.

[1811] A "navigation system" is a device or software that calculates and provides directions to a destination.

[1812] The "means for setting a destination" refers to a device or software that has the function of determining a destination based on the user's schedule information and inputting it into the navigation system.

[1813] This system streamlines user schedule management and automates necessary information acquisition and communication. It also recognizes the user's emotions and dynamically changes messages and follow-up conversations based on those emotions. This system, especially when applied to autonomous vehicles, improves the in-car user experience.

[1814] System configuration

[1815] The system consists of the following main components:

[1816] 1. Server: Serves as the central control unit, monitoring schedule information, retrieving contact information, generating and sending messages, analyzing replies, making reservations, updating schedules, and providing notifications. It also uses an emotion engine to recognize the user's emotional state and dynamically respond accordingly.

[1817] 2. Terminal: A device operated by the user that communicates with the server, sends and receives messages, collects emotional data, and works with the navigation system of the autonomous vehicle.

[1818] 3. User: The entity that uses the system, inputs schedule information, and receives updated schedules and notifications.

[1819] Program processing

[1820] Monitoring schedule information and collecting emotion data

[1821] The server periodically scans the schedule database to detect unconfirmed schedules for which the date, time, and location have not yet been set. The device also collects facial expression data from the user via voice input, text messages, or the camera, and sends this data to the server.

[1822] Obtaining information about the other person and creating a message

[1823] The server retrieves the contact information of the person previously entered from a database based on the pending schedule entry. Next, it analyzes past conversation history and the user's emotional data and generates an appropriate message using natural language processing. For example, if it determines that the user is feeling stressed, it generates a message such as, "Could you please suggest a location and time for our next lunch?"

[1824] Sending automated messages

[1825] The generated message will be sent to the other party via the device, which saves the user the trouble of manually sending the message.

[1826] Parsing reply messages

[1827] When the other person sends a reply message, the device receives the reply and sends the contents to the server, which analyzes the received message and extracts the location and time information of the reply.

[1828] Make the necessary reservations

[1829] If necessary, the server accesses the reservation site and automatically makes a reservation, entering information such as the date and time, number of people, and any special requests, and confirming the reservation.

[1830] New schedule updates and notifications

[1831] Once the reservation is complete, the server updates the schedule database with the new information. The device then sends the updated information and a notification that the reservation is complete to the user. The other person also receives a similar notification.

[1832] Linking with navigation systems

[1833] It works in conjunction with the navigation system of the autonomous vehicle to set the destination based on the user's schedule. To do this, schedule data is input into the navigation system so that the vehicle automatically heads to the destination.

[1834] Specific examples

[1835] A user schedules a business meeting, but the date, time, and location are not yet determined.

[1836] The server detects pending schedules and obtains the client's contact information. At the same time, the device collects the user's facial expression data and sends it to the server.

[1837] The server references past interactions and emotional data to generate a message saying, "We need to decide on a meeting location and time. Can you make any suggestions?" and sends it to the client via the device.

[1838] The client replies, "How about a conference room in the ABC building at 2pm?"

[1839] The terminal receives this reply and sends it to the server.

[1840] The server parses the reply and confirms the proposed location and time.

[1841] The server accesses the conference room reservation site and makes a reservation for 2 p.m.

[1842] The server updates the confirmed schedule and notifies the user and client.

[1843] Because the user has enabled automatic conversation generation, the message sent to the client is "Relax and have fun" instead of "Your booking is complete, we're looking forward to it."

[1844] Prompt Sentence Examples

[1845] "Generate a message to ask the user to suggest a location and time for an unscheduled event."

[1846] "Generate a message to help the user relax if they are currently feeling stressed."

[1847] As described above, this system significantly reduces the user's workload through the automation of schedule management and communication, and can respond in a way that takes the user's feelings into consideration. Furthermore, by linking with the navigation system of an autonomous vehicle, it can improve the efficiency and comfort of travel.

[1848] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1849] Step 1:

[1850] The server periodically scans the schedule book database. It takes all entries in the schedule book database as input and detects unconfirmed schedules (those for which the date, time, and location have not been set). It generates a list of unconfirmed schedules as output. This data processing or data calculation checks the fields of the schedule entries and lists the entries with unconfirmed fields.

[1851] Step 2:

[1852] The server retrieves the contact information of the other party that was previously input based on the pending schedule from the database. It uses the list of pending schedules as input and accesses the contact information database of the other party. It generates a list containing the contact information of the other party as output. In this process, it searches the database for the contact information of the other party that corresponds to each schedule entry and compiles it into a list.

[1853] Step 3:

[1854] The server collects facial expression data from the user via voice input, text messages, or a camera, and analyzes it with an emotion engine. The inputs are voice, text, and facial expression data from the user. The output is an analyzed emotional state (e.g., stress, joy). This process uses a generative AI model to analyze the emotional data and performs calculations to identify the user's emotional state.

[1855] Step 4:

[1856] The server uses natural language processing to generate an appropriate message based on the pending schedule, the other person's contact information, and the analyzed emotional state. Details of the pending schedule, contact information, and emotional state are used as input. The generated message is obtained as output. This process utilizes natural language processing with a generative AI model, and generates messages while also referencing past conversation history.

[1857] Step 5:

[1858] The server sends the generated message to the other party using a communication method. It uses the generated message and the other party's contact information as input. It obtains the sending completion status of the message sent to the other party as output. This process uses the messaging service API to perform the specific operations to send the message.

[1859] Step 6:

[1860] The device receives the message that the other party has replied to and sends it to the server. The reply message from the other party is used as input. The reply message sent to the server is obtained as output. This process uses the device's message receiving function to transfer the received message to the server via the network.

[1861] Step 7:

[1862] The server analyzes the reply message and extracts the location and time information of the reply. It uses the reply message as input and obtains the extracted location and time information as output. This process uses natural language processing technology to analyze the message content and performs specific calculations to extract the necessary information.

[1863] Step 8:

[1864] The server accesses the reservation site based on the extracted information and automatically executes the reservation. The extracted location and time information is used as input. The reservation completion status and reservation confirmation information are obtained as output. This process automatically inputs information using the reservation site's API and confirms the reservation.

[1865] Step 9:

[1866] The server updates the schedule book database with the new schedule information and sends a notification. It uses the reservation confirmation information and the new schedule information as input. It obtains the updated schedule and the notification sending completion status as output. This process performs the specific operations of updating the database and sending the notification using the notification service API.

[1867] Step 10:

[1868] The server works with the autonomous vehicle's navigation system to set a destination based on the user's schedule. It uses new schedule information as input and reflects the set destination information in the navigation system as output. This process uses the navigation system's API to perform specific operations to set the destination.

[1869] Step 11:

[1870] If the user has enabled automatic conversation generation, the server generates a follow-up conversation based on the user's emotional data and schedule information and sends it to the other party. The server uses the user's emotional data and schedule information as input. The generated follow-up conversation is obtained as output. This process uses a generative AI model to generate a follow-up conversation and sends it using the messaging service API.

[1871] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, t...

Claims

1. a means for monitoring schedule information; a means for retrieving pre-entered contact information from a database; and A means for analyzing past conversation history and generating messages using natural language processing; means for transmitting the generated message using a communication means; means for receiving and analyzing the reply message; A means of accessing the booking site and making a booking; A means of updating and notifying new schedule information; a means for automatically generating conversations and sending follow-up conversations; A system including:

2. 2. The system of claim 1, wherein the means for monitoring schedule information periodically scans a schedule book database.

3. 2. The system of claim 1, wherein said means for making a reservation comprises inputting information including date and time, number of people, and special requests, and obtaining reservation confirmation information.

Images